How Long Before 100x Better HDD Energy Efficiency?
Over the past 25 years disk sizes have grown as shown in this diagram:
The growth rate, as can be seen, is very linear. As can be seen in the diagram, we’ve risen from a 1GB HDD to a 100GB HDD in 10 years, or a 10GB HDD to a 1TB HDD in… 10 years.
Thus, every 10 years storage capacity becomes approximately 100 times more energy efficient to use as an archive.
Meanwhile, the fall in price for a HDD has been huge, but exponential. Back in 1980 you’d be looking at a staggering $233,000 for a GB of storage space, where as today a GB costs less than 20 cents. Placing sample data of HDD prices from 1998 to 2004, I produced the following:
So, growth in disk size is linear, whilst the fall in disk $/GB exhibits exponential decay.
Overall, this makes for very good archiving economics on HDD.
Better Disk Electrical Usage Economies
The amount of electricity used by a disk drive (regardless of storage capacity) has also fallen, though nowhere near as dramatically as the capacity per HDD has risen.
Let’s take a modern HDD: e.g., the Hitachi 1TB Deskstar 7K1000.B gives “43% energy savings when disk is on standby”. That’s 5.2 watts on standy instead of 8.4 watts. Therefore, in archive usage, if a disk is idle for 700 hours in a month, it’ll use 3.64 kWh.
This compares favourably to most typical HDDs 8 years ago; e.g., the 30.7GB Maxtor, which drew 6.8 watts on standby and 12.9 watts when seeking. In archive scenarios, we calculate this as a 30% energy draw saving.
Whilst 30% energy efficiency savings are never to be turned away, the fact that the Hitachi stores 1TB of data (33x the amount of data) is far more significant. Our conclusion? In the long run, more important than paying a premium for the latest “green technology” is buying a solution that allows the modular addition of capacity; a solution that allows the use of the best available technology at the time of that addition.
Taking the trend of increased disk space in a HDD unit and falling energy usage, it can be seen that 100x energy efficiency is being achieved every ~9 years.
New Technologies: 2.5” Drives and SSDs
All general electricity usage improvements on 3.5” drives are good but perhaps more exciting are the new 2.5” drives coming out and SSD technology in general.
A 2.5” drive uses 1.4 watts for Read/Write (e.g. Fujitsu 500GB, Hitachi Travelstar 5K500.B). No word on their standby electrical usage – but even taking the worst case of 1.4 watts that’s a significant improvement over 3.5” drives sucking 5.2 watts on standby.
Plugging the figures for 2.5” drives vs 3.5” drives into our electricity usage formulas results in a slightly higher electricity cost for a 2.5” drives configuration. Why? Because at the moment they store only 50% of the data, and because therefore one ends up using more CPUs, RAID controllers, etc. However, it’s getting close, and it wont be long before 2.5” drives will provide better TCO archive economics. OM therefore predict that within 8 years we’ll have 100 times better energy efficiency from HDDs!
SSDs are a longer term prospect, but a very promising one. Their electricity draw is significantly lower again, and people like (whatever the reality behind it) the concept of non-moving parts. I’d personally be VERY cautious about this technology up front – I believe it’ll take many years before the reliability of SSD will be proven as a suitable long-term storage medium and until then many people will get stung by storage that gets corrupted over time.
Optimum Time Between Disk Recycling Taking Into Account Energy Usage
When should an old unit of storage be rolled out in favour of a (presumably) larger capacity storage unit?
Let’s take this to be the point where a years energy usage to keep the old unit running is greater than the cost to buy a similar amount of storage in a new technology.
Projecting this into the future:
1. If current trends continue then disk sizes will be 4 times current sizes in 3 years time.
2. Calculations show that a 100TB MatrixStore cluster costs in the region of £2,271 per annum in electricity (including cooling, HDDs, CPUs); this is based on an 8p per kWh price for electricity and 24TB nodes. The purchase cost of 100TB (that’s 220TB before Raid and Mirroring) is £120k.
If we then assume that electricity costs rise at 10% per annum, and that rack space is NOT of a premium, we end up with a scenario where the electricity cost for a 100TB of storage “crosses over” the cost for a similar amount of storage after approximately 7 or 8 years.
At that point the new 100TB archive will arrive in 1/25th of the space, and will use 1/25th of the electricity.
Now, since rack space generally IS of a premium, we project rolling out old hardware every 6 years to be a good ballpark time-frame. (NB: this does not take into account the cost of moving the data).
Alternative Solutions: Tape
Changing tact a little here, let’s briefly discuss why tape based solutions might not be as green as you think.
Tapes need to be stored at the correct temperature and often are physically very space consuming. However, the main “non-green” factor of tape archives comes in if they need to be rolled out in their entirety, due to any of the following reasons:
- because of hardware maintenance costs rise for systems that are outdated
- because of higher manpower cost related to maintaining old tape formats / sizes compared to newer solutions
- because new demands on data usage require an updated solution
(to name but a few)
We do not believe it is unfair to suggest that tape solutions often get rolled out every 5 years or so.
When it comes to rolling out a tape solution, it may well be required to move all their data from old tape formats to new tape formats; a labour intensive exercise that is neither “green” nor economically.
In comparing a tape archive to a HDD archive; the cost of transferring data at the end of the tapes lifespan may well exceed the energy cost of keeping that data on spinning disk (and therefore easy to transfer).
Based on trends from the first 30 years of HDDs:
- Disk capacity rises by approximately 158.5% per annum
- Electricity usage per GB falls a hundredfold every nine years
- Economically, the overriding most important aspect of a disk-based archive is being able to roll in/roll out new/old technology as appropriate
- Being able to take advantage of SSD/other emerging technologies is advantageous
- With economics typical in our example, hardware should be rolled out every 5-7 years – possibly sooner if rack space is of a premium or electricity prices rises greater than 10% pa
- Alternative solutions often have hidden energy and “green” costs related to manpower required and frequent technology form factor redundancy
- As the costs have fallen, new demands for HDD data storage and usage have come in and will continue to do so
Comparing keeping an archive on disk to the cost of doing the same 10 years ago, that cost has fallen rapidly in terms of hardware, manpower and electrical usage. There is no reason not to believe that trend will continue.