Lately I have been writing a lot about the role of flash memory and disk drives in the future of storage infrastructures (here and here a couple of examples). And latest news is simply confirming that it will be all about Flash for primary workloads and Object Storage (and disks) for the rest.
Reality is that Flash is quickly becoming already a no-brainer when it comes to IOPS per GB in primary data scenarios, but pure capacity is a totally different thing and it will continue to be deployed through disks for a long time yet. Even if you find a good deal, and flash price will continue to shrink, the difference in terms of $/GB will still be huge in the next few years.
Your mileage may vary
$/GB is/was the parameter used in the past to calculate the cost of storage with traditional arrays. But things have become a lot fuzzier with all the modern data services and compression techniques.
$/GB could vary substantially from user to user and it often depends on the type of data you are storing, its compressibility and other factors strictly related to overall efficiency of the storage system and to how it is used.
Last week SanDisk launched a new box, a sort of “JBOF” array (Just a Bunch Of Flash). It’s amazingly dense and cheap (less than $2 per GB, they say) but if you add data protection (a RAID or a file system, like CEPH or ZFS for example), things could change. Just think about mirroring it, the $2/GB price tag suddenly becomes $4 per GB!
Reality check
Just to make it easier for every one, I’m going to expose facts about $/GB and $/IOPS using the easiest possible way. I will compare prices that I found in a recent HGST price list. These are list prices for one single disk purchase (which means that the final price could be a lot less if you are an OEM or you buy in quantity). One of the prices comes from Amazon (and this is a consumer-grade product which means, also in this case, that the price can be much lower for the vendor/reseller).
These are the 4 products:
Performance:
– SAS: HGST ULTRASTAR C10K1800 – 2,5”/10K 900GB SAS – $350
– SSD : HGST s841 – 2.5” SSD SAS, MLC 15nm, 800GB- $2440
Capacity:
– SATA1: HGST Ultrastar He8 – 3,5” SATA 7.2K RPM, 6TB – $298
– SATA2: HGST Deskstar – 3,5” 5.4K RPM – around $150 on Amazon.com
First of all, it is important to understand that we are comparing simple components (raw material), and any operation you perform on the front-end of an array produces several operations in the backend. RAID, block-size optimization, compression/recompression, deduplication, bus or CPU limits, and so on, limit the number of IOPS that are effectively seen by the hosts.
Let’s do the math:
[table id=1 /]
* considering a 50000 IOPS 4k random workload (which, again, aren’t the IOPS you obtain from your array)
It’s plain to see that SSD wins hands down if the comparison is about $/IOPS and IOPS/GB. But on the other hand, there is a great abyss between the 0.037-0.049 $/GB registered by SATA drives and the others.
And the loser is…
It’s also quite clear that the big loser here is the 10K SAS drive. It doesn’t come out the winner in any comparison and the technology won’t be developing that much in the future, results can only get worse over time.
At the same time, the consumer-grade SATA $/GB is almost 100 times better than Flash. Yes, 100!
To be fair, a consumer-grade flash disk (w/ SATA interface) is only 10x the $/GB of SATA, but even with a Y/Y price slash of 40% it will take 5/6 years to have comparable prices, without considering that we already have much more denser hard disks in the pipeline.
In the near future we will be seeing many more (petabyte-scale) storage systems using consumer-grade (low cost) SATA drives coupled with modern data protection techniques (like distributed Erasure Coding for example) which help to manage multiple disk faults and longer rebuild time. For example HDS, with its HCP S10, is already doing it.
Closing the circle (…and stating the obvious)
Flash wins in IOPS, HDD wins in Capacity, and this won’t be changing anytime soon.
All-Flash storage systems are cost effective because flash allows to use in-line deduplication and compression at virtually no cost. Their predictability is very high and it’s good for high-demanding primary workloads.
At the same time, hybrid systems are still better than All-Flash when the demand for IOPS is not too high. In this case, the “IOPS/GB” shown in the above table/chart is an important metric. If you want to build a balanced IOPS/TB system you still need Flash+SATA. This is why most SMB customers are still going hybrid and companies like Tegile and Nimble are doing so well.
Last but not least, high capacity systems will continue to be nearly All-Disk for a long time. And I say nearly because today most object storage solutions use RAM for metadata and cache. Tomorrow, with very high capacity disks, it could be cheaper to store some information on a relatively small SSDs.
I think what this really highlights is the soon-to-be death of “enterprise” SSD – the price difference is simply too high to sustain it.
When you scale up the price of enterprise SSDs to cover an array, you end up with comparisons like these 2, each costing around $10000 for the drives:
6 x 800GB enterprise SSDs for 3.2TB of usable space at RAID5 with 1 hot spare
40 x 500GB consumer SSDs for 8TB of usable space at RAID10 with 8 hot spares
If you are a disk array manufacturer and you’re reliant on the high-end enterprise flash drives, then you already look very expensive, and soon you’re going to start looking unreliable, compared to masses of consumer drives.
Ewan,
thanks for commenting.
My POV is that enterprise SSDs differs from consumer-grade SSD in many aspects. Durability, performance, predictability, and so on.
I agree with you that price is very different but most vendors don’t apply the same guarantee to consumer-grade flash for enterprise/datacenter workloads or for more than 1year! which means that you’ll end in buying many more drives to sustain fails and performance degradations over time. This without considering rack space and power consumption.
They certainly differ in a number of ways, but the very low cost drives are improving much faster than the high-end ones.
For example, look at the Crucial MX200, it’s got an endurance of “only” 87GB writes per day for it’s 500GB model, but that’s up from 40GB per day on the MX100 model released last year. It’s available for $200 for 500GB.
IBM, EMC, and the like won’t be giving 3 year unlimited warranties on eMLC SSDs any time soon, but Pure, Nimble and the other new “SSD native” or hybrid array manufacturers seem to manage..
I agree though there’s more too it than simply dropping SLC based drives now, but if I was a manufacturer who simply had to ship SLC to make their product work, I’d be very concerned.
Pure, SolidFire, and all other startups are using MLC drives very efficiently (this is why they can allow to offer a lifetime warranty… when covered by support). traditional arrays don’t. that’s all. 😉
Great article, My comment is on this statement… “it will be all about Flash for primary workloads and Object Storage (and disks) for the rest.” I disagree. The $/TB of tape is too attractive for archive and disaster recovery (for certain) as well as for backup (for the foreseeable future) to ignore. For those environments, you can’t beat tape. So my question is, when do we start seeing FLAPE systems? 😉