Last week, at SFD17, I attended a couple of very insightful presentations from Starwind and Intel about NVMe-oF and Intel’s SPDK. I think we are on the verge of a major change in the way datacenter storage will be organized, managed and accessed.
The SAN, as we know it, is dead
The traditional storage array and, more in general, the concept of storage area network have been showing their limits for a while now but nobody was ready for a radical change. Yes, HCI has changed the way we represent storage in a virtual environment. Scale-out architectures and integrations with upper layers make storage easier to deploy, consume and manage. And there are many more, and common, features than in the past but most of the basic concepts remain identical to what was originally available with the first SAN implementations in the mid ’90s.
In the last twenty years or so, we have seen many evolutionary steps of the original concept and technology: SCSI, iSCSI, FC, FCoE, bigger better hardware, virtualization, data protection techniques, data footprint efficiency, flash memory and so on. Thanks to all these advancements, primary storage has been commoditized and simplified but has never reached the level of flexibility required by next-gen cloud infrastructures.
The massive introduction of flash memory brought about the first limitations of traditional storage array architectures. And, for example, NVMe helped to remove many layers between the media and the controller… but this was just the beginning.
NVMe-oF is a game changer
NVMe allows to take full advantage of modern media types, (both flash and memory class storage) by reducing CPU load and latency to access data. Thanks to NVMe, the industry is now able to significantly reduce latency from milliseconds to hundreds of microseconds or less. Long story short, an SSD with an NVMe interface is faster and less resource hungry at the host level than identical devices equipped with SAS or SATA interface. But, again, this is just a more efficient way to access the media from the host.
NVMe-oF is a totally different story. In fact, it extends NVMe and allows devices to be accessed remotely. You can use it to substitute FC or iSCSI as transport protocol for a SAN, but there is much more than that. The real beauty of it is that you can access a remote device, installed on a different system, as if it were locally! A demonstration of this technology was presented last week by Starwind at Storage Field Day 17. They used Intel SPDK (Storage Performance Development Kit) to optimize resource consumption at the host level while reaching the limit of the devices involved in the demo (around 500K+ Iops at 500-ish uS of latency for an Optane-based PCI card!). In a word, impressive!
Thinking about composable storage
Now that was just a proof of concept, but let’s try to think for a moment about the next step. Up to now, composable infrastructures were no more than a fancy new implementation of the old blade servers. I’ve never found them really interesting, especially because of the limitation in the size of these boxes. With NVMe-oF this will change dramatically!
The second demo from Starwind was quite simple but showed the potential of the technology. Borrowing storage resources from other servers across the datacenter and building your virtual storage array by taking advantage of powerful local CPUs for data protection and optimization, is just amazing! And it is even more compelling if you think that the performance impact is negligible when compared to local media access.
Composable storage infrastructures are a step forward for hyperconvergence too, making performance metrics and predictability more similar to what you can get from a high-end array at a fraction of the cost.
Furthermore, the next step for NVMe is called NVMe-oTCP. To make a quick comparison, it is like FC and iSCSI. You trade some of the performance to get huge flexibility in terms of storage network layout!
Closing the circle
There are several vendors working on NVMe-oF, the standard is not really 100% standard yet, but it has happened for all first-generation implementations of any protocols. We will soon get to a point where NVMe, NVMe-oF and later NVMe-oTCP, will become the first choice for every enterprise grade storage solution. Getting the flexibility and composability described above will take longer, but I’m 100% sure that we will get there, and companies like Starwind have already demonstrated it. It’s not a bad idea to keep an eye on them to see how their product evolves…
Intel did a great job with its SPDK, not only because it removes all the bottlenecks to access NVMe devices but also because it is open source and can be used with non-intel devices too. I’m not a developer, and their presentation at SFD17 was quite technical (here the link to the video), but it looks like this is the way to go when it comes to developing a solution based on NVMe.
Disclaimer: I was invited to Storage Field Day 17 by GestaltIT and they paid for travel and accommodation, I have not been compensated for my time and am not obliged to blog. Furthermore, the content is not reviewed, approved or edited by any other person than the Juku team.