vSphere 6 Storage – The Future of Storage at Scale

Warning – This blog is not an engineering deep dive on everything VVOLs. There will be many of those launched at VMware Partner Exchange 2015 with incredible info.

This blog will be about the future of VMware Storage at Scale. First off congrats to VMware for the release of vSphere 6. In this release there are many new features. Some of these features are that MonsterVM’s can now scale to 128 vCPUs, 4tb RAM, hosts can run 480 CPU’s + 12tb RAM, Fault Tolerance support 4 VCPU (VMFT puts a VM into synchronous VMotion consistency lock so a workload can survive a reactive failure), Cross vCenter vMotion (this provides greater scale), Long Distance vMotion enhancements, and a “tech preview” of VMware API’s for IO Filters (VAIO). The theme here is ironically parallel to SolidFire’s consolidate, automate, and scale market message (SolidFire would add with guaranteed performance for every VM).

One last big point on vSphere 6; you’ll see enhanced service level management to Storage I/O controls. You might get the impression that this would fully resolve the need to use Quality of Service (QoS) in storage. That might be the case in small to medium designs but At-Scale you’ll find strong arguments supported from Cormac Hogan, Josh Atwell, Aaron Delp, and Dave Wright here, here, and here that SIOC+QoS is still the way to design workload assurances in Enterprise and Service Providers.

On to the main theme of this blog…

In the Enterprise and Service Provider, organizations are looking for more than just scripts and new ways to manage storage constructs like LUNS. The future will be unlocked at scale with vSphere Virtual Volumes (VVOLs) and new ways VMware vRealize will fully automate storage like never before. I’ll outline some of the constructs on 3 layers;

Storage administrator management
vSphere administrators management
Service catalog management of storage through policy automation finally “vRealized”.

Caveat #1 – This blog is for Enterprise and Service Providers at Scale which means 1000’s to 100,000’s of VM’s and application instances. This blog also addresses SMB and Mid-Enterprise through Service/Cloud Providers like CenturyLink, SunGuard, Calligo, and others. That is undeniable trend in IT for those markets is now to leverage Service/Cloud Providers rather than build their own. Service/Cloud Providers crave for new storage automation and therefore provide new efficiencies in providing IaaS to SMB/Mid-Enterprise.

At Enterprise and Service Provider Scale manual admin of storage isn’t reasonable although troubleshooting and manual task still persist. Automation is KING here at every level and that includes Powershell, Puppet, Chef, and the crown jewel for VMware.. VMware vRealize Automation. Hold up. Managing storage from vRA? I thought it was all going to be VVOLs?! Read below….

Storage Automation for Service Catalog Management

I’m putting vRA Storage Automation first as a way to help everyone know that in At-Scale Environments this is the 1st place you need to assess; Storage Automation at the application automation level. This is also my best attempt to not bury the lead of this story. VVOLs will get a lot of attention and that might seem to be the lead story. Not everyone will have VVOLs immediately (probably be iteratively through many providers through 2015 and early 2016) and you’d be left without ways to start meaningful storage automation for At-Scale environments. You’ll notice above that we have an application blueprint that we can create storage policies against. The “1-K-2K-5K-STD” are SolidFire QoS minimum, maximum, and burst I/O settings but you could call out any unique setting to a storage solution. This integration was the created by Rawlinson Rivera (@punchingclouds) with the help of some crazy smart vCO guys. It’s available now via the VMware Solutions Exchange here called “vRealize Automation and Storage Policy based Management” Basically it provides end users to present on demand storage capabilities for integrated array capabilities. This can be done through vSphere tagging or VVOLs, etc.

Here’s a peak at SolidFire’s newest “Tech Preview” of vRealize Automation with SolidFire Storage you’ll find at booth 616 at PEX2015. Note: You’ll see QoS, snaps, replication, and SRM protection capabilities all exposed as selectable options for application builds. Boom!

For more info on this tech preview video contact me.

Storage Automation for vSphere Admins

I separate this out as a middle layer of Storage Automation for vSphere 6 because vCenter management is always a core part of operating the Software Defined Data Center. The main theme is really just get all storage capabilities exposed as storage policies and start leveraging SPBM as mentioned above. vSphere 6 will provide standard LUN and VVOL ways to operate in this environment so don’t feel like you have to have VVOLs to get to SPBM benefits. Above I’ve shown an example of SolidFire exposing QoS settings for MongoDB, Standard VM’s, and Performance VM’s. You should be thinking about the association of VM’s to storage policies rather than just manually creating and managing each of those attributes per VM. The future with vSphere 6 is SPBM and automation!

Storage Automation for Storage Admins

Ah yes, finally we can obsess about managing LUN’s and disks right? vSphere 6 will provide you the flexibility to utilize Virtual Volumes (VVOLs) and/or traditional LUNs, etc. This is key because you’ll be able to mix and match to give you flexibility for implementation across a variety of storage solutions. Below you’ll see the high level on VVOLs associating VM’s through SPBM to Protocol Endpoints (PE’s) to Storage Containers. Having published capabilities a this storage layer will be huge and also vary a lot from storage vendors. Not all will have Dedupe, Encryption, or features like QoS. The key in the future here is the VASA provider. Study up over time on how each vendor leverages the Control Plane for truly becoming Software Defined Storage.