If you have worked in IT for more than a few years you probably know that it is not easy to get all the different components of an IT Infrastructure to work nicely together. If you have found or even created a combination that works you will want to stick with that set as long as possible. Storage is the foundation of your virtualization setup and in most cases it is also one of the most challenging parts of your IT infrastructure in terms of getting the right performance and availability for the right price. Therefore I can understand that the idea alone of moving to a different storage platform can be daunting.

The first step to take to get an answer to the question ‘how do I know if my organization is ready for Software Defined Storage?‘ is to make a list of all the storage systems and storage types you currently use and what you use them for. You might for example have a NetApp FAS or HP Lefthand system as your primary storage system to run your VMWare virtualization platform. You might use a couple of Windows Servers for serving files to your Windows workstations and you could be storing your Veeam backups on an HP 2000 array and you recently bought a Synology NAS since the HP array ran out of space. Oh and you almost forgot you where using a few Intel SSD of a few years old to run your development and infrastructure test environment on.

The nice thing about Software Defined Storage is that it is very flexible.

The nice thing about Software Defined Storage, that is if you use vendor independent Software Defined Storage (but that is for another blog), is that it is very flexible. That flexibility is something you should handle carefully so you don’t accidentally throw away the baby with the bathwater. You can use Software Defined Storage to make your storage very cheap by choosing to use consumer grade hardware. You can use Software Defined Storage to start out very small, and grow gradually. You can also use Software Defined Storage to get a lot more performance for your intended use than if you spend the same amount of money on good old vendor based storage solutions, but still use enterprise hardware. You can also use re-purpose your old server hardware to offer storage now.

If you want to be successful in implementing Software Defined Storage in your own organization it is smartest to start out when it is time to replace or expand one of your storage needs. In most cases it’s best not to start with an application that needs full performance and full availability. You will want to feel comfortable and at ease with your new storage product before you put it in a location that always has to work. One of the easy targets to start using Software Defined Storage is in backup storage systems. Especially if your current backup space is running out of capacity and you are using ‘quick fix’ expansions like a Qnap or Synology NAS to store your enterprise backups. But you could also start out by buying a small setup for running your IT infrastructure test environment on. If you are currently experiencing performance or scalability problems with your tier one, production, storage environment it might still be interesting to take a look at some of the software defined storage solutions, but you might want to ask for expert advice from Software Defined Storage integrators with a few years of experience.

Hardware is much more important than the software you are going to use mostly…

After you have selected the storage system you want to replace or expand you can start looking at the software options… and the hardware options of course. Yes, in storage hardware is in most cases much more important than the software you use. If you use unreliable hardware, or hardware that does not meet the expected performance you require your total storage solution, no matter what wonderful storage software you use, your total solution may turn out a total failure. And unlike when you buy a total solution from a vendor, the result will be as good as the people and process that have put it together. The good part of this of course is that when you have put a system together with your own team, and it works as expected or better, your team deserves all the glory. If you are going to need good quality enterprise storage hardware you are best off by getting in contact with a company that sells software defined solutions and does not sell closed solutions.

The best way to start is have a look at the software options for your use-case, make a list of them and see which types of hardware you might need for them. Software Defined Storage options that you could have a look at that I consider Enterprise Ready are: Open-E JovianDSS, OSNexus QuantaStor (includes ZFS, Gluster and Ceph), Compuverde vNAS, NexentaStor, Hedvig, VMWare VSAN, or Linbit DRBD. Of course new options emerge every day and some of them are really promising, but after looking at new Software Defined Storage solutions for over a decade my advice is to not try any new options unless you have a lot of time and enterprise equipment on your hands to build the experience you are going to need to see what works and what doesn’t. If you do decide to go that road, let me know and I might be able to give you a few hints in the right direction.

Users might be tempted to use your Proof of Concept for production purposes if it works for them…

When you have reviewed the software options and decided which ones might be fit for your organization it’s time to build a Proof of Concept or PoC. Build a smaller version or a starter version of your envisioned storage solution and test it hard and long. This is best done in an environment that is as separated from your production environment as possible. The real danger with storage PoC’s is that when it works functionally and the performance is good or better than what your users are used to they might be tempted to use it for production purposes right then and there. Then when you try to stress the limits of your PoC and it breaks and worst case data is lost at best your user will have lost a lot of confidence in software defined storage. If you are building something that is not backup storage, make sure to include the backup and recovery options in your PoC. Take your time in running a PoC, let it run for a longer period of time while you focus on other things. That way you can see how stable your storage really is. A good duration for a storage system is at least a few weeks. When you are using an combination of hardware and software you have not used before, a month or more is usually a good idea. During that whole period, measure, monitor and administer your complete PoC environment as if it was production, this will also give you a feel of how good it is to maintain. And please don’t overdo it, if you are currently only looking at your storage system only once a week that should be enough for your software defined solution too. You can declare your PoC as successful when it performs as predicted, it is as stable as predicted and as maintainable as predicted. Of course measured by the goals and objectives you set before implementing your PoC.

After a successful PoC it is time to introduce your new setup to your business users. How this can be done is dependent on the  business IT requirements of course. My personal preference is to set a ‘beta period’ during which you give extra attention to all the aspects of the new setup, but especially to the storage. Have a close look at how it performs under real load. Keep a close eye on logging options and if you see anything you do not understand, look them up or ask your software vendor or integrator in that moment. It will not be the first time that something fundamentally wrong in your setup could have been detected in this early stage. Things as simple as a weak connector, a typo in the configuration or software bug that stands out on your platform, can break any storage system and cause severe problem when not dealt with soon. At the end of the ‘beta period’, when you are confident with your setup and everything is running smoothly you could do an official launch to your organization and possibly to the public too. As stated earlier in this blog: building and running a high quality Software Defined Storage Solution is an accomplishment and it is most definitely also a compliment of the team that integrated it in to the organization.

This blog was originally posted on koendejonge.nl personal blog site.
It has also been posted on LinkedIn.

About

Koen de Jonge is owner and leader of ProcoliX and Inprove IT Infrastructure Solutions. ProcoliX delivers IT Infrastructure as a Service (Managed Hosting, IaaS). Inprove delivers IT Infrastructure Solutions with a very strong focus on Software Defined Solutions.

Inprove IT Infrastructure Solutions is Elite partner of HGST, Nexenta partner since 2008, Compuverde partner, Partner of OSNexus, VMware partner, Linbit partner, Hedvig partner, partner of DataON Storage, and partner many more interesting IT Infrastructure Software and Hardware vendors.