I/O

Welcome!

For a first post I figure I'd talk about one of the main issues I've found while virtualizing machines within any of the technologies from VMware ESX , Citrix Xenserver, ... I/O capabilities of the storage where these virtual files or partitions reside whether connected by IP networks (NFS, iSCSI), Fiber channel, and local storage. Storage medium usually consists of either SATA, SAS, SCSI, FC, and SSD. I'm going to make an attempt to speak about these different technologies.

First let me dis-arm the idea where FC is faster then iSCSI and NFS is the worst. This all depends on how it's implemented. When iSCSI is configured to use 10Gb/s networks it can easily surpass 4Gb or even 8Gb FC just as NFS can easily be as fast as the other technologies. The real difference becomes whether or not multi-path is enabled and if the storage is capable of these performance levels in the first place. Multi-path brings a couple benefits; the first being the aggregate bandwidth of all the connections added together. If mating the 2 appropriate technologies each I/O request made can be channeled through a separate path. The other benefit can be realized from not having a single point of failure. Typically if the technology supports this configuration it will have the ability of failing over to another path, or re-issuing I/O requests if the request never comes back with an acknowledgement. If configured properly your virtual machines will not crash but simply hang for a short period of time then regain activity when the requests have timed out. I will speak to this in greater detail in a later post of how to configure some of these technologies.

Another issue relating to storage is how it itself is configured. I've noticed a huge difference in whether SAS or SATA or if it is configured as a RAID5 or a RAID 10 or 50. I'll let Wikipedia ( http://en.wikipedia.org/wiki/RAID ) define RAID for me :-) however understanding the differences and trade-offs of each type can lead you to disaster or complete success even within the same disk technology. What I mean by this - it is possible to see a RAID 50 or RAID 10 SATA storage array achieve closely the same performance level as a RAID 5 SAS array. Let's say for example a SATA disk is capable of 100 iops (input output operations per second) and a SAS disk is capable of 175 iops. Keep in mind there are other contributing factors but these are averages.

If we take a 10x SATA 1TB disks RAID 10 and show for every spindle capacity we actually have 200 iops
RAID 10 = double the spindles for 5TB capacity = 2 x iops = 1000iops for 5TB or storage

Next we take 9x SAS 600GB disks RAID 5 and show for every spindle capacity we actually have 155iops
RAID 5 = 1.125 spindles for 4.8TB capacity = .889 x iops = 1400iops for 4.8TB of storage

In this example we can see SAS still leads by 140% however the cost difference could be an interesting story. From this example we could also see if we configure 6 SATA disks in a RAID 5 for 5TB of capacity our performance is substantially less.
RAID 5 = 1.2 spindles for 5TB capacity = .833 x iops = 500iops for same amount of storage.

This is substantially less then even our RAID 10 configuration. There are some other considerations due to relative read and write performance of each technology. Write performance comparing a RAID 5 and 10 could alone even out the numbers in the above equations between SAS and SATA. In a RAID 5 all bits must be written to all disks, the parity bit calculated and written. This must happen for each I/O request and can only happen in succession. When the process is applied to multi array type RAID levels as in RAID 10 or 50 these operations can happen simultaneously with most current generation controllers. This can also improve the performance of the overall system. Considering this with the above equations we could potentially realize an additional 25%-75% penalty depending on the amount of writes.

Considering a virtualized environment where we are not simply dealing with documents and SQL databases, we are dealing with virtual disks and every read and write occurring corresponds with virtual disk blocks the virtual operating system is changing. The point here is these disk blocks can be larger then the standard I/O chunks we have been used to traditionally dealing with.

Bottom line - when planning a storage system for virtualization consider the number of virtual machines and each machine's I/O requirement in the physical world or the performance level desired and add about 10%-20% for the extra virtualized layer in between.