Planning storage is a simple thing, you go to your Storage Admin’s and say, I need x amount of LUNs of this size please for my ESX servers and they NO, we only do xGB size LUN’s, or they breath thought their teeth like a motor mechanic or plumber and say, Storage doesn’t grow on trees you know, we don’t have much left, are you sure you really need all that space, etc.
But I digress. 🙂
The majority of people know and understand that you can have a maximum of 256 LUNs per ESX host, yes that is correct 256, that means that your cluster can have almost 1/2 a petabyte of storage attached to it. :O
however the fact is you will reaching that maximum will be quite difficult, I shudder to think of the reboot times due to HBA enumeration.
The limit that you will most likely reach is “Maximum Paths”; this limit is set at 1024 per server.
wow that seems a large number you say how will I ever reach that; unfortunately the answer is quite easily.
here is a quick formula to aid you in Max Path / LUN numbers
number of HBA’s * number of Ports * number of connections to fabric form San / Max Path per server.
So a Quick example. if you have hosts that have two dual port HBA’s that connect to a SAN with four uplink connections, (average now a days), your formula will read:
2 * 2 * 4 /1024 = 64
this means that the maximum number of LUNs that can be attached to a single ESX host is 64. this too is not an insignificant number, however and here is the crux of the issue, 6 LUNs can be very quickly eaten up if you are dealing with MSCS which cross boxes. Remember there is a requirement that your Cluster shared storage be on RDM’s.
So for example if your Cluster has several high IO resource groups you would create a RDM for each mapped Drive, you can see where I am going now.
Now couple this with a requirement for point in time backup of groups of servers. Whoooooo, what is this and where did this requirement come from, well obviously it came via the back door, over the afternoon drinking session 😀
You know the sort of conversation, My boss was talking to your boss about the conversation he overheard the SAN guys having about consistent pools of storage being able to be snapshotted and the backup are kept in point of time synchronicity and the next thing you know it has become a requirement (all post design sign off, of course).
So out comes the Fag paper, and the design starts. but wait, where is the Virtualisation TA, oh he is on leave, he will be back on Monday, OK we need to get this sorted, but remember to run it by him on Monday, (emmmm, guess what, yep they never did)
So the “Technical” meeting starts and before long there are another 20 RDM,s that need to be created. this goes along and suddenly the Virtualisation team requests some new storage for their VMFS partitions and 2 of the 5 LUNs they have requested fail to attach and you discover that you are at 66 LUNs.
Quick and dirty solution is to pull 2 of the Connections out of the back of each of the ESX servers to reduce number of paths from 8 to 16.
And then you finally start the proper “Technical Review” meeting.
Well that was my Doh moment for yesterday, I wonder what today will bring :S