Hyper-V Scenario: Diskless Hyper-V Node

Motivation/Goal:

While sketching out the move from stand alone Hyper-V hosts to a clustered environment, one of the ideas that we tossed around was going for diskless nodes, having them boot via iSCSI Remote Boot from our SAN. The primary reason for us being having all HDDs centralised in the SAN, making monitoring harddrive failures easier and getting the flexibility to move the disks around depending on IO needs.

Equipment:

As NIC we used a Intel x520 DA SFP+ 10Gbit Dualport card, as Storage the Equallogic PS 6xxx series. Some of the preparation will eventually be different with other vendor adapters (I assume), so thats why I mention the equipment. As Operation System we used Windows 2008 R2 Datacenter Edition (Service Pack 1 level). Feel free to add your experience with other equipment.

Preparation:

Per default (the test scenario was conducted Q1/12), the Intel NIC we used (x520) had a ROM that doesn't support iSCSI Remote boot, so you will have to flash the card with the corresponding ROM (there are versions for PXE, iSCSI and so on). For that we prepared a USB Stick with DOS, so the tool to flash could run from it. Depending how the stick reports to the BIOS, you cannot partition a USB stick. Keep that in mind when you get one, as DOS (or more precisely FAT used by DOS) doesn't support partitions greater 2GB / 4GB (http://support.microsoft.com/kb/118335). Refer to http://www.intel.com/support/network/adapter/pro100/sb/CS-023748.htm for the Flash Utility and the ROMs. After flashing you can enable the iSCSI ROM with the bootutil and configure the iSCSI Target which you want to use as system disk. Another thing you have to keep in mind is that while the NIC can print during the boot process that the drive was recognized and connected, Windows setup might show you no available disks. This is most likely the missing NIC driver (e.g. the 10 GBit NIc drivers from Intel aren't included with the Windows media). You have to add them during the Windows setup and then refresh the disks and it will show up.

Result:

The general setup worked just fine, booting from iSCSI was no problem. However, as we intended to use the host for Hyper-V, we stumbled across one issue that ultimately made the scenario not working for us: The NIC used for boot from iSCSI isn't showing up in Hyper-V Virtual Network Manager, so you cannot use it for a virtual switch (screenshot). Sacrificing a whole 10GBit port for the system drive wasn't worth it in our scenario. It might be an option for environments build with 1GBit though, where both number of ports is usually higher (quadport NICs) and the cost per port is lower.

Research:

There is not a lot of information about boot from iSCSI on the internet, at least not that I found it. I did some inquiring on the Intel forum to find out if the limitation is coming from the NIC driver or the operating system. I got no technical info that would allow a sophisticated answer to that. There is one document (found here http://www.microsoft.com/en-us/download/details.aspx?id=18986) with some technical insight, but not enough to answer it definitely. The comments on page 66 make me believe though (no changes required to NIC driver) that it is a windows limitation. So I will give it a try again now that Windows 2012 is out and will edit the article with my findings.