- Vmware Cluster Setup
- Vmware Failover Cluster Shared Disk
- Vmware Linux Cluster Shared Disk
- Vmware Windows Cluster Shared Disk
In vSphere 6.0, you can configure two or more VMs running Windows Server Failover Clustering (or MSCS for older Windows OSes), using common, shared virtual disks (RDM) among them AND still be able to successfully vMotion any of the clustered nodes without inducing failure.
- Again, shared storage available to cluster nodes, Windows Server 2003, 2008, and 2012, will be defined by VMware for VMware Workstation. Windows Server 2008 at the physical level supports FC, iSCSI, and SAS for shared storage. Whatever VMware allows, you can use with Windows Server 2008.
- Creating a Windows cluster: part 2 – Configuring shared disk in the OS Continuing along from the previous installment of this series (Using iSCSI to connect to Shared Storage) we’ll take a look at configuring the disks you have attached to your prospective cluster nodes.
Applies to: Windows Server 2019, Windows Server 2016, Windows Server 2012 R2, Windows Server 2012
You need the following hardware to create a failover cluster. To be supported by Microsoft, all hardware must be certified for the version of Windows Server that you are running, and the complete failover cluster solution must pass all tests in the Validate a Configuration Wizard. For more information about validating a failover cluster, see Validate Hardware for a Failover Cluster.
Servers: We recommend that you use a set of matching computers that contain the same or similar components.
Network adapters and cable (for network communication): If you use iSCSI, each network adapter should be dedicated to either network communication or iSCSI, not both.
In the network infrastructure that connects your cluster nodes, avoid having single points of failure. For example, you can connect your cluster nodes by multiple, distinct networks. Alternatively, you can connect your cluster nodes with one network that's constructed with teamed network adapters, redundant switches, redundant routers, or similar hardware that removes single points of failure.
Note
If you connect cluster nodes with a single network, the network will pass the redundancy requirement in the Validate a Configuration Wizard. However, the report from the wizard will include a warning that the network should not have single points of failure.
Device controllers or appropriate adapters for the storage:
- Serial Attached SCSI or Fibre Channel: If you are using Serial Attached SCSI or Fibre Channel, in all clustered servers, all elements of the storage stack should be identical. It's required that the multipath I/O (MPIO) software be identical and that the Device Specific Module (DSM) software be identical. It's recommended that the mass-storage device controllers— the host bus adapter (HBA), HBA drivers, and HBA firmware—that are attached to cluster storage be identical. If you use dissimilar HBAs, you should verify with the storage vendor that you are following their supported or recommended configurations.
- iSCSI: If you are using iSCSI, each clustered server should have one or more network adapters or HBAs that are dedicated to the cluster storage. The network you use for iSCSI should not be used for network communication. In all clustered servers, the network adapters you use to connect to the iSCSI storage target should be identical, and we recommend that you use Gigabit Ethernet or higher.
Storage: You must use Storage Spaces Direct or shared storage that's compatible with Windows Server 2012 R2 or Windows Server 2012. You can use shared storage that's attached, and you can also use SMB 3.0 file shares as shared storage for servers that are running Hyper-V that are configured in a failover cluster. For more information, see Deploy Hyper-V over SMB.
In most cases, attached storage should contain multiple, separate disks (logical unit numbers, or LUNs) that are configured at the hardware level. For some clusters, one disk functions as the disk witness (described at the end of this subsection). Other disks contain the files required for the clustered roles (formerly called clustered services or applications). Storage requirements include the following:
To use the native disk support included in Failover Clustering, use basic disks, not dynamic disks.
We recommend that you format the partitions with NTFS. If you use Cluster Shared Volumes (CSV), the partition for each of those must be NTFS.
Note
If you have a disk witness for your quorum configuration, you can format the disk with either NTFS or Resilient File System (ReFS).
For the partition style of the disk, you can use either master boot record (MBR) or GUID partition table (GPT).
A disk witness is a disk in the cluster storage that's designated to hold a copy of the cluster configuration database. A failover cluster has a disk witness only if this is specified as part of the quorum configuration. For more information, see Understanding Quorum in Storage Spaces Direct.
Hardware requirements for Hyper-V
If you are creating a failover cluster that includes clustered virtual machines, the cluster servers must support the hardware requirements for the Hyper-V role. Hyper-V requires a 64-bit processor that includes the following:
- Hardware-assisted virtualization. This is available in processors that include a virtualization option—specifically processors with Intel Virtualization Technology (Intel VT) or AMD Virtualization (AMD-V) technology.
- Hardware-enforced Data Execution Prevention (DEP) must be available and enabled. Specifically, you must enable Intel XD bit (execute disable bit) or AMD NX bit (no execute bit).
For more information about the Hyper-V role, see Hyper-V Overview.
Deploying storage area networks with failover clusters
When deploying a storage area network (SAN) with a failover cluster, follow these guidelines:
Confirm compatibility of the storage: Confirm with manufacturers and vendors that the storage, including drivers, firmware, and software used for the storage, are compatible with failover clusters in the version of Windows Server that you are running.
Isolate storage devices, one cluster per device: Servers from different clusters must not be able to access the same storage devices. In most cases, a LUN used for one set of cluster servers should be isolated from all other servers through LUN masking or zoning.
Consider using multipath I/O software or teamed network adapters: In a highly available storage fabric, you can deploy failover clusters with multiple host bus adapters by using multipath I/O software or network adapter teaming (also called load balancing and failover, or LBFO). This provides the highest level of redundancy and availability. For Windows Server 2012 R2 or Windows Server 2012, your multipath solution must be based on Microsoft Multipath I/O (MPIO). Your hardware vendor will typically supply an MPIO device-specific module (DSM) for your hardware, although Windows Server includes one or more DSMs as part of the operating system.
For more information about LBFO, see NIC Teaming Overview in the Windows Server Technical Library.
Important
Host bus adapters and multipath I/O software can be very version sensitive. If you are implementing a multipath solution for your cluster, work closely with your hardware vendor to choose the correct adapters, firmware, and software for the version of Windows Server that you are running.
Consider using Storage Spaces: If you plan to deploy serial attached SCSI (SAS) clustered storage that's configured using Storage Spaces, see Deploy Clustered Storage Spaces for the requirements.
More information
I’ve always been a fan of scale-out storage architecture, I’ve always said that The future of storage is Scale Out, and I’ve spent a fair amount of time studying software-only solutions like Ceph. The new solution from Microsoft, Storage Spaces Direct, seems like another great technology that will be soon available to us, so I decided to test it in my lab.
Vmware Cluster Setup
Storage Spaces Direct
Storage Spaces Direct is a new share-nothing scale-out storage solution developed by Microsoft, that will be soon available as part of Windows Server 2016. If you want to learn more about it, this one is a really good starting page. To test this solution using Windows Server 2016 Technology Preview 5, I’ve decided to run the entire solution nested in my VMware lab. There are some configurations and steps that need to be taken to make it run inside virtual machines. The final result is going to be a 4 nodes cluster, as this is the minimum amount of nodes that is required.
Storage Spaces Direct can be shortened as S2D, and it’s the name you will see in this article.
The virtual machines
For my lab, I’ve built 4 different virtual machines, with these hardware specifications:
4 vCPU
16 GB RAM
System disk 40 GB
Two network connections
In addition to this, I’ve added 4 hard disks. They are all connected to a dedicated SCSI controller, that is configured with Physical SCSI Bus Sharing:
This is paramount to guarantee the correct identification of the disks by the Storage Spaces wizards.
The 4 disks connected to this controller has then to be created as Thick Provision Eager Zeroed, otherwise the Physical SCSI Bus Sharing cannot be used. So, be careful about the storage consumption as these disks are going to be completely inflated from the beginning. Then, I needed to configure the 30GB disk as an SSD. On a virtual machine, this can be done by adding:
to the VMX configuration file. With this parameters, the Guest OS can recognize the disk as an SSD, and use it later as the caching tier for Storage Spaces Direct. Windows 2016 also properly identifies the other disks. Without the Physical Bus sharing in fact, there will be for each disk this error during the cluster validation:
In fact, without setting this option, during the validation of a new node in the cluster, you will see this error in the Storage Spaces Direct configuration:
By using the proper bus sharing, the disks are correctly identified:
This can also be appreciated using Powershell. With regular disks and no BUS Sharing, this is the output:
With thick disks and Physical Bus Sharing, this is the output:
Windows 2016 TP5 is installed on all the four nodes, and they are all joined to my domain. There are two networks on each node, and the final configuration is like this:
We are now ready to build the cluster.
Build the cluster using powershell
In order to speed up things, and get a consistent result, I’ve decided to build the new cluster using Powershell. Also, you will see later there are some steps that need specific options that may not be available via the graphical interface.
First, on each of the four nodes we install the needed components:
Then, we go and we create the new cluster:
If we then proceed and validate the cluster, either via the graphical Failover Cluster Manager or again with Powershell using:
we will notice that the section “Storage Spaces Direct” has a result of Failed. The reason is this one:
But as I said, there is a workaround.
Next step, we check the Cluster network, and we configure the two available networks to be available for clients (frontend) and for cluster internal communications (backend):
The tests added in TP5 run as we said some SCSI commands, that fail on a virtual disk. We can work around this by turning off automatic configuration and skip eligibility checks when enabling S2D, and then manually create the storage pool and storage tiers afterwards:
Then, we create a new storage pool:
and we configure all the virtual disks by making them appear as proper HDD, as Storage Spaces Direct as seen in the error above, accepts only SSD and HDD. Before we have this situation:
With this command we force these disks to be marked as HDD:
If we check again the available disks, we have this new situation:
The final result is the pool correctly created and ready to be consumed:
Virtual disks and volumes
Now that the cluster is created, it’s time to create our first volume and use it. For this part of the post, I’ll go back to the graphical interface, so I can explain a little bit the different available options. To start, with the pool selected, we start the wizard to create a New Virtual Disk. After selecting S2D as the storage pool to be used, we give the virtual disk a name and select to use tiers:
We accept enclosure awareness, and we configure the storage layout as this: mirror for Faster Tier and parity for Standard Tier, and for the resiliency settings two-way mirror for Faster Tier and Single Parity for Standard Tier.
Then, we configure Faster Tier size at 50GB, Standard Tier size at 500 GB, and we disable the Read Cache. We confirm all the selections and the disk is created. Before closing the wizard, we select the option to immediately create a volume: file system will be ReFS and it will use the entire size of the virtual disk:
Last step of this part, we select the virtual disk and use the command “Add to Cluster Shared Volumes”.
File server and shares
Now, as we want to have at the end of the test a working file share where we can drop our files, we need to create a role in the cluster, in this case a File Server. A simple powershell one-liner is all we need:
Then, we create the share. In the nodes of the cluster, there is a mount point for the newly created volume, in C:ClusterStorageVolume1. We will use this location to create our new share:
The share can be reached now over the network using the UNC path SOFSRepository, and we can read and write data to it.
Vmware Failover Cluster Shared Disk
To test the resiliency of S2D, I’ve done this simple test. I started to copy some large ISO files to the share, and while the copy was going on I powered off directly from vSphere the node ssd3, at the time owner of the File Server role. The role is immediately passed to ssd2, and the file copy goes on without any interruption.
Vmware Linux Cluster Shared Disk
Final notes
Vmware Windows Cluster Shared Disk
I’ve really enjoyed the time I’ve spent to play with Storage Spaces Direct. The issues to make it work in a virtualized environment are not important, as in a production environment I’m expecting people to use physical servers among those listed in the hardware compatibility list that Microsoft is preparing. The configuration of the solution is really simple, and the failover capabilities are really reliable. When Windows 2016 will become Generally Available later this year, I’m expecting many IT admins to start thinking of it as a new possible solution to create a scale-out storage, especially in situations where SMB3 is the needed protocol.