Blog

What is vSAN HCI Mesh and its Policy?

Definition:

HCI Mesh is a distinctive approach that uses software to disaggregate compute and storage resources. By combining multiple autonomous vSAN clusters into a native, cross-cluster architecture, HCI Mesh allows resources to be disaggregated and stranded capacity to be utilized. In essence, vSAN enables the mounting of datastores from other vSAN clusters (servers) in the vCenter inventory, without fundamentally altering the existing HCI model or requiring specialized hardware. With HCI Mesh, a cluster with surplus compute resources can now access additional storage capacity from a remote vSAN cluster.

Policy:

According to the policy, a virtual machine (VM) can only be located entirely on either a remote vSAN datastore or other datastores. If the VM needs to be spread across multiple datastores, the remote vSAN datastore cannot be included, regardless of the other datastores used.

The following combinations are not supported for locating virtual machine files:

1. VMFS + remote vSAN
2. remote vSAN DS 1 + remote vSAN DS 2
3. local vSAN + remote vSAN

Problem Statement

Prior to VMware Cloud Director 10.4.2, if a VMware Cloud Director storage policy included a mix of different types of datastores, such as local VMFS, shared VMFS, local vSAN, or remote vSAN, the VMware Cloud Director placement engine could select any datastore within the policy to place the virtual machine files and disk. However, this posed a challenge if the placement engine chose a remote vSAN datastore, VMware Cloud Director did not necessarily adhere to the vSAN HCI Mesh Policy (remote vSAN) of keeping all virtual machine files in the same datastore, resulting in scattered VM files that went against the remote vSAN policies.

Apart from violating the remote vSAN policy, another drawback was the potential performance issues that could arise. If a virtual machine’s disks were scattered across various types of datastores, especially on remote vSAN datastore, it could lead to significant IOPS or latency problems with certain disks.

How the Problem Statement has been Resolved?

From VMware Cloud Director 10.4.2 onwards, the VMware Cloud Director VM placement engine has been enhanced to take into account the vSAN HCI Mesh Placement Policy (remote vSAN) when placing VMs and their files. If the placement engine selects a remote vSAN datastore within the storage policy, VMware Cloud Director will guarantee that all VM files are stored on the same remote vSAN datastore in accordance with the vSAN HCI Mesh placement policy.

To accommodate different scenarios when using vSAN Remote Datastore with the storage policy that adheres to the vSAN HCI Mesh placement policy, there are several advanced configurations available. These configurations can alter the behaviour of the VMware Cloud Director VM placement engine when making decisions about where to place Virtual Machine files.

1. fabric.storage.placement.prefer.non.remote.vsan
   • When the value of fabric.storage.placement.prefer.non.remote.vsan is set to true, it indicates that the administrator has instructed VMware Cloud Director to prioritize non-remote vSAN datastores during VM placement. In this case, VMware Cloud Director will only use remote vSAN datastores if all other datastores are either unavailable or fully utilized.
   • Conversely, when this value is set to false, VMware Cloud Director will not lower the priority of remote vSAN datastores and will treat them the same as other datastores.
     • It is important to note that this flag is set to false by default.
   • CMT commands to control this feature in VMware Cloud Director: ./cell-management-tool manage-config -n fabric.storage.placement.prefer.non.remote.vsan -v true
2. fabric.storage.placement.keep.vmdks.together.on.remote.vsan
   • When the value of fabric.storage.placement.keep.vmdks.together.on.remote.vsan is set to true, VMware Cloud Director will make an effort to place all of a virtual machine’s disks (vmdks) on a single container when using a remote vSAN datastore. For example, if a remote vSAN is preferred and in accordance with the vSAN HCI mesh policy, VMware Cloud Director will place the entire VM on that remote vSAN datastore.
   • On the other hand, if this value is set to false, it means that VMware Cloud Director will not comply with the vSAN HCI Mesh VM placement policy.
     • Please take note that this flag is set to true by default. Additionally, if a storage policy that includes a remote vSAN is being used, this flag must always be set to true.
   • CMT commands to control this feature in VMware Cloud Director: ./cell-management-tool manage-config -n fabric.storage.placement.keep.vmdks.together.on.remote.vsan-v true

In this manner, VMware Cloud Director 10.4.2 has been improved in terms of how it utilizes remote vSAN datastores and how it can comply with the vSAN HCI Mesh Storage Placement Policies.

Please be advised that this report is intended for informational purposes only and represents our best effort to provide accurate and useful insights.

Source

Originally posted on April 25, 2023 @ 6:54 pm