VM Limit per Host – VSAN

According to the VSAN maximums, there is a 100 VM limit per host in a VSAN 5.5 cluster and 200 in a VSAN 6 cluster. This seems to be a soft limit as I was recently able to deploy 999 VMs in to a 4 node VSAN 5.5 cluster (with one host acting as a dedicated HA node, so not running any compute). I got to ~333 VMs per host before I reached the 3000 component limit (which is a hard limit) on each host. The below is a screen grab of vsan.check_limits from RVC:

Max_Components

I reached out to the GSS guys at VMware who quite rightly pointed out that 100 VMs per host is the supported maximum, so any more than that would not be supported.

So…  in the event we have (for example) a 4 host VSAN cluster with 512GB RAM per host…

  • Node A has 300 VMs each with 1GB RAM and 1 x 10GB Disk
  • Node B has 4 x VMs with 24GB RAM each and 1 x 10GB Disk
  • Node C has 146 x VM with 1GB RAM and 1 x 10GB Disk
  • Node C has 300 VMs each with 1GB RAM and 1 x 10GB Disk

In this example Failure to Tolerate = 0, each VM is only using a handful of components and the cluster is under no stress.

DRS will not migrate VMs as the cluster is under no stress and no alarms would be generated as no vobs will be triggered, however it is not a supported solution as there would be 750 VMs in the cluster.

This got me thinking about solutions to this problem, thanks to some great posts by @DuncanYB (What’s New in vSphere 5.5 for DRS and Limit the Amount of Eggs in a Single Basket) and the guys at VMware GSS we found a DRS advanced setting that will limit the number of VMs on a single host:

In the DRS Advanced Options, create a new Option “LimitVMsPerESXHost” with a value of 100.

LimitVMsPerESXHost

Time for a Feature Request I think…

2 comments

  1. Hey Brian,

    I’ve not had a chance to deploy 6.1 yet, however I have spoke with a few of the VMware VSAN guys about it…

    In the ROBO setup the remote site vSphere cluster is managed centrally by a Virtual Center server (in the head office) over standard WAN links.

    Each remote site can host two ESXi servers running Virtual SAN allowing the RAID1 of disk objects to be fulfilled (Failures to Tolerate = 1), in the head office there is a witness host for each remote site, the witness is a virtualised ESXi host.

    All IO is carried out on the two ROBO hosts… not the central witness host, performance should be as good as a full VSAN cluster. No IO traffic will be sent across the WAN, so WAN latency shouldn’t impact performance. This is the same concept as “Data Locality” in a VSAN stretched cluster.

  2. vCrooky – any similair tests on vSAN 6.1 as yet? I would be interested to know results based on the branch 2-node vSAN clusters

Leave a Reply

Your email address will not be published. Required fields are marked *