A four node cluster with a single VM (with a single VMDK less that 255GB), deployed with a Storage Policy of “Failures to Tolerate = 1” and “Disks to Stripe = 1” would look like this:
Recently I added an ESXi host to a VSAN cluster (HP DL380 G9 with 2 x 800GB SSD and 6 x 4TB Magnetic) however when creating the disk group only 2 disks were available for use.
After some investigation it turns out the other disks already had partitions, this was confirmed by running the following command:
Running the PartedUtil getptbl /vmfs/devices/disks/naa.5000xxxxxxxxx command fails with the error: “Error: Can’t have a partition outside the disk!”
To resolve this (and allow the disks to be added to a VSAN disk group) I ran the following:
partedUtil setptbl /vmfs/devices/disks/naa.5000xxxxxxxxxxx msdos
Then reboot the host.
***This will destroy any data already on the disks***
Always validate the network configuration. The VSAN Misconfiguration detected error is by far the most common error seen when configuring VSAN. Normally this means that either the port group has not been successfully configured for Virtual SAN or multicast has not been set up properly.
On Cisco switches, unless an IGMP Snooping Carrier has been configured OR IGMP snooping has been explicitly disabled on the ports used for Virtual SAN, configuration will generally fail. In the default configuration it is simply not configured, and therefore, even if the network admin says it is configured properly it may not be configured at all, double check it to avoid any pain.
ESXi 5.5 and above stores coredumps on a datastore attached to the host, it can also create a vsantraces directory. Both of these can lock the datastore and prevent it from being deleted.
To check for and remove the coredump file do the following:
esxcli system coredump file list Path Active Configured Size ------------------------------------- ------ ---------- --------- /vmfs/volumes/xx/vmkdump/xxx.dumpfile false false 702545920 /vmfs/volumes/xxx/vmkdump/xxx.dumpfile true true 702545920 /vmfs/volumes/xxx/vmkdump/xxx.dumpfile false false 702545920
The output shows that there are 3 dump files which are blocking the datastore. Only the owning ESXi host can disable and delete them, so you have to find out which ESXi is responsible for each file:
When creating a virtual machine VSAN Storage Based Policy, you may experience one or more of these symptoms:
- No matching Virtual SAN (VSAN) datastores are listed in the matching resources page of the Storage Based Policy creation wizard.
- You are unable to create the storage based policy.
This issue may occur due to inconsistent data between the Storage Management Service (SMS) and the Storage Based Policy Manager (SPBM).
VSAN cannot use disks with existing partitions on them, this post describes how to delete partitions on a disk (in this case an SSD).
To obtain a list of disk device names that can be managed by
partedUtil, run this command:
Example of output under ESXi/ESX 4.0 and later:
mpx.vmhba0:C0:T0:L0 <– disk device
mpx.vmhba0:C0:T0:L0:1 <– partition 1
mpx.vmhba0:C0:T0:L0:2 <– partition 2
mpx.vmhba0:C0:T0:L0:3 <– partition 3
mpx.vmhba0:C0:T0:L0:5 <– partition 5
naa.60060160205010004265efd36125df11 <– disk device
naa.60060160205010004265efd36125df11:1 <– partition 1
Number of Disk Stripes Per Object
This defines the number of physical disks across which each replica of a storage object is striped. This impacts read and write operations as below:
Since all writes go to SSD (write buffer), the value of an increased stripe width may or may not improve performance. This is because there is no guarantee that the new stripe will use a different SSD; the new stripe may be placed on a HDD in the same disk group and thus the new strip will use the same SSD. The only occasion where an increased stripe width could add value is when there are many writes to destage from SSD to HDD. In this case, having a stripe could improve destage performance.
One of the most important concepts to understand in VSAN is the notion of storage objects and components. Virtual Machines deployed on a vsanDatastore may have 4 different kinds of storage objects associated with it:
- The Virtual Machine home or “namespace directory”
- A swap object (if the virtual machine is powered on)
- Virtual disks/VMDKs
- Delta-disks created for snapshots. Each delta-disk is an object.
Of these four objects, the virtual machine namespace may need a little further explanation. All virtual machines files, excluding virtual machine disks, deltas (snapshots) and swap, reside in an area called the virtual machine namespace on VSAN. The sorts of files in here can be the .vmx, the .log files, .vmdk & snapshot deltas descriptors files and everything else one would expect to find in a VM home directory.