VMware: Shrink VMDKs by removing zeroed blocks

VMDK size is one of the main aspects to consider for administrators when deploying new VMs since it could introduce several hazards in storage space usage. By introducing thin provisioning VMware mitigated space exhaustion issues due to the fact that this provisioning mechanism allocates only blocks that are effectively used by the operating system residing on VMDK itself preventing in this way storage to be pre-emptively filled by overzealous admins planning for big virtual machine disks.

But what about shrinking size of those VMDKs that were already provisioned using thick provisioning? They usually consume a lot of storage space and not always their thick allocation is justified. Usually thick eager is the allocation mechanism choosen for I/O intensive virtual machines due to the fact that by zeroing blocks during VMDK creation, and not when operating system issues the first write on a block, introduce a little performance increase (just on first write though).

As most of you certainly know thick lazy zeroed is another allocation mechanism that pre-emptively reserve storage space required by the virtual machine VMDK. Difference between thick lazy and thick eager is that the former reserve whole VMDK size on storage without zeroing blocks, which will be zeroed at first write.

Another crucial aspect to consider is that ESXi is not aware of space freed by the upper operating system due to file deletion. This space cannot be reclaimed by underlying hypervisor and because of this VMDK cannot be "downsized automatically".

Let's clarify this with an example: suppose that we have a virtual machine whose VMDK is thin provisioned.
Operating system is 5GB in size. We install 2GB of additional software. Operating system will report a space consumption of 5+2=7GB. VMDK will be the same (5+2=7GB) size.
We later decide that additional software just installed is useless so we delete/uninstall it. Operating system will report a space consumption of 7-2=5GB. VMDK will be unchanged with a size of 7GB.
This is because ESXi is not aware that space is currently unused by the guest OS and cannot reclaim it.

By shrinking VMDKs this space can be reclaimed. A prior important condition for unused space reclamation is that operating system zeroes currently unused blocks, this means that for shrink to be the most effective guest operating system should fill all currently unused space with zeroes. This can be done in Windows guests by using softwares like Hard Disk Scrubber or SDelete and in Linux guest with dd if=/dev/zero of=/<destination_path>/placeholderfile bs=8192 && rm -rf /<destination_path>/placeholderfile command.

Since unused space is reclaimed after a shrink operation VMDK will be converted from thin, eager zeroed, lazy zeroed to a thin provisioned VMDK.

Let's now delve into how shrinking is performed. A common method is using Storage vMotion. When storage vMotioning a virtual machine resulting disk format can be choosen. Another method is using VMware Converter by converting guest OS like a classic P2V conversion. Pros of those two methods is that they can be performed while VM is powered on.

A third method to shrink a VMDK is by Using vmkfstools.

This method requires the VM to be powered off during shrinking process and you to connect to ESXi host using SSH to run some commands.

For article purpouse I created a VM with a thick provisioned VMDK. Following image show you space usage reported at Linux guest level:

Guest OS reports a total space of 60GB of which 2.3 GB are currently used.

Since it's thick provisioned VMDK is also 60GB in size

To shrink VMDK using vmkfstools the following command is used:

vmkfstools --punchzero <path_to_VMDK_to_shrink>.vmdk

--punchzero option is quite self-explicative, it basically removes all zeroed blocks in a VMDK by freeing up space.

As you can expect, based on previous explanation, the resultant VMDK will be shrinked down to the size of effective guest OS space usage.

That's all!!