VSphere | Advanced snapshot trobuleshooting - Part 1 - The ESXi shell is your friend

I am writing this article as an extension to my previous post A VM is showing disk size of 0B where I will attempt to make a general explanation on how to troubleshoot and successfully solve snapshots and consolidation problems in your vSphere with a number of examples. Because the general article was so big, I decided to split it into three parts. This Part 1, is where I am describing the commands which are used to solve most of the snapshot problems and in the next two parts I will describe the process of troubleshooting via examples.



Quick jump to:
Part 2 | Part 3

---

Warning / Disclaimer!!

The steps and procedures described in this article contain low level commands and/or information on editing configuration files directly on ESXi hosts, all of which should be done by professionals.
Further more, the examples here can only be used as a general guidelines on how to trobuleshoot the described problems, and do not provide the exact solution. I am not responsible for any damages done to your infrastructure by reproducing these steps. Proceed at your own risk!

---

Before proceeding further it is highly recommended to get familiar as to how snapshots work:

Also this guide is not a direct solution, but it can serve only as a general guideline on how to solve them.

Useful SSH commands for snapshot troubleshooter:

Get a list of all of the VMs
# vim-cmd vmsvc/getallvms

If tou know the VM you are looking for but you merely want to see some quick info then pipe that with grep. This is particularly important if you want to find out the VM ID, and the datastore where its files are.
# vim-cmd vmsvc/getallvms | grep "VM Name"

Create snapshot via the CLI:
# vim-cmd vmsvc/snapshot.create [VmId] [snapshotName] [snapshotDescription]

List the VM snapshots:
# vim-cmd vmsvc/snapshot.get [VMId]

Delete all snapshots - this is like a manual consolidation:
# vim-cmd vmsvc/snapshot.removeall [VMID]

While deleting snapshots is in progress:

To monitor the VMDK snapshot and base disks which are currently being updated, run this watch command while inside the VM folder:
# watch -d 'ls -luth | grep -E "delta|flat|sesparse"'

If in doubt that something is stuck, you can monitor the timestamp update of the delta disks with:
# ls -lrth |grep -E "flat|delta|sesparse"


Also take note:

The datastores are located in /vmfs/volumes

The datastore names are actually aliases. When changing location into the datastore it is advisable to use the datastore real name, as opposed to the alias. Like for example:

# ls -l /vmfs/volumes

drwxr-xr-t    1 root     root          2240 Jan 19 11:50 516c1a57-449b7efb-17ee-b8ac6f7f3c70
drwxr-xr-t    1 root     root          3220 Mar  9 11:45 4ac65a74-838fa7e6-2f8c-00265554a8d6
drwxr-xr-t    1 root     root          2240 Mar  9 15:22 516c1a57-449b7efb-17ee-b8ac6f7f3c70
lrwxr-xr-x    1 root     root            35 Mar 10 16:22 Datastore1 -> 4ac65a74-838fa7e6-2f8c-00265554a8d6
lrwxr-xr-x    1 root     root            35 Mar 10 16:22 v7000 10K SAS 2 -> 5114ce6c-2efe905e-81bf-d8d385a3f488
lrwxr-xr-x    1 root     root            35 Mar 10 16:22 v7000-XCP-Dev-Env -> 516c1a57-449b7efb-17ee-b8ac6f7f3c70


To cd into v7000-XCP-Dev-Env it is reccomended to use:
# cp /vmfs/volumes/516c1a57-449b7efb-17ee-b8ac6f7f3c70


Next in Part 2 we will take a look at our first example.