Friday, March 11, 2016

VSphere | Advanced snapshot trobuleshooting - Part 3 - Example: Invalid snapshot configuration

Unable to consolidate because of invalid snapshot configuration

This is Part 3 of the short tutorial series:
vSphere | Advanced snapshot troubleshooting

This is much more advanced than our previous example.  You will see these two errors if you attempt to consolidate, clone or migrate the VM:

Detected and invalid snapshot configuration

or

... vmdk was not found


Quick jump to:
Part 1 | Part 2


Warning / Disclaimer!!

The steps and procedures described in this article contain low level commands and/or information on editing configuration files directly on ESXi hosts, all of which should be done by professionals.
Further more, the examples here can only be used as a general guidelines on how to trobuleshoot the described problems, and do not provide the exact solution. I am not responsibly for any damages done to your infrastructure by reproducing these steps. Proceed at your own risk!





The devil dwels in the details. Go into the VM folder and list the files inside:

# cd /vmfs/volumes/516c1a57-449b7efb-17ee-b8ac6f7f3c70/CP\ Content

... let's see:

# /vmfs/volumes/516c1a57-449b7efb-17ee-b8ac6f7f3c70/CP Content # ls -l
total 1049404432
-rw-r--r--    1 root     root            73 Dec 11 10:28 CP Content-8be54c27.hlog
-rw-------    1 root     root     12884901888 Apr 10  2014 CP Content-8be54c27.vswp.40947
-rw-r--r--    1 root     root            13 Feb 14 23:56 CP Content-aux.xml
-rw-------    1 root     root       3277312 Feb 19 03:03 CP Content-ctk.vmdk
-rw-------    1 root     root     53687091200 Feb 16 15:46 CP Content-flat.vmdk
-rw-------    1 root     root          8684 Feb 16 15:46 CP Content.nvram
-rw-------    1 root     root           583 Feb 19 03:03 CP Content.vmdk
-rw-r--r--    1 root     root            80 Feb 19 03:03 CP Content.vmsd
-rwxr-xr-x    1 root     root          3675 Mar  8 14:10 CP Content.vmx
-rw-r--r--    1 root     root          3043 Aug 27  2013 CP Content.vmxf
-rw-------    1 root     root       4915712 Feb 19 03:03 CP Content_1-ctk.vmdk
-rw-------    1 root     root     322122547200 Feb 16 15:46 CP Content_1-flat.vmdk
-rw-------    1 root     root           588 Feb 19 03:03 CP Content_1.vmdk
-rw-------    1 root     root       5325312 Feb 15 05:29 CP Content_2-000001-ctk.vmdk
-rw-------    1 root     root       1335296 Feb 14 04:41 CP Content_2-000001-delta.vmdk
-rw-------    1 root     root           423 Feb 19 03:03 CP Content_2-000001.vmdk
-rw-------    1 root     root       5325312 Feb 15 05:29 CP Content_2-000002-ctk.vmdk
-rw-------    1 root     root       1335296 Feb 14 11:48 CP Content_2-000002-delta.vmdk
-rw-------    1 root     root           430 Feb 19 03:03 CP Content_2-000002.vmdk
-rw-------    1 root     root       5325312 Feb 15 05:29 CP Content_2-000003-ctk.vmdk
-rw-------    1 root     root       1335296 Feb 14 11:50 CP Content_2-000003-delta.vmdk
-rw-------    1 root     root           430 Feb 19 03:03 CP Content_2-000003.vmdk
-rw-------    1 root     root       5325312 Feb 15 05:29 CP Content_2-000004-ctk.vmdk
-rw-------    1 root     root      18112512 Feb 14 15:50 CP Content_2-000004-delta.vmdk
-rw-------    1 root     root           430 Feb 19 03:03 CP Content_2-000004.vmdk
-rw-------    1 root     root       5325312 Feb 15 05:29 CP Content_2-000005-ctk.vmdk
-rw-------    1 root     root      18112512 Feb 14 15:51 CP Content_2-000005-delta.vmdk
-rw-------    1 root     root           430 Feb 19 03:03 CP Content_2-000005.vmdk
-rw-------    1 root     root       5325312 Feb 15 05:29 CP Content_2-000006-ctk.vmdk
-rw-------    1 root     root      18112512 Feb 14 19:52 CP Content_2-000006-delta.vmdk
-rw-------    1 root     root           430 Feb 19 03:03 CP Content_2-000006.vmdk
-rw-------    1 root     root       5325312 Feb 15 05:29 CP Content_2-000007-ctk.vmdk
-rw-------    1 root     root      18112512 Feb 14 19:53 CP Content_2-000007-delta.vmdk
-rw-------    1 root     root           430 Feb 19 03:03 CP Content_2-000007.vmdk
-rw-------    1 root     root       5325312 Feb 15 05:29 CP Content_2-000008-ctk.vmdk
-rw-------    1 root     root      18112512 Feb 14 19:54 CP Content_2-000008-delta.vmdk
-rw-------    1 root     root           430 Feb 19 03:03 CP Content_2-000008.vmdk
-rw-------    1 root     root       5325312 Feb 15 05:29 CP Content_2-000009-ctk.vmdk
-rw-------    1 root     root      18112512 Feb 14 23:55 CP Content_2-000009-delta.vmdk
-rw-------    1 root     root           430 Feb 19 03:03 CP Content_2-000009.vmdk
-rw-------    1 root     root       5325312 Feb 16 15:46 CP Content_2-000010-ctk.vmdk
-rw-------    1 root     root     538206208 Feb 16 15:46 CP Content_2-000010-delta.vmdk
-rw-------    1 root     root           430 Feb 19 03:03 CP Content_2-000010.vmdk
-rw-------    1 root     root       5325312 Feb 18 22:56 CP Content_2-000011-ctk.vmdk
-rw-------    1 root     root       1335296 Feb 18 22:56 CP Content_2-000011-delta.vmdk
-rw-------    1 root     root           430 Feb 19 03:03 CP Content_2-000011.vmdk
-rw-------    1 root     root       5325312 Feb 18 23:02 CP Content_2-000012-ctk.vmdk
-rw-------    1 root     root       1335296 Feb 18 23:02 CP Content_2-000012-delta.vmdk
-rw-------    1 root     root           430 Feb 19 03:03 CP Content_2-000012.vmdk
-rw-------    1 root     root       5325312 Feb 18 23:02 CP Content_2-000013-ctk.vmdk
-rw-------    1 root     root       1335296 Feb 18 23:02 CP Content_2-000013-delta.vmdk
-rw-------    1 root     root           430 Feb 19 03:03 CP Content_2-000013.vmdk
-rw-------    1 root     root       5325312 Feb 19 03:03 CP Content_2-000014-ctk.vmdk
-rw-------    1 root     root       1335296 Feb 19 03:03 CP Content_2-000014-delta.vmdk
-rw-------    1 root     root           430 Feb 19 03:03 CP Content_2-000014.vmdk
-rw-------    1 root     root       5325312 Feb 19 05:17 CP Content_2-000015-ctk.vmdk
-rw-------    1 root     root       1335296 Feb 19 03:03 CP Content_2-000015-delta.vmdk
-rw-------    1 root     root           407 Feb 19 05:17 CP Content_2-000015.vmdk
-rw-------    1 root     root       5325312 Feb 14 04:41 CP Content_2-ctk.vmdk
-rw-------    1 root     root     697932185600 Dec 11 12:11 CP Content_2-flat.vmdk
-rw-------    1 root     root           589 Feb 19 03:03 CP Content_2.vmdk
-rw-r--r--    1 root     root         58070 Apr 11  2014 vmware-10.log
-rw-r--r--    1 root     root      29684335 Jan 15  2015 vmware-11.log
-rw-r--r--    1 root     root        344598 Jan 15  2015 vmware-12.log
-rw-r--r--    1 root     root      30097536 Dec 11 10:28 vmware-13.log
-rw-r--r--    1 root     root       7250756 Feb 14 13:04 vmware-14.log
-rw-r--r--    1 root     root         58084 Apr 11  2014 vmware-9.log
-rw-r--r--    1 root     root       1623017 Feb 16 15:46 vmware.log


Hmmm... delta disks. Nothing unusual.
Let's take a closer look. The error message for consolidation says: "Detected an invalid snapshot configuration". While the other error when attempting to clone the VM in attempt to resolve says that the disk file is missing. If we take a look at our VM files we can see that indeed, that particular disk file is missing - in our case: "CP Content-000001.vmdk"

Lets edit the VM file. It's the file with a .vmx extension. In our case that's the "CP Content.vmx" and navigate to the disks section. By the file name that looks like our first disk.

# vi CP\ Content.vmx
... from here on I can only show you with screenshots. The scsi part of this config file is where our disk files are described.
Like for example: scsi0:0 is for the first disk, scsi0:1 is for the second and so on.



We can see that scsi0:0.filename = "CP Content-000001.vmdk" and our file is missing i.e. the we only have CP Content.vmdk available and no delta disks for that disk. Go on and edit that line to match our present disk file.

This is how it should look like:


WARNING:
While pointing to disk files in the .vmx config file, you should always match the highest possible disk available for that scsi disk. In our case we only have the main disk and we didn't have a delta disk. This usually happens if snapshot consolidation is in progress, but it is interrupted somehow. However if for example we had "CP Content-000056.vmdk" in the VM directory but the config file points to a nonexistent disk like "CP Content-000100.vmdk" then we should edit the line and set it to point to that "CP Content-000056.vmdk" file. It is always the highest numbered disk in the working chain of snapshots that is the present disk. I know that this must be confusing for some of you. Read on more on the VM ware knowledge base on how snapshots work to understand this better.

After this, try and consolidate the VM and hopefully it should work.


No comments:

Post a Comment