I wanted to blog here about a silly mistake I made yesterday.
I need to expand a VM’s disk and was using the ol’ vmkfstools -X. However, I obviously do this all too flippantly and forgot to check if there was any snapshots. oops. That sunken feeling.
I corrupted the vmdk and the snapshot vmsd file.
Here’s how I fixed it all.
Firstly I couldn’t run anything against the original vmdk file, as it kept saying the file was locked by the host. I unregistered the VM from VC, put the ESX server into maintenance mode (which moved all the VMs off to other hosts in the clustering – fully enabled DRS) and power cycled the host. This remove the lock on the file thankfully.
The original vmdk had been 20GB before I tried expanding it to 30GB. So I created a new 20GB vmdk:
# vmkfstools -c 20G -d thick silver1.vmdk
I then copied the contents of the broken 30GB disk into the new one:
# dd if=silver-flat.vmdk of=silver1-flat.vmdk bs=1048576 count=20480
Moved the old vmdk and renamed the new one:
# vmkfstools -E silver.vmdk silver.vmdk.old
# vmkfstools -E silver1.vmdk silver.vmdk
All the snapshots rely upon a CID and a parentCID in their -00000x.vmdk files. I opened both snapshot disks to record the details.
# cat silver-000001.vmdk
# cat silver-000002.vmdk
Basically, 000002 parentCID is 000001′s CID. In turn 000001 parentID was set to point to the original vmdk. So I had to edit the new vmdk file and set this CID.
# vi silver.vmdk
I was then able to power on the VM. Phew!
However the snapshots were still screwed up. When I tries to commit the snapshots with:
# vmware-cmd silver.vmx removesnapshots
It thought it didn’t have any snapshots, but it was clearly using the two additional vmdk disks. The vmsd file was also corrupted. So I renamed the existing one:
# mv silver.vmsd silver.vmsd.old
Then I created a new snapshot, to re-generate a new vmsd file:
# vmware-cmd silver.vmx createsnapshot recreate_vmsd
And finally I could commit the snapshots:
# vmware-cmd silver.vmx removesnapshots
Now all that was left was to increase the disk:
#vmkfstools -X 30G silver.vmdk
I guess I won’t be doing that in a hurry again 
Related posts:
