I am still periodically having this issue with the Checkpoints not merging. Today I tried to do a bit of further research and found that the vmms.exe process on the owner Hyper-V server has a .vmcx and a .vmrs file opened in the Snapshot folder of the offending VM (while the VM is shutdown). These files being opened prevent a Storage Migration of the VM to a different folder. In trying to manually move folders, etc., somehow Hyper-V lost the configuration for the VM, so I ended up doing a restore. But next time this happens, I'd like to see if these same two files (the .vmcx and the .vmrs) are open on another Hyper-V host in the cluster after a migration between hosts. I don't see any problems with permissions - it just seems like a bug in Hyper-V for the vmms.exe to not be able to automatically merge the Checkpoint as it should, but it somehow keeps these files open.
My goal isn't to prevent this Checkpoint merge issue, but just to try to find a way to quickly recover from it without having to remove the VM from Failover Cluster & from Hyper-V and having to manually add it back as a "new" VM. (I could backup and restore the VM or export & import, but that would take a while).
Windows Server 2019 Hyper-V Failover Cluster .avhdx files remaining after backup
Hello!
I have a customer with a three node Windows Server 2019 Hyper-V Failover Cluster. When I backup VM's w/ Veeam, I sometimes get left-over .avhdx files that don't automatically merge. With additional backups, sometimes the VM keeps adding additional .avhdx files - other times there remains only a single .avhdx file. Hyper-V Manager doesn't show any Checkpoints. And when I try to look at the Hyper-V Settings in Failover Cluster manager, the MMC crashes with the error "MMC has detected an error in a snap-in".
I have a process to manually fix these VM's:
- Remove VM from Cluster [this is necessary because the Failover Cluster MMC crashes and I can't edit the VM disks in Hyper-V Manager when the VM is part of a cluster]
- Shutdown VM [this is a pain]
- Manually merge .avhdx files in chain back to parent [Merge-VHD or AZSBTools]
- Fix disk in Hyper-V Manager Settings back to .vhdx
- Rejoin Cluster
The really interesting thing is that [Get-WinEvent Microsoft-Windows-Hyper-V-* | Where-Object{$_.Message -like "<VMName>"} ] shows "background disk merge has been started" and "background disk merge has been finished successfully" for the VM in question. Yet the .avhdx had been created at some point and not cleaned up automatically.
I'd love to be able to fix these VM's without having to shutdown the VM. I'd really love to be able to not have these .avhdx files remain and have the system automatically merge them.
Any suggestions? I'd certainly be glad to hear them!
Dave
8 answers
Sort by: Most helpful
-
Dave Baddorf 20 Reputation points
2024-02-05T15:49:15.7933333+00:00 -
Deleted
This answer has been deleted due to a violation of our Code of Conduct. The answer was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
Comments have been turned off. Learn more
-
Dave Baddorf 20 Reputation points
2024-02-28T15:56:21.14+00:00 Dear diary, I am not sure why I keep thinking that I can figure out a way to fix these checkpoints without removing the VM and re-adding it. I can't run a shared-nothing migration to another Hyper-V - the broken checkpoint won't allow the migration. I can't even run a live-migration within the cluster when the checkpoint is messed up. I can manually merge the .avhdx to the .vhdx and replace the Hard Drive in Hyper-V settings. That leaves the VM intact, but it is still broken under the hood because I can't open Settings in Failover Cluster Manger without a crash of the Cluster Manger Console. At least this doesn't work consistently. Again, I need to remove the VM and re-add. I know that no one will read this diary entry, but I guess it's helpful to get it all out. Dave