Windows 2019 server - fs freeze does not work properly - access is denied to IVssWriterCallback
By running a backup in Proxmox VE, version 9.1.4 and I sometimes get an error that a volume cannot be added to snapshot set, error that is shown in vzdump, but not consistently.
By checking the error logs in Even Viewer I can see there is Access is denied. Basically this error is very persistent and it happens right after fs-freeze is done for every Win Server 2019
Error from WIndows Event Viewer:
Source: VSS
Event ID: 8194
Description:
Volume Shadow Copy Service error: Unexpected error querying for the IVssWriterCallback interface. hr = 0x80070005, Access is denied.
. This is often caused by incorrect security settings in either the writer or requestor process.
To reproduce this error, it is quite simple, install Windows Server 2019 and install latest Qemu Guest agent and run a backup job from PVE.
Vm is running Windows server 2019. I tried 0.271 and 0.285.
I am cross referencing some relevant information related to this issue:
https://kb.msp360.com/backup/errors/vss-error-8194-access-denied
https://forum.proxmox.com/threads/virtio-and-qemu-ga.57599/
Workaround is to allow NetworkService Full access to DCOM, but this is quite sloppy workaround as any service running under NetworkService get full access to DCOM.
Update1:
On the other hand, I have several Windows Server 2019 fully patched without any GPO applied(a few of them not even in domain) with the same issue.
This is the description from Gemini:
When the VSS freeze command is issued, the following failure chain occurs:
Identity Confusion: The QEMU VSS provider (a DLL) is hosted by the guest agent process (qemu-ga.exe). However, the system does not have an AppID (Application ID) linked to this executable in the Registry.
Permission Defaulting: Because no specific security policy exists for the QEMU AppID, Windows falls back to Machine-Default permissions.
Account Blockage: The VSS service often operates or initiates calls via the NETWORK SERVICE or SYSTEM accounts. Under hardened rules, "Machine-Default" settings are too restrictive to allow these accounts to perform "Local Activation" of the COM server.
The Result: The system logs Event ID 10016, stating that "Local Activation permission... to the user NT AUTHORITY\NETWORK SERVICE" was denied.
Gemini prepared a powershell script that fixes this issue. I have tested it on 2 Win Server 2019 and it fixes "Access is denied" in Event viewer. I will try it out on a VM that is in domain.