CPU Hotplug (Host Model) Causes the Windows VM to BSOD

Host environment

  • Operating system: Linux
  • OS/kernel version: 5.14
  • Architecture: x86_64 (x64) -- Intel(R) Xeon(R) CPU E3-1230 v6
  • QEMU flavor: qemu-system-x86_64
  • QEMU version: 8.0.0, 9.1.0, 9.1.1, 9.1.50 (master)
  • QEMU command line:
./qemu \
    -enable-kvm \
     -cpu host,hv-time=on,hv-relaxed=on,hv-vapic=on,hv-spinlocks=0x1fff,hv-vpindex=on,hv-runtime=on,hv-synic=on,hv-stimer=on,hv-stimer-direct=on,hv-reset=on,hv-frequencies=on,hv-tlbflush=on,hv-ipi=on,hv-evmcs=on \
     -m size=4G \
     -object memory-backend-ram,size=4G,id=m0 \
     -smp 2,maxcpus=8,sockets=1,dies=1,cores=8,threads=1 \
     -numa node,nodeid=0,cpus=0-7,memdev=m0 \
     -no-user-config \
     -hda /mdrab/image.qcow2 \
     -cdrom /mdrab/2019.iso \
     -rtc base=localtime \
     -global kvm-pit.lost_tick_policy=discard \
     -qmp unix:/tmp/addcpu-sock,server=on,wait=off \
     -device usb-tablet \
     -vnc 10.10.102.161:10001 \
     -global ICH9-LPC.noreboot=off \
     -watchdog-action reset \
    -usb

Emulated/Virtualized environment

  • Operating system: Windows
  • OS/kernel version: Server 2019
  • Architecture: x64

Description of problem

The QEMU runs on a host with the Intel(R) Xeon(R) CPU E3-1230 v6 CPU which supports Software Guard Extension (SGX). I start a VM with Windows Server 2019 inside and with -cpu host,.... When I attempts to hotplug additional CPU (when the VM is running), the OS issues a bug check 0x3e (MULTIPROCESSOR_CONFIGURATION_NOT_SUPPORTED). The problem is that the newly hotplugged CPU is not evaluated as "equivalent enough" compared to the already present CPUs. I did some more digging and reverse engineering and it looks like the CPU being hotplugged has SGX turned off. This seems to be fixed when the VM reboots.

I tried to disable SGX through -cpu host,-sgx which helps (the VM successfully accepts the hotplugged CPU), however, +sgx does not help (seems to have no effect on the CPU being hotplugged).

My goal is to be able to hotplug CPUs even when the host CPU supports SGX.

I tested with QEMU 8.0.0, 9.1.0, 9.1.1 and 9.1.50 (current master) but with no luck.

Steps to reproduce

  1. Create a simple Windows VM,
  2. start the VM,
  3. use qpm-shell to hotplug a CPU (https://www.qemu.org/docs/master/system/cpu-hotplug.html).

I can provide you the VM as well but its image (QCOW2) has around 10G in size.

Best regards Martin Dráb