VM execution fails for tianocore edk2 ovmf uefi based image on windows whpx
Host environment
- Operating system: Windows 10 21H2 19044.1586
- OS/kernel version: 21H2 19044.1586
- Architecture: x86_64
- QEMU flavor: qemu-system-x86_64
- QEMU version:
QEMU emulator version 6.2.0 (v6.2.0-11889-g5b72bf03f5-dirty)
QEMU emulator version 7.0.0 (v7.0.0-11902-g1d935f4a02-dirty)
- QEMU command line:
qemu-system-x86_64 -cpu qemu64 -m 4096 -machine "type=q35,kernel-irqchip=off" -accel whpx -smp "sockets=1,cores=8,threads=1" -boot c -drive "index=0,if=pflash,format=raw,readonly=on,file=C:/vol/scoop_01/scoopg/apps/qemu/current/share/edk2-x86_64-code.fd"
Emulated/Virtualized environment
- Operating system: installed of Fedora-36 beta
- OS/kernel version: 5.17.0-0.rc5
- Architecture: x86_64
Description of problem
Cannot do a UEFI tianocore boot of image with linux installation.
I think the BIOS/UEFI/firmware when run inside a virtual-machine should be oblivious to the type of hypervisor, just probe and enable the emulated hardware. Maybe WHPX is not enabling pflash devices properly.
My goal is to create a 40Gb fedora linux image with a on-image UEFI boot sequence that I can
- native boot using ventoy (works)
- boot using kvm/qemu in linux (works)
- boot using whpx/qemu in windows (no success yet)
My original sequence of steps to reproduce was.
- Under Linux, in qemu-vm, create a bootable linux image by installing from the fedora livecd installer
- Confirm qemu-VM/fedora installation/UEFI boot works fine under Linux/kvm/qemu. One can see tianocore logo booting up.
- reboot to windows
- attempt to boot with analogous windows qemu command. confirm boot failure and error message
- remove
-accel whpx
and rerun, confirm boot succeeds with tianocore image, albeit un-accelarated
It turns out the image creation is not required.
The below works under linux
XDG_RUNTIME_DIR=/run/user/1000 qemu-system-x86_64 -cpu qemu64 -m 4096 -machine "type=q35" -accel "kvm" -smp "sockets=1,cores=8,threads=1" -boot d -drive "index=0,if=pflash,format=raw,readonly=on,file=/usr/share/edk2/ovmf/OVMF_CODE.fd" -drive "index=1,if=pflash,format=raw,file=/vol/15KJ_Images/vstorage/OVMF_VARS.fd" -drive "index=2,format=raw,file=/vol/15KJ_Images/transcend/m02_lnx.raw.img.vtoy" -device "virtio-vga-gl" -display "gtk,gl=on" -rtc "base=utc" -net "user" -device "virtio-net,netdev=vmnic" -netdev "user,id=vmnic,net=192.168.20.0/24,dns=192.168.20.3,dhcpstart=192.168.20.15" -qmp tcp:0:5955,server,nowait
The below does not work under windows
qemu-system-x86_64 -cpu qemu64 -m 4096 -machine "type=q35,kernel-irqchip=off" -accel whpx -smp "sockets=1,cores=8,threads=1" -boot d -drive "index=0,if=pflash,format=raw,readonly=on,file=C:/vol/scoop_01/scoopg/apps/qemu/current/share/edk2-x86_64-code.fd" -drive "index=1,if=pflash,format=raw,file=E:/vstorage/OVMF_VARS.fd" -drive "index=2,if=virtio,media=disk,format=raw,file=H:\m01_lnx.raw.img" -drive "index=3,if=virtio,media=disk,format=raw,file=H:\gkpics01.raw.img" -drive "index=4,if=virtio,media=disk,format=vhdx,file=E:\test\sgdata.vhdx" -display gtk -vga virtio -rtc base=utc -netdev user,id=vmnic1,net=192.168.20.0/24,dns=192.168.20.3,dhcpstart=192.168.20.15 -device virtio-net,netdev=vmnic1 -qmp "tcp:127.0.0.1:5955,server,nowait"
:
Windows Hypervisor Platform accelerator is operational
qemu-system-x86_64: WHPX: Failed to emulate MMIO access with EmulatorReturnStatus: 2
qemu-system-x86_64: WHPX: Failed to exec a virtual processor
The image does boot if one removes the hardware hypervisor argument -accel whpx
Steps to reproduce
The full qemu command with disk images is not required. Just the accel whpx and the pflash devices are sufficient.
- Confirm that the VM does not execute with the command
qemu-system-x86_64 -cpu qemu64 -m 4096 -machine "type=q35,kernel-irqchip=off" -accel whpx -boot c -drive "index=0,if=pflash,format=raw,readonly=on,file=C:/vol/scoop_01/scoopg/apps/qemu/current/share/edk2-x86_64-code.fd"
- Confirm that the VM does execute and tianocore logo shoes up when
-accel whpx
is removed.
Additional information
- In the planned changes of Fedora 37, going forward, fedora installer will no longer support installing fresh to machines with legacy BIOS and will necessarily require UEFI boot. This means that there is urgency in allowing this mode of booting.
Ref
- It looks like this bug has been known for a while
- 2019-Jan claims that 'HAX and WHPX do not support ROMD at all' https://listman.redhat.com/archives/libvir-list/2019-January/msg01363.html
- 20210702 https://edk2.groups.io/g/discuss/topic/83941304
- 20200929 qemu-5.1.0 https://lore.kernel.org/qemu-devel/160142368077.3192.15854935917101554025.malone@chaenomeles.canonical.com/ Previous launchpad bug https://bugs.launchpad.net/bugs/1821595
- Maybe related to #858 qemu-system-x86_64: WHPX: Unexpected VP exit code 4
- 2021-Aug Perhaps DUP of #513 Qemu/WHPX fails on applying UEFI firmware with -pflash
Workaround
Bypass UEFI boot, using a regular nonUEFI grub install on a tiny boot disk.
- Use dd create a 128Mb raw img virtual-disk
- Use qemu to boot a VM, using a fedora livecd with this attached as drive with index=0
- Use fdisk to put an dosmbr on the disk with a single Linux partition,
- Use mkfs.ext4 to format it as ext4
- mount the partition at /mnt/a1
- execute
grub2-install --skip-fs-probe --boot-directory=/mnt/a1 target=i386-pc /dev/vda
- create the following file /mnt/a1/grub2/grub.cfg
echo "executing grub2.cfg on msdos boot partition
sleep 3
insmod part_gpt
search --no-floppy --fs-uuid --set=dev <UUID OF /boot parition>
set root=($dev)
configfile ($dev)/grub2/grub.cfg
- unmount, poweroff vm
- restart vm, this time boot this drive with index=0, and the main raw disk image at index=1
- qemu VM should boot normally into the custom-install on the main raw disk image.
Linux kernel does not care if it is booted via UEFI or not. i was not sure of this, but thought to give it a try. I initially thought Linux would be like Windows, wherein a UEFI windows installation would not be bootable if boot method is changed.