Skip to content

Live migrating vms with disk attached fails with CPU feature error

Software environment

  • Operating system: Fedora
  • Architecture: x86
  • kernel version: 6.12 and 6.15
  • libvirt version: 11.0.0 and 11.6.0
  • Hypervisor and version: Fedora 42/Fedora 43

When migrating a vm with a disk attached from Fedora 42 libvirt 11.0.0-3/qemu-9.2.4-2 to Fedora 43 beta libvirt 11.6.0-1/qemu-10.1.0-7 with command

virsh migrate vmname --change-protection --p2p --persistent --live --verbose --undefinesource --abort-on-error qemu+tcp://destination/system

error: operation failed: guest CPU doesn't match specification: extra features: ht

Migrating the same vm from Fedora 42 to Fedora 42 and Fedora 43 to Fedora 43 works fine. Only migrations from Fedora 42 to Fedora 43 errors.

If the disk is removed from the vm, the live vm migration succeeds without error.

Reproducible: Always

Steps to Reproduce:

  1. Create a vm with a disk
  2. Live migrate the vm from Fedora 42 libvirt 11.0.0-3/qemu-9.2.4-2 to Fedora 43 libvirt 11.6.0-1/qemu-10.1.0-7

Actual Results: error: operation failed: guest CPU doesn't match specification: extra features: ht

Expected Results: Live migration should succeed without error

Additional Information: Looking at the xmldump of a running vm with a disk attached on Fedora 42 and Fedora 43, there is a difference in the cpu features listed.

In Fedora 42 "ht" is missing from the cpu feature policy section of the xml where in Fedora 43, the exact same vm shows "ht".

Fedora 42 qemu-9.2.4/libvirt 11.0.0:
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Skylake-Server-noTSX-IBRS</model>
    <topology sockets='1' dies='1' clusters='1' cores='2' threads='1'/>
    <feature policy='disable' name='mpx'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='pku'/>
  </cpu>

Fedora 43 qemu-10.1.0/libvirt 11.6.0:
  <cpu mode='custom' match='exact' check='full'>
    <model fallback='forbid'>Skylake-Server-noTSX-IBRS</model>
    <topology sockets='1' dies='1' clusters='1' cores='2' threads='1'/>
    <feature policy='disable' name='mpx'/>
    <feature policy='require' name='hypervisor'/>
    <feature policy='require' name='ht'/>
    <feature policy='require' name='pku'/>
  </cpu>

If the disk is removed from the vm, ht always appears.

The raw xml of the offline vm:

<domain type='kvm'>
 <name>testvm</name>
 <uuid>eed2dcc0-674e-4d69-91a6-d51100f4c686</uuid>
 <metadata>
   <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
     <libosinfo:os id="http://redhat.com/rhel/9-unknown"/>
   </libosinfo:libosinfo>
 </metadata>
 <memory unit='KiB'>8388608</memory>
 <currentMemory unit='KiB'>8388608</currentMemory>
 <vcpu placement='static'>2</vcpu>
 <os>
   <type arch='x86_64' machine='pc-q35-9.2'>hvm</type>
   <bootmenu enable='no'/>
   <smbios mode='sysinfo'/>
 </os>
 <features>
   <acpi/>
   <apic/>
   <pvspinlock state='on'/>
   <vmport state='off'/>
 </features>
 <cpu mode='custom' match='exact' check='partial'>
   <model fallback='allow'>Skylake-Server-noTSX-IBRS</model>
   <topology sockets='1' dies='1' clusters='1' cores='2' threads='1'/>
 </cpu>
 <clock offset='utc'>
   <timer name='kvmclock' present='yes'/>
 </clock>
 <on_poweroff>destroy</on_poweroff>
 <on_reboot>restart</on_reboot>
 <on_crash>restart</on_crash>
 <pm>
   <suspend-to-mem enabled='no'/>
   <suspend-to-disk enabled='no'/>
 </pm>
 <devices>
   <emulator>/usr/bin/qemu-system-x86_64</emulator>
   <disk type='file' device='disk'>
     <driver name='qemu' type='qcow2' cache='none'/>
     <source file='/var/lib/libvirt/images/testvm.qcow2'/>
     <target dev='vda' bus='virtio'/>
     <boot order='1'/>
     <address type='pci' domain='0x0000' bus='0x05' slot='0x00' function='0x0'/>
   </disk>
   <controller type='usb' index='0' model='qemu-xhci' ports='15'>
     <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/>
   </controller>
   <controller type='pci' index='0' model='pcie-root'/>
   <controller type='pci' index='1' model='pcie-root-port'>
     <model name='pcie-root-port'/>
     <target chassis='1' port='0x10'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/>
   </controller>
   <controller type='pci' index='2' model='pcie-root-port'>
     <model name='pcie-root-port'/>
     <target chassis='2' port='0x11'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
   </controller>
   <controller type='pci' index='3' model='pcie-root-port'>
     <model name='pcie-root-port'/>
     <target chassis='3' port='0x12'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/>
   </controller>
   <controller type='pci' index='4' model='pcie-root-port'>
     <model name='pcie-root-port'/>
     <target chassis='4' port='0x13'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/>
   </controller>
   <controller type='pci' index='5' model='pcie-root-port'>
     <model name='pcie-root-port'/>
     <target chassis='5' port='0x14'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x4'/>
   </controller>
   <controller type='pci' index='6' model='pcie-root-port'>
     <model name='pcie-root-port'/>
     <target chassis='6' port='0x15'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x5'/>
   </controller>
   <controller type='pci' index='7' model='pcie-root-port'>
     <model name='pcie-root-port'/>
     <target chassis='7' port='0x16'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x6'/>
   </controller>
   <controller type='pci' index='8' model='pcie-root-port'>
     <model name='pcie-root-port'/>
     <target chassis='8' port='0x17'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x7'/>
   </controller>
   <controller type='pci' index='9' model='pcie-root-port'>
     <model name='pcie-root-port'/>
     <target chassis='9' port='0x18'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0' multifunction='on'/>
   </controller>
   <controller type='pci' index='10' model='pcie-root-port'>
     <model name='pcie-root-port'/>
     <target chassis='10' port='0x19'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x1'/>
   </controller>
   <controller type='pci' index='11' model='pcie-root-port'>
     <model name='pcie-root-port'/>
     <target chassis='11' port='0x1a'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x2'/>
   </controller>
   <controller type='pci' index='12' model='pcie-root-port'>
     <model name='pcie-root-port'/>
     <target chassis='12' port='0x1b'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x3'/>
   </controller>
   <controller type='pci' index='13' model='pcie-root-port'>
     <model name='pcie-root-port'/>
     <target chassis='13' port='0x1c'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x4'/>
   </controller>
   <controller type='pci' index='14' model='pcie-root-port'>
     <model name='pcie-root-port'/>
     <target chassis='14' port='0x1d'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x5'/>
   </controller>
   <controller type='sata' index='0'>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/>
   </controller>
   <controller type='virtio-serial' index='0'>
     <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/>
   </controller>
   <interface type='network'>
     <mac address='52:54:00:c5:5d:0d'/>
     <source network='vlan2002'/>
     <model type='virtio'/>
     <boot order='10'/>
     <rom bar='on'/>
     <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/>
   </interface>
   <serial type='pty'>
     <target type='isa-serial' port='0'>
       <model name='isa-serial'/>
     </target>
   </serial>
   <console type='pty'>
     <target type='serial' port='0'/>
   </console>
   <channel type='unix'>
     <target type='virtio' name='org.qemu.guest_agent.0'/>
     <address type='virtio-serial' controller='0' bus='0' port='1'/>
   </channel>
   <channel type='spicevmc'>
     <target type='virtio' name='com.redhat.spice.0'/>
     <address type='virtio-serial' controller='0' bus='0' port='2'/>
   </channel>
   <input type='tablet' bus='usb'>
     <address type='usb' bus='0' port='1'/>
   </input>
   <input type='mouse' bus='ps2'/>
   <input type='keyboard' bus='ps2'/>
   <graphics type='spice' autoport='yes'>
     <listen type='address'/>
     <image compression='off'/>
   </graphics>
   <audio id='1' type='spice'/>
   <video>
     <model type='virtio' heads='1' primary='yes'/>
     <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
   </video>
   <redirdev bus='usb' type='spicevmc'>
     <address type='usb' bus='0' port='2'/>
   </redirdev>
   <redirdev bus='usb' type='spicevmc'>
     <address type='usb' bus='0' port='3'/>
   </redirdev>
   <watchdog model='itco' action='none'/>
   <memballoon model='none'/>
   <rng model='virtio'>
     <backend model='random'>/dev/urandom</backend>
     <address type='pci' domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
   </rng>
 </devices>
</domain>

Both source and destination servers have working hyperthreading.

Edited by Jack Low
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information