Random kernel panic (2/3) in github macOS runner: IO-APIC + timer doesn't work!
Host environment
- Operating system: macOS 13.7.2
- OS/kernel version: Darwin Mac-1740261768089.local 22.6.0 Darwin Kernel Version 22.6.0: Fri Nov 15 17:21:49 PST 2024; root:xnu-8796.141.3.709.7~2/RELEASE_X86_64 x86_64
- Architecture: x86_64
- QEMU flavor: qemu-system-x86_64
- QEMU version: qemu--9.2.0
- QEMU command line:
qemu-system-x86_64 \
-name \
test \
-m \
1024 \
-cpu \
host \
-machine \
q35,accel=hvf \
-smp \
1,sockets=1,cores=1,threads=1 \
-drive \
if=pflash,format=raw,readonly=on,file=/usr/local/share/qemu/edk2-x86_64-code.fd \
-drive \
file=/Users/runner/.vmnet-helper/vms/test/disk.img,if=virtio,format=raw,discard=on \
-drive \
file=/Users/runner/.vmnet-helper/vms/test/cidata.iso,id=cdrom0,if=none,format=raw,readonly=on \
-device \
virtio-scsi-pci,id=scsi0 \
-device \
scsi-cd,bus=scsi0.0,drive=cdrom0 \
-netdev \
dgram,id=net1,local.type=fd,local.str=3 \
-device \
virtio-net-pci,netdev=net1,mac=1a:ad:75:f7:ca:a2 \
-monitor \
none \
-serial \
file:/Users/runner/.vmnet-helper/vms/test/serial.log \
-nographic \
-nodefaults
Emulated/Virtualized environment
- Operating system: Ubuntu 24.10
- OS/kernel version: Linux test 6.11.0-14-generic #15-Ubuntu SMP PREEMPT_DYNAMIC Fri Jan 10 23:48:25 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
- Architecture: x86_64
Description of problem
Random kernel panic (2/3 runs average) with this traceback:
[ 0.020000] Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report. Then try booting with the 'noapic' option.
[ 0.020000] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 6.11.0-14-generic #15-Ubuntu
[ 0.020000] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS edk2-stable202408-prebuilt.qemu.org 08/13/2024
[ 0.020000] Call Trace:
[ 0.020000] <TASK>
[ 0.020000] show_stack+0x49/0x60
[ 0.020000] dump_stack_lvl+0x5f/0x90
[ 0.020000] dump_stack+0x10/0x18
[ 0.020000] panic+0x16a/0x328
[ 0.020000] check_timer+0x4d1/0x570
[ 0.020000] setup_IO_APIC+0x1e5/0x210
[ 0.020000] apic_intr_mode_init+0xd0/0xf0
[ 0.020000] x86_late_time_init+0x24/0x40
[ 0.020000] start_kernel+0x3f9/0x4a0
[ 0.020000] x86_64_start_reservations+0x24/0x30
[ 0.020000] x86_64_start_kernel+0xf2/0x100
[ 0.020000] common_startup_64+0x13e/0x141
[ 0.020000] </TASK>
[ 0.020000] ---[ end Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report. Then try booting with the 'noapic' option. ]---
Steps to reproduce
- Start qemu in macos-13 github runner
Additional information
Example failed build: https://github.com/nirs/vmnet-helper/actions/runs/13477646025/job/37658748139
serial.log:
3h3hBdsDxe: failed to load Boot0001 "UEFI QEMU QEMU CD-ROM " from PciRoot(0x0)/Pci(0x1,0x0)/Scsi(0x0,0x0): Not Found
BdsDxe: loading Boot0002 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x3,0x0)
BdsDxe: starting Boot0002 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x3,0x0)
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
[ 0.000000] Linux version 6.11.0-14-generic (buildd@lcy02-amd64-032) (x86_64-linux-gnu-gcc-14 (Ubuntu 14.2.0-4ubuntu2) 14.2.0, GNU ld (GNU Binutils for Ubuntu) 2.43.1) #15-Ubuntu SMP PREEMPT_DYNAMIC Fri Jan 10 23:48:25 UTC 2025 (Ubuntu 6.11.0-14.15-generic 6.11.0)
[ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-6.11.0-14-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0
[ 0.000000] KERNEL supported cpus:
[ 0.000000] Intel GenuineIntel
[ 0.000000] AMD AuthenticAMD
[ 0.000000] Hygon HygonGenuine
[ 0.000000] Centaur CentaurHauls
[ 0.000000] zhaoxin Shanghai
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000007fffff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000000800000-0x0000000000807fff] ACPI NVS
[ 0.000000] BIOS-e820: [mem 0x0000000000808000-0x000000000080afff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000080b000-0x000000000080bfff] ACPI NVS
[ 0.000000] BIOS-e820: [mem 0x000000000080c000-0x0000000000810fff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000000811000-0x00000000008fffff] ACPI NVS
[ 0.000000] BIOS-e820: [mem 0x0000000000900000-0x000000003ee41fff] usable
[ 0.000000] BIOS-e820: [mem 0x000000003ee42000-0x000000003ef02fff] reserved
[ 0.000000] BIOS-e820: [mem 0x000000003ef03000-0x000000003f8ecfff] usable
[ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1.
[ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1.
[ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1.
[ 0.000000] NR_IRQS: 524544, nr_irqs: 256, preallocated irqs: 16
[ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.000000] Console: colour dummy device 80x25
[ 0.000000] printk: legacy console [tty1] enabled
[ 0.000000] printk: legacy console [ttyS0] enabled
[ 0.000000] ACPI: Core revision 20240322
[ 0.000000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns
[ 0.001000] APIC: Switch to symmetric I/O mode setup
[ 0.003000] x2apic: IRQ remapping doesn't support X2APIC mode
[ 0.011000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.013000] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
[ 0.013000] ...trying to set up timer (IRQ0) through the 8259A ...
[ 0.013000] ..... (found apic 0 pin 2) ...
[ 0.014000] ....... failed.
[ 0.014000] ...trying to set up timer as Virtual Wire IRQ...
[ 0.018000] ..... failed.
[ 0.018000] ...trying to set up timer as ExtINT IRQ...
[ 0.020000] ..... failed :(.
[ 0.020000] Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report. Then try booting with the 'noapic' option.
[ 0.020000] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 6.11.0-14-generic #15-Ubuntu
[ 0.020000] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS edk2-stable202408-prebuilt.qemu.org 08/13/2024
[ 0.020000] Call Trace:
[ 0.020000] <TASK>
[ 0.020000] show_stack+0x49/0x60
[ 0.020000] dump_stack_lvl+0x5f/0x90
[ 0.020000] dump_stack+0x10/0x18
[ 0.020000] panic+0x16a/0x328
[ 0.020000] check_timer+0x4d1/0x570
[ 0.020000] setup_IO_APIC+0x1e5/0x210
[ 0.020000] apic_intr_mode_init+0xd0/0xf0
[ 0.020000] x86_late_time_init+0x24/0x40
[ 0.020000] start_kernel+0x3f9/0x4a0
[ 0.020000] x86_64_start_reservations+0x24/0x30
[ 0.020000] x86_64_start_kernel+0xf2/0x100
[ 0.020000] common_startup_64+0x13e/0x141
[ 0.020000] </TASK>
[ 0.020000] ---[ end Kernel panic - not syncing: IO-APIC + timer doesn't work! Boot with apic=debug and send a report. Then try booting with the 'noapic' option. ]---
Same Ubuntu image never fail with vfkit vm on the same macos-13 github runners.