Skip to content

Random kernel panic (2/3) in github macOS runner: IO-APIC + timer doesn't work!

Host environment

  • Operating system: macOS 13.7.2
  • OS/kernel version: Darwin Mac-1740261768089.local 22.6.0 Darwin Kernel Version 22.6.0: Fri Nov 15 17:21:49 PST 2024; root:xnu-8796.141.3.709.7~2/RELEASE_X86_64 x86_64
  • Architecture: x86_64
  • QEMU flavor: qemu-system-x86_64
  • QEMU version: qemu--9.2.0
  • QEMU command line:
qemu-system-x86_64 \
    -name \
    test \
    -m \
    1024 \
    -cpu \
    host \
    -machine \
    q35,accel=hvf \
    -smp \
    1,sockets=1,cores=1,threads=1 \
    -drive \
    if=pflash,format=raw,readonly=on,file=/usr/local/share/qemu/edk2-x86_64-code.fd \
    -drive \
    file=/Users/runner/.vmnet-helper/vms/test/disk.img,if=virtio,format=raw,discard=on \
    -drive \
    file=/Users/runner/.vmnet-helper/vms/test/cidata.iso,id=cdrom0,if=none,format=raw,readonly=on \
    -device \
    virtio-scsi-pci,id=scsi0 \
    -device \
    scsi-cd,bus=scsi0.0,drive=cdrom0 \
    -netdev \
    dgram,id=net1,local.type=fd,local.str=3 \
    -device \
    virtio-net-pci,netdev=net1,mac=1a:ad:75:f7:ca:a2 \
    -monitor \
    none \
    -serial \
    file:/Users/runner/.vmnet-helper/vms/test/serial.log \
    -nographic \
    -nodefaults

Emulated/Virtualized environment

  • Operating system: Ubuntu 24.10
  • OS/kernel version: Linux test 6.11.0-14-generic #15-Ubuntu SMP PREEMPT_DYNAMIC Fri Jan 10 23:48:25 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
  • Architecture: x86_64

Description of problem

Random kernel panic (2/3 runs average) with this traceback:

[    0.020000] Kernel panic - not syncing: IO-APIC + timer doesn't work!  Boot with apic=debug and send a report.  Then try booting with the 'noapic' option.
[    0.020000] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 6.11.0-14-generic #15-Ubuntu
[    0.020000] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS edk2-stable202408-prebuilt.qemu.org 08/13/2024
[    0.020000] Call Trace:
[    0.020000]  <TASK>
[    0.020000]  show_stack+0x49/0x60
[    0.020000]  dump_stack_lvl+0x5f/0x90
[    0.020000]  dump_stack+0x10/0x18
[    0.020000]  panic+0x16a/0x328
[    0.020000]  check_timer+0x4d1/0x570
[    0.020000]  setup_IO_APIC+0x1e5/0x210
[    0.020000]  apic_intr_mode_init+0xd0/0xf0
[    0.020000]  x86_late_time_init+0x24/0x40
[    0.020000]  start_kernel+0x3f9/0x4a0
[    0.020000]  x86_64_start_reservations+0x24/0x30
[    0.020000]  x86_64_start_kernel+0xf2/0x100
[    0.020000]  common_startup_64+0x13e/0x141
[    0.020000]  </TASK>
[    0.020000] ---[ end Kernel panic - not syncing: IO-APIC + timer doesn't work!  Boot with apic=debug and send a report.  Then try booting with the 'noapic' option. ]---

Steps to reproduce

  1. Start qemu in macos-13 github runner

Additional information

Example failed build: https://github.com/nirs/vmnet-helper/actions/runs/13477646025/job/37658748139

serial.log:

3h3hBdsDxe: failed to load Boot0001 "UEFI QEMU QEMU CD-ROM " from PciRoot(0x0)/Pci(0x1,0x0)/Scsi(0x0,0x0): Not Found
BdsDxe: loading Boot0002 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x3,0x0)
BdsDxe: starting Boot0002 "UEFI Misc Device" from PciRoot(0x0)/Pci(0x3,0x0)
EFI stub: Loaded initrd from LINUX_EFI_INITRD_MEDIA_GUID device path
[    0.000000] Linux version 6.11.0-14-generic (buildd@lcy02-amd64-032) (x86_64-linux-gnu-gcc-14 (Ubuntu 14.2.0-4ubuntu2) 14.2.0, GNU ld (GNU Binutils for Ubuntu) 2.43.1) #15-Ubuntu SMP PREEMPT_DYNAMIC Fri Jan 10 23:48:25 UTC 2025 (Ubuntu 6.11.0-14.15-generic 6.11.0)
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-6.11.0-14-generic root=LABEL=cloudimg-rootfs ro console=tty1 console=ttyS0
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Hygon HygonGenuine
[    0.000000]   Centaur CentaurHauls
[    0.000000]   zhaoxin   Shanghai  
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000007fffff] usable
[    0.000000] BIOS-e820: [mem 0x0000000000800000-0x0000000000807fff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x0000000000808000-0x000000000080afff] usable
[    0.000000] BIOS-e820: [mem 0x000000000080b000-0x000000000080bfff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000000080c000-0x0000000000810fff] usable
[    0.000000] BIOS-e820: [mem 0x0000000000811000-0x00000000008fffff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x0000000000900000-0x000000003ee41fff] usable
[    0.000000] BIOS-e820: [mem 0x000000003ee42000-0x000000003ef02fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000003ef03000-0x000000003f8ecfff] usable
[    0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1.
[    0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1.
[    0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1.
[    0.000000] NR_IRQS: 524544, nr_irqs: 256, preallocated irqs: 16
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] Console: colour dummy device 80x25
[    0.000000] printk: legacy console [tty1] enabled
[    0.000000] printk: legacy console [ttyS0] enabled
[    0.000000] ACPI: Core revision 20240322
[    0.000000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns
[    0.001000] APIC: Switch to symmetric I/O mode setup
[    0.003000] x2apic: IRQ remapping doesn't support X2APIC mode
[    0.011000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.013000] ..MP-BIOS bug: 8254 timer not connected to IO-APIC
[    0.013000] ...trying to set up timer (IRQ0) through the 8259A ...
[    0.013000] ..... (found apic 0 pin 2) ...
[    0.014000] ....... failed.
[    0.014000] ...trying to set up timer as Virtual Wire IRQ...
[    0.018000] ..... failed.
[    0.018000] ...trying to set up timer as ExtINT IRQ...
[    0.020000] ..... failed :(.
[    0.020000] Kernel panic - not syncing: IO-APIC + timer doesn't work!  Boot with apic=debug and send a report.  Then try booting with the 'noapic' option.
[    0.020000] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Not tainted 6.11.0-14-generic #15-Ubuntu
[    0.020000] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS edk2-stable202408-prebuilt.qemu.org 08/13/2024
[    0.020000] Call Trace:
[    0.020000]  <TASK>
[    0.020000]  show_stack+0x49/0x60
[    0.020000]  dump_stack_lvl+0x5f/0x90
[    0.020000]  dump_stack+0x10/0x18
[    0.020000]  panic+0x16a/0x328
[    0.020000]  check_timer+0x4d1/0x570
[    0.020000]  setup_IO_APIC+0x1e5/0x210
[    0.020000]  apic_intr_mode_init+0xd0/0xf0
[    0.020000]  x86_late_time_init+0x24/0x40
[    0.020000]  start_kernel+0x3f9/0x4a0
[    0.020000]  x86_64_start_reservations+0x24/0x30
[    0.020000]  x86_64_start_kernel+0xf2/0x100
[    0.020000]  common_startup_64+0x13e/0x141
[    0.020000]  </TASK>
[    0.020000] ---[ end Kernel panic - not syncing: IO-APIC + timer doesn't work!  Boot with apic=debug and send a report.  Then try booting with the 'noapic' option. ]---

Same Ubuntu image never fail with vfkit vm on the same macos-13 github runners.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information