qemu-system-x86_64: Linux kernel warning when CONFIG_NUMA_EMU is enabled
Host environment
- Operating system: Linux
- OS/kernel version: SMP Debian 5.10.46-5
- Architecture: x86_64
- QEMU flavor: qemu-system-x86_64
- QEMU version:
QEMU emulator version 6.2.0 (v6.2.0-dirty), commit44f28df24767cf9dca1ddc9b23157737c4cbb645 - QEMU command line:
./qemu-system-x86_64 -m 4G -smp 2 -kernel ~/linux-stable/arch/x86/boot/bzImage -append "console=ttyS0 root=/dev/sda earlyprintk=serial" -enable-kvm -nographic -snapshot
Emulated/Virtualized environment
- Operating system: Linux
- OS/kernel version:
v5.16.9 - Architecture: x86
Description of problem
When Linux kernel is run on qemu 6.2, it prints a warning when NUMA_EMU is used. When the same kernel is run on qemu 6.1.1 (54e1f5be86dd11744e45da8be6afad01d01d59e7) or earlier, no such warning is printed.
[ 0.341924] smpboot: CPU0: Intel QEMU Virtual CPU version 2.5+ (family: 0xf, model: 0x6b, stepping: 0x1)
[ 0.342371] Performance Events: unsupported Netburst CPU model 107 no PMU driver, software events only.
[ 0.343302] rcu: Hierarchical SRCU implementation.
[ 0.344470] smp: Bringing up secondary CPUs ...
[ 0.345349] x86: Booting SMP configuration:
[ 0.345945] .... node #1, CPUs: #1
[ 0.014099] ------------[ cut here ]------------
[ 0.014099] sched: CPU #1's llc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
[ 0.014099] WARNING: CPU: 1 PID: 0 at arch/x86/kernel/smpboot.c:423 topology_sane.isra.0+0x62/0x70
[ 0.014099] Modules linked in:
[ 0.014099] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.16.9 #6
[ 0.014099] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.15.0-0-g2dd4b9b3f840-prebuilt.qemu.org 04/01/2014
[ 0.014099] RIP: 0010:topology_sane.isra.0+0x62/0x70
[ 0.014099] Code: 80 3d e1 72 a1 01 00 75 f6 48 83 ec 08 4c 89 da 44 89 d6 48 c7 c7 c0 cd f4 8b 88 44 24 07 c6 05 c3 72 a1 01 01 e8 3c 16 b7 00 <0f> 0b 0f b6 44 24 07 48 83 c4 08 c3 66 90 48 8b 0d 21 95 a3 01 0f
[ 0.014099] RSP: 0000:ffffa8c3006a3ed8 EFLAGS: 00010086
[ 0.014099] RAX: 0000000000000000 RBX: ffffa335fdc15480 RCX: 0000000000000000
[ 0.014099] RDX: 0000000000000002 RSI: 00000000ffffffea RDI: 00000000ffffffff
[ 0.014099] RBP: ffffa3353dc15480 R08: ffffffff8c335ac8 R09: 00000000ffffdfff
[ 0.014099] R10: ffffffff8c255ae0 R11: ffffffff8c255ae0 R12: 0000000000000001
[ 0.014099] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000001
[ 0.014099] FS: 0000000000000000(0000) GS:ffffa335fdc00000(0000) knlGS:0000000000000000
[ 0.014099] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 0.014099] CR2: 0000000000000000 CR3: 0000000112a0c000 CR4: 00000000000006e0
[ 0.014099] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 0.014099] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 0.014099] Call Trace:
[ 0.014099] <TASK>
[ 0.014099] set_cpu_sibling_map+0x16a/0x560
[ 0.014099] start_secondary+0x42/0xf0
[ 0.014099] secondary_startup_64_no_verify+0xc2/0xcb
[ 0.014099] </TASK>
[ 0.014099] Kernel panic - not syncing: panic_on_warn set ...
[ 0.014099] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.16.9 #6
[ 0.014099] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.15.0-0-g2dd4b9b3f840-prebuilt.qemu.org 04/01/2014
[ 0.014099] Call Trace:
[ 0.014099] <TASK>
[ 0.014099] dump_stack_lvl+0x34/0x44
[ 0.014099] panic+0xef/0x2a6
[ 0.014099] ? topology_sane.isra.0+0x62/0x70
[ 0.014099] __warn.cold+0x26/0x30
[ 0.014099] ? topology_sane.isra.0+0x62/0x70
[ 0.014099] report_bug+0x9a/0xc0
[ 0.014099] handle_bug+0x3c/0x60
[ 0.014099] exc_invalid_op+0x14/0x70
[ 0.014099] asm_exc_invalid_op+0x12/0x20
[ 0.014099] RIP: 0010:topology_sane.isra.0+0x62/0x70
[ 0.014099] Code: 80 3d e1 72 a1 01 00 75 f6 48 83 ec 08 4c 89 da 44 89 d6 48 c7 c7 c0 cd f4 8b 88 44 24 07 c6 05 c3 72 a1 01 01 e8 3c 16 b7 00 <0f> 0b 0f b6 44 24 07 48 83 c4 08 c3 66 90 48 8b 0d 21 95 a3 01 0f
[ 0.014099] RSP: 0000:ffffa8c3006a3ed8 EFLAGS: 00010086
[ 0.014099] RAX: 0000000000000000 RBX: ffffa335fdc15480 RCX: 0000000000000000
[ 0.014099] RDX: 0000000000000002 RSI: 00000000ffffffea RDI: 00000000ffffffff
[ 0.014099] RBP: ffffa3353dc15480 R08: ffffffff8c335ac8 R09: 00000000ffffdfff
[ 0.014099] R10: ffffffff8c255ae0 R11: ffffffff8c255ae0 R12: 0000000000000001
[ 0.014099] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000001
[ 0.014099] ? topology_sane.isra.0+0x62/0x70
[ 0.014099] set_cpu_sibling_map+0x16a/0x560
[ 0.014099] start_secondary+0x42/0xf0
[ 0.014099] secondary_startup_64_no_verify+0xc2/0xcb
[ 0.014099] </TASK>
[ 0.014099] ---[ end Kernel panic - not syncing: panic_on_warn set ... ]---Steps to reproduce
- Check out the Linux kernel:
git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
cd linux-stable
git checkout v5.16.9- Generate the configuration file:
make defconfig
./scripts/config -e NUMA_EMU -e CMDLINE_BOOL --set-str CMDLINE "numa=fake=2 panic_on_warn=1" -d CONFIG_CMDLINE_OVERRIDE- Build the kernel
make -j32- Run qemu and wait for a couple of seconds:
./qemu-system-x86_64 -m 4G -smp 2 -kernel ~/linux-stable/arch/x86/boot/bzImage -append "console=ttyS0 root=/dev/sda earlyprintk=serial" -enable-kvm -nographic -snapshotAdditional information
With explicit NUMA configuration, it boots fine:
./qemu-system-x86_64 -m 4G -smp 2 -object memory-backend-ram,size=8G,id=m0 -numa node,cpus=0-1,nodeid=0,memdev=m0 -kernel ~/linux-stable/arch/x86/boot/bzImage -append "console=ttyS0 root=/dev/sda earlyprintk=serial" -enable-kvm -nographic -snapshotOn the host machine:
$ numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53
node 0 size: 95259 MB
node 0 free: 1767 MB
node 1 cpus: 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71
node 1 size: 96757 MB
node 1 free: 2407 MB
node distances:
node 0 1
0: 10 21
1: 21 10