**Disclosure:** I found and analysed this with the help of an AI assistant (Anthropic Claude), and the attached reproducer was written with it. I re-ran the steps and checked every result below myself on the host described. The reproducer is attached only to demonstrate the problem; it is not offered for inclusion in QEMU. ## Host environment - Operating system: Ubuntu 24.04.5 LTS (WSL 2 on Windows) - OS/kernel version: Linux 6.18.40.1-microsoft-standard-WSL2 #1 SMP PREEMPT_DYNAMIC x86_64 GNU/Linux - Architecture: x86_64, 12 host CPUs - QEMU flavor: qemu-system-arm, qemu-system-aarch64 - QEMU version: QEMU emulator version 11.1.50 (v11.1.0-1862-gf7ada39eda), built from master f7ada39eda (30 Sep 2026) - QEMU command line: ``` ./qemu-system-arm -M virt -cpu max -smp 4 -display none \ -semihosting-config enable=on -kernel wfe-sev-lock-arm ./qemu-system-aarch64 -M virt -cpu max -smp 4 -display none \ -semihosting-config enable=on -kernel wfe-sev-lock-aarch64 ``` ## Emulated/Virtualized environment - Operating system: bare-metal test program (attached, built against tests/tcg minilib and boot.S) - OS/kernel version: n/a (first seen with an SMP 32-bit Linux 4.1 guest, see below) - Architecture: Arm, AArch32 and AArch64, TCG with MTTCG (the default) ## Description of problem Now that WFE really halts the vCPU (Cortex-M since d238858bff in 11.0, A-profile since 60e7ee5bb7/ 01b223f895 in 11.1), an SMP guest whose lock waiters sleep in WFE and whose releaser executes SEV can hang. One waiter stays halted after its WFE even though the lock has been released to it. This is the 32-bit Arm Linux ticket spinlock: `arch_spin_lock()` waits in `wfe()` and `arch_spin_unlock()` ends with `dsb_sev()`. Inside `spin_lock_irqsave()` interrupts are masked, so if no other event arrives the waiter never wakes. The sleep side runs under the BQL. `qemu_process_cpu_events()` clears `exit_request` and then loops (system/cpus.c:405-415): ``` while (cpu_thread_is_idle(cpu)) { ... qemu_cond_wait(cpu->halt_cond, &bql); } ``` `cpu_thread_is_idle()` reaches `arm_cpu_has_work()`, which for a WFE halt reads `env.event_register` (target/arm/cpu.c:161). The wake side runs without the BQL. `HELPER(sev)` (target/arm/tcg/op_helper.c:471-481) runs on the releasing vCPU's thread, sets `target_cpu->env.event_register = true` for every CPU, and calls `qemu_cpu_kick()` for the others. The kick does `qemu_cond_broadcast(cpu->halt_cond)` (system/cpus.c:444) and the TCG kick, which only sets `exit_request`. The halt loop above has already cleared that flag and doesn't check it again. So if the SEV lands after the sleeper has evaluated `cpu_thread_is_idle()` but before it is blocked in `qemu_cond_wait()`, the broadcast has no waiter. `event_register` is now true, but the thread sleeps until some unrelated kick. This is the usual lost wake-up of a condition variable that is signalled without holding its mutex. The other wakers I looked at hold the BQL: `cpu_interrupt()` asserts it (system/cpus.c:269), and the WFxT timer callback runs in the main loop. As far as I can see, SEV is the only guest-reachable path that changes another vCPU's `has_work()` state from a vCPU thread without the BQL. (In review of "accel/tcg: Make halt-to-exec transition explicit" v1 06/11, the Arm SEV helper was mentioned as already kicking the CPU. That is true, but the kick's broadcast is exactly what gets lost here.) ## Steps to reproduce 1. Build the attached programs against a QEMU source tree. They use the tests/tcg minilib, boot.S and kernel.ld: ``` QEMU=/path/to/qemu.git \ ARM_CC=arm-linux-gnueabi-gcc \ A64_CC=aarch64-none-elf-gcc \ ./build.sh ``` I used Ubuntu's arm-linux-gnueabi-gcc 13.3.0 and Arm GNU Toolchain 14.2.rel1 aarch64-none-elf-gcc. Any aarch64 gcc that can build tests/tcg/aarch64/system should do. 2. Run either command line above. CPU0 starts CPUs 1-3 with PSCI CPU_ON. Each worker takes one WFE/SEV ticket lock 20000 times, with interrupts masked and no timer or event stream. CPU0 only watches. If no worker makes progress for 500 ms, it prints the lock state, executes SEV itself to recover, and carries on. After 20 stalls, or at the end of a run with any stall, it prints FAIL and exits 1; a clean run prints PASS and exits 0. 3. On master, typical output is: ``` 4 CPUs, 3 contending workers, 20000 iterations each STALL 1: no progress for 500 ms; lock owner 949 next 952 cpu1 holds the next ticket 949 and is still waiting: lost wake-up ... FAIL: giving up after 20 stalls ``` While it is stalled, all three worker vCPU threads are blocked in the futex wait on `halt_cond`, and each guest PC is just after its WFE. ## Additional information Results on this host, each run with MTTCG and `-smp` as shown: | build | guest | \-smp | runs that stalled | |-------|-------|-------|-------------------| | master f7ada39 | arm | 3 | 7/30 | | master f7ada39 | arm | 4 | 16/30 | | master f7ada39 | aarch64 | 3 | 11/30 | | master f7ada39 | aarch64 | 4 | 25/30 | | master f7ada39, `-accel tcg,thread=single` | arm, aarch64 | 4 | 0/6, 0/6 | | master + BQL taken in `HELPER(sev)` (local experiment) | the four MTTCG rows | | 0/120 | A later re-check, run serially, stalled in 12/12 arm and 9/9 aarch64 runs (-smp 4) on master. With the experiment, 6/6 arm and 9/9 aarch64 runs passed. The local experiment, wrapping the body of `HELPER(sev)` in `BQL_LOCK_GUARD()`, removes the hang. It costs about 1.5-1.7x on a loop that does an uncontended SEV every iteration: median 232 ms vs 399 ms at -smp 4, 837 ms vs 1294 ms at -smp 8. 32-bit Linux does a SEV on every `spin_unlock()`, so a real fix might prefer to take the BQL only when the target vCPU is halted, or to make the halt wait recheck `event_register` safely. I haven't tried either. Where this was found: an out-of-tree SMP board model booting a 32-bit Linux 4.1 kernel hung in 9 of 32 boots. Both vCPUs were halted after the WFE in `_raw_spin_lock_irqsave()` (clockevents lock, IRQs masked, no tick pending), with the lock word reading `next == owner + 2`. With the experiment above, 0 of 40 boots hung. On a stock `virt` Linux guest a pending timer interrupt usually rescues the waiter, so I'd expect a lost wake-up there to show up as an occasional delay of up to one tick rather than a hang. This probably affects stable-11.1, and Cortex-M SMP boards since 11.0. Possibly related, but with different causes: #4044 (SEV was a NOP on v6K, fixed by dd0d695306) and #4580 (WFxT/WFE deadline units). [build.sh](/uploads/37e03ebbe82169a6714db61e0b401286/build.sh) [wfe-sev-lock-arm.c](/uploads/0293aa6efeac0b3f9dcdfce9356e533b/wfe-sev-lock-arm.c) [wfe-sev-lock-aarch64.c](/uploads/d91a852fe4ec2f59bf55801d1ba236c3/wfe-sev-lock-aarch64.c)