Skip to content

Plugin scoreboard deadlock (plugin.lock vs start_exclusive)

Host environment

  • QEMU version: 9rc4->tip
  • QEMU command line: gdb --args ~/qemu-org/bin/qemu-riscv64 -d plugin -plugin /home/rehn/source/qemu/build/contrib/plugins/libhotblocks.so,inline=true /home/rehn/source/jdk/vanilla/build/zbb/images/jdk/bin/java -jar /home/rehn/Downloads/dacapo-9.12-MR1-bach.jar -n 1 jython

Description of problem

Deadlock

In frame 9 the thread grabs the plugin.lock, and starts to wait for other cpus to enter exclusive idle.

#7  0x00005555555a1295 in start_exclusive () at ../hw/core/cpu-common.c:199
#8  plugin_grow_scoreboards__locked (cpu=0x7fff0c2b4720) at ../plugins/core.c:238
#9  qemu_plugin_vcpu_init_hook (cpu=0x7fff0c2b4720) at ../plugins/core.c:258

The other thread just finished a TB and do the callback to the plugin, so it will not become exclusive idle until it finishes. That callback tries to create a new 'scoreboard', but plugin.lock is already taken.

#7  qemu_plugin_scoreboard_new (element_size=element_size@entry=8) at ../plugins/api.c:464
#8  0x00007ffff7fb973d in vcpu_tb_trans (id=<optimized out>, tb=0x555555858d60) at /home/rehn/source/qemu/contrib/plugins/hotblocks.c:125
#9  0x00005555557394f1 in qemu_plugin_tb_trans_cb (cpu=<optimized out>, tb=0x555555858d60) at ../plugins/core.c:418

Locally I'm using this fix, reverse order so we enter exclusive idle before grabbing the plugin.lock:

diff --git a/plugins/core.c b/plugins/core.c
index 1e58a57bf1..0e41c4ef22 100644
--- a/plugins/core.c
+++ b/plugins/core.c
@@ -236,4 +236,2 @@ static void plugin_grow_scoreboards__locked(CPUState *cpu)
 
-    /* cpus must be stopped, as tb might still use an existing scoreboard. */
-    start_exclusive();
     struct qemu_plugin_scoreboard *score;
@@ -244,3 +242,2 @@ static void plugin_grow_scoreboards__locked(CPUState *cpu)
     tb_flush(cpu);
-    end_exclusive();
 }
@@ -250,2 +247,4 @@ void qemu_plugin_vcpu_init_hook(CPUState *cpu)
     bool success;
+    /* cpus must be stopped, as tb might still use an existing scoreboard. */
+    start_exclusive();
 
@@ -259,2 +258,3 @@ void qemu_plugin_vcpu_init_hook(CPUState *cpu)
     qemu_rec_mutex_unlock(&plugin.lock);
+    end_exclusive();

Steps to reproduce

Run command a few times and get 'unlucky'

Edited by Robbin Ehn
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information