**Package:** `modernc.org/libc` **Affected:** `___lock` / `___unlock` in `libc_musl.go`, byte-identical in `v1.72.5` and the current release `v1.74.1`. ## Summary `___lock`/`___unlock` implement a C mutex over an `int32` lock word plus a Go-side handoff map `locks map[uintptr]*lock`. When an unlocker runs in the window after a contending locker has incremented the lock word but before it has registered itself in the map, the unlocker creates a throwaway handoff object, releases and deletes it, and drops the waiter count to zero. The waiter then finds the map entry gone, allocates a *fresh* lock, and blocks on it forever — the handoff it was waiting for already happened and was discarded. The lock word is left non-zero, so every subsequent `___lock` on that address also blocks. The process wedges permanently at zero CPU. ## The interleaving ``` T1 ___lock(&w) w: 0 -> 1, ==1 => fast path, returns holding the lock T2 ___lock(&w) w: 1 -> 2, !=1 => slow path ... about to take locksMu, has NOT registered in `locks` yet T1 ___unlock(&w) w: 2 -> 1, !=0 => slow path locksMu.Lock(); locks[&w] == nil (T2 not registered) => l = &lock{waiters:1}; l.Lock(); l.Unlock() => waiters-- == 0 => delete(locks, &w) // handoff discarded locksMu.Unlock() T2 (resumes) locksMu.Lock(); locks[&w] == nil (deleted) => l = &lock{}; locks[&w] = l; l.Lock(); waiters++ locksMu.Unlock() l.Lock() // BLOCKS FOREVER — nothing will unlock this l ``` `w` is left at 1, so later `___lock(&w)` callers also take the slow path and pile up behind the dead `l`. ## Reproducer Self-contained, no dependencies — `go run main.go`, ~5 s, prints the verdict. It transcribes `___lock`/`___unlock` **verbatim** from `libc_musl.go`. The only adaptations are the C `uintptr` lock word becoming `*int32` and one scheduling hook on the slow path so the interleaving is *forced* rather than raced for — no locking logic is changed. Moving the hook to fire *after* registration (i.e. the ordering the code comment assumes) makes it complete without hanging, so this is the interleaving itself, not a doctored transcription. ```go // Minimal, self-contained reproducer for a lost-wakeup deadlock in // modernc.org/libc's musl lock emulation (___lock / ___unlock, libc_musl.go). // // `go run main.go` — Exit 0 = bug reproduced (deadlock), Exit 1 = not present. // // ___lock / ___unlock are transcribed VERBATIM from modernc.org/libc@v1.74.1 // (byte-identical in v1.72.5). Only adaptations: the C `p uintptr` lock word // becomes a `*int32`, the map is keyed by that pointer, and one scheduling hook // is inserted on lock's slow path — at the exact point the race turns on — so // the interleaving is forced instead of raced for. No locking logic is changed. package main import ( "fmt" "os" "sync" "sync/atomic" "time" ) // ---- transcribed from libc_musl.go (verbatim except as noted above) ---- type lock struct { sync.Mutex waiters int } var ( locksMu sync.Mutex locks = map[*int32]*lock{} ) // hook: no-op in libc; here it pauses lock's slow path at the post-increment / // pre-register point. Fast-path acquirers never reach it. var lockHook = func() {} func ___lock(p *int32) { if atomic.AddInt32(p, 1) == 1 { return } lockHook() // foo was already acquired by some other C thread. locksMu.Lock() l := locks[p] if l == nil { l = &lock{} locks[p] = l l.Lock() } l.waiters++ locksMu.Unlock() l.Lock() // Wait for T1 to release foo. } func ___unlock(p *int32) { if atomic.AddInt32(p, -1) == 0 { return } // Some other C thread is waiting for foo. locksMu.Lock() l := locks[p] if l == nil { // We are T1 and we got the locksMu locked before T2. l = &lock{waiters: 1} l.Lock() } l.Unlock() // Release foo, T2 may now lock it. l.waiters-- if l.waiters == 0 { // we are T2 delete(locks, p) } locksMu.Unlock() } // ---- driver: force the unlock-before-register interleaving ---- func main() { var counter int32 paused := make(chan struct{}) resume := make(chan struct{}) lockHook = func() { lockHook = func() {} // fire once, for the second (contending) locker only close(paused) <-resume } // T1 acquires on the fast path (counter 0 -> 1) and holds the lock. ___lock(&counter) // T2 contends: counter 1 -> 2, enters the slow path, pauses in the hook // BEFORE it has registered itself in the `locks` map. t2done := make(chan struct{}) go func() { ___lock(&counter) close(t2done) }() <-paused // T1 unlocks into that window. It finds locks[p] == nil (T2 not yet // registered), so it creates a throwaway handoff, unlocks it, drops waiters // to 0, and DELETES the map entry — the handoff is discarded. ___unlock(&counter) close(resume) // T2 resumes: locks[p] == nil (deleted), so it makes a fresh lock and blocks // on it forever. Nothing will ever unlock it. select { case <-t2done: fmt.Println("NO DEADLOCK: T2 acquired the lock — bug not present in this build.") os.Exit(1) case <-time.After(5 * time.Second): fmt.Println("DEADLOCK REPRODUCED: T2's ___lock never returned.") fmt.Printf(" lock word counter = %d (non-zero: the lock looks held, but no goroutine holds it)\n", atomic.LoadInt32(&counter)) fmt.Println(" Every subsequent ___lock on this address would also block.") os.Exit(0) } } ``` Output on current `libc`: ``` DEADLOCK REPRODUCED: T2's ___lock never returned. lock word counter = 1 (non-zero: the lock looks held, but no goroutine holds it) Every subsequent ___lock on this address would also block. ``` ## How it shows up in practice Two goroutines each running a `modernc.org/quickjs` VM, both evaluating a JavaScript `Date` local-time getter (`getFullYear`, `getMonth`, `getDate`): ``` JS Date.getFullYear() -> quickjs getTimezoneOffset -> libc localtime_r -> __secs_to_zone -> __lock(&<timezone lock>) ``` Both VMs contend on the single process-global timezone lock. Correct, balanced usage — but two concurrent callers can hit the window above. Observed in production as an indexer that hung with 8 goroutines parked in `modernc.org/libc.___lock` for 35 minutes at zero CPU. UTC-only paths (`toISOString`) never take the lock, so it stays hidden until concurrent code touches a local-time getter. ## Suggested fix direction The handoff must not be torn down until the waiter has observed it. Options: - In `___unlock`, when `locks[p] == nil`, do **not** synthesize-and-delete a handoff; instead leave a record the waiter can find (or make the waiter re-check the lock word and retry rather than block unconditionally). - Or gate the lock/unlock pair on a per-address condition variable keyed in `locks`, so a wakeup delivered before the wait is not lost.