Skip to content

Crash: fatal error: concurrent map read and map write

Summary

One of our k8s runners crashed with the error message: fatal error: concurrent map read and map write. From our investigations this looks like it was caused by the dependency github.com/moby/spdystream. There seems to be an open PR for that dependency, that may fix this issue: https://github.com/moby/spdystream/pull/91.

Steps to reproduce

Reproducing is hard, because this seems to be a concurrency bug.

Relevant logs and/or screenshots

The following is the start of the stack trace and to our knowledge the relevant goroutine that causes the crash in the github.com/moby/spdystream dependency.

job log 
... (usual logs)
fatal error: concurrent map read and map write

goroutine ********* [running]:
github.com/moby/spdystream.(*Connection).handlePingFrame(0xc02c9afc20?, 0xc0008faf90?)
	/go/pkg/mod/github.com/moby/spdystream@v0.2.0/connection.go:618 +0x3b
github.com/moby/spdystream.(*Connection).frameHandler(0xc003329b80, 0x2600f80?, 0xc016dd93b0?)
	/go/pkg/mod/github.com/moby/spdystream@v0.2.0/connection.go:432 +0x17f
github.com/moby/spdystream.(*Connection).Serve.func2(0xc0017df7a8?)
	/go/pkg/mod/github.com/moby/spdystream@v0.2.0/connection.go:331 +0x66
created by github.com/moby/spdystream.(*Connection).Serve
	/go/pkg/mod/github.com/moby/spdystream@v0.2.0/connection.go:327 +0x71

... (rest of the stack trace omitted, because it is too long and not relevant)

Environment description

The runner is a k8s runner.

Used GitLab Runner version

v15.11.0

Possible fixes

The open PR https://github.com/moby/spdystream/pull/91 for github.com/moby/spdystream may fix this issue. If the upstream project does not fix the issue, you may need to fork the project and apply a fix.