Skip to content

Flaky test due to eviction of catfile cache hanging

https://gitlab.com/gitlab-org/gitaly/-/jobs/1688130203

panic: goroutines running: found unexpected goroutines:
[Goroutine 9007 in state chan receive, 6 minutes, with gitlab.com/gitlab-org/gitaly/v14/internal/command.New.func2 on top of the stack:
goroutine 9007 [chan receive, 6 minutes]:
gitlab.com/gitlab-org/gitaly/v14/internal/command.New.func2()
	/builds/gitlab-org/gitaly/internal/command/command.go:222 +0x5d
created by gitlab.com/gitlab-org/gitaly/v14/internal/command.New
	/builds/gitlab-org/gitaly/internal/command/command.go:221 +0x1216

 Goroutine 9009 in state chan receive, 6 minutes, with gitlab.com/gitlab-org/gitaly/v14/internal/git/catfile.(*ProcessCache).getOrCreateProcess.func3 on top of the stack:
goroutine 9009 [chan receive, 6 minutes]:
gitlab.com/gitlab-org/gitaly/v14/internal/git/catfile.(*ProcessCache).getOrCreateProcess.func3()
	/builds/gitlab-org/gitaly/internal/git/catfile/cache.go:291 +0x54
created by gitlab.com/gitlab-org/gitaly/v14/internal/git/catfile.(*ProcessCache).getOrCreateProcess
	/builds/gitlab-org/gitaly/internal/git/catfile/cache.go:290 +0x8f4

 Goroutine 9006 in state IO wait, 6 minutes, with internal/poll.runtime_pollWait on top of the stack:
goroutine 9006 [IO wait, 6 minutes]:
internal/poll.runtime_pollWait(0x7f753462fd80, 0x72)
	/usr/local/go/src/runtime/netpoll.go:229 +0x89
internal/poll.(*pollDesc).wait(0xc000cced38, 0xc00052a000, 0x1)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:84 +0xbd
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:89
internal/poll.(*FD).Read(0xc000cced20, {0xc00052a000, 0x8000, 0x8000})
	/usr/local/go/src/internal/poll/fd_unix.go:167 +0x419
os.(*File).read(...)
	/usr/local/go/src/os/file_posix.go:32
os.(*File).Read(0xc0008040c8, {0xc00052a000, 0x8000, 0x8000})
	/usr/local/go/src/os/file.go:119 +0xc8
io.copyBuffer({0x1e2b200, 0xc000f17180}, {0x1e317e0, 0xc0008040c8}, {0x0, 0x0, 0x0})
	/usr/local/go/src/io/io.go:423 +0x28b
io.Copy(...)
	/usr/local/go/src/io/io.go:382
os/exec.(*Cmd).writerDescriptor.func1()
	/usr/local/go/src/os/exec/exec.go:311 +0x65
os/exec.(*Cmd).Start.func1(0xc00007fae0)
	/usr/local/go/src/os/exec/exec.go:441 +0x36
created by os/exec.(*Cmd).Start
	/usr/local/go/src/os/exec/exec.go:440 +0xd54

 Goroutine 9008 in state chan receive, 6 minutes, with gitlab.com/gitlab-org/gitaly/v14/internal/git/catfile.newObjectReader.func1 on top of the stack:
goroutine 9008 [chan receive, 6 minutes]:
gitlab.com/gitlab-org/gitaly/v14/internal/git/catfile.newObjectReader.func1()
	/builds/gitlab-org/gitaly/internal/git/catfile/object_reader.go:90 +0x66
created by gitlab.com/gitlab-org/gitaly/v14/internal/git/catfile.newObjectReader
	/builds/gitlab-org/gitaly/internal/git/catfile/object_reader.go:89 +0x69c
]
	panic: found running child processes 28789:
    PID COMMAND
  28789 /builds/gitlab-org/gitaly/_build/deps/git/install/bin/git --git-dir /tmp/gitaly-2897639034/734384428/storages.d/default/@hashed/4b/ae/4baea9b3fd248865ee2dee5842d70458ea1fdd5f4d9e84bbf38a526997ecb682.git -c core.fsyncObjectFiles=true -c gc.auto=0 -c core.autocrlf=input cat-file --batch --end-of-options


goroutine 1 [running]:
gitlab.com/gitlab-org/gitaly/v14/internal/testhelper.mustHaveNoChildProcess()
	/builds/gitlab-org/gitaly/internal/testhelper/leakage.go:60 +0x1a5
panic({0x13a6020, 0xc000aca100})
	/usr/local/go/src/runtime/panic.go:1047 +0x266
gitlab.com/gitlab-org/gitaly/v14/internal/testhelper.mustHaveNoGoroutines()
	/builds/gitlab-org/gitaly/internal/testhelper/leakage.go:36 +0x5cf
gitlab.com/gitlab-org/gitaly/v14/internal/testhelper.Run.func1({0x0, 0x0, 0xc000866380}, 0x0)
	/builds/gitlab-org/gitaly/internal/testhelper/configure.go:79 +0x2c1
gitlab.com/gitlab-org/gitaly/v14/internal/testhelper.Run(0x400, {0x0, 0x0, 0x0})
	/builds/gitlab-org/gitaly/internal/testhelper/configure.go:80 +0x55
gitlab.com/gitlab-org/gitaly/v14/internal/gitaly/service/commit.TestMain(...)
	/builds/gitlab-org/gitaly/internal/gitaly/service/commit/testhelper_test.go:20
main.main()
	_testmain.go:173 +0x269
FAIL	gitlab.com/gitlab-org/gitaly/v14/internal/gitaly/service/commit	455.705s
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information