Gitaly bundle isolation not working in CI
Discovered recently, see https://gitlab.com/gitlab-org/gitlab-ce/commits/gitaly-ruby-isolation-failure and build https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/50208205.
1) Gitlab::Git::Repository#delete_all_refs_except deletes all refs except those with the specified prefixes
Failure/Error: raise CommandError.new(e)
Gitlab::Git::CommandError:
2:undefined method `refs' for #<Gitaly::DeleteRefsRequest:0x00000000045e3d88>
# ./lib/gitlab/git/repository.rb:1361:in `rescue in gitaly_migrate'
This error is very odd. It's hard to read but what this is is an exception inside gitaly-ruby, where we call the Gitaly::DeleteRefsRequest#refs
method which is defined in gitaly-proto 0.83.0, as bundled with gitaly-ruby, but which is not defined in gitaly-proto 0.78.0, as bundled at the top level.
@zj and I spent time looking at this and we believe what is happening here is that gitaly-ruby is running with gitaly-proto 0.78.0 as specified in (CE) Gemfile.lock, not 0.83.0 as specified in tmp/tests/gitaly/ruby/Gemfile.lock.
This test failure was fixed when we updated gitaly-proto at the top-level. That strongly suggests that gitaly-ruby was picking up the top-level Gemfile instead of it's own Gemfile.
I don't think this is a problem for production installations; it's about the way we manage gems for gitlab and gitaly in CI.