Skip to content

"Maximum time in concurrency queue reached" when using git ssh for one specific repository on gitlab.com

Summary

Sometimes for multiple hours / days we cannot use git ssh on gitlab.com for one specific repository because every attempt to do so will result in the following error message after a few seconds of waiting:

15:38 $ git fetch
remote:
remote: ========================================================================
remote:
remote: ERROR: maximum time in concurrency queue reached

remote:
remote: ========================================================================
remote:
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

We have tested the following things and all of them will result in the same behavior:

  • Use different user accounts
  • Access from different locations (ec2 in us-east, ec2 in europe)

The issue was first observed early in december and since then would occasionally happen - at the moment ssh access is working again.
I have documented some times when it was not working in this forum thread.

If the error would only last a few minutes it would not be a problem, but the error often persists for multiple hours (and once even through the weekend) and does not allow anyone from anywhere to use the repository with the ssh endpoint. The HTTP endpoint is not affected.

I will update the forum thread when the issue occurs again.

Steps to reproduce

The issue was only observed on our main private monorepo. The project path is shopiq/core - a gitlab employee might be able to further diagnose this.

A few informations about our repository

  • Size is about 100MB
  • About 50 branches
  • 1000+ tags where some tags are moved to other refs sometimes
  • The repository has multiple root commits because it was created by merging multiple single project repositories
  • No usage of git lfs
  • One submodule

Example Project

What is the current bug behavior?

What is the expected correct behavior?

No error when using git via ssh.

Relevant logs and/or screenshots

Output of checks

This bug happens on GitLab.com

Possible fixes