The scheduler failed to assign job to the runner, please try again or contact system administrator

Summary

Sporadic accurences of runner failures since 12.5.2

Seems like the commit that was made to

Make jobs/request to be resillient

did exactly the opposite of what is was intended to do.

fd24f258

Steps to reproduce

Happens sporadidly (How one can reproduce the issue - this is very important)

Example Project

(If possible, please create an example project here on GitLab.com that exhibits the problematic behavior, and link to it here in the bug report)

(If you are using an older version of GitLab, this will also determine whether the bug is fixed in a more recent version)

Relevant logs and/or screenshots

(Paste any relevant logs - please use code blocks (```) to format console output, logs, and code as it's tough to read otherwise.)

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info

(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:env:info)
System information
System:
Current User:   git
Using RVM:      no
Ruby Version:   2.6.3p62
Gem Version:    2.7.9
Bundler Version:1.17.3
Rake Version:   12.3.3
Redis Version:  3.2.12
Git Version:    2.22.0
Sidekiq Version:5.2.7
Go Version:     unknown
GitLab information
Version:        12.5.2
Revision:       49482945d28
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     PostgreSQL
DB Version:     10.7
URL:            https://url
HTTP Clone URL: https://url/some-group/some-project.git
SSH Clone URL:  git@url:some-group/some-project.git
Using LDAP:     no
Using Omniauth: yes
Omniauth Providers: saml
GitLab Shell
Version:        10.2.0
Repository storage paths:

• default: /data/gitlab/git/repositories
• gitaly-02: /data/gitlab/git/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git

(For installations from source run and paste the output of: sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production)

Possible fixes

Disabling the ci_doom_build feature flag.

fd24f258

(If you can, link to the line of code that might be responsible for the problem)