Dependency proxy fails when using a bot user

Summary

After !129697 (merged) was deployed to GitLab.com, we started to see 403 errors in jobs that use the dependency proxy, e.g. gitlab-org/quality/engineering-productivity/master-broken-incidents#4132 (comment 1640594844).

ERROR: Job failed: failed to pull image "gitlab.com:443/gitlab-org/dependency_proxy/containers/ruby:3.0-alpine3.16" with specified policies [always]: Error response from daemon: Head "https://gitlab.com:443/v2/gitlab-org/dependency_proxy/containers/ruby/manifests/3.0-alpine3.16": error parsing HTTP 403 response body: no error details found in HTTP response body: "{\"message\":\"access forbidden\",\"status\":\"error\",\"http_status\":403}" (manager.go:237:0s)

A manual retry resolves the problem which seem to indicate that there's an issue with dependency proxy authorization for bot users (@gitlab-bot in this case).

Steps to reproduce

Find a pipeline created by @gitlab-bot and look at the failed jobs.

Example Project

What is the current bug behavior?

403 errors when authenticating/authorizing with the dependency proxy.

What is the expected correct behavior?

No authenticating/authorizing errors with the dependency proxy.

Relevant logs and/or screenshots

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info


(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check

(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)
(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)
(we will only investigate if the tests are passing)