Registry and Repo basic auth no longer working

Summary

After upgrading GitLab (Docker installation) to 17.11.0, accessing the Container Registry via Docker client fails with 500 error on /jwt/auth endpoint if no Authorization header is provided. This used to return 401, but now raises a TypeError due to a nil token. Or i may be mistaken...

Additionally, users who previously used https instead of SSH links to clone the repo and used their username and password instead of Personal Access Tokens also experience 500 error when pulling or pushing changes to repo.

Steps to reproduce

Configure GitLab with Container Registry.
Do not perform docker login (use already previously configured credentials).

Run this command:

docker pull <self-hosted-gitlab-host>/<namespace>/<project>/image:tag

Error obtained:

Error response from daemon: Get "https://<self-hosted-gitlab-host>:5050/v2/": received unexpected HTTP status: 500 Internal Server Error

Observe GitLab logs (e.g., /var/log/gitlab/gitlab-rails/production.log).

Example Project

Internal self-managed instance - not publicly accessible.

What is the current bug behavior?

Instead of authorizing client with previous credentials, GitLab raises:

TypeError (no implicit conversion of nil into String)

Specifically in:

lib/authn/tokens/personal_access_token.rb:7:in `start_with?'

This results in a 500 Internal Server Error on the /jwt/auth endpoint.

Related log:

gitlab  | ==> /var/log/gitlab/gitlab-workhorse/current <==
gitlab  | {"backend_id":"rails","content_type":"text/plain; charset=utf-8","correlation_id":"01JS80KXM4SQ0TRN56RR54GBQD","duration_ms":10,"host":"dcc-gitlab.addiko.com","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","read_bytes":373,"referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","route_id":"default","status":401,"system":"http","time":"2025-04-19T21:52:32Z","ttfb_ms":10,"uri":"/v2/","user_agent":"docker/27.3.1 go/go1.22.7 git-commit/41ca978 kernel/6.8.0-1015-aws os/linux arch/amd64 UpstreamClient(Docker-Client/27.3.1 \\(linux\\))","written_bytes":0}
gitlab  |
gitlab  | ==> /var/log/gitlab/nginx/gitlab_access.log <==
gitlab  | 172.31.18.120 - - [19/Apr/2025:21:52:32 +0000] "GET /v2/ HTTP/1.1" 401 5 "" "docker/27.3.1 go/go1.22.7 git-commit/41ca978 kernel/6.8.0-1015-aws os/linux arch/amd64 UpstreamClient(Docker-Client/27.3.1 \x5C(linux\x5C))" -
gitlab  |
gitlab  | ==> /var/log/gitlab/gitlab-rails/production_json.log <==
gitlab  | {"method":"GET","path":"/jwt/auth","format":"html","controller":"JwtController","action":"auth","status":500,"time":"2025-04-19T21:52:32.449Z","params":[{"key":"account","value":"dcc"},{"key":"scope","value":"repository:mtodoric/docker-images/ansible:pull"},{"key":"service","value":"dependency_proxy"}],"correlation_id":"01JS80KXMNBAF457A1M1A736W5","meta.caller_id":"JwtController#auth","meta.feature_category":"container_registry","remote_ip":"172.31.18.120","ua":"docker/27.3.1 go/go1.22.7 git-commit/41ca978 kernel/6.8.0-1015-aws os/linux arch/amd64 UpstreamClient(Docker-Client/27.3.1 \\(linux\\))","request_urgency":"low","target_duration_s":5,"redis_calls":9,"redis_duration_s":0.000904,"redis_read_bytes":1508,"redis_write_bytes":493,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.000904,"redis_feature_flag_read_bytes":1508,"redis_feature_flag_write_bytes":493,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040306,"mem_objects":15393,"mem_bytes":1859612,"mem_mallocs":3128,"mem_total_bytes":2475332,"pid":4577,"worker_id":"puma_0","rate_limiting_gates":[],"exception.class":"TypeError","exception.message":"no implicit conversion of nil into String","exception.backtrace":["lib/authn/tokens/personal_access_token.rb:7:in `start_with?'","lib/authn/tokens/personal_access_token.rb:7:in `prefix?'","lib/authn/agnostic_token_identifier.rb:30:in `block in token_type'","lib/authn/agnostic_token_identifier.rb:30:in `each'","lib/authn/agnostic_token_identifier.rb:30:in `find'","lib/authn/agnostic_token_identifier.rb:30:in `token_type'","lib/authn/agnostic_token_identifier.rb:26:in `token?'","lib/gitlab/auth.rb:232:in `user_with_password_for_git'","lib/gitlab/auth.rb:105:in `find_for_git_client'","app/controllers/jwt_controller.rb:45:in `block in authenticate_project_or_user'","actionpack (7.0.8.7) lib/action_controller/metal/http_authentication.rb:107:in `authenticate'","actionpack (7.0.8.7) lib/action_controller/metal/http_authentication.rb:97:in `authenticate_with_http_basic'","app/controllers/jwt_controller.rb:44:in `authenticate_project_or_user'","activesupport (7.0.8.7) lib/active_support/callbacks.rb:400:in `block in make_lambda'","activesupport (7.0.8.7) lib/active_support/callbacks.rb:199:in `block (2 levels) in halting'","actionpack (7.0.8.7) lib/abstract_controller/callbacks.rb:34:in `block (2 levels) in <module:Callbacks>'","activesupport (7.0.8.7) lib/active_support/callbacks.rb:200:in `block in halting'","activesupport (7.0.8.7) lib/active_support/callbacks.rb:595:in `block in invoke_before'","activesupport (7.0.8.7) lib/active_support/callbacks.rb:595:in `each'","activesupport (7.0.8.7) lib/active_support/callbacks.rb:595:in `invoke_before'","activesupport (7.0.8.7) lib/active_support/callbacks.rb:116:in `block in run_callbacks'","sentry-rails (5.22.1) lib/sentry/rails/controller_transaction.rb:34:in `block in sentry_around_action'","sentry-ruby (5.22.1) lib/sentry/hub.rb:108:in `with_child_span'","sentry-ruby (5.22.1) lib/sentry-ruby.rb:503:in `with_child_span'","sentry-rails (5.22.1) lib/sentry/rails/controller_transaction.rb:18:in `sentry_around_action'","activesupport (7.0.8.7) lib/active_support/callbacks.rb:127:in `block in run_callbacks'","activesupport (7.0.8.7) lib/active_support/callbacks.rb:138:in `run_callbacks'","actionpack (7.0.8.7) lib/abstract_controller/callbacks.rb:233:in `process_action'","actionpack (7.0.8.7) lib/action_controller/metal/rescue.rb:23:in `process_action'","actionpack (7.0.8.7) lib/action_controller/metal/instrumentation.rb:67:in `block in process_action'","activesupport (7.0.8.7) lib/active_support/notifications.rb:206:in `block in instrument'","activesupport (7.0.8.7) lib/active_support/notifications/instrumenter.rb:24:in `instrument'","activesupport (7.0.8.7) lib/active_support/notifications.rb:206:in `instrument'","actionpack (7.0.8.7) lib/action_controller/metal/instrumentation.rb:66:in `process_action'","actionpack (7.0.8.7) lib/action_controller/metal/params_wrapper.rb:259:in `process_action'","activerecord (7.0.8.7) lib/active_record/railties/controller_runtime.rb:27:in `process_action'","actionpack (7.0.8.7) lib/abstract_controller/base.rb:151:in `process'","actionview (7.0.8.7) lib/action_view/rendering.rb:39:in `process'","actionpack (7.0.8.7) lib/action_controller/metal.rb:188:in `dispatch'","actionpack (7.0.8.7) lib/action_controller/metal.rb:249:in `block in dispatch'","lib/gitlab/middleware/action_controller_static_context.rb:23:in `call'","actionpack (7.0.8.7) lib/action_controller/metal.rb:249:in `dispatch'","actionpack (7.0.8.7) lib/action_dispatch/routing/route_set.rb:49:in `dispatch'","actionpack (7.0.8.7) lib/action_dispatch/routing/route_set.rb:32:in `serve'","actionpack (7.0.8.7) lib/action_dispatch/journey/router.rb:50:in `block in serve'","actionpack (7.0.8.7) lib/action_dispatch/journey/router.rb:32:in `each'","actionpack (7.0.8.7) lib/action_dispatch/journey/router.rb:32:in `serve'","actionpack (7.0.8.7) lib/action_dispatch/routing/route_set.rb:852:in `call'","gitlab-experiment (0.9.1) lib/gitlab/experiment/middleware.rb:19:in `call'","omniauth (2.1.2) lib/omniauth/strategy.rb:202:in `call!'","omniauth (2.1.2) lib/omniauth/strategy.rb:169:in `call'","flipper (0.28.3) lib/flipper/middleware/memoizer.rb:72:in `memoized_call'","flipper (0.28.3) lib/flipper/middleware/memoizer.rb:37:in `call'","lib/gitlab/middleware/sidekiq_shard_awareness_validation.rb:20:in `block in call'","lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'","lib/gitlab/middleware/sidekiq_shard_awareness_validation.rb:20:in `call'","lib/gitlab/middleware/memory_report.rb:13:in `call'","lib/gitlab/middleware/speedscope.rb:13:in `call'","lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call'","lib/gitlab/middleware/go.rb:21:in `call'","lib/gitlab/etag_caching/middleware.rb:21:in `call'","lib/gitlab/middleware/query_analyzer.rb:11:in `block in call'","lib/gitlab/database/query_analyzer.rb:83:in `within'","lib/gitlab/middleware/query_analyzer.rb:11:in `call'","lib/ci/job_token/middleware.rb:11:in `call'","batch-loader (2.0.5) lib/batch_loader/middleware.rb:11:in `call'","rack-attack (6.7.0) lib/rack/attack.rb:103:in `call'","apollo_upload_server (2.1.6) lib/apollo_upload_server/middleware.rb:19:in `call'","lib/gitlab/middleware/multipart.rb:173:in `call'","rack-attack (6.7.0) lib/rack/attack.rb:127:in `call'","warden (1.2.9) lib/warden/manager.rb:36:in `block in call'","warden (1.2.9) lib/warden/manager.rb:34:in `catch'","warden (1.2.9) lib/warden/manager.rb:34:in `call'","rack-cors (2.0.2) lib/rack/cors.rb:102:in `call'","rack (2.2.13) lib/rack/tempfile_reaper.rb:15:in `call'","rack (2.2.13) lib/rack/etag.rb:27:in `call'","rack (2.2.13) lib/rack/conditional_get.rb:27:in `call'","rack (2.2.13) lib/rack/head.rb:12:in `call'","actionpack (7.0.8.7) lib/action_dispatch/http/permissions_policy.rb:38:in `call'","actionpack (7.0.8.7) lib/action_dispatch/http/content_security_policy.rb:39:in `call'","lib/gitlab/middleware/read_only/controller.rb:50:in `call'","lib/gitlab/middleware/read_only.rb:18:in `call'","lib/gitlab/middleware/unauthenticated_session_expiry.rb:18:in `call'","rack (2.2.13) lib/rack/session/abstract/id.rb:266:in `context'","rack (2.2.13) lib/rack/session/abstract/id.rb:260:in `call'","actionpack (7.0.8.7) lib/action_dispatch/middleware/cookies.rb:704:in `call'","lib/gitlab/middleware/same_site_cookies.rb:27:in `call'","actionpack (7.0.8.7) lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'","activesupport (7.0.8.7) lib/active_support/callbacks.rb:99:in `run_callbacks'","actionpack (7.0.8.7) lib/action_dispatch/middleware/callbacks.rb:26:in `call'","sentry-rails (5.22.1) lib/sentry/rails/rescued_exception_interceptor.rb:14:in `call'","actionpack (7.0.8.7) lib/action_dispatch/middleware/debug_exceptions.rb:28:in `call'","lib/gitlab/middleware/path_traversal_check.rb:35:in `call'","lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'","sentry-ruby (5.22.1) lib/sentry/rack/capture_exceptions.rb:30:in `block (2 levels) in call'","sentry-ruby (5.22.1) lib/sentry/hub.rb:269:in `with_session_tracking'","sentry-ruby (5.22.1) lib/sentry-ruby.rb:416:in `with_session_tracking'","sentry-ruby (5.22.1) lib/sentry/rack/capture_exceptions.rb:21:in `block in call'","sentry-ruby (5.22.1) lib/sentry/hub.rb:59:in `with_scope'","sentry-ruby (5.22.1) lib/sentry-ruby.rb:396:in `with_scope'","sentry-ruby (5.22.1) lib/sentry/rack/capture_exceptions.rb:20:in `call'","actionpack (7.0.8.7) lib/action_dispatch/middleware/show_exceptions.rb:29:in `call'","lib/gitlab/middleware/basic_health_check.rb:25:in `call'","lograge (0.11.2) lib/lograge/rails_ext/rack/logger.rb:15:in `call_app'","railties (7.0.8.7) lib/rails/rack/logger.rb:25:in `block in call'","activesupport (7.0.8.7) lib/active_support/tagged_logging.rb:99:in `block in tagged'","activesupport (7.0.8.7) lib/active_support/tagged_logging.rb:37:in `tagged'","activesupport (7.0.8.7) lib/active_support/tagged_logging.rb:99:in `tagged'","railties (7.0.8.7) lib/rails/rack/logger.rb:25:in `call'","actionpack (7.0.8.7) lib/action_dispatch/middleware/remote_ip.rb:93:in `call'","lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'","lib/gitlab/middleware/request_context.rb:15:in `call'","lib/gitlab/middleware/webhook_recursion_detection.rb:15:in `call'","request_store (1.7.0) lib/request_store/middleware.rb:19:in `call'","rack (2.2.13) lib/rack/method_override.rb:24:in `call'","rack (2.2.13) lib/rack/runtime.rb:22:in `call'","rack-timeout (0.7.0) lib/rack/timeout/core.rb:154:in `block in call'","rack-timeout (0.7.0) lib/rack/timeout/support/timeout.rb:19:in `timeout'","rack-timeout (0.7.0) lib/rack/timeout/core.rb:153:in `call'","config/initializers/fix_local_cache_middleware.rb:11:in `call'","lib/gitlab/middleware/compressed_json.rb:44:in `call'","actionpack (7.0.8.7) lib/action_dispatch/middleware/executor.rb:14:in `call'","lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'","rack (2.2.13) lib/rack/sendfile.rb:110:in `call'","lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'","lib/gitlab/metrics/requests_rack_middleware.rb:83:in `call'","gitlab-labkit (0.37.0) lib/labkit/middleware/rack.rb:22:in `block in call'","gitlab-labkit (0.37.0) lib/labkit/context.rb:35:in `with_context'","gitlab-labkit (0.37.0) lib/labkit/middleware/rack.rb:21:in `call'","actionpack (7.0.8.7) lib/action_dispatch/middleware/request_id.rb:26:in `call'","actionpack (7.0.8.7) lib/action_dispatch/middleware/host_authorization.rb:131:in `call'","railties (7.0.8.7) lib/rails/engine.rb:530:in `call'","railties (7.0.8.7) lib/rails/railtie.rb:226:in `public_send'","railties (7.0.8.7) lib/rails/railtie.rb:226:in `method_missing'","lib/gitlab/middleware/release_env.rb:12:in `call'","rack (2.2.13) lib/rack/urlmap.rb:74:in `block in call'","rack (2.2.13) lib/rack/urlmap.rb:58:in `each'","rack (2.2.13) lib/rack/urlmap.rb:58:in `call'","puma (6.5.0) lib/puma/configuration.rb:279:in `call'","puma (6.5.0) lib/puma/request.rb:99:in `block in handle_request'","puma (6.5.0) lib/puma/thread_pool.rb:389:in `with_force_shutdown'","puma (6.5.0) lib/puma/request.rb:98:in `handle_request'","puma (6.5.0) lib/puma/server.rb:468:in `process_client'","puma (6.5.0) lib/puma/server.rb:249:in `block in run'","puma (6.5.0) lib/puma/thread_pool.rb:166:in `block in spawn_thread'"],"db_duration_s":0.00115,"view_duration_s":0.0,"duration_s":0.01674}
gitlab  |
gitlab  | ==> /var/log/gitlab/gitlab-rails/production.log <==
gitlab  |
gitlab  | TypeError (no implicit conversion of nil into String):
gitlab  |
gitlab  | lib/authn/tokens/personal_access_token.rb:7:in `start_with?'
gitlab  | lib/authn/tokens/personal_access_token.rb:7:in `prefix?'
gitlab  | lib/authn/agnostic_token_identifier.rb:30:in `block in token_type'
gitlab  | lib/authn/agnostic_token_identifier.rb:30:in `each'
gitlab  | lib/authn/agnostic_token_identifier.rb:30:in `find'
gitlab  | lib/authn/agnostic_token_identifier.rb:30:in `token_type'
gitlab  | lib/authn/agnostic_token_identifier.rb:26:in `token?'
gitlab  | lib/gitlab/auth.rb:232:in `user_with_password_for_git'
gitlab  | lib/gitlab/auth.rb:105:in `find_for_git_client'
gitlab  | app/controllers/jwt_controller.rb:45:in `block in authenticate_project_or_user'
gitlab  | actionpack (7.0.8.7) lib/action_controller/metal/http_authentication.rb:107:in `authenticate'
gitlab  | actionpack (7.0.8.7) lib/action_controller/metal/http_authentication.rb:97:in `authenticate_with_http_basic'
gitlab  | app/controllers/jwt_controller.rb:44:in `authenticate_project_or_user'
gitlab  | activesupport (7.0.8.7) lib/active_support/callbacks.rb:400:in `block in make_lambda'
gitlab  | activesupport (7.0.8.7) lib/active_support/callbacks.rb:199:in `block (2 levels) in halting'
gitlab  | actionpack (7.0.8.7) lib/abstract_controller/callbacks.rb:34:in `block (2 levels) in <module:Callbacks>'
gitlab  | activesupport (7.0.8.7) lib/active_support/callbacks.rb:200:in `block in halting'
gitlab  | activesupport (7.0.8.7) lib/active_support/callbacks.rb:595:in `block in invoke_before'
gitlab  | activesupport (7.0.8.7) lib/active_support/callbacks.rb:595:in `each'
gitlab  | activesupport (7.0.8.7) lib/active_support/callbacks.rb:595:in `invoke_before'
gitlab  | activesupport (7.0.8.7) lib/active_support/callbacks.rb:116:in `block in run_callbacks'
gitlab  | sentry-rails (5.22.1) lib/sentry/rails/controller_transaction.rb:34:in `block in sentry_around_action'
gitlab  | sentry-ruby (5.22.1) lib/sentry/hub.rb:108:in `with_child_span'
gitlab  | sentry-ruby (5.22.1) lib/sentry-ruby.rb:503:in `with_child_span'
gitlab  | sentry-rails (5.22.1) lib/sentry/rails/controller_transaction.rb:18:in `sentry_around_action'
gitlab  | activesupport (7.0.8.7) lib/active_support/callbacks.rb:127:in `block in run_callbacks'
gitlab  | activesupport (7.0.8.7) lib/active_support/callbacks.rb:138:in `run_callbacks'
gitlab  | actionpack (7.0.8.7) lib/abstract_controller/callbacks.rb:233:in `process_action'
gitlab  | actionpack (7.0.8.7) lib/action_controller/metal/rescue.rb:23:in `process_action'
gitlab  | actionpack (7.0.8.7) lib/action_controller/metal/instrumentation.rb:67:in `block in process_action'
gitlab  | activesupport (7.0.8.7) lib/active_support/notifications.rb:206:in `block in instrument'
gitlab  | activesupport (7.0.8.7) lib/active_support/notifications/instrumenter.rb:24:in `instrument'
gitlab  | activesupport (7.0.8.7) lib/active_support/notifications.rb:206:in `instrument'
gitlab  | actionpack (7.0.8.7) lib/action_controller/metal/instrumentation.rb:66:in `process_action'
gitlab  | actionpack (7.0.8.7) lib/action_controller/metal/params_wrapper.rb:259:in `process_action'
gitlab  | activerecord (7.0.8.7) lib/active_record/railties/controller_runtime.rb:27:in `process_action'
gitlab  | actionpack (7.0.8.7) lib/abstract_controller/base.rb:151:in `process'
gitlab  | actionview (7.0.8.7) lib/action_view/rendering.rb:39:in `process'
gitlab  | actionpack (7.0.8.7) lib/action_controller/metal.rb:188:in `dispatch'
gitlab  | actionpack (7.0.8.7) lib/action_controller/metal.rb:249:in `block in dispatch'
gitlab  | lib/gitlab/middleware/action_controller_static_context.rb:23:in `call'
gitlab  | actionpack (7.0.8.7) lib/action_controller/metal.rb:249:in `dispatch'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/routing/route_set.rb:49:in `dispatch'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/routing/route_set.rb:32:in `serve'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/journey/router.rb:50:in `block in serve'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/journey/router.rb:32:in `each'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/journey/router.rb:32:in `serve'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/routing/route_set.rb:852:in `call'
gitlab  | gitlab-experiment (0.9.1) lib/gitlab/experiment/middleware.rb:19:in `call'
gitlab  | omniauth (2.1.2) lib/omniauth/strategy.rb:202:in `call!'
gitlab  | omniauth (2.1.2) lib/omniauth/strategy.rb:169:in `call'
gitlab  | flipper (0.28.3) lib/flipper/middleware/memoizer.rb:72:in `memoized_call'
gitlab  | flipper (0.28.3) lib/flipper/middleware/memoizer.rb:37:in `call'
gitlab  | lib/gitlab/middleware/sidekiq_shard_awareness_validation.rb:20:in `block in call'
gitlab  | lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'
gitlab  | lib/gitlab/middleware/sidekiq_shard_awareness_validation.rb:20:in `call'
gitlab  | lib/gitlab/middleware/memory_report.rb:13:in `call'
gitlab  | lib/gitlab/middleware/speedscope.rb:13:in `call'
gitlab  | lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call'
gitlab  | lib/gitlab/middleware/go.rb:21:in `call'
gitlab  | lib/gitlab/etag_caching/middleware.rb:21:in `call'
gitlab  | lib/gitlab/middleware/query_analyzer.rb:11:in `block in call'
gitlab  | lib/gitlab/database/query_analyzer.rb:83:in `within'
gitlab  | lib/gitlab/middleware/query_analyzer.rb:11:in `call'
gitlab  | lib/ci/job_token/middleware.rb:11:in `call'
gitlab  | batch-loader (2.0.5) lib/batch_loader/middleware.rb:11:in `call'
gitlab  | rack-attack (6.7.0) lib/rack/attack.rb:103:in `call'
gitlab  | apollo_upload_server (2.1.6) lib/apollo_upload_server/middleware.rb:19:in `call'
gitlab  | lib/gitlab/middleware/multipart.rb:173:in `call'
gitlab  | rack-attack (6.7.0) lib/rack/attack.rb:127:in `call'
gitlab  | warden (1.2.9) lib/warden/manager.rb:36:in `block in call'
gitlab  | warden (1.2.9) lib/warden/manager.rb:34:in `catch'
gitlab  | warden (1.2.9) lib/warden/manager.rb:34:in `call'
gitlab  | rack-cors (2.0.2) lib/rack/cors.rb:102:in `call'
gitlab  | rack (2.2.13) lib/rack/tempfile_reaper.rb:15:in `call'
gitlab  | rack (2.2.13) lib/rack/etag.rb:27:in `call'
gitlab  | rack (2.2.13) lib/rack/conditional_get.rb:27:in `call'
gitlab  | rack (2.2.13) lib/rack/head.rb:12:in `call'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/http/permissions_policy.rb:38:in `call'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/http/content_security_policy.rb:39:in `call'
gitlab  | lib/gitlab/middleware/read_only/controller.rb:50:in `call'
gitlab  | lib/gitlab/middleware/read_only.rb:18:in `call'
gitlab  | lib/gitlab/middleware/unauthenticated_session_expiry.rb:18:in `call'
gitlab  | rack (2.2.13) lib/rack/session/abstract/id.rb:266:in `context'
gitlab  | rack (2.2.13) lib/rack/session/abstract/id.rb:260:in `call'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/middleware/cookies.rb:704:in `call'
gitlab  | lib/gitlab/middleware/same_site_cookies.rb:27:in `call'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
gitlab  | activesupport (7.0.8.7) lib/active_support/callbacks.rb:99:in `run_callbacks'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/middleware/callbacks.rb:26:in `call'
gitlab  | sentry-rails (5.22.1) lib/sentry/rails/rescued_exception_interceptor.rb:14:in `call'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/middleware/debug_exceptions.rb:28:in `call'
gitlab  | lib/gitlab/middleware/path_traversal_check.rb:35:in `call'
gitlab  | lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'
gitlab  | sentry-ruby (5.22.1) lib/sentry/rack/capture_exceptions.rb:30:in `block (2 levels) in call'
gitlab  | sentry-ruby (5.22.1) lib/sentry/hub.rb:269:in `with_session_tracking'
gitlab  | sentry-ruby (5.22.1) lib/sentry-ruby.rb:416:in `with_session_tracking'
gitlab  | sentry-ruby (5.22.1) lib/sentry/rack/capture_exceptions.rb:21:in `block in call'
gitlab  | sentry-ruby (5.22.1) lib/sentry/hub.rb:59:in `with_scope'
gitlab  | sentry-ruby (5.22.1) lib/sentry-ruby.rb:396:in `with_scope'
gitlab  | sentry-ruby (5.22.1) lib/sentry/rack/capture_exceptions.rb:20:in `call'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/middleware/show_exceptions.rb:29:in `call'
gitlab  | lib/gitlab/middleware/basic_health_check.rb:25:in `call'
gitlab  | lograge (0.11.2) lib/lograge/rails_ext/rack/logger.rb:15:in `call_app'
gitlab  | railties (7.0.8.7) lib/rails/rack/logger.rb:25:in `block in call'
gitlab  | activesupport (7.0.8.7) lib/active_support/tagged_logging.rb:99:in `block in tagged'
gitlab  | activesupport (7.0.8.7) lib/active_support/tagged_logging.rb:37:in `tagged'
gitlab  | activesupport (7.0.8.7) lib/active_support/tagged_logging.rb:99:in `tagged'
gitlab  | railties (7.0.8.7) lib/rails/rack/logger.rb:25:in `call'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/middleware/remote_ip.rb:93:in `call'
gitlab  | lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'
gitlab  | lib/gitlab/middleware/request_context.rb:15:in `call'
gitlab  | lib/gitlab/middleware/webhook_recursion_detection.rb:15:in `call'
gitlab  | request_store (1.7.0) lib/request_store/middleware.rb:19:in `call'
gitlab  | rack (2.2.13) lib/rack/method_override.rb:24:in `call'
gitlab  | rack (2.2.13) lib/rack/runtime.rb:22:in `call'
gitlab  | rack-timeout (0.7.0) lib/rack/timeout/core.rb:154:in `block in call'
gitlab  | rack-timeout (0.7.0) lib/rack/timeout/support/timeout.rb:19:in `timeout'
gitlab  | rack-timeout (0.7.0) lib/rack/timeout/core.rb:153:in `call'
gitlab  | config/initializers/fix_local_cache_middleware.rb:11:in `call'
gitlab  | lib/gitlab/middleware/compressed_json.rb:44:in `call'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/middleware/executor.rb:14:in `call'
gitlab  | lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'
gitlab  | rack (2.2.13) lib/rack/sendfile.rb:110:in `call'
gitlab  | lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'
gitlab  | lib/gitlab/metrics/requests_rack_middleware.rb:83:in `call'
gitlab  | gitlab-labkit (0.37.0) lib/labkit/middleware/rack.rb:22:in `block in call'
gitlab  | gitlab-labkit (0.37.0) lib/labkit/context.rb:35:in `with_context'
gitlab  | gitlab-labkit (0.37.0) lib/labkit/middleware/rack.rb:21:in `call'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/middleware/request_id.rb:26:in `call'
gitlab  | actionpack (7.0.8.7) lib/action_dispatch/middleware/host_authorization.rb:131:in `call'
gitlab  | railties (7.0.8.7) lib/rails/engine.rb:530:in `call'
gitlab  | railties (7.0.8.7) lib/rails/railtie.rb:226:in `public_send'
gitlab  | railties (7.0.8.7) lib/rails/railtie.rb:226:in `method_missing'
gitlab  | lib/gitlab/middleware/release_env.rb:12:in `call'
gitlab  | rack (2.2.13) lib/rack/urlmap.rb:74:in `block in call'
gitlab  | rack (2.2.13) lib/rack/urlmap.rb:58:in `each'
gitlab  | rack (2.2.13) lib/rack/urlmap.rb:58:in `call'
gitlab  | puma (6.5.0) lib/puma/configuration.rb:279:in `call'
gitlab  | puma (6.5.0) lib/puma/request.rb:99:in `block in handle_request'
gitlab  | puma (6.5.0) lib/puma/thread_pool.rb:389:in `with_force_shutdown'
gitlab  | puma (6.5.0) lib/puma/request.rb:98:in `handle_request'
gitlab  | puma (6.5.0) lib/puma/server.rb:468:in `process_client'
gitlab  | puma (6.5.0) lib/puma/server.rb:249:in `block in run'
gitlab  | puma (6.5.0) lib/puma/thread_pool.rb:166:in `block in spawn_thread'
gitlab  |
gitlab  | ==> /var/log/gitlab/gitlab-workhorse/current <==
gitlab  | {"backend_id":"rails","content_type":"text/html; charset=utf-8","correlation_id":"01JS80KXMNBAF457A1M1A736W5","duration_ms":49,"host":"dcc-gitlab.addiko.com","level":"info","method":"GET","msg":"access","proto":"HTTP/1.1","read_bytes":519,"referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","route":"","route_id":"default","status":500,"system":"http","time":"2025-04-19T21:52:32Z","ttfb_ms":49,"uri":"/jwt/auth?account=dcc\u0026scope=repository%3Amtodoric%2Fdocker-images%2Fansible%3Apull\u0026service=dependency_proxy","user_agent":"docker/27.3.1 go/go1.22.7 git-commit/41ca978 kernel/6.8.0-1015-aws os/linux arch/amd64 UpstreamClient(Docker-Client/27.3.1 \\(linux\\))","written_bytes":1624}
gitlab  |
gitlab  | ==> /var/log/gitlab/nginx/gitlab_access.log <==
gitlab  | 172.31.18.120 - dcc [19/Apr/2025:21:52:32 +0000] "GET /jwt/auth?account=dcc&scope=repository%3Amtodoric%2Fdocker-images%2Fansible%3Apull&service=dependency_proxy HTTP/1.1" 500 1624 "" "docker/27.3.1 go/go1.22.7 git-commit/41ca978 kernel/6.8.0-1015-aws os/linux arch/amd64 UpstreamClient(Docker-Client/27.3.1 \x5C(linux\x5C))" -

What is the expected correct behavior?

I should be able to login with same credentials I've used before. Credentials have not changed, 2FA is not enabled.

Relevant logs and/or screenshots

{
  "exception.class": "TypeError",
  "exception.message": "no implicit conversion of nil into String",
  "controller": "JwtController",
  "action": "auth",
  "status": 500,
  "params": [
    {"key": "account", "value": "<username>"},
    {"key": "scope", "value": "repository:<namespace>/<docker-images>/ansible:pull"},
    {"key": "service", "value": "dependency_proxy"}
  ]
}

Output of checks

Expand for output related to GitLab environment info

System information
System:
Proxy:          no
Current User:   git
Using RVM:      no
Ruby Version:   3.2.5
Gem Version:    3.6.6
Bundler Version:2.6.5
Rake Version:   13.0.6
Redis Version:  7.2.7
Sidekiq Version:7.3.9
Go Version:     unknown

GitLab information
Version:        17.11.0-ee
Revision:       b7c061ce59a
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     PostgreSQL
DB Version:     16.8
URL:            https://<self-hosted>
HTTP Clone URL: https://<self-hosted>some-group/some-project.git
SSH Clone URL:  ssh://git@<self-hosted>/some-group/some-project.git
Elasticsearch:  no
Geo:            no
Using LDAP:     no
Using Omniauth: yes
Omniauth Providers: bitbucket

GitLab Shell
Version:        14.41.0
Repository storages:
- default:      unix:/var/opt/gitlab/gitaly/gitaly.socket
GitLab Shell path:              /opt/gitlab/embedded/service/gitlab-shell

Gitaly
- default Address:      unix:/var/opt/gitlab/gitaly/gitaly.socket
- default Version:      17.11.0
- default Git Version:  2.49.0.gl1

Expand for output related to the GitLab application check

Checking GitLab subtasks ...

Checking GitLab Shell ...

GitLab Shell: ... GitLab Shell version >= 14.41.0 ? ... OK (14.41.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-shell-check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Gitaly ...

Gitaly: ... default ... OK

Checking Gitaly ... Finished

Checking Sidekiq ...

Sidekiq: ... Running? ... yes
Number of Sidekiq processes (cluster/worker) ... 1/1

Checking Sidekiq ... Finished

Checking Incoming Email ...

Incoming Email: ... Reply by email is disabled in config/gitlab.yml

Checking Incoming Email ... Finished

Checking LDAP ...

LDAP: ... LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab App ...

Database config exists? ... yes
Tables are truncated? ... skipped
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Cable config exists? ... yes
Resque config exists? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Systemd unit files or init script exist? ... skipped (omnibus-gitlab has neither init script nor systemd units)
Systemd unit files or init script up-to-date? ... skipped (omnibus-gitlab has neither init script nor systemd units)
Projects have namespace: ...
4/8 ... yes
22/10 ... yes
4/11 ... yes
12/12 ... yes
13/13 ... yes
36/14 ... yes
36/15 ... yes
22/16 ... yes
22/17 ... yes
38/18 ... yes
38/19 ... yes
38/20 ... yes
38/21 ... yes
38/22 ... yes
38/23 ... yes
38/24 ... yes
38/25 ... yes
38/26 ... yes
38/27 ... yes
38/28 ... yes
4/29 ... yes
53/30 ... yes
53/31 ... yes
53/32 ... yes
53/33 ... yes
53/34 ... yes
53/35 ... yes
53/36 ... yes
53/37 ... yes
53/38 ... yes
53/40 ... yes
53/41 ... yes
38/42 ... yes
4/43 ... yes
4/45 ... yes
53/47 ... yes
4/48 ... yes
4/50 ... yes
38/53 ... yes
53/54 ... yes
53/55 ... yes
53/56 ... yes
53/57 ... yes
4/58 ... yes
4/60 ... yes
4/61 ... yes
4/62 ... yes
53/63 ... yes
4/64 ... yes
4/65 ... yes
4/67 ... yes
53/68 ... yes
53/69 ... yes
38/70 ... yes
4/71 ... yes
36/72 ... yes
112/73 ... yes
53/74 ... yes
22/75 ... yes
22/76 ... yes
22/77 ... yes
24/78 ... yes
4/79 ... yes
4/80 ... yes
4/81 ... yes
4/82 ... yes
129/83 ... yes
4/84 ... yes
4/86 ... yes
4/88 ... yes
4/89 ... yes
149/90 ... yes
149/93 ... yes
4/104 ... yes
149/105 ... yes
4/106 ... yes
149/107 ... yes
149/108 ... yes
Redis version >= 6.2.14? ... yes
Ruby version >= 3.0.6 ? ... yes (3.2.5)
Git user has default SSH configuration? ... yes
Active users: ... 29
Is authorized keys file accessible? ... yes
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes
Elasticsearch version 7.x-8.x or OpenSearch version 1.x ... skipped (Advanced Search is disabled)
All migrations must be finished before doing a major upgrade ... skipped (Advanced Search is disabled)

Checking GitLab App ... Finished


Checking GitLab subtasks ... Finished

Possible fixes

Authenticate via Private Access Token with read_registry role. Use token value instead of password in docker login command.

Edited Apr 22, 2025 by Marko Todoric