Parallel matrix job name lengths are not properly handled when they exceed 255 characters

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Close this issue

Summary

Job names have a limit of 255 characters. If they exceed this character limit, they fail with:

Unable to run pipeline
  jobs:<job_name> name is too long (maximum is 255 characters)

However, job names generated from parallel:matrix configurations do not produce this same error. If a parallel:matrix generated job name were to exceed the 255 character limit, the pipeline silently fails to be created in the background. This is due to a PG error like so:

{
  exception.cause_class: "PG::StringDataRightTruncation",
  exception.class: "ActiveRecord::ValueTooLong",
  exception.message: "PG::StringDataRightTruncation: ERROR:  value too long for type character varying(255)"
}

We need to implement a check for parallel:matrix generated job names and surface errors to the end user when they exceed the 255 character limit. Currently all pipelines that would result in this scenario will fail to be created with no reporting to the end user.

Steps to reproduce

Attempt to kick off a pipeline via push or manually with a .gitlab-ci.yml like so:

test_job:
  script:
    - echo "Running test"
  parallel:
    matrix:
      - VARIANT: "alpha"
        OPTIONS: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"

What is the current bug behavior?

Pipelines with parallel:matrix generated job names that exceed 255 characters will silently fail to be created in the background due to a PG error.

What is the expected correct behavior?

Pipelines with parallel:matrix generated job names that exceed 255 characters are generated with an error presented in the UI, as they do for non parallel:matrix generated jobs.

Relevant logs and/or screenshots

Exception SQL

/*application:sidekiq,correlation_id:01KCSKK9J7QG7EADNXWZA65GD7,jid:1e44974278fb1a145242700a,endpoint_id:CreatePipelineWorker,db_config_database:gitlabhq_production_sidekiq_urgent,db_config_name:ci*/ INSERT INTO "p_ci_builds" ("status", "created_at", "updated_at", "name", "stage_idx", "tag", "ref", "type", "when", "retried", "protected", "processed", "scheduling_type", "stage_id", "partition_id", "commit_id", "project_id", "user_id", "lock_version") VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19) RETURNING "id"

Exception backtrace

[activerecord (7.2.3) lib/active_record/connection_adapters/postgresql_adapter.rb:899:in `exec_params', activerecord (7.2.3) lib/active_record/connection_adapters/postgresql_adapter.rb:899:in `block (2 levels) in exec_no_cache', activerecord (7.2.3) lib/active_record/connection_adapters/abstract_adapter.rb:1024:in `block in with_raw_connection', activesupport (7.2.3) lib/active_support/concurrency/null_lock.rb:9:in `synchronize', activerecord (7.2.3) lib/active_record/connection_adapters/abstract_adapter.rb:993:in `with_raw_connection', activerecord (7.2.3) lib/active_record/connection_adapters/postgresql_adapter.rb:898:in `block in exec_no_cache', activesupport (7.2.3) lib/active_support/notifications/instrumenter.rb:58:in `instrument', activerecord (7.2.3) lib/active_record/connection_adapters/abstract_adapter.rb:1141:in `log', activerecord (7.2.3) lib/active_record/connection_adapters/postgresql_adapter.rb:897:in `exec_no_cache', activerecord (7.2.3) lib/active_record/connection_adapters/postgresql_adapter.rb:877:in `execute_and_clear', marginalia (1.11.1) lib/marginalia.rb:91:in `execute_and_clear_with_marginalia', activerecord (7.2.3) lib/active_record/connection_adapters/postgresql/database_statements.rb:66:in `internal_exec_query', activerecord (7.2.3) lib/active_record/connection_adapters/abstract/database_statements.rb:159:in `exec_insert', activerecord (7.2.3) lib/active_record/connection_adapters/postgresql/database_statements.rb:85:in `exec_insert', activerecord (7.2.3) lib/active_record/connection_adapters/abstract/database_statements.rb:197:in `insert', activerecord (7.2.3) lib/active_record/connection_adapters/abstract/query_cache.rb:27:in `insert', lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `public_send', lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `block in write_using_load_balancer', lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write', lib/gitlab/database/load_balancing/load_balancer.rb:235:in `retry_with_backoff', lib/gitlab/database/load_balancing/load_balancer.rb:131:in `read_write', lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer', lib/gitlab/database/load_balancing/connection_proxy.rb:61:in `block (2 levels) in <class:ConnectionProxy>', activerecord (7.2.3) lib/active_record/persistence.rb:258:in `block in _insert_record', lib/gitlab/database/load_balancing/setup.rb:57:in `block in setup_connection_proxy', activerecord (7.2.3) lib/active_record/persistence.rb:251:in `_insert_record', activerecord (7.2.3) lib/active_record/persistence.rb:928:in `block in _create_record', lib/gitlab/database/load_balancing/setup.rb:57:in `block in setup_connection_proxy', activerecord (7.2.3) lib/active_record/persistence.rb:925:in `_create_record', activerecord (7.2.3) lib/active_record/counter_cache.rb:201:in `_create_record', activerecord (7.2.3) lib/active_record/locking/optimistic.rb:84:in `_create_record', activerecord (7.2.3) lib/active_record/encryption/encryptable_record.rb:184:in `_create_record', activerecord (7.2.3) lib/active_record/attribute_methods/dirty.rb:240:in `_create_record', activerecord (7.2.3) lib/active_record/callbacks.rb:445:in `block in _create_record', activesupport (7.2.3) lib/active_support/callbacks.rb:110:in `run_callbacks', activesupport (7.2.3) lib/active_support/callbacks.rb:914:in `_run_create_callbacks', activerecord (7.2.3) lib/active_record/callbacks.rb:445:in `_create_record', activerecord (7.2.3) lib/active_record/timestamp.rb:116:in `_create_record', activerecord (7.2.3) lib/active_record/persistence.rb:896:in `create_or_update', activerecord (7.2.3) lib/active_record/callbacks.rb:441:in `block in create_or_update', activesupport (7.2.3) lib/active_support/callbacks.rb:121:in `block in run_callbacks', activerecord (7.2.3) lib/active_record/autosave_association.rb:372:in `around_save_collection_association', activesupport (7.2.3) lib/active_support/callbacks.rb:130:in `block in run_callbacks', activesupport (7.2.3) lib/active_support/callbacks.rb:141:in `run_callbacks', activesupport (7.2.3) lib/active_support/callbacks.rb:914:in `_run_save_callbacks', activerecord (7.2.3) lib/active_record/callbacks.rb:441:in `create_or_update', activerecord (7.2.3) lib/active_record/timestamp.rb:127:in `create_or_update', activerecord (7.2.3) lib/active_record/persistence.rb:393:in `save', activerecord (7.2.3) lib/active_record/validations.rb:48:in `save', activerecord (7.2.3) lib/active_record/transactions.rb:362:in `block in save', activerecord (7.2.3) lib/active_record/transactions.rb:418:in `block (2 levels) in with_transaction_returning_status', activerecord (7.2.3) lib/active_record/connection_adapters/abstract/database_statements.rb:359:in `transaction', lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `public_send', lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `block in write_using_load_balancer', lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write', lib/gitlab/database/load_balancing/load_balancer.rb:235:in `retry_with_backoff', lib/gitlab/database/load_balancing/load_balancer.rb:131:in `read_write', lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer', lib/gitlab/database/load_balancing/connection_proxy.rb:78:in `transaction', activerecord (7.2.3) lib/active_record/transactions.rb:414:in `block in with_transaction_returning_status', lib/gitlab/database/load_balancing/setup.rb:57:in `block in setup_connection_proxy', activerecord (7.2.3) lib/active_record/transactions.rb:410:in `with_transaction_returning_status', activerecord (7.2.3) lib/active_record/transactions.rb:362:in `save', activerecord (7.2.3) lib/active_record/suppressor.rb:52:in `save', state_machines (0.100.4) lib/state_machines/machine.rb:569:in `block in save', state_machines (0.100.4) lib/state_machines/transition_collection.rb:174:in `block in run_actions', state_machines (0.100.4) lib/state_machines/transition_collection.rb:193:in `catch_exceptions', state_machines (0.100.4) lib/state_machines/transition_collection.rb:172:in `run_actions', state_machines (0.100.4) lib/state_machines/transition_collection.rb:157:in `run_callbacks', state_machines (0.100.4) lib/state_machines/transition_collection.rb:240:in `run_callbacks', state_machines (0.100.4) lib/state_machines/transition_collection.rb:75:in `block (2 levels) in perform', state_machines (0.100.4) lib/state_machines/transition_collection.rb:75:in `catch', state_machines (0.100.4) lib/state_machines/transition_collection.rb:75:in `block in perform', state_machines (0.100.4) lib/state_machines/transition_collection.rb:208:in `within_transaction', state_machines (0.100.4) lib/state_machines/transition_collection.rb:74:in `perform', state_machines-activerecord (0.100.0) lib/state_machines/integrations/active_record.rb:432:in `around_save', state_machines (0.100.4) lib/state_machines/machine.rb:569:in `save', activerecord (7.2.3) lib/active_record/associations/collection_association.rb:381:in `insert_record', activerecord (7.2.3) lib/active_record/associations/has_many_association.rb:63:in `insert_record', activerecord (7.2.3) lib/active_record/autosave_association.rb:414:in `block in save_collection_association', activerecord (7.2.3) lib/active_record/autosave_association.rb:403:in `each', activerecord (7.2.3) lib/active_record/autosave_association.rb:403:in `save_collection_association', activerecord (7.2.3) lib/active_record/autosave_association.rb:195:in `block in add_autosave_association_callbacks', activerecord (7.2.3) lib/active_record/autosave_association.rb:168:in `instance_eval', activerecord (7.2.3) lib/active_record/autosave_association.rb:168:in `block in define_non_cyclic_method', activesupport (7.2.3) lib/active_support/callbacks.rb:362:in `block in make_lambda', activesupport (7.2.3) lib/active_support/callbacks.rb:208:in `call', activesupport (7.2.3) lib/active_support/callbacks.rb:564:in `block in invoke_after', activesupport (7.2.3) lib/active_support/callbacks.rb:564:in `each', activesupport (7.2.3) lib/active_support/callbacks.rb:564:in `invoke_after', activesupport (7.2.3) lib/active_support/callbacks.rb:111:in `run_callbacks', activesupport (7.2.3) lib/active_support/callbacks.rb:914:in `_run_create_callbacks', activerecord (7.2.3) lib/active_record/callbacks.rb:445:in `_create_record', activerecord (7.2.3) lib/active_record/timestamp.rb:116:in `_create_record', activerecord (7.2.3) lib/active_record/persistence.rb:896:in `create_or_update', activerecord (7.2.3) lib/active_record/callbacks.rb:441:in `block in create_or_update', activesupport (7.2.3) lib/active_support/callbacks.rb:121:in `block in run_callbacks', activerecord (7.2.3) lib/active_record/autosave_association.rb:372:in `around_save_collection_association', activesupport (7.2.3) lib/active_support/callbacks.rb:130:in `block in run_callbacks', activesupport (7.2.3) lib/active_support/callbacks.rb:141:in `run_callbacks', activesupport (7.2.3) lib/active_support/callbacks.rb:914:in `_run_save_callbacks', activerecord (7.2.3) lib/active_record/callbacks.rb:441:in `create_or_update', activerecord (7.2.3) lib/active_record/timestamp.rb:127:in `create_or_update', activerecord (7.2.3) lib/active_record/persistence.rb:393:in `save', activerecord (7.2.3) lib/active_record/validations.rb:48:in `save', activerecord (7.2.3) lib/active_record/transactions.rb:362:in `block in save', activerecord (7.2.3) lib/active_record/transactions.rb:418:in `block (2 levels) in with_transaction_returning_status', activerecord (7.2.3) lib/active_record/connection_adapters/abstract/database_statements.rb:359:in `transaction', lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `public_send', lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `block in write_using_load_balancer', lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write', lib/gitlab/database/load_balancing/load_balancer.rb:235:in `retry_with_backoff', lib/gitlab/database/load_balancing/load_balancer.rb:131:in `read_write', lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer', lib/gitlab/database/load_balancing/connection_proxy.rb:78:in `transaction', activerecord (7.2.3) lib/active_record/transactions.rb:414:in `block in with_transaction_returning_status', lib/gitlab/database/load_balancing/setup.rb:57:in `block in setup_connection_proxy', activerecord (7.2.3) lib/active_record/transactions.rb:410:in `with_transaction_returning_status', activerecord (7.2.3) lib/active_record/transactions.rb:362:in `save', activerecord (7.2.3) lib/active_record/suppressor.rb:52:in `save', state_machines (0.100.4) lib/state_machines/machine.rb:569:in `block in save', state_machines (0.100.4) lib/state_machines/transition_collection.rb:174:in `block in run_actions', state_machines (0.100.4) lib/state_machines/transition_collection.rb:193:in `catch_exceptions', state_machines (0.100.4) lib/state_machines/transition_collection.rb:172:in `run_actions', state_machines (0.100.4) lib/state_machines/transition_collection.rb:157:in `run_callbacks', state_machines (0.100.4) lib/state_machines/transition_collection.rb:240:in `run_callbacks', state_machines (0.100.4) lib/state_machines/transition_collection.rb:75:in `block (2 levels) in perform', state_machines (0.100.4) lib/state_machines/transition_collection.rb:75:in `catch', state_machines (0.100.4) lib/state_machines/transition_collection.rb:75:in `block in perform', state_machines (0.100.4) lib/state_machines/transition_collection.rb:208:in `within_transaction', state_machines (0.100.4) lib/state_machines/transition_collection.rb:74:in `perform', state_machines-activerecord (0.100.0) lib/state_machines/integrations/active_record.rb:432:in `around_save', state_machines (0.100.4) lib/state_machines/machine.rb:569:in `save', activerecord (7.2.3) lib/active_record/associations/collection_association.rb:381:in `insert_record', activerecord (7.2.3) lib/active_record/associations/has_many_association.rb:63:in `insert_record', activerecord (7.2.3) lib/active_record/autosave_association.rb:414:in `block in save_collection_association', activerecord (7.2.3) lib/active_record/autosave_association.rb:403:in `each', activerecord (7.2.3) lib/active_record/autosave_association.rb:403:in `save_collection_association', activerecord (7.2.3) lib/active_record/autosave_association.rb:195:in `block in add_autosave_association_callbacks', activerecord (7.2.3) lib/active_record/autosave_association.rb:168:in `instance_eval', activerecord (7.2.3) lib/active_record/autosave_association.rb:168:in `block in define_non_cyclic_method', activesupport (7.2.3) lib/active_support/callbacks.rb:362:in `block in make_lambda', activesupport (7.2.3) lib/active_support/callbacks.rb:208:in `call', activesupport (7.2.3) lib/active_support/callbacks.rb:564:in `block in invoke_after', activesupport (7.2.3) lib/active_support/callbacks.rb:564:in `each', activesupport (7.2.3) lib/active_support/callbacks.rb:564:in `invoke_after', activesupport (7.2.3) lib/active_support/callbacks.rb:111:in `run_callbacks', activesupport (7.2.3) lib/active_support/callbacks.rb:914:in `_run_create_callbacks', activerecord (7.2.3) lib/active_record/callbacks.rb:445:in `_create_record', activerecord (7.2.3) lib/active_record/timestamp.rb:116:in `_create_record', activerecord (7.2.3) lib/active_record/persistence.rb:896:in `create_or_update', activerecord (7.2.3) lib/active_record/callbacks.rb:441:in `block in create_or_update', activesupport (7.2.3) lib/active_support/callbacks.rb:121:in `block in run_callbacks', activerecord (7.2.3) lib/active_record/autosave_association.rb:372:in `around_save_collection_association', activesupport (7.2.3) lib/active_support/callbacks.rb:130:in `block in run_callbacks', activesupport (7.2.3) lib/active_support/callbacks.rb:141:in `run_callbacks', activesupport (7.2.3) lib/active_support/callbacks.rb:914:in `_run_save_callbacks', activerecord (7.2.3) lib/active_record/callbacks.rb:441:in `create_or_update', activerecord (7.2.3) lib/active_record/timestamp.rb:127:in `create_or_update', activerecord (7.2.3) lib/active_record/persistence.rb:426:in `save!', activerecord (7.2.3) lib/active_record/validations.rb:54:in `save!', activerecord (7.2.3) lib/active_record/transactions.rb:366:in `block in save!', activerecord (7.2.3) lib/active_record/transactions.rb:418:in `block (2 levels) in with_transaction_returning_status', activerecord (7.2.3) lib/active_record/connection_adapters/abstract/database_statements.rb:359:in `transaction', lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `public_send', lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `block in write_using_load_balancer', lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write', lib/gitlab/database/load_balancing/load_balancer.rb:235:in `retry_with_backoff', lib/gitlab/database/load_balancing/load_balancer.rb:131:in `read_write', lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer', lib/gitlab/database/load_balancing/connection_proxy.rb:78:in `transaction', activerecord (7.2.3) lib/active_record/transactions.rb:414:in `block in with_transaction_returning_status', lib/gitlab/database/load_balancing/setup.rb:57:in `block in setup_connection_proxy', activerecord (7.2.3) lib/active_record/transactions.rb:410:in `with_transaction_returning_status', activerecord (7.2.3) lib/active_record/transactions.rb:366:in `save!', activerecord (7.2.3) lib/active_record/suppressor.rb:56:in `save!', state_machines (0.100.4) lib/state_machines/machine.rb:573:in `block in save!', state_machines (0.100.4) lib/state_machines/transition_collection.rb:174:in `block in run_actions', state_machines (0.100.4) lib/state_machines/transition_collection.rb:193:in `catch_exceptions', state_machines (0.100.4) lib/state_machines/transition_collection.rb:172:in `run_actions', state_machines (0.100.4) lib/state_machines/transition_collection.rb:157:in `run_callbacks', state_machines (0.100.4) lib/state_machines/transition_collection.rb:240:in `run_callbacks', state_machines (0.100.4) lib/state_machines/transition_collection.rb:75:in `block (2 levels) in perform', state_machines (0.100.4) lib/state_machines/transition_collection.rb:75:in `catch', state_machines (0.100.4) lib/state_machines/transition_collection.rb:75:in `block in perform', state_machines (0.100.4) lib/state_machines/transition_collection.rb:208:in `within_transaction', state_machines (0.100.4) lib/state_machines/transition_collection.rb:74:in `perform', state_machines-activerecord (0.100.0) lib/state_machines/integrations/active_record.rb:432:in `around_save', state_machines (0.100.4) lib/state_machines/machine.rb:573:in `save!', lib/gitlab/ci/pipeline/chain/create.rb:20:in `block (4 levels) in perform!', app/models/concerns/cross_database_modification.rb:91:in `block in transaction', activerecord (7.2.3) lib/active_record/connection_adapters/abstract/transaction.rb:616:in `block in within_new_transaction', activesupport (7.2.3) lib/active_support/concurrency/null_lock.rb:9:in `synchronize', activerecord (7.2.3) lib/active_record/connection_adapters/abstract/transaction.rb:613:in `within_new_transaction', activerecord (7.2.3) lib/active_record/connection_adapters/abstract/database_statements.rb:361:in `transaction', lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `public_send', lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `block in write_using_load_balancer', lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write', lib/gitlab/database/load_balancing/load_balancer.rb:235:in `retry_with_backoff', lib/gitlab/database/load_balancing/load_balancer.rb:131:in `read_write', lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer', lib/gitlab/database/load_balancing/connection_proxy.rb:78:in `transaction', activerecord (7.2.3) lib/active_record/transactions.rb:234:in `block in transaction', lib/gitlab/database/load_balancing/setup.rb:57:in `block in setup_connection_proxy', activerecord (7.2.3) lib/active_record/transactions.rb:233:in `transaction', lib/gitlab/database.rb:407:in `transaction', app/models/concerns/cross_database_modification.rb:82:in `transaction', activerecord (7.2.3) lib/active_record/transactions.rb:354:in `transaction', lib/gitlab/ci/pipeline/chain/create.rb:19:in `block (3 levels) in perform!', app/models/concerns/ci/bulk_insertable_tags.rb:13:in `with_bulk_insert_tags', lib/gitlab/ci/pipeline/chain/create.rb:18:in `block (2 levels) in perform!', app/models/concerns/bulk_insertable_associations.rb:54:in `with_bulk_insert', lib/gitlab/ci/pipeline/chain/create.rb:17:in `block in perform!', lib/gitlab/ci/pipeline/logger.rb:33:in `instrument', lib/gitlab/ci/pipeline/logger.rb:43:in `instrument_once_with_sql', lib/gitlab/ci/pipeline/chain/create.rb:12:in `perform!', lib/gitlab/ci/pipeline/chain/sequence.rb:20:in `block in build!', lib/gitlab/ci/pipeline/chain/sequence.rb:16:in `each', lib/gitlab/ci/pipeline/chain/sequence.rb:16:in `build!', app/services/ci/create_pipeline_service.rb:108:in `execute', app/workers/create_pipeline_worker.rb:31:in `perform', vendor/gems/sidekiq/lib/sidekiq/processor.rb:220:in `execute_job', vendor/gems/sidekiq/lib/sidekiq/processor.rb:185:in `block (4 levels) in process', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', ee/lib/gitlab/sidekiq_middleware/set_session/server.rb:21:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/current_organization/server.rb:21:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/identity/restore.rb:12:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/resource_usage_limit/middleware.rb:16:in `perform', lib/gitlab/sidekiq_middleware/resource_usage_limit/server.rb:8:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:35:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/skip_jobs.rb:51:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/throttling/middleware.rb:39:in `perform', lib/gitlab/sidekiq_middleware/throttling/server.rb:8:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/pause_control/strategies/base.rb:31:in `perform', lib/gitlab/sidekiq_middleware/pause_control/strategy_handler.rb:22:in `perform', lib/gitlab/sidekiq_middleware/pause_control/server.rb:8:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform', lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:44:in `perform', lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/concurrency_limit/middleware.rb:44:in `perform', lib/gitlab/sidekiq_middleware/concurrency_limit/server.rb:8:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/click_house/migration_support/sidekiq_middleware.rb:7:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context', lib/gitlab/sidekiq_middleware/worker_context/server.rb:19:in `block in call', lib/gitlab/application_context.rb:177:in `block in use', gitlab-labkit (1.0.1) lib/labkit/context.rb:43:in `with_context', lib/gitlab/application_context.rb:177:in `use', lib/gitlab/application_context.rb:99:in `with_context', lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `block in call', lib/gitlab/database/query_analyzer.rb:94:in `within', lib/gitlab/sidekiq_middleware/query_analyzer.rb:7:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/set_ip_address.rb:7:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/server_metrics.rb:112:in `block in call', lib/gitlab/sidekiq_middleware/server_metrics.rb:145:in `block in instrument', lib/gitlab/metrics/background_transaction.rb:33:in `run', lib/gitlab/sidekiq_middleware/server_metrics.rb:145:in `instrument', lib/gitlab/sidekiq_middleware/server_metrics.rb:111:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `block in call', gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:66:in `enabling_request_store', gems/gitlab-safe_request_store/lib/gitlab/safe_request_store.rb:59:in `ensure_request_store', lib/gitlab/sidekiq_middleware/request_store_middleware.rb:7:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', gitlab-labkit (1.0.1) lib/labkit/middleware/sidekiq/server.rb:22:in `block in call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:180:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', gitlab-labkit (1.0.1) lib/labkit/middleware/sidekiq/user_experience_sli/server.rb:16:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', gitlab-labkit (1.0.1) lib/labkit/middleware/sidekiq/context/server.rb:16:in `block in call', gitlab-labkit (1.0.1) lib/labkit/context.rb:43:in `with_context', gitlab-labkit (1.0.1) lib/labkit/middleware/sidekiq/context/server.rb:15:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke', gitlab-labkit (1.0.1) lib/labkit/middleware/sidekiq/server.rb:21:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/monitor.rb:10:in `block in call', lib/gitlab/sidekiq_daemon/monitor.rb:46:in `within_job', lib/gitlab/sidekiq_middleware/monitor.rb:9:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:10:in `block in call', lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled', lib/gitlab/sidekiq_middleware/shard_awareness_validator.rb:9:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', lib/gitlab/sidekiq_middleware/size_limiter/server.rb:13:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', marginalia (1.11.1) lib/marginalia/sidekiq_instrumentation.rb:9:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', sentry-sidekiq (5.23.0) lib/sentry/sidekiq/sentry_context_middleware.rb:54:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', vendor/gems/sidekiq/lib/sidekiq/job/interrupt_handler.rb:9:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:183:in `block in traverse', vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:26:in `track', vendor/gems/sidekiq/lib/sidekiq/metrics/tracking.rb:134:in `call', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:182:in `traverse', vendor/gems/sidekiq/lib/sidekiq/middleware/chain.rb:173:in `invoke', vendor/gems/sidekiq/lib/sidekiq/processor.rb:184:in `block (3 levels) in process', vendor/gems/sidekiq/lib/sidekiq/processor.rb:145:in `block (6 levels) in dispatch', vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:118:in `local', vendor/gems/sidekiq/lib/sidekiq/processor.rb:144:in `block (5 levels) in dispatch', vendor/gems/sidekiq/lib/sidekiq/rails.rb:27:in `block in call', activesupport (7.2.3) lib/active_support/reloader.rb:77:in `block in wrap', activesupport (7.2.3) lib/active_support/execution_wrapper.rb:91:in `wrap', activesupport (7.2.3) lib/active_support/reloader.rb:74:in `wrap', vendor/gems/sidekiq/lib/sidekiq/rails.rb:26:in `call', vendor/gems/sidekiq/lib/sidekiq/processor.rb:139:in `block (4 levels) in dispatch', vendor/gems/sidekiq/lib/sidekiq/processor.rb:281:in `stats', vendor/gems/sidekiq/lib/sidekiq/processor.rb:134:in `block (3 levels) in dispatch', lib/gitlab/sidekiq_logging/structured_logger.rb:21:in `call', vendor/gems/sidekiq/lib/sidekiq/processor.rb:133:in `block (2 levels) in dispatch', vendor/gems/sidekiq/lib/sidekiq/job_retry.rb:85:in `global', vendor/gems/sidekiq/lib/sidekiq/processor.rb:132:in `block in dispatch', vendor/gems/sidekiq/lib/sidekiq/job_logger.rb:40:in `prepare', vendor/gems/sidekiq/lib/sidekiq/processor.rb:131:in `dispatch', vendor/gems/sidekiq/lib/sidekiq/processor.rb:183:in `block (2 levels) in process', vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `handle_interrupt', vendor/gems/sidekiq/lib/sidekiq/processor.rb:182:in `block in process', vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `handle_interrupt', vendor/gems/sidekiq/lib/sidekiq/processor.rb:181:in `process', vendor/gems/sidekiq/lib/sidekiq/processor.rb:86:in `process_one', vendor/gems/sidekiq/lib/sidekiq/processor.rb:76:in `run', vendor/gems/sidekiq/lib/sidekiq/component.rb:10:in `watchdog', vendor/gems/sidekiq/lib/sidekiq/component.rb:19:in `block in safe_thread']

Output of checks

This bug happens on GitLab.com

Possible fixes

Patch release information for backports

If the bug fix needs to be backported in a patch release to a version under the maintenance policy, please follow the steps on the patch release runbook for GitLab engineers.

Refer to the internal "Release Information" dashboard for information about the next patch release, including the targeted versions, expected release date, and current status.

High-severity bug remediation

To remediate high-severity issues requiring an internal release for single-tenant SaaS instances, refer to the internal release process for engineers.

Edited Dec 26, 2025 by 🤖 GitLab Bot 🤖