ExpireBuildInstanceArtifactsWorker - ActiveRecord::RecordNotUnique: PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "ci_builds_pkey"
Summary
We have nearly every day some build jobs where the ExpireBuildInstanceArtifactsWorker job will fail due of duplicate entry in postgres db
Steps to reproduce
Well not really sure. We didn't change anything (beside usual GitLab updates) in our building system. It seems to be that
some builds are adding the duplicate entry in ci_builds table.
Example Project
N/A
What is the current bug behavior?
Builds jobs are adding a duplicate entry in ci_buildstable hence ExpireBuildInstanceArtifactsWorker will error out with following error message:
ActiveRecord::RecordNotUnique: PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "ci_builds_pkey" DETAIL: Key (id)=(14487) already exists. : UPDATE "ci_builds" SET "artifacts...What is the expected correct behavior?
Just a unique db entry so that ExpireBuildInstanceArtifactsWorker can do it's job
Relevant logs and/or screenshots
Log shows more detailed error message...sadly we cannot find the place where the duplicate entry is set:
2018-03-15_13:53:17.57085 2018-03-15T13:53:17.570Z 547 TID-ot1lu47xg WARN: {"context":"Job raised exception","job":{"class":"ExpireBuildInstanceArtifactsWorker","args":[14487],"retry":3,"queue":"expire_build_instance_artifacts","jid":"f79102ae23651ec929ecd568","created_at":1521121815.7162015,"enqueued_at":1521121997.5072944,"error_message":"PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"ci_builds_pkey\"\nDETAIL: Key (id)=(14487) already exists.\n: UPDATE \"ci_builds\" SET \"artifacts_file\" = NULL, \"artifacts_metadata\" = NULL, \"updated_at\" = '2018-03-15 13:51:43.777271' WHERE \"ci_builds\".\"id\" = 14487","error_class":"ActiveRecord::RecordNotUnique","failed_at":1521121819.3550558,"retry_count":2,"retried_at":1521121903.782153},"jobstr":"{\"class\":\"ExpireBuildInstanceArtifactsWorker\",\"args\":[14487],\"retry\":3,\"queue\":\"expire_build_instance_artifacts\",\"jid\":\"f79102ae23651ec929ecd568\",\"created_at\":1521121815.7162015,\"enqueued_at\":1521121997.5072944,\"error_message\":\"PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \\\"ci_builds_pkey\\\"\\nDETAIL: Key (id)=(14487) already exists.\\n: UPDATE \\\"ci_builds\\\" SET \\\"artifacts_file\\\" = NULL, \\\"artifacts_metadata\\\" = NULL, \\\"updated_at\\\" = '2018-03-15 13:51:43.777271' WHERE \\\"ci_builds\\\".\\\"id\\\" = 14487\",\"error_class\":\"ActiveRecord::RecordNotUnique\",\"failed_at\":1521121819.3550558,\"retry_count\":2,\"retried_at\":1521121903.782153}"}
2018-03-15_13:53:17.57141 2018-03-15T13:53:17.570Z 547 TID-ot1lu47xg WARN: ActiveRecord::RecordNotUnique: PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "ci_builds_pkey"
2018-03-15_13:53:17.57155 DETAIL: Key (id)=(14487) already exists.
2018-03-15_13:53:17.57168 : UPDATE "ci_builds" SET "artifacts_file" = NULL, "artifacts_metadata" = NULL, "updated_at" = '2018-03-15 13:53:17.565351' WHERE "ci_builds"."id" = 14487
2018-03-15_13:53:17.57183 2018-03-15T13:53:17.570Z 547 TID-ot1lu47xg WARN: /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/peek-pg-1.3.0/lib/peek/views/pg.rb:17:in `async_exec'
2018-03-15_13:53:17.57195 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/peek-pg-1.3.0/lib/peek/views/pg.rb:17:in `async_exec'
2018-03-15_13:53:17.57212 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `block in exec_no_cache'
2018-03-15_13:53:17.57222 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'
2018-03-15_13:53:17.57234 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activesupport-4.2.10/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
2018-03-15_13:53:17.57243 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'
2018-03-15_13:53:17.57253 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `exec_no_cache'
2018-03-15_13:53:17.57271 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/postgresql_adapter.rb:584:in `execute_and_clear'
2018-03-15_13:53:17.57280 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/postgresql/database_statements.rb:173:in `exec_delete'
2018-03-15_13:53:17.57292 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract/database_statements.rb:114:in `update'
2018-03-15_13:53:17.57301 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract/query_cache.rb:14:in `update'
2018-03-15_13:53:17.57309 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/relation.rb:88:in `_update_record'
2018-03-15_13:53:17.57317 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/persistence.rb:515:in `_update_record'
2018-03-15_13:53:17.57326 /opt/gitlab/embedded/service/gitlab-rails/config/initializers/active_record_locking.rb:16:in `_update_record'
2018-03-15_13:53:17.57336 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/attribute_methods/dirty.rb:129:in `_update_record'
2018-03-15_13:53:17.57346 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/callbacks.rb:310:in `block in _update_record'
2018-03-15_13:53:17.57358 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
2018-03-15_13:53:17.57369 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
2018-03-15_13:53:17.57380 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
2018-03-15_13:53:17.57388 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
2018-03-15_13:53:17.57402 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_update_callbacks'
2018-03-15_13:53:17.57411 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/callbacks.rb:310:in `_update_record'
2018-03-15_13:53:17.57424 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/timestamp.rb:70:in `_update_record'
2018-03-15_13:53:17.57435 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/persistence.rb:504:in `create_or_update'
2018-03-15_13:53:17.57444 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/callbacks.rb:302:in `block in create_or_update'
2018-03-15_13:53:17.57466 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:117:in `call'
2018-03-15_13:53:17.57476 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
2018-03-15_13:53:17.57483 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:505:in `call'
2018-03-15_13:53:17.57491 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
2018-03-15_13:53:17.57498 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activesupport-4.2.10/lib/active_support/callbacks.rb:778:in `_run_save_callbacks'
2018-03-15_13:53:17.57507 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/callbacks.rb:302:in `create_or_update'
2018-03-15_13:53:17.57517 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/persistence.rb:120:in `save'
2018-03-15_13:53:17.57525 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/validations.rb:37:in `save'
2018-03-15_13:53:17.57532 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/attribute_methods/dirty.rb:21:in `save'
2018-03-15_13:53:17.57543 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/transactions.rb:286:in `block (2 levels) in save'
2018-03-15_13:53:17.57550 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/transactions.rb:351:in `block in with_transaction_returning_status'
2018-03-15_13:53:17.57558 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `block in transaction'
2018-03-15_13:53:17.57568 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract/transaction.rb:184:in `within_new_transaction'
2018-03-15_13:53:17.57577 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/connection_adapters/abstract/database_statements.rb:213:in `transaction'
2018-03-15_13:53:17.57584 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/transactions.rb:220:in `transaction'
2018-03-15_13:53:17.57592 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/transactions.rb:348:in `with_transaction_returning_status'
2018-03-15_13:53:17.57600 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/transactions.rb:286:in `block in save'
2018-03-15_13:53:17.57608 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/transactions.rb:301:in `rollback_active_record_state!'
2018-03-15_13:53:17.57616 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.10/lib/active_record/transactions.rb:285:in `save'
2018-03-15_13:53:17.57625 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/state_machines-activerecord-0.4.0/lib/state_machines/integrations/active_record.rb:501:in `block in save'
2018-03-15_13:53:17.57633 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/state_machines-activerecord-0.4.0/lib/state_machines/integrations/active_record.rb:520:in `block in around_save'
2018-03-15_13:53:17.57640 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/state_machines-0.4.0/lib/state_machines/transition_collection.rb:150:in `block in run_actions'
2018-03-15_13:53:17.57650 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/state_machines-0.4.0/lib/state_machines/transition_collection.rb:170:in `catch_exceptions'
2018-03-15_13:53:17.57660 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/state_machines-0.4.0/lib/state_machines/transition_collection.rb:148:in `run_actions'
2018-03-15_13:53:17.57669 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/state_machines-0.4.0/lib/state_machines/transition_collection.rb:133:in `run_callbacks'
2018-03-15_13:53:17.57679 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/state_machines-0.4.0/lib/state_machines/transition_collection.rb:212:in `run_callbacks'
2018-03-15_13:53:17.57686 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/state_machines-0.4.0/lib/state_machines/transition_collection.rb:63:in `block (2 levels) in perform'
2018-03-15_13:53:17.57694 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/state_machines-0.4.0/lib/state_machines/transition_collection.rb:63:in `catch'
2018-03-15_13:53:17.57702 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/state_machines-0.4.0/lib/state_machines/transition_collection.rb:63:in `block in perform'
2018-03-15_13:53:17.57712 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/state_machines-0.4.0/lib/state_machines/transition_collection.rb:186:in `within_transaction'
2018-03-15_13:53:17.57787 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/state_machines-0.4.0/lib/state_machines/transition_collection.rb:62:in `perform'
2018-03-15_13:53:17.57803 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/state_machines-activerecord-0.4.0/lib/state_machines/integrations/active_record.rb:520:in `around_save'
2018-03-15_13:53:17.57811 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/state_machines-activerecord-0.4.0/lib/state_machines/integrations/active_record.rb:501:in `save'
2018-03-15_13:53:17.57819 /opt/gitlab/embedded/service/gitlab-rails/app/models/ci/build.rb:377:in `erase_artifacts!'
2018-03-15_13:53:17.57828 /opt/gitlab/embedded/service/gitlab-rails/app/workers/expire_build_instance_artifacts_worker.rb:13:in `perform'
2018-03-15_13:53:17.57837 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:188:in `execute_job'
2018-03-15_13:53:17.57847 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:170:in `block (2 levels) in process'
2018-03-15_13:53:17.57857 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
2018-03-15_13:53:17.57868 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_status/server_middleware.rb:5:in `call'
2018-03-15_13:53:17.57880 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2018-03-15_13:53:17.57888 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/request_store_middleware.rb:6:in `call'
2018-03-15_13:53:17.57897 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2018-03-15_13:53:17.57904 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/shutdown.rb:52:in `call'
2018-03-15_13:53:17.57912 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2018-03-15_13:53:17.57920 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/server/active_record.rb:15:in `call'
2018-03-15_13:53:17.57928 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2018-03-15_13:53:17.57935 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sentry-raven-2.5.3/lib/raven/integrations/sidekiq.rb:7:in `call'
2018-03-15_13:53:17.57942 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
2018-03-15_13:53:17.57951 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:133:in `invoke'
2018-03-15_13:53:17.57959 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:169:in `block in process'
2018-03-15_13:53:17.57967 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:141:in `block (6 levels) in dispatch'
2018-03-15_13:53:17.57976 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/job_retry.rb:97:in `local'
2018-03-15_13:53:17.57985 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:140:in `block (5 levels) in dispatch'
2018-03-15_13:53:17.57997 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq.rb:36:in `block in <module:Sidekiq>'
2018-03-15_13:53:17.58008 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:136:in `block (4 levels) in dispatch'
2018-03-15_13:53:17.58020 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:204:in `stats'
2018-03-15_13:53:17.58029 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:131:in `block (3 levels) in dispatch'
2018-03-15_13:53:17.58039 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/job_logger.rb:7:in `call'
2018-03-15_13:53:17.58047 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:130:in `block (2 levels) in dispatch'
2018-03-15_13:53:17.58056 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/job_retry.rb:72:in `global'
2018-03-15_13:53:17.58064 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:129:in `block in dispatch'
2018-03-15_13:53:17.58073 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/logging.rb:44:in `with_context'
2018-03-15_13:53:17.58081 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/logging.rb:38:in `with_job_hash_context'
2018-03-15_13:53:17.58089 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:128:in `dispatch'
2018-03-15_13:53:17.58098 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:168:in `process'
2018-03-15_13:53:17.58107 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:85:in `process_one'
2018-03-15_13:53:17.58113 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:73:in `run'
2018-03-15_13:53:17.58122 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/util.rb:16:in `watchdog'
2018-03-15_13:53:17.58128 /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/util.rb:25:in `block in safe_thread'
==> /var/log/gitlab/postgresql/current <==
2018-03-15_13:53:17.56835 ERROR: duplicate key value violates unique constraint "ci_builds_pkey"
2018-03-15_13:53:17.56857 DETAIL: Key (id)=(14487) already exists.
2018-03-15_13:53:17.56869 STATEMENT: UPDATE "ci_builds" SET "artifacts_file" = NULL, "artifacts_metadata" = NULL, "updated_at" = '2018-03-15 13:53:17.565351' WHERE "ci_builds"."id" = 14487
==> /var/log/gitlab/gitlab-rails/production.log <==
Removing artifacts for build 14487...
Output of checks
Bug happens on the official GitLab docker image since ce-10.5.0
Results of GitLab environment info
Expand for output related to GitLab environment info
System information System: Current User: git Using RVM: no Ruby Version: 2.3.6p384 Gem Version: 2.6.13 Bundler Version:1.13.7 Rake Version: 12.3.0 Redis Version: 3.2.11 Git Version: 2.14.3 Sidekiq Version:5.0.5 Go Version: unknown GitLab information Version: 10.5.4 Revision: 8d768f9 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: http://gitlab.dev-mc.de HTTP Clone URL: http://gitlab.dev-mc.de/some-group/some-project.git SSH Clone URL: git@gitlab.dev-mc.de:some-group/some-project.git Using LDAP: no Using Omniauth: no GitLab Shell Version: 6.0.3 Repository storage paths: - default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
Checking GitLab Shell ... GitLab Shell version >= 6.0.3 ? ... OK (6.0.3) Repo base directory exists? default... yes Repo storage directories are symlinks? default... no Repo paths owned by git:root, or git:git? default... yes Repo paths access is drwxrws---? default... yes hooks directories in repos are links: ... 32/112 ... ok 32/113 ... ok 101/114 ... ok 52/115 ... ok 32/116 ... ok 51/117 ... ok 51/118 ... ok 51/119 ... ok 51/120 ... ok 53/121 ... ok 50/122 ... ok 32/123 ... ok 32/124 ... ok 104/125 ... ok 48/126 ... ok 37/127 ... ok 99/128 ... ok 37/129 ... ok 37/130 ... ok 37/132 ... ok 37/133 ... ok 106/134 ... ok 45/135 ... ok 44/136 ... ok 44/137 ... ok 37/138 ... ok 32/139 ... ok 46/140 ... ok 37/141 ... ok 37/142 ... repository is empty 37/143 ... ok 37/144 ... ok 34/145 ... ok 34/146 ... ok 34/147 ... ok 34/148 ... ok 34/149 ... ok 34/150 ... ok 34/151 ... ok 34/152 ... ok 34/153 ... ok 34/154 ... ok 34/155 ... ok 43/156 ... ok 32/157 ... ok 37/158 ... ok 37/159 ... ok 37/160 ... ok 37/161 ... ok 39/162 ... ok 79/163 ... ok 34/164 ... ok 42/165 ... ok 42/166 ... ok 32/167 ... ok 32/168 ... ok 32/169 ... ok 74/170 ... ok 42/171 ... ok 42/172 ... ok 37/173 ... ok 54/174 ... ok 35/175 ... ok 35/176 ... ok 103/177 ... ok 32/178 ... ok 35/179 ... ok 35/180 ... ok 54/181 ... ok 35/182 ... ok 54/183 ... ok 54/184 ... ok 40/186 ... ok 56/187 ... ok 56/188 ... ok 38/189 ... ok 34/190 ... ok 57/191 ... ok 57/192 ... ok 56/193 ... ok 33/194 ... ok 33/195 ... ok 33/196 ... ok 33/197 ... ok 33/198 ... ok 69/199 ... ok 40/200 ... ok 41/201 ... ok 41/202 ... ok 34/203 ... ok 34/204 ... ok 55/206 ... ok 59/207 ... ok 59/208 ... ok 58/209 ... ok 54/210 ... ok 58/211 ... ok 58/212 ... ok 32/213 ... ok 32/214 ... ok 32/215 ... ok 58/216 ... ok 51/217 ... ok 51/218 ... ok 63/219 ... ok 63/221 ... ok 73/222 ... ok 73/223 ... ok 66/228 ... ok 33/229 ... ok 69/230 ... ok 36/234 ... ok 36/236 ... ok 36/237 ... ok 36/238 ... ok 36/239 ... ok 56/240 ... ok 62/244 ... ok 36/245 ... ok 2/246 ... ok 58/248 ... ok 63/249 ... ok 59/250 ... ok 70/251 ... ok 70/252 ... ok 70/253 ... ok 70/254 ... ok 36/255 ... ok 72/257 ... ok 72/258 ... ok 72/259 ... ok 72/260 ... ok 73/263 ... repository is empty 74/264 ... ok 36/267 ... ok 77/268 ... ok 77/269 ... ok 34/270 ... ok 78/271 ... ok 78/272 ... ok 78/273 ... ok 36/274 ... ok 79/275 ... ok 80/280 ... ok 36/281 ... ok 34/283 ... ok 81/285 ... ok 2/286 ... ok 33/287 ... ok 82/288 ... ok 82/289 ... ok 82/290 ... ok 83/291 ... ok 62/292 ... ok 66/295 ... ok 84/296 ... ok 84/297 ... ok 84/298 ... ok 57/299 ... ok 87/305 ... ok 62/307 ... ok 36/314 ... ok 36/316 ... ok 36/317 ... ok 90/318 ... ok 58/319 ... ok 61/320 ... ok 61/322 ... ok 22/323 ... ok 22/324 ... ok 32/325 ... ok 32/326 ... repository is empty 61/328 ... ok 61/329 ... ok 22/331 ... ok 62/332 ... ok 36/333 ... ok 32/339 ... ok 36/340 ... ok 62/341 ... ok 32/342 ... ok 62/343 ... ok 71/344 ... ok 61/345 ... ok 2/346 ... ok 95/348 ... ok 32/349 ... repository is empty 96/350 ... ok 62/351 ... ok 61/352 ... ok 96/354 ... ok 32/355 ... ok 32/358 ... repository is empty 95/359 ... ok 95/360 ... ok 36/361 ... ok 99/362 ... ok 61/363 ... ok 36/364 ... ok 100/365 ... ok 100/366 ... ok 101/367 ... ok 36/369 ... ok 36/370 ... ok 103/371 ... ok 104/373 ... ok 105/375 ... ok 105/376 ... ok 105/377 ... ok 62/379 ... ok 62/380 ... ok 106/381 ... ok 36/382 ... ok 57/383 ... ok 107/384 ... ok 36/385 ... ok 36/386 ... ok 36/387 ... ok 61/388 ... ok 36/389 ... ok 32/390 ... ok 34/391 ... ok 36/392 ... ok 22/393 ... ok 109/394 ... ok 109/395 ... ok 36/396 ... ok 36/398 ... ok 36/400 ... ok 102/401 ... ok 36/402 ... repository is empty Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Redis available via internal API: OK Access to /var/opt/gitlab/.ssh/authorized_keys: OK gitlab-shell self-check successful Checking GitLab Shell ... Finished Checking Sidekiq ... Running? ... yes Number of Sidekiq processes ... 1 Checking Sidekiq ... Finished Reply by email is disabled in config/gitlab.yml Checking LDAP ... LDAP is disabled in config/gitlab.yml Checking LDAP ... Finished Checking GitLab ... Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 32/112 ... yes 32/113 ... yes 101/114 ... yes 52/115 ... yes 32/116 ... yes 51/117 ... yes 51/118 ... yes 51/119 ... yes 51/120 ... yes 53/121 ... yes 50/122 ... yes 32/123 ... yes 32/124 ... yes 104/125 ... yes 48/126 ... yes 37/127 ... yes 99/128 ... yes 37/129 ... yes 37/130 ... yes 37/132 ... yes 37/133 ... yes 106/134 ... yes 45/135 ... yes 44/136 ... yes 44/137 ... yes 37/138 ... yes 32/139 ... yes 46/140 ... yes 37/141 ... yes 37/142 ... yes 37/143 ... yes 37/144 ... yes 34/145 ... yes 34/146 ... yes 34/147 ... yes 34/148 ... yes 34/149 ... yes 34/150 ... yes 34/151 ... yes 34/152 ... yes 34/153 ... yes 34/154 ... yes 34/155 ... yes 43/156 ... yes 32/157 ... yes 37/158 ... yes 37/159 ... yes 37/160 ... yes 37/161 ... yes 39/162 ... yes 79/163 ... yes 34/164 ... yes 42/165 ... yes 42/166 ... yes 32/167 ... yes 32/168 ... yes 32/169 ... yes 74/170 ... yes 42/171 ... yes 42/172 ... yes 37/173 ... yes 54/174 ... yes 35/175 ... yes 35/176 ... yes 103/177 ... yes 32/178 ... yes 35/179 ... yes 35/180 ... yes 54/181 ... yes 35/182 ... yes 54/183 ... yes 54/184 ... yes 40/186 ... yes 56/187 ... yes 56/188 ... yes 38/189 ... yes 34/190 ... yes 57/191 ... yes 57/192 ... yes 56/193 ... yes 33/194 ... yes 33/195 ... yes 33/196 ... yes 33/197 ... yes 33/198 ... yes 69/199 ... yes 40/200 ... yes 41/201 ... yes 41/202 ... yes 34/203 ... yes 34/204 ... yes 55/206 ... yes 59/207 ... yes 59/208 ... yes 58/209 ... yes 54/210 ... yes 58/211 ... yes 58/212 ... yes 32/213 ... yes 32/214 ... yes 32/215 ... yes 58/216 ... yes 51/217 ... yes 51/218 ... yes 63/219 ... yes 63/221 ... yes 73/222 ... yes 73/223 ... yes 66/228 ... yes 33/229 ... yes 69/230 ... yes 36/234 ... yes 36/236 ... yes 36/237 ... yes 36/238 ... yes 36/239 ... yes 56/240 ... yes 62/244 ... yes 36/245 ... yes 2/246 ... yes 58/248 ... yes 63/249 ... yes 59/250 ... yes 70/251 ... yes 70/252 ... yes 70/253 ... yes 70/254 ... yes 36/255 ... yes 72/257 ... yes 72/258 ... yes 72/259 ... yes 72/260 ... yes 73/263 ... yes 74/264 ... yes 36/267 ... yes 77/268 ... yes 77/269 ... yes 34/270 ... yes 78/271 ... yes 78/272 ... yes 78/273 ... yes 36/274 ... yes 79/275 ... yes 80/280 ... yes 36/281 ... yes 34/283 ... yes 81/285 ... yes 2/286 ... yes 33/287 ... yes 82/288 ... yes 82/289 ... yes 82/290 ... yes 83/291 ... yes 62/292 ... yes 66/295 ... yes 84/296 ... yes 84/297 ... yes 84/298 ... yes 57/299 ... yes 87/305 ... yes 62/307 ... yes 36/314 ... yes 36/316 ... yes 36/317 ... yes 90/318 ... yes 58/319 ... yes 61/320 ... yes 61/322 ... yes 22/323 ... yes 22/324 ... yes 32/325 ... yes 32/326 ... yes 61/328 ... yes 61/329 ... yes 22/331 ... yes 62/332 ... yes 36/333 ... yes 32/339 ... yes 36/340 ... yes 62/341 ... yes 32/342 ... yes 62/343 ... yes 71/344 ... yes 61/345 ... yes 2/346 ... yes 95/348 ... yes 32/349 ... yes 96/350 ... yes 62/351 ... yes 61/352 ... yes 96/354 ... yes 32/355 ... yes 32/358 ... yes 95/359 ... yes 95/360 ... yes 36/361 ... yes 99/362 ... yes 61/363 ... yes 36/364 ... yes 100/365 ... yes 100/366 ... yes 101/367 ... yes 36/369 ... yes 36/370 ... yes 103/371 ... yes 104/373 ... yes 105/375 ... yes 105/376 ... yes 105/377 ... yes 62/379 ... yes 62/380 ... yes 106/381 ... yes 36/382 ... yes 57/383 ... yes 107/384 ... yes 36/385 ... yes 36/386 ... yes 36/387 ... yes 61/388 ... yes 36/389 ... yes 32/390 ... yes 34/391 ... yes 36/392 ... yes 22/393 ... yes 109/394 ... yes 109/395 ... yes 36/396 ... yes 36/398 ... yes 36/400 ... yes 102/401 ... yes 36/402 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.3.5 ? ... yes (2.3.6) Git version >= 2.9.5 ? ... yes (2.14.3) Git user has default SSH configuration? ... yes Active users: ... 15 Checking GitLab ... Finished
Possible fixes
Honestly no idea right now. Maybe as reference another build related bug we encounterd as well since 10.5.0:
Removing the duplicate entry by hand will solve this issue for the particular job.
Edited by Lars Greiss