Database Deadlock with UpdateMergeRequestsWorker on GitLab 12.1 and Postgres 10.7
Summary
UpdateMergeRequestsWorker running for hours until Sidekiq is restarted.
Steps to reproduce
Not entirely sure. We've seen this happen on our most heavily used repositories, but not sure what triggers it.
Example Project
None at the moment.
What is the current bug behavior?
We see that the UpdateMergeRequestsWorker is blocked on several projects (for up to 7-8 hours). The affected MRs also do not reflect the latest pushed commits.
What is the expected correct behavior?
No blocking should happen.
Relevant logs and/or screenshots
I have narrowed it down via the Database logs as well as sending SIGTTIN to Sidekiq instances.
2019-07-29 11:54:27 UTC:xx.xx.xx.xx(38170):gitlab@gitlabhq_production:[16727]:ERROR: deadlock detected
2019-07-29 11:54:27 UTC:xx.xx.xx.xx(38170):gitlab@gitlabhq_production:[16727]:DETAIL: Process 16727 waits for ShareLock on transaction 72790439; blocked by process 10876.
Process 10876 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 16727.
Process 16727: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 10876: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 319923 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 319923)
2019-07-29 11:54:27 UTC:xx.xx.xx.xx(38170):gitlab@gitlabhq_production:[16727]:HINT: See server log for query details.
2019-07-29 11:54:27 UTC:xx.xx.xx.xx(38170):gitlab@gitlabhq_production:[16727]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:54:27 UTC:xx.xx.xx.xx(38170):gitlab@gitlabhq_production:[16727]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:54:27 UTC:xx.xx.xx.xx(38170):gitlab@gitlabhq_production:[16727]:LOG: could not send data to client: Connection timed out
2019-07-29 11:54:27 UTC:xx.xx.xx.xx(38170):gitlab@gitlabhq_production:[16727]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:55:51 UTC:xx.xx.xx.xx(60424):gitlab@gitlabhq_production:[20410]:ERROR: deadlock detected
2019-07-29 11:55:51 UTC:xx.xx.xx.xx(60424):gitlab@gitlabhq_production:[20410]:DETAIL: Process 20410 waits for ShareLock on transaction 72800923; blocked by process 1254.
Process 1254 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 20410.
Process 20410: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 1254: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 319956 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 319956)
2019-07-29 11:55:51 UTC:xx.xx.xx.xx(60424):gitlab@gitlabhq_production:[20410]:HINT: See server log for query details.
2019-07-29 11:55:51 UTC:xx.xx.xx.xx(60424):gitlab@gitlabhq_production:[20410]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:55:51 UTC:xx.xx.xx.xx(60424):gitlab@gitlabhq_production:[20410]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:55:51 UTC:xx.xx.xx.xx(60424):gitlab@gitlabhq_production:[20410]:LOG: could not send data to client: Connection timed out
2019-07-29 11:55:51 UTC:xx.xx.xx.xx(60424):gitlab@gitlabhq_production:[20410]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:55:52 UTC:xx.xx.xx.xx(54420):gitlab@gitlabhq_production:[24071]:ERROR: deadlock detected
2019-07-29 11:55:52 UTC:xx.xx.xx.xx(54420):gitlab@gitlabhq_production:[24071]:DETAIL: Process 24071 waits for ShareLock on transaction 72800923; blocked by process 1254.
Process 1254 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 24071.
Process 24071: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 1254: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 319956 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 319956)
2019-07-29 11:55:52 UTC:xx.xx.xx.xx(54420):gitlab@gitlabhq_production:[24071]:HINT: See server log for query details.
2019-07-29 11:55:52 UTC:xx.xx.xx.xx(54420):gitlab@gitlabhq_production:[24071]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:55:52 UTC:xx.xx.xx.xx(54420):gitlab@gitlabhq_production:[24071]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:55:52 UTC:xx.xx.xx.xx(54420):gitlab@gitlabhq_production:[24071]:LOG: could not send data to client: Connection timed out
2019-07-29 11:55:52 UTC:xx.xx.xx.xx(54420):gitlab@gitlabhq_production:[24071]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:ERROR: deadlock detected
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:DETAIL: Process 454 waits for ShareLock on transaction 72849258; blocked by process 12706.
Process 12706 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 454.
Process 454: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 12706: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 320144 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 320144)
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:HINT: See server log for query details.
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:LOG: could not send data to client: Connection timed out
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:FATAL: connection to client lost
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:ERROR: deadlock detected
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:DETAIL: Process 25621 waits for ShareLock on transaction 72849258; blocked by process 12706.
Process 12706 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 25621.
Process 25621: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 12706: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 320144 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 320144)
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:HINT: See server log for query details.
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:LOG: could not send data to client: Connection timed out
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:FATAL: connection to client lost
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:ERROR: deadlock detected
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:DETAIL: Process 2667 waits for ShareLock on transaction 72870619; blocked by process 18135.
Process 18135 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 2667.
Process 2667: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 18135: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 320232 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 320232)
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:HINT: See server log for query details.
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:LOG: could not send data to client: Connection timed out
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:FATAL: connection to client lost
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:ERROR: deadlock detected
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:DETAIL: Process 3154 waits for ShareLock on transaction 72870619; blocked by process 18135.
Process 18135 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 3154.
Process 3154: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 18135: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 320232 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 320232)
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:HINT: See server log for query details.
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:LOG: could not send data to client: Connection timed out
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:FATAL: connection to client lost
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:ERROR: deadlock detected
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:DETAIL: Process 9826 waits for ShareLock on transaction 72870619; blocked by process 18135.
Process 18135 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 9826.
Process 9826: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 18135: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 320232 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 320232)
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:HINT: See server log for query details.
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:LOG: could not send data to client: Connection timed out
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:FATAL: connection to client lost
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:ERROR: deadlock detected
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:DETAIL: Process 18215 waits for ShareLock on transaction 72870619; blocked by process 18135.
Process 18135 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 18215.
Process 18215: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 18135: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 320232 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 320232)
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:HINT: See server log for query details.
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:LOG: could not send data to client: Connection timed out
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:FATAL: connection to client lost
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:ERROR: deadlock detected
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:DETAIL: Process 4291 waits for ShareLock on transaction 72870619; blocked by process 18135.
Process 18135 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 4291.
Process 4291: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 18135: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 320232 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 320232)
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:HINT: See server log for query details.
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:LOG: could not send data to client: Connection timed out
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:FATAL: connection to client lost
I can see three general kinds of stack traces:
/opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request.rb:144
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/peek.rb:18:in `async_exec_params'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/peek.rb:18:in `exec_params'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:611:in `block (2 levels) in exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:610:in `block in exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:581:in `block (2 levels) in log'
/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:580:in `block in log'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:571:in `log'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:609:in `exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:596:in `execute_and_clear'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:94:in `exec_delete'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:170:in `update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/query_cache.rb:21:in `update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:197:in `_update_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:696:in `_update_row'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/config_initializers_active_record_locking.rb:13:in `_update_row'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:719:in `_update_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/attribute_methods/dirty.rb:134:in `_update_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:350:in `block in _update_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:132:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:816:in `_run_update_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:350:in `_update_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/timestamp.rb:114:in `_update_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:705:in `create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:342:in `block in create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:132:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:816:in `_run_save_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:342:in `create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:275:in `save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/validations.rb:46:in `save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:310:in `block (2 levels) in save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:387:in `block in with_transaction_returning_status'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:265:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:385:in `with_transaction_returning_status'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:310:in `block in save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:325:in `rollback_active_record_state!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:309:in `save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/suppressor.rb:44:in `save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:509:in `block in save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:528:in `block in around_save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:150:in `block in run_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:170:in `catch_exceptions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:148:in `run_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:133:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:224:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:128:in `block in run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:371:in `before'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:356:in `block (2 levels) in before'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/callback.rb:188:in `run_methods'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/callback.rb:183:in `block in run_methods'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/timeless.rb:12:in `timeless'
/opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request.rb:144:in `block (2 levels) in <class:MergeRequest>'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/eval_helpers.rb:79:in `evaluate_method'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/callback.rb:181:in `run_methods'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/callback.rb:159:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:355:in `block in before'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:354:in `catch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:354:in `before'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:190:in `block in run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:292:in `block in pausable'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:292:in `catch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:292:in `pausable'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:190:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:127:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:212:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:63:in `block (2 levels) in perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:63:in `catch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:63:in `block in perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:182:in `block in within_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:171:in `block in within_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/machine.rb:1868:in `block in within_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:555:in `block in transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `block in transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:554:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/machine.rb:1868:in `within_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:170:in `within_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:181:in `within_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:62:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:528:in `around_save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:509:in `save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:154:in `block (2 levels) in run_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:154:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:154:in `block in run_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:170:in `catch_exceptions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:148:in `run_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:60:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:163:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/event.rb:155:in `fire'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/event.rb:219:in `block in add_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/machine.rb:729:in `block (2 levels) in define_helper'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:112:in `block in reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:105:in `each'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:105:in `reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:24:in `refresh_merge_requests!'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/ee/merge_requests/refresh_service.rb:13:in `block in refresh_merge_requests!'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/ee/merge_requests/refresh_service.rb:43:in `update_approvers'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/ee/merge_requests/refresh_service.rb:12:in `refresh_merge_requests!'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:11:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/app/workers/update_merge_requests_worker.rb:18:in `block in perform'
/opt/gitlab/embedded/lib/ruby/2.6.0/benchmark.rb:293:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/app/workers/update_merge_requests_worker.rb:17:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in `execute_job'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in `block (2 levels) in process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sidekiq_middleware.rb:15:in `block in call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/transaction.rb:57:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sidekiq_middleware.rb:15:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/correlation_logger.rb:10:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/gitlab-labkit-0.3.0/lib/labkit/correlation/correlation_id.rb:18:in `use_id'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/correlation_logger.rb:9:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/memory_killer.rb:18:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in `invoke'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in `block in process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in `local'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:43:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/reloader.rb:73:in `block in wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/reloader.rb:72:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:42:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in `stats'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_logging/structured_logger.rb:16:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in `global'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in `block in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in `with_context'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in `dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in `process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in `process_one'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in `run'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in `watchdog'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in `block in safe_thread'
/opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request_diff.rb:156
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/peek.rb:18:in `async_exec_params'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/peek.rb:18:in `exec_params'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:611:in `block (2 levels) in exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:610:in `block in exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:581:in `block (2 levels) in log'
/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:580:in `block in log'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:571:in `log'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:609:in `exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:596:in `execute_and_clear'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:94:in `exec_delete'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:170:in `update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/query_cache.rb:21:in `update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/relation.rb:337:in `update_all'
/opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request_diff.rb:156:in `save_git_content'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in save_git_content'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `save_git_content'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:426:in `block in make_lambda'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:236:in `block in halting_and_conditional'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:517:in `block in invoke_after'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:517:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:517:in `invoke_after'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:133:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:816:in `_run_create_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:346:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/timestamp.rb:102:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:705:in `create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:342:in `block in create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:132:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:816:in `_run_save_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:342:in `create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:308:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/validations.rb:52:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:315:in `block in save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:387:in `block in with_transaction_returning_status'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:265:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:385:in `with_transaction_returning_status'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:315:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/suppressor.rb:48:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:513:in `block in save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:528:in `block in around_save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:150:in `block in run_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:170:in `catch_exceptions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:148:in `run_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:133:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:212:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:63:in `block (2 levels) in perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:63:in `catch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:63:in `block in perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:186:in `within_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:62:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:528:in `around_save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:513:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:375:in `insert_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/has_many_association.rb:36:in `insert_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:362:in `block (2 levels) in _create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:458:in `replace_on_target'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:283:in `add_to_target'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:361:in `block in _create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:135:in `block in transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `block in transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:134:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:359:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/has_many_association.rb:124:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/association.rb:194:in `create!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_proxy.rb:366:in `create!'
/opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request.rb:698:in `block in create_merge_request_diff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/gitaly_client.rb:284:in `allow_n_plus_1_calls'
/opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request.rb:697:in `create_merge_request_diff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in create_merge_request_diff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `create_merge_request_diff'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/reload_diffs_service.rb:12:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request.rb:753:in `reload_diff'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:107:in `block in reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:105:in `each'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:105:in `reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:24:in `refresh_merge_requests!'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/ee/merge_requests/refresh_service.rb:13:in `block in refresh_merge_requests!'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/ee/merge_requests/refresh_service.rb:43:in `update_approvers'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/ee/merge_requests/refresh_service.rb:12:in `refresh_merge_requests!'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:11:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/app/workers/update_merge_requests_worker.rb:18:in `block in perform'
/opt/gitlab/embedded/lib/ruby/2.6.0/benchmark.rb:293:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/app/workers/update_merge_requests_worker.rb:17:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in `execute_job'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in `block (2 levels) in process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sidekiq_middleware.rb:15:in `block in call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/transaction.rb:57:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sidekiq_middleware.rb:15:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/correlation_logger.rb:10:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/gitlab-labkit-0.3.0/lib/labkit/correlation/correlation_id.rb:18:in `use_id'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/correlation_logger.rb:9:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/memory_killer.rb:18:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in `invoke'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in `block in process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in `local'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:43:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/reloader.rb:73:in `block in wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/reloader.rb:72:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:42:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in `stats'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_logging/structured_logger.rb:16:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in `global'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in `block in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in `with_context'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in `dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in `process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in `process_one'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in `run'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in `watchdog'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in `block in safe_thread'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/approval_rules/finalize_service.rb:29
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/peek.rb:18:in `async_exec_params'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/peek.rb:18:in `exec_params'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:611:in `block (2 levels) in exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:610:in `block in exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:581:in `block (2 levels) in log'
/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:580:in `block in log'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:571:in `log'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:609:in `exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:596:in `execute_and_clear'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:81:in `exec_query'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:130:in `exec_insert'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:115:in `exec_insert'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:162:in `insert'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/query_cache.rb:21:in `insert'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:187:in `_insert_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:734:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/counter_cache.rb:184:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/locking/optimistic.rb:70:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/attribute_methods/dirty.rb:140:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:346:in `block in _create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:98:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:816:in `_run_create_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:346:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/timestamp.rb:102:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:705:in `create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:342:in `block in create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:132:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:816:in `_run_save_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:342:in `create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:308:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/validations.rb:52:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:315:in `block in save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:387:in `block in with_transaction_returning_status'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:265:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:385:in `with_transaction_returning_status'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:315:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/suppressor.rb:48:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/has_many_through_association.rb:84:in `save_through_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/has_many_through_association.rb:45:in `insert_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:439:in `block (2 levels) in concat_records'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:458:in `replace_on_target'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:283:in `add_to_target'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:437:in `block in concat_records'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:435:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:435:in `concat_records'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/has_many_association.rb:117:in `concat_records'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/has_many_through_association.rb:27:in `concat_records'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:120:in `block in concat'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:135:in `block in transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:265:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:134:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:120:in `concat'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/has_many_through_association.rb:21:in `concat'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:415:in `replace_records'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:251:in `block in replace'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:135:in `block in transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:265:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:134:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:251:in `replace'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:41:in `writer'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/builder/association.rb:116:in `users='
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/approval_rules/finalize_service.rb:29:in `block in merge_group_members_into_users'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/relation/delegation.rb:71:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/relation/delegation.rb:71:in `each'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/approval_rules/finalize_service.rb:28:in `merge_group_members_into_users'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/approval_rules/finalize_service.rb:16:in `block in execute'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `block in transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/approval_rules/finalize_service.rb:14:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/ee/merge_requests/post_merge_service.rb:11:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/merge_service.rb:90:in `after_merge'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in after_merge'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `after_merge'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/merge_service.rb:25:in `block in execute'
/opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request.rb:1151:in `in_locked_state'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in in_locked_state'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `in_locked_state'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/merge_service.rb:23:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/app/workers/merge_worker.rb:12:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in `execute_job'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in `block (2 levels) in process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sidekiq_middleware.rb:15:in `block in call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/transaction.rb:57:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sidekiq_middleware.rb:15:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/correlation_logger.rb:10:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/gitlab-labkit-0.3.0/lib/labkit/correlation/correlation_id.rb:18:in `use_id'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/correlation_logger.rb:9:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/memory_killer.rb:18:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in `invoke'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in `block in process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in `local'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:43:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/reloader.rb:73:in `block in wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/reloader.rb:72:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:42:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in `stats'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_logging/structured_logger.rb:16:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in `global'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in `block in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in `with_context'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in `dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in `process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in `process_one'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in `run'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in `watchdog'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in `block in safe_thread'
Output of checks
(If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com)
Results of GitLab environment info
Expand for output related to GitLab environment info
$ sudo gitlab-rake gitlab:env:info
System information System: Ubuntu 16.04 Proxy: no Current User: git Using RVM: no Ruby Version: 2.6.3p62 Gem Version: 2.7.9 Bundler Version:1.17.3 Rake Version: 12.3.2 Redis Version: 3.2.12 Git Version: 2.21.0 Sidekiq Version:5.2.7 Go Version: unknown
GitLab information Version: 12.1.1-ee Revision: b49d6585894 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 10.7 URL: https://example.com HTTP Clone URL: https://example,com/some-group/some-project.git SSH Clone URL: git@example.com:some-group/some-project.git Elasticsearch: yes Geo: no Using LDAP: yes Using Omniauth: yes Omniauth Providers:
GitLab Shell Version: 9.3.0 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
sudo gitlab-rake gitlab:check SANITIZE=true Checking GitLab subtasks ...Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 9.3.0 ? ... OK (9.3.0) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... no # Sidekiq running on separate instance Try fixing it: sudo -u git -H RAILS_ENV=production bin/background_jobs start For more information see: doc/install/installation.md in section "Install Init Script" see log/sidekiq.log for possible errors Please fix the error above and rerun the checks.
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Checking Reply by email ...
IMAP server credentials are correct? ... no # Mail processed on Sidekiq instances Try fixing it: An error occurred: Net::OpenTimeout: Timeout to open TCP connection to mail.gitlab.internal:143 (exceeds 30 seconds) Check that the information in config/gitlab.yml is correct For more information see: doc/administration/reply_by_email.md Please fix the error above and rerun the checks. Init.d configured correctly? ... skipped MailRoom running? ... skipped
Checking Reply by email ... Finished
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... Server: ldapmain LDAP authentication... Success LDAP users with access to your GitLab server (only showing the first 100 results) User output sanitized. Found 100 users of 100 limit.
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... # All yes Redis version >= 2.8.0? ... yes Ruby version >= 2.5.3 ? ... yes (2.6.3) Git version >= 2.21.0 ? ... yes (2.21.0) Git user has default SSH configuration? ... no # alternate login mechanism Try fixing it: mkdir ~/gitlab-check-backup-1564450701 sudo mv /var/opt/gitlab/.ssh/id_rsa.pub ~/gitlab-check-backup-1564450701 sudo mv /var/opt/gitlab/.ssh/id_rsa ~/gitlab-check-backup-1564450701 For more information see: doc/ssh/README.md in section "SSH on the GitLab server" Please fix the error above and rerun the checks. Active users: ... XXXX Elasticsearch version 5.6 - 6.x? ... yes (6.8.1)
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished
Possible fixes
I can't seem to identify where the SELECT... FOR UPDATE
is being issued, or if that's something introduced in Rails 5.2?