Database Deadlock with UpdateMergeRequestsWorker on GitLab 12.1 and Postgres 10.7
### Summary
UpdateMergeRequestsWorker running for hours until Sidekiq is restarted.
### Steps to reproduce
Not entirely sure. We've seen this happen on our most heavily used repositories, but not sure what triggers it.
### Example Project
None at the moment.
### What is the current *bug* behavior?
We see that the UpdateMergeRequestsWorker is blocked on several projects (for up to 7-8 hours). The affected MRs also do not reflect the latest pushed commits.
### What is the expected *correct* behavior?
No blocking should happen.
### Relevant logs and/or screenshots
I have narrowed it down via the Database logs as well as sending SIGTTIN to Sidekiq instances.
```
2019-07-29 11:54:27 UTC:xx.xx.xx.xx(38170):gitlab@gitlabhq_production:[16727]:ERROR: deadlock detected
2019-07-29 11:54:27 UTC:xx.xx.xx.xx(38170):gitlab@gitlabhq_production:[16727]:DETAIL: Process 16727 waits for ShareLock on transaction 72790439; blocked by process 10876.
Process 10876 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 16727.
Process 16727: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 10876: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 319923 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 319923)
2019-07-29 11:54:27 UTC:xx.xx.xx.xx(38170):gitlab@gitlabhq_production:[16727]:HINT: See server log for query details.
2019-07-29 11:54:27 UTC:xx.xx.xx.xx(38170):gitlab@gitlabhq_production:[16727]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:54:27 UTC:xx.xx.xx.xx(38170):gitlab@gitlabhq_production:[16727]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:54:27 UTC:xx.xx.xx.xx(38170):gitlab@gitlabhq_production:[16727]:LOG: could not send data to client: Connection timed out
2019-07-29 11:54:27 UTC:xx.xx.xx.xx(38170):gitlab@gitlabhq_production:[16727]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:55:51 UTC:xx.xx.xx.xx(60424):gitlab@gitlabhq_production:[20410]:ERROR: deadlock detected
2019-07-29 11:55:51 UTC:xx.xx.xx.xx(60424):gitlab@gitlabhq_production:[20410]:DETAIL: Process 20410 waits for ShareLock on transaction 72800923; blocked by process 1254.
Process 1254 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 20410.
Process 20410: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 1254: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 319956 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 319956)
2019-07-29 11:55:51 UTC:xx.xx.xx.xx(60424):gitlab@gitlabhq_production:[20410]:HINT: See server log for query details.
2019-07-29 11:55:51 UTC:xx.xx.xx.xx(60424):gitlab@gitlabhq_production:[20410]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:55:51 UTC:xx.xx.xx.xx(60424):gitlab@gitlabhq_production:[20410]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:55:51 UTC:xx.xx.xx.xx(60424):gitlab@gitlabhq_production:[20410]:LOG: could not send data to client: Connection timed out
2019-07-29 11:55:51 UTC:xx.xx.xx.xx(60424):gitlab@gitlabhq_production:[20410]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:55:52 UTC:xx.xx.xx.xx(54420):gitlab@gitlabhq_production:[24071]:ERROR: deadlock detected
2019-07-29 11:55:52 UTC:xx.xx.xx.xx(54420):gitlab@gitlabhq_production:[24071]:DETAIL: Process 24071 waits for ShareLock on transaction 72800923; blocked by process 1254.
Process 1254 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 24071.
Process 24071: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 1254: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 319956 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 319956)
2019-07-29 11:55:52 UTC:xx.xx.xx.xx(54420):gitlab@gitlabhq_production:[24071]:HINT: See server log for query details.
2019-07-29 11:55:52 UTC:xx.xx.xx.xx(54420):gitlab@gitlabhq_production:[24071]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:55:52 UTC:xx.xx.xx.xx(54420):gitlab@gitlabhq_production:[24071]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:55:52 UTC:xx.xx.xx.xx(54420):gitlab@gitlabhq_production:[24071]:LOG: could not send data to client: Connection timed out
2019-07-29 11:55:52 UTC:xx.xx.xx.xx(54420):gitlab@gitlabhq_production:[24071]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:ERROR: deadlock detected
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:DETAIL: Process 454 waits for ShareLock on transaction 72849258; blocked by process 12706.
Process 12706 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 454.
Process 454: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 12706: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 320144 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 320144)
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:HINT: See server log for query details.
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:LOG: could not send data to client: Connection timed out
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:18 UTC:xx.xx.xx.xx(26510):gitlab@gitlabhq_production:[454]:FATAL: connection to client lost
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:ERROR: deadlock detected
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:DETAIL: Process 25621 waits for ShareLock on transaction 72849258; blocked by process 12706.
Process 12706 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 25621.
Process 25621: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 12706: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 320144 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 320144)
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:HINT: See server log for query details.
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:LOG: could not send data to client: Connection timed out
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:19 UTC:xx.xx.xx.xx(39092):gitlab@gitlabhq_production:[25621]:FATAL: connection to client lost
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:ERROR: deadlock detected
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:DETAIL: Process 2667 waits for ShareLock on transaction 72870619; blocked by process 18135.
Process 18135 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 2667.
Process 2667: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 18135: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 320232 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 320232)
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:HINT: See server log for query details.
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:LOG: could not send data to client: Connection timed out
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:21 UTC:xx.xx.xx.xx(42874):gitlab@gitlabhq_production:[2667]:FATAL: connection to client lost
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:ERROR: deadlock detected
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:DETAIL: Process 3154 waits for ShareLock on transaction 72870619; blocked by process 18135.
Process 18135 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 3154.
Process 3154: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 18135: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 320232 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 320232)
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:HINT: See server log for query details.
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:LOG: could not send data to client: Connection timed out
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:22 UTC:xx.xx.xx.xx(43306):gitlab@gitlabhq_production:[3154]:FATAL: connection to client lost
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:ERROR: deadlock detected
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:DETAIL: Process 9826 waits for ShareLock on transaction 72870619; blocked by process 18135.
Process 18135 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 9826.
Process 9826: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 18135: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 320232 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 320232)
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:HINT: See server log for query details.
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:LOG: could not send data to client: Connection timed out
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:23 UTC:xx.xx.xx.xx(45926):gitlab@gitlabhq_production:[9826]:FATAL: connection to client lost
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:ERROR: deadlock detected
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:DETAIL: Process 18215 waits for ShareLock on transaction 72870619; blocked by process 18135.
Process 18135 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 18215.
Process 18215: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 18135: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 320232 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 320232)
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:HINT: See server log for query details.
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:LOG: could not send data to client: Connection timed out
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:24 UTC:xx.xx.xx.xx(10504):gitlab@gitlabhq_production:[18215]:FATAL: connection to client lost
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:ERROR: deadlock detected
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:DETAIL: Process 4291 waits for ShareLock on transaction 72870619; blocked by process 18135.
Process 18135 waits for ExclusiveLock on tuple (14143,9) of relation 19362 of database 16400; blocked by process 4291.
Process 4291: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
Process 18135: UPDATE "merge_requests" SET "latest_merge_request_diff_id" = 320232 WHERE (id = 146781 AND COALESCE(latest_merge_request_diff_id, 0) < 320232)
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:HINT: See server log for query details.
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:CONTEXT: while locking tuple (14143,9) in relation "merge_requests"
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:LOG: could not send data to client: Connection timed out
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:STATEMENT: SELECT "merge_requests".* FROM "merge_requests" WHERE "merge_requests"."id" = 146781 LIMIT 1 FOR UPDATE
2019-07-29 11:56:25 UTC:xx.xx.xx.xx(18098):gitlab@gitlabhq_production:[4291]:FATAL: connection to client lost
```
I can see three general kinds of stack traces:
#### /opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request.rb:144
```
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/peek.rb:18:in `async_exec_params'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/peek.rb:18:in `exec_params'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:611:in `block (2 levels) in exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:610:in `block in exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:581:in `block (2 levels) in log'
/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:580:in `block in log'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:571:in `log'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:609:in `exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:596:in `execute_and_clear'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:94:in `exec_delete'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:170:in `update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/query_cache.rb:21:in `update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:197:in `_update_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:696:in `_update_row'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/config_initializers_active_record_locking.rb:13:in `_update_row'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:719:in `_update_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/attribute_methods/dirty.rb:134:in `_update_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:350:in `block in _update_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:132:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:816:in `_run_update_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:350:in `_update_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/timestamp.rb:114:in `_update_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:705:in `create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:342:in `block in create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:132:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:816:in `_run_save_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:342:in `create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:275:in `save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/validations.rb:46:in `save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:310:in `block (2 levels) in save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:387:in `block in with_transaction_returning_status'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:265:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:385:in `with_transaction_returning_status'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:310:in `block in save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:325:in `rollback_active_record_state!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:309:in `save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/suppressor.rb:44:in `save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:509:in `block in save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:528:in `block in around_save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:150:in `block in run_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:170:in `catch_exceptions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:148:in `run_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:133:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:224:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:128:in `block in run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:371:in `before'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:356:in `block (2 levels) in before'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/callback.rb:188:in `run_methods'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/callback.rb:183:in `block in run_methods'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/timeless.rb:12:in `timeless'
/opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request.rb:144:in `block (2 levels) in <class:MergeRequest>'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/eval_helpers.rb:79:in `evaluate_method'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/callback.rb:181:in `run_methods'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/callback.rb:159:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:355:in `block in before'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:354:in `catch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:354:in `before'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:190:in `block in run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:292:in `block in pausable'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:292:in `catch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:292:in `pausable'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:190:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:127:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:212:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:63:in `block (2 levels) in perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:63:in `catch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:63:in `block in perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:182:in `block in within_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:171:in `block in within_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/machine.rb:1868:in `block in within_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:555:in `block in transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `block in transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:554:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/machine.rb:1868:in `within_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:170:in `within_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:181:in `within_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:62:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:528:in `around_save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:509:in `save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:154:in `block (2 levels) in run_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:154:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:154:in `block in run_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:170:in `catch_exceptions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:148:in `run_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:60:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition.rb:163:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/event.rb:155:in `fire'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/event.rb:219:in `block in add_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/machine.rb:729:in `block (2 levels) in define_helper'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:112:in `block in reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:105:in `each'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:105:in `reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:24:in `refresh_merge_requests!'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/ee/merge_requests/refresh_service.rb:13:in `block in refresh_merge_requests!'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/ee/merge_requests/refresh_service.rb:43:in `update_approvers'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/ee/merge_requests/refresh_service.rb:12:in `refresh_merge_requests!'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:11:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/app/workers/update_merge_requests_worker.rb:18:in `block in perform'
/opt/gitlab/embedded/lib/ruby/2.6.0/benchmark.rb:293:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/app/workers/update_merge_requests_worker.rb:17:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in `execute_job'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in `block (2 levels) in process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sidekiq_middleware.rb:15:in `block in call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/transaction.rb:57:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sidekiq_middleware.rb:15:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/correlation_logger.rb:10:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/gitlab-labkit-0.3.0/lib/labkit/correlation/correlation_id.rb:18:in `use_id'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/correlation_logger.rb:9:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/memory_killer.rb:18:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in `invoke'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in `block in process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in `local'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:43:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/reloader.rb:73:in `block in wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/reloader.rb:72:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:42:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in `stats'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_logging/structured_logger.rb:16:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in `global'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in `block in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in `with_context'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in `dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in `process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in `process_one'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in `run'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in `watchdog'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in `block in safe_thread'
```
#### /opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request_diff.rb:156
```
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/peek.rb:18:in `async_exec_params'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/peek.rb:18:in `exec_params'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:611:in `block (2 levels) in exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:610:in `block in exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:581:in `block (2 levels) in log'
/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:580:in `block in log'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:571:in `log'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:609:in `exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:596:in `execute_and_clear'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:94:in `exec_delete'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:170:in `update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/query_cache.rb:21:in `update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/relation.rb:337:in `update_all'
/opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request_diff.rb:156:in `save_git_content'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in save_git_content'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `save_git_content'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:426:in `block in make_lambda'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:236:in `block in halting_and_conditional'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:517:in `block in invoke_after'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:517:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:517:in `invoke_after'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:133:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:816:in `_run_create_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:346:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/timestamp.rb:102:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:705:in `create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:342:in `block in create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:132:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:816:in `_run_save_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:342:in `create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:308:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/validations.rb:52:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:315:in `block in save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:387:in `block in with_transaction_returning_status'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:265:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:385:in `with_transaction_returning_status'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:315:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/suppressor.rb:48:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:513:in `block in save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:528:in `block in around_save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:150:in `block in run_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:170:in `catch_exceptions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:148:in `run_actions'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:133:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:212:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:63:in `block (2 levels) in perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:63:in `catch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:63:in `block in perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:186:in `within_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-0.5.0/lib/state_machines/transition_collection.rb:62:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:528:in `around_save'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/state_machines-activerecord-0.5.1/lib/state_machines/integrations/active_record.rb:513:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:375:in `insert_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/has_many_association.rb:36:in `insert_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:362:in `block (2 levels) in _create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:458:in `replace_on_target'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:283:in `add_to_target'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:361:in `block in _create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:135:in `block in transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `block in transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:134:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:359:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/has_many_association.rb:124:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/association.rb:194:in `create!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_proxy.rb:366:in `create!'
/opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request.rb:698:in `block in create_merge_request_diff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/gitaly_client.rb:284:in `allow_n_plus_1_calls'
/opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request.rb:697:in `create_merge_request_diff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in create_merge_request_diff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `create_merge_request_diff'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/reload_diffs_service.rb:12:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request.rb:753:in `reload_diff'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:107:in `block in reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:105:in `each'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:105:in `reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `reload_merge_requests'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:24:in `refresh_merge_requests!'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/ee/merge_requests/refresh_service.rb:13:in `block in refresh_merge_requests!'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/ee/merge_requests/refresh_service.rb:43:in `update_approvers'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/ee/merge_requests/refresh_service.rb:12:in `refresh_merge_requests!'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/refresh_service.rb:11:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/app/workers/update_merge_requests_worker.rb:18:in `block in perform'
/opt/gitlab/embedded/lib/ruby/2.6.0/benchmark.rb:293:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/app/workers/update_merge_requests_worker.rb:17:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in `execute_job'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in `block (2 levels) in process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sidekiq_middleware.rb:15:in `block in call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/transaction.rb:57:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sidekiq_middleware.rb:15:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/correlation_logger.rb:10:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/gitlab-labkit-0.3.0/lib/labkit/correlation/correlation_id.rb:18:in `use_id'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/correlation_logger.rb:9:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/memory_killer.rb:18:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in `invoke'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in `block in process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in `local'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:43:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/reloader.rb:73:in `block in wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/reloader.rb:72:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:42:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in `stats'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_logging/structured_logger.rb:16:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in `global'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in `block in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in `with_context'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in `dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in `process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in `process_one'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in `run'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in `watchdog'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in `block in safe_thread'
```
#### /opt/gitlab/embedded/service/gitlab-rails/ee/app/services/approval_rules/finalize_service.rb:29
```
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/peek.rb:18:in `async_exec_params'
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/peek.rb:18:in `exec_params'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:611:in `block (2 levels) in exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:610:in `block in exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:581:in `block (2 levels) in log'
/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:580:in `block in log'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/notifications/instrumenter.rb:23:in `instrument'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract_adapter.rb:571:in `log'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:609:in `exec_no_cache'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql_adapter.rb:596:in `execute_and_clear'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:81:in `exec_query'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:130:in `exec_insert'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/postgresql/database_statements.rb:115:in `exec_insert'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:162:in `insert'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/query_cache.rb:21:in `insert'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:187:in `_insert_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:734:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/counter_cache.rb:184:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/locking/optimistic.rb:70:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/attribute_methods/dirty.rb:140:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:346:in `block in _create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:98:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:816:in `_run_create_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:346:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/timestamp.rb:102:in `_create_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:705:in `create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:342:in `block in create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:132:in `run_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/callbacks.rb:816:in `_run_save_callbacks'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/callbacks.rb:342:in `create_or_update'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/persistence.rb:308:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/validations.rb:52:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:315:in `block in save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:387:in `block in with_transaction_returning_status'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:265:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:385:in `with_transaction_returning_status'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:315:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/suppressor.rb:48:in `save!'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/has_many_through_association.rb:84:in `save_through_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/has_many_through_association.rb:45:in `insert_record'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:439:in `block (2 levels) in concat_records'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:458:in `replace_on_target'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:283:in `add_to_target'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:437:in `block in concat_records'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:435:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:435:in `concat_records'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/has_many_association.rb:117:in `concat_records'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/has_many_through_association.rb:27:in `concat_records'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:120:in `block in concat'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:135:in `block in transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:265:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:134:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:120:in `concat'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/has_many_through_association.rb:21:in `concat'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:415:in `replace_records'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:251:in `block in replace'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:135:in `block in transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:265:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:134:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:251:in `replace'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/collection_association.rb:41:in `writer'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/associations/builder/association.rb:116:in `users='
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/approval_rules/finalize_service.rb:29:in `block in merge_group_members_into_users'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/relation/delegation.rb:71:in `each'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/relation/delegation.rb:71:in `each'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/approval_rules/finalize_service.rb:28:in `merge_group_members_into_users'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/approval_rules/finalize_service.rb:16:in `block in execute'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `block in transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/transaction.rb:239:in `block in within_new_transaction'
/opt/gitlab/embedded/lib/ruby/2.6.0/monitor.rb:230:in `mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/transaction.rb:236:in `within_new_transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/connection_adapters/abstract/database_statements.rb:267:in `transaction'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activerecord-5.2.3/lib/active_record/transactions.rb:212:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/approval_rules/finalize_service.rb:14:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/ee/app/services/ee/merge_requests/post_merge_service.rb:11:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/merge_service.rb:90:in `after_merge'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in after_merge'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `after_merge'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/merge_service.rb:25:in `block in execute'
/opt/gitlab/embedded/service/gitlab-rails/app/models/merge_request.rb:1151:in `in_locked_state'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in in_locked_state'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `in_locked_state'
/opt/gitlab/embedded/service/gitlab-rails/app/services/merge_requests/merge_service.rb:23:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `block in execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/method_call.rb:36:in `measure'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/instrumentation.rb:161:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/app/workers/merge_worker.rb:12:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in `execute_job'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in `block (2 levels) in process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sidekiq_middleware.rb:15:in `block in call'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/transaction.rb:57:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sidekiq_middleware.rb:15:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/correlation_logger.rb:10:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/gitlab-labkit-0.3.0/lib/labkit/correlation/correlation_id.rb:18:in `use_id'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/correlation_logger.rb:9:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/request_store_middleware.rb:8:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/memory_killer.rb:18:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in `invoke'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in `block in process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in `local'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:43:in `block in call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/reloader.rb:73:in `block in wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/execution_wrapper.rb:87:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/activesupport-5.2.3/lib/active_support/reloader.rb:72:in `wrap'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/rails.rb:42:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in `stats'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_logging/structured_logger.rb:16:in `call'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in `global'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in `block in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in `with_context'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in `dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in `process'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in `process_one'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in `run'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in `watchdog'
/opt/gitlab/embedded/lib/ruby/gems/2.6.0/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in `block in safe_thread'
```
### Output of checks
(If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com)
#### Results of GitLab environment info
<details>
<summary>Expand for output related to GitLab environment info</summary>
<pre>
$ sudo gitlab-rake gitlab:env:info
System information
System: Ubuntu 16.04
Proxy: no
Current User: git
Using RVM: no
Ruby Version: 2.6.3p62
Gem Version: 2.7.9
Bundler Version:1.17.3
Rake Version: 12.3.2
Redis Version: 3.2.12
Git Version: 2.21.0
Sidekiq Version:5.2.7
Go Version: unknown
GitLab information
Version: 12.1.1-ee
Revision: b49d6585894
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: PostgreSQL
DB Version: 10.7
URL: https://example.com
HTTP Clone URL: https://example,com/some-group/some-project.git
SSH Clone URL: git@example.com:some-group/some-project.git
Elasticsearch: yes
Geo: no
Using LDAP: yes
Using Omniauth: yes
Omniauth Providers:
GitLab Shell
Version: 9.3.0
Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories
GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell
Git: /opt/gitlab/embedded/bin/git
</pre>
</details>
#### Results of GitLab application Check
<details>
<summary>Expand for output related to the GitLab application check</summary>
<pre>
sudo gitlab-rake gitlab:check SANITIZE=true
Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 9.3.0 ? ... OK (9.3.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... no # Sidekiq running on separate instance
Try fixing it:
sudo -u git -H RAILS_ENV=production bin/background_jobs start
For more information see:
doc/install/installation.md in section "Install Init Script"
see log/sidekiq.log for possible errors
Please fix the error above and rerun the checks.
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Checking Reply by email ...
IMAP server credentials are correct? ... no # Mail processed on Sidekiq instances
Try fixing it:
An error occurred: Net::OpenTimeout: Timeout to open TCP connection to mail.gitlab.internal:143 (exceeds 30 seconds)
Check that the information in config/gitlab.yml is correct
For more information see:
doc/administration/reply_by_email.md
Please fix the error above and rerun the checks.
Init.d configured correctly? ... skipped
MailRoom running? ... skipped
Checking Reply by email ... Finished
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... Server: ldapmain
LDAP authentication... Success
LDAP users with access to your GitLab server (only showing the first 100 results)
User output sanitized. Found 100 users of 100 limit.
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ... # All yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.6.3)
Git version >= 2.21.0 ? ... yes (2.21.0)
Git user has default SSH configuration? ... no # alternate login mechanism
Try fixing it:
mkdir ~/gitlab-check-backup-1564450701
sudo mv /var/opt/gitlab/.ssh/id_rsa.pub ~/gitlab-check-backup-1564450701
sudo mv /var/opt/gitlab/.ssh/id_rsa ~/gitlab-check-backup-1564450701
For more information see:
doc/ssh/README.md in section "SSH on the GitLab server"
Please fix the error above and rerun the checks.
Active users: ... XXXX
Elasticsearch version 5.6 - 6.x? ... yes (6.8.1)
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished
</pre>
</details>
### Possible fixes
I can't seem to identify where the `SELECT... FOR UPDATE` is being issued, or if that's something introduced in Rails 5.2?
issue