After merging a large MR and then immediately clicking `Revert`, request to branches is pending for 30s
Summary
This causes the E2E test to fail for reverting a large MR. This can be seen when running this test against the GDK, although sometimes it will pass as it completes in around ~55 seconds.
The request to branches was pending
for around 30 seconds
And then the following requests are fired and take ~20-30 seconds
The issue seems to be caused by clicking merge and then immediately clicking revert, if we do this, the call to /branches
will be pending for a ~30 seconds. If you wait a few seconds and click Revert
the request to branches will complete very quickly.
Failing job(s): https://gitlab.com/gitlab-org/gitlab/-/jobs/5787267965
Failing spec(s): browser_ui/3_create/merge_request/revert/reverting_merge_request_spec.rb
Corresponding test case(s): https://gitlab.com/gitlab-org/gitlab/-/quality/test_cases/347709
Other instances of this failure
- 2023-12-06: https://gitlab.com/gitlab-org/gitlab/-/jobs/5697468103 (https://gitlab.com/gitlab-org/gitlab/-/pipelines/1098023695)
- 2023-12-09: https://gitlab.com/gitlab-org/gitlab/-/jobs/5716533027 (https://gitlab.com/gitlab-org/gitlab/-/pipelines/1101031899)
- 2023-12-09: https://gitlab.com/gitlab-org/gitlab/-/jobs/5718052133 (https://gitlab.com/gitlab-org/gitlab/-/pipelines/1101358845)
- 2023-12-18: https://gitlab.com/gitlab-org/gitlab/-/jobs/5771537530 (https://gitlab.com/gitlab-org/gitlab/-/pipelines/1110628603)
Stack trace
Failure/Error: merge_request.revert_change!
Net::ReadTimeout:
Net::ReadTimeout with #<TCPSocket:(closed)>
Screenshot / HTML page
Logs
./gitlab-ee-f54100bd/logs/sidekiq/current:{"severity":"ERROR","time":"2023-12-19T17:43:22.585Z","worker":"MergeRequestMergeabilityCheckWorker","message":"Failed to check mergeability of merge request: Failed to obtain a lock","merge_request_id":2,"retry":0}
./gitlab-ee-f54100bd/logs/sidekiq/current:{"severity":"ERROR","time":"2023-12-19T17:43:22.749Z","worker":"MergeRequestMergeabilityCheckWorker","message":"Failed to check mergeability of merge request: Failed to obtain a lock","merge_request_id":2,"retry":0}
./gitlab-ee-f54100bd/logs/sidekiq/current:{"severity":"ERROR","time":"2023-12-19T17:50:56.805Z","worker":"MergeRequestMergeabilityCheckWorker","message":"Failed to check mergeability of merge request: Failed to obtain a lock","merge_request_id":3,"retry":0}
./gitlab-ee-f54100bd/logs/sidekiq/current:{"severity":"ERROR","time":"2023-12-19T17:50:56.927Z","worker":"MergeRequestMergeabilityCheckWorker","message":"Failed to check mergeability of merge request: Failed to obtain a lock","merge_request_id":3,"retry":0}
./gitlab-ee-f54100bd/logs/sidekiq/current:{"severity":"ERROR","time":"2023-12-19T17:52:27.697Z","worker":"MergeRequestMergeabilityCheckWorker","message":"Failed to check mergeability of merge request: Failed to obtain a lock","merge_request_id":4,"retry":0}
./gitlab-ee-f54100bd/logs/sidekiq/current:{"severity":"ERROR","time":"2023-12-19T17:52:27.909Z","worker":"MergeRequestMergeabilityCheckWorker","message":"Failed to check mergeability of merge request: Failed to obtain a lock","merge_request_id":4,"retry":0}
./gitlab-ee-f54100bd/logs/gitlab-rails/application_json.log:{"severity":"ERROR","time":"2023-12-19T17:34:55.697Z","correlation_id":"01HJ1JA5SZ4FJAGD1GJFDA3QRR","meta.caller_id":"POST /api/:version/groups","meta.remote_ip":"172.19.0.4","meta.feature_category":"groups_and_projects","meta.user":"root","meta.user_id":1,"meta.client_id":"user/1","message":"Cannot obtain an exclusive lease. There must be another instance already in execution."}
./gitlab-ee-f54100bd/logs/gitlab-rails/application_json.log:{"severity":"ERROR","time":"2023-12-19T17:34:56.648Z","correlation_id":"01HJ1JA6S0HHRZ9PJZB8P1JPFH","meta.caller_id":"POST /api/:version/groups","meta.remote_ip":"172.19.0.4","meta.feature_category":"groups_and_projects","meta.user":"root","meta.user_id":1,"meta.client_id":"user/1","message":"Cannot obtain an exclusive lease. There must be another instance already in execution."}
./gitlab-ee-f54100bd/logs/gitlab-rails/application_json.log:{"severity":"ERROR","time":"2023-12-19T17:35:03.967Z","correlation_id":"01HJ1JABX6NS664Y1CHG7BPAK0","meta.caller_id":"RepositoryImportWorker","meta.remote_ip":"172.19.0.5","meta.feature_category":"importers","meta.user":"root","meta.user_id":1,"meta.project":"gitlab-qa-sandbox-group-3/qa-test-2023-12-19-17-34-55-9bfdfb58b2cd2f54/importedproject-e5a4c2f27b9bead3","meta.root_namespace":"gitlab-qa-sandbox-group-3","meta.client_id":"user/1","meta.root_caller_id":"Import::GitlabProjectsController#create","message":"Cannot obtain an exclusive lease. There must be another instance already in execution."}
./gitlab-ee-f54100bd/logs/gitlab-rails/application_json.log:{"severity":"ERROR","time":"2023-12-19T17:35:32.082Z","correlation_id":"01HJ1JABX6NS664Y1CHG7BPAK0","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"172.19.0.5","meta.feature_category":"source_code_management","meta.user":"root","meta.user_id":1,"meta.project":"gitlab-qa-sandbox-group-3/qa-test-2023-12-19-17-34-55-9bfdfb58b2cd2f54/importedproject-e5a4c2f27b9bead3","meta.root_namespace":"gitlab-qa-sandbox-group-3","meta.client_id":"user/1","meta.root_caller_id":"Import::GitlabProjectsController#create","message":"Cannot obtain an exclusive lease. There must be another instance already in execution.","lease_key":"namespace:namespaces_root_statistics:2","class_name":"Namespace::AggregationSchedule","lease_timeout":300}
./gitlab-ee-f54100bd/logs/gitlab-rails/application_json.log:{"severity":"ERROR","time":"2023-12-19T17:43:16.253Z","correlation_id":"01HJ1JR968NTF7DQER2224BCJ9","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"172.19.0.5","meta.feature_category":"source_code_management","meta.user":"root","meta.user_id":1,"meta.project":"gitlab-qa-sandbox-group-3/qa-test-2023-12-19-17-34-55-9bfdfb58b2cd2f54/importedproject-7323c3e6bdb123bc","meta.root_namespace":"gitlab-qa-sandbox-group-3","meta.client_id":"user/1","meta.root_caller_id":"Import::GitlabProjectsController#create","message":"Cannot obtain an exclusive lease. There must be another instance already in execution.","lease_key":"namespace:namespaces_root_statistics:2","class_name":"Namespace::AggregationSchedule","lease_timeout":300}
./gitlab-ee-f54100bd/logs/gitlab-rails/application_json.log:{"severity":"ERROR","time":"2023-12-19T17:50:05.855Z","correlation_id":"01HJ1K5XKV3HEV054Y2CR3VRRQ","meta.caller_id":"RepositoryImportWorker","meta.remote_ip":"172.19.0.5","meta.feature_category":"importers","meta.user":"root","meta.user_id":1,"meta.project":"gitlab-qa-sandbox-group-3/qa-test-2023-12-19-17-34-55-9bfdfb58b2cd2f54/importedproject-182102d9513a3426","meta.root_namespace":"gitlab-qa-sandbox-group-3","meta.client_id":"user/1","meta.root_caller_id":"Import::GitlabProjectsController#create","message":"Cannot obtain an exclusive lease. There must be another instance already in execution."}
./gitlab-ee-f54100bd/logs/gitlab-rails/application_json.log:{"severity":"ERROR","time":"2023-12-19T17:50:34.174Z","correlation_id":"f48dbd3e00a6f9c14ebd9dd328929c24","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"172.19.0.5","meta.feature_category":"source_code_management","meta.user":"root","meta.user_id":1,"meta.project":"gitlab-qa-sandbox-group-3/qa-test-2023-12-19-17-34-55-9bfdfb58b2cd2f54/importedproject-e5a4c2f27b9bead3","meta.root_namespace":"gitlab-qa-sandbox-group-3","meta.client_id":"user/1","meta.root_caller_id":"Import::GitlabProjectsController#create","message":"Cannot obtain an exclusive lease. There must be another instance already in execution.","lease_key":"namespace:namespaces_root_statistics:2","class_name":"Namespace::AggregationSchedule","lease_timeout":300}
./gitlab-ee-f54100bd/logs/gitlab-rails/application_json.log:{"severity":"ERROR","time":"2023-12-19T17:51:30.688Z","correlation_id":"01HJ1K8G3083F062FKS7GBBYN3","meta.caller_id":"RepositoryImportWorker","meta.remote_ip":"172.19.0.5","meta.feature_category":"importers","meta.user":"root","meta.user_id":1,"meta.project":"gitlab-qa-sandbox-group-3/qa-test-2023-12-19-17-34-55-9bfdfb58b2cd2f54/importedproject-8496866ce9549d51","meta.root_namespace":"gitlab-qa-sandbox-group-3","meta.client_id":"user/1","meta.root_caller_id":"Import::GitlabProjectsController#create","message":"Cannot obtain an exclusive lease. There must be another instance already in execution."}