Pipeline view broken in version 10.7.1 (and some other dependent pages like MergeRequest)
<!--- Please read this! Before opening a new issue, make sure to search for keywords in the issues filtered by the "regression" or "bug" label. For the Community Edition issue tracker: - https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name%5B%5D=regression - https://gitlab.com/gitlab-org/gitlab-ce/issues?label_name%5B%5D=bug For the Enterprise Edition issue tracker: - https://gitlab.com/gitlab-org/gitlab-ee/issues?label_name%5B%5D=regression - https://gitlab.com/gitlab-org/gitlab-ee/issues?label_name%5B%5D=bug and verify the issue you're about to submit isn't a duplicate. ---> ### Summary After updating to version 10.7.1, the pipeline page failed in timeout. Same thing when opening a MergeRequest (error 503). ### Steps to reproduce Upgrade Gitlab-CE from 10.7.0 to 10.7.1, wait few minutes (the problem doesn't occur from the beginning, but after a while) then open the Pipelines page of any project (see screenshot), or the detail of a merge request (error 503). ### What is the current *bug* behavior? Timeout or error 503 ### What is the expected *correct* behavior? Display the list of Pipelines ### Relevant logs and/or screenshots ![image](/uploads/9a0033488ada9d35b5b484b1d3f86e8b/image.png) An error occurring frequently in log: ``` Gitlab::Git::CommandError (4:Deadline Exceeded): lib/gitlab/git/repository.rb:1376:in `rescue in gitaly_migrate' lib/gitlab/git/repository.rb:1369:in `gitaly_migrate' lib/gitlab/git/commit.rb:234:in `batch_by_oid' app/models/repository.rb:114:in `commits_by' app/models/commit.rb:102:in `block (2 levels) in lazy' app/models/commit.rb:99:in `each' app/models/commit.rb:99:in `block in lazy' app/models/ci/pipeline.rb:346:in `latest?' app/serializers/base_serializer.rb:14:in `represent' app/serializers/merge_request_serializer.rb:14:in `represent' app/helpers/issuables_helper.rb:45:in `serialize_issuable' app/views/projects/merge_requests/show.html.haml:21:in `_app_views_projects_merge_requests_show_html_haml___1245166347478095933_69981244587480' app/controllers/projects/merge_requests_controller.rb:53:in `block (2 levels) in show' app/controllers/projects/merge_requests_controller.rb:38:in `show' lib/gitlab/i18n.rb:50:in `with_locale' lib/gitlab/i18n.rb:56:in `with_user_locale' app/controllers/application_controller.rb:326:in `set_locale' lib/gitlab/middleware/multipart.rb:95:in `call' lib/gitlab/request_profiler/middleware.rb:14:in `call' lib/gitlab/middleware/go.rb:17:in `call' lib/gitlab/etag_caching/middleware.rb:11:in `call' lib/gitlab/middleware/read_only/controller.rb:28:in `call' lib/gitlab/middleware/read_only.rb:16:in `call' lib/gitlab/request_context.rb:18:in `call' lib/gitlab/metrics/requests_rack_middleware.rb:27:in `call' lib/gitlab/middleware/release_env.rb:10:in `call' Completed 503 Service Unavailable in 30081ms (Views: 0.5ms | ActiveRecord: 13.8ms) ``` #### Results of GitLab environment info <details> <summary>Expand for output related to GitLab environment info</summary> <pre> System information System: Ubuntu 14.04 Current User: git Using RVM: no Ruby Version: 2.3.6p384 Gem Version: 2.6.13 Bundler Version:1.13.7 Rake Version: 12.3.0 Redis Version: 3.2.11 Git Version: 2.14.3 Sidekiq Version:5.0.5 Go Version: unknown GitLab information Version: 10.7.1 Revision: 0d49bb8 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: http://gitlab.akema.fr HTTP Clone URL: http://gitlab.akema.fr/some-group/some-project.git SSH Clone URL: ssh://git@gitlab.akema.fr:33622/some-group/some-project.git Using LDAP: yes Using Omniauth: no GitLab Shell Version: 7.1.2 Repository storage paths: - default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git </pre> </details> #### Results of GitLab application Check <details> <summary>Expand for output related to the GitLab application check</summary> <pre> Checking GitLab Shell ... GitLab Shell version >= 7.1.2 ? ... OK (7.1.2) Repo base directory exists? default... yes Repo storage directories are symlinks? default... no Repo paths owned by git:root, or git:git? default... yes Repo paths access is drwxrws---? default... yes hooks directories in repos are links: ... 2/1 ... ok 5/16 ... ok 5/17 ... ok 5/19 ... ok 8/21 ... ok 31/22 ... ok 8/23 ... ok 16/25 ... ok 7/26 ... ok 8/27 ... ok 7/29 ... ok 8/30 ... ok 8/31 ... ok 16/32 ... ok 16/33 ... ok 8/36 ... ok 5/37 ... ok 14/39 ... ok 14/43 ... ok 18/44 ... ok 7/45 ... ok 2/47 ... ok 8/52 ... ok 8/53 ... ok 14/54 ... ok 20/55 ... ok 21/56 ... ok 22/57 ... ok 22/59 ... ok 30/61 ... repository is empty 8/62 ... repository is empty 14/63 ... repository is empty 14/64 ... ok 22/66 ... ok 14/67 ... ok 31/69 ... ok 8/70 ... ok 16/71 ... ok 12/72 ... ok 14/73 ... repository is empty 14/74 ... ok 14/75 ... ok 14/76 ... ok 14/77 ... ok 43/78 ... ok 5/79 ... ok 44/80 ... ok 8/81 ... ok 46/82 ... ok 46/83 ... ok 46/84 ... ok 5/87 ... ok 48/89 ... ok 8/90 ... repository is empty 5/91 ... ok 51/93 ... ok 8/94 ... ok 54/95 ... repository is empty 54/96 ... ok 8/97 ... ok 56/98 ... ok Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Redis available via internal API: OK Access to /var/opt/gitlab/.ssh/authorized_keys: OK gitlab-shell self-check successful Checking GitLab Shell ... Finished Checking Sidekiq ... Running? ... yes Number of Sidekiq processes ... 1 Checking Sidekiq ... Finished Checking Reply by email ... IMAP server credentials are correct? ... yes Init.d configured correctly? ... skipped MailRoom running? ... skipped Checking Reply by email ... Finished Checking LDAP ... Server: ldapmain LDAP authentication... Anonymous. No `bind_dn` or `password` configured LDAP users with access to your GitLab server (only showing the first 100 results) [... removed ....] Checking LDAP ... Finished Checking GitLab ... Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 2/1 ... yes 5/16 ... yes 5/17 ... yes 5/19 ... yes 8/21 ... yes 31/22 ... yes 8/23 ... yes 16/25 ... yes 7/26 ... yes 8/27 ... yes 7/29 ... yes 8/30 ... yes 8/31 ... yes 16/32 ... yes 16/33 ... yes 8/36 ... yes 5/37 ... yes 14/39 ... yes 14/43 ... yes 18/44 ... yes 7/45 ... yes 2/47 ... yes 8/52 ... yes 8/53 ... yes 14/54 ... yes 20/55 ... yes 21/56 ... yes 22/57 ... yes 22/59 ... yes 30/61 ... yes 8/62 ... yes 14/63 ... yes 14/64 ... yes 22/66 ... yes 14/67 ... yes 31/69 ... yes 8/70 ... yes 16/71 ... yes 12/72 ... yes 14/73 ... yes 14/74 ... yes 14/75 ... yes 14/76 ... yes 14/77 ... yes 43/78 ... yes 5/79 ... yes 44/80 ... yes 8/81 ... yes 46/82 ... yes 46/83 ... yes 46/84 ... yes 5/87 ... yes 48/89 ... yes 8/90 ... yes 5/91 ... yes 51/93 ... yes 8/94 ... yes 54/95 ... yes 54/96 ... yes 8/97 ... yes 56/98 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.3.5 ? ... yes (2.3.6) Git version >= 2.9.5 ? ... yes (2.14.3) Git user has default SSH configuration? ... yes Active users: ... 21 Checking GitLab ... Finished </pre> </details> ### Possible fixes Just downgrade to 10.7.0 and all is fine
issue