Pipeline view broken in version 10.7.1 (and some other dependent pages like MergeRequest)
Summary
After updating to version 10.7.1, the pipeline page failed in timeout. Same thing when opening a MergeRequest (error 503).
Steps to reproduce
Upgrade Gitlab-CE from 10.7.0 to 10.7.1, wait few minutes (the problem doesn't occur from the beginning, but after a while) then open the Pipelines page of any project (see screenshot), or the detail of a merge request (error 503).
What is the current bug behavior?
Timeout or error 503
What is the expected correct behavior?
Display the list of Pipelines
Relevant logs and/or screenshots
An error occurring frequently in log:
Gitlab::Git::CommandError (4:Deadline Exceeded):
lib/gitlab/git/repository.rb:1376:in `rescue in gitaly_migrate'
lib/gitlab/git/repository.rb:1369:in `gitaly_migrate'
lib/gitlab/git/commit.rb:234:in `batch_by_oid'
app/models/repository.rb:114:in `commits_by'
app/models/commit.rb:102:in `block (2 levels) in lazy'
app/models/commit.rb:99:in `each'
app/models/commit.rb:99:in `block in lazy'
app/models/ci/pipeline.rb:346:in `latest?'
app/serializers/base_serializer.rb:14:in `represent'
app/serializers/merge_request_serializer.rb:14:in `represent'
app/helpers/issuables_helper.rb:45:in `serialize_issuable'
app/views/projects/merge_requests/show.html.haml:21:in `_app_views_projects_merge_requests_show_html_haml___1245166347478095933_69981244587480'
app/controllers/projects/merge_requests_controller.rb:53:in `block (2 levels) in show'
app/controllers/projects/merge_requests_controller.rb:38:in `show'
lib/gitlab/i18n.rb:50:in `with_locale'
lib/gitlab/i18n.rb:56:in `with_user_locale'
app/controllers/application_controller.rb:326:in `set_locale'
lib/gitlab/middleware/multipart.rb:95:in `call'
lib/gitlab/request_profiler/middleware.rb:14:in `call'
lib/gitlab/middleware/go.rb:17:in `call'
lib/gitlab/etag_caching/middleware.rb:11:in `call'
lib/gitlab/middleware/read_only/controller.rb:28:in `call'
lib/gitlab/middleware/read_only.rb:16:in `call'
lib/gitlab/request_context.rb:18:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:27:in `call'
lib/gitlab/middleware/release_env.rb:10:in `call'
Completed 503 Service Unavailable in 30081ms (Views: 0.5ms | ActiveRecord: 13.8ms)
Results of GitLab environment info
Expand for output related to GitLab environment info
System information System: Ubuntu 14.04 Current User: git Using RVM: no Ruby Version: 2.3.6p384 Gem Version: 2.6.13 Bundler Version:1.13.7 Rake Version: 12.3.0 Redis Version: 3.2.11 Git Version: 2.14.3 Sidekiq Version:5.0.5 Go Version: unknown
GitLab information Version: 10.7.1 Revision: 0d49bb8 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: http://gitlab.akema.fr HTTP Clone URL: http://gitlab.akema.fr/some-group/some-project.git SSH Clone URL: ssh://git@gitlab.akema.fr:33622/some-group/some-project.git Using LDAP: yes Using Omniauth: no
GitLab Shell Version: 7.1.2 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
Checking GitLab Shell ...
GitLab Shell version >= 7.1.2 ? ... OK (7.1.2) Repo base directory exists? default... yes Repo storage directories are symlinks? default... no Repo paths owned by git:root, or git:git? default... yes Repo paths access is drwxrws---? default... yes hooks directories in repos are links: ... 2/1 ... ok 5/16 ... ok 5/17 ... ok 5/19 ... ok 8/21 ... ok 31/22 ... ok 8/23 ... ok 16/25 ... ok 7/26 ... ok 8/27 ... ok 7/29 ... ok 8/30 ... ok 8/31 ... ok 16/32 ... ok 16/33 ... ok 8/36 ... ok 5/37 ... ok 14/39 ... ok 14/43 ... ok 18/44 ... ok 7/45 ... ok 2/47 ... ok 8/52 ... ok 8/53 ... ok 14/54 ... ok 20/55 ... ok 21/56 ... ok 22/57 ... ok 22/59 ... ok 30/61 ... repository is empty 8/62 ... repository is empty 14/63 ... repository is empty 14/64 ... ok 22/66 ... ok 14/67 ... ok 31/69 ... ok 8/70 ... ok 16/71 ... ok 12/72 ... ok 14/73 ... repository is empty 14/74 ... ok 14/75 ... ok 14/76 ... ok 14/77 ... ok 43/78 ... ok 5/79 ... ok 44/80 ... ok 8/81 ... ok 46/82 ... ok 46/83 ... ok 46/84 ... ok 5/87 ... ok 48/89 ... ok 8/90 ... repository is empty 5/91 ... ok 51/93 ... ok 8/94 ... ok 54/95 ... repository is empty 54/96 ... ok 8/97 ... ok 56/98 ... ok Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Reply by email ...
IMAP server credentials are correct? ... yes Init.d configured correctly? ... skipped MailRoom running? ... skipped
Checking Reply by email ... Finished
Checking LDAP ...
Server: ldapmain LDAP authentication... Anonymous. No
bind_dnorpasswordconfigured LDAP users with access to your GitLab server (only showing the first 100 results) [... removed ....]Checking LDAP ... Finished
Checking GitLab ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 2/1 ... yes 5/16 ... yes 5/17 ... yes 5/19 ... yes 8/21 ... yes 31/22 ... yes 8/23 ... yes 16/25 ... yes 7/26 ... yes 8/27 ... yes 7/29 ... yes 8/30 ... yes 8/31 ... yes 16/32 ... yes 16/33 ... yes 8/36 ... yes 5/37 ... yes 14/39 ... yes 14/43 ... yes 18/44 ... yes 7/45 ... yes 2/47 ... yes 8/52 ... yes 8/53 ... yes 14/54 ... yes 20/55 ... yes 21/56 ... yes 22/57 ... yes 22/59 ... yes 30/61 ... yes 8/62 ... yes 14/63 ... yes 14/64 ... yes 22/66 ... yes 14/67 ... yes 31/69 ... yes 8/70 ... yes 16/71 ... yes 12/72 ... yes 14/73 ... yes 14/74 ... yes 14/75 ... yes 14/76 ... yes 14/77 ... yes 43/78 ... yes 5/79 ... yes 44/80 ... yes 8/81 ... yes 46/82 ... yes 46/83 ... yes 46/84 ... yes 5/87 ... yes 48/89 ... yes 8/90 ... yes 5/91 ... yes 51/93 ... yes 8/94 ... yes 54/95 ... yes 54/96 ... yes 8/97 ... yes 56/98 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.3.5 ? ... yes (2.3.6) Git version >= 2.9.5 ? ... yes (2.14.3) Git user has default SSH configuration? ... yes Active users: ... 21
Checking GitLab ... Finished
Possible fixes
Just downgrade to 10.7.0 and all is fine
