503 shown when trying to view a commit
Summary
Each time one of our users try to view a commit in the web UI a 503 Service Unavailable is shown.
Steps to reproduce
- Goto any project and press Commits -> All commits are shown (e.g. .../acp/meta-acp-test/commits/master)
- Press a commit header to try to view commit details (try to go to e.g. .../acp/meta-acp-test/commit/bc8c27e4e00fba50906ce249f954b79948d3d0fe) -> 503 Service Unavailable is shown
What is the expected correct behavior?
Expecting to see the commit details page
Relevant logs and/or screenshots
From production.log
Started GET "/acp/meta-acp-test/commit/bc8c27e4e00fba50906ce249f954b79948d3d0fe" for 192.168.15.30 at 2018-06-18 14:02:46 +0200 Processing by Projects::CommitController#show as HTML Parameters: {"namespace_id"=>"acp", "project_id"=>"meta-acp-test", "id"=>"bc8c27e4e00fba50906ce249f954b79948d3d0fe"}
Gitlab::Git::CommandError (8:):
lib/gitlab/git/repository.rb:1380:in rescue in gitaly_migrate' lib/gitlab/git/repository.rb:1373:in
gitaly_migrate'
lib/gitlab/git/commit.rb:234:in batch_by_oid' app/models/repository.rb:121:in
commits_by'
app/models/commit.rb:102:in block (2 levels) in lazy' app/models/commit.rb:99:in
each'
app/models/commit.rb:99:in block in lazy' app/views/projects/commit/_commit_box.html.haml:66:in
block in _app_views_projects_commit__commit_box_html_haml___2108594324849825822_69847954724280'
app/views/projects/commit/_commit_box.html.haml:65:in each' app/views/projects/commit/_commit_box.html.haml:65:in
_app_views_projects_commit__commit_box_html_haml___2108594324849825822_69847954724280'
app/views/projects/commit/show.html.haml:11:in _app_views_projects_commit_show_html_haml__4491424344252851011_69847951010860' app/controllers/projects/commit_controller.rb:25:in
block (2 levels) in show'
app/controllers/projects/commit_controller.rb:24:in show' lib/gitlab/i18n.rb:50:in
with_locale'
lib/gitlab/i18n.rb:56:in with_user_locale' app/controllers/application_controller.rb:334:in
set_locale'
lib/gitlab/middleware/multipart.rb:95:in call' lib/gitlab/request_profiler/middleware.rb:14:in
call'
ee/lib/gitlab/jira/middleware.rb:15:in call' lib/gitlab/middleware/go.rb:17:in
call'
lib/gitlab/etag_caching/middleware.rb:11:in call' lib/gitlab/middleware/read_only/controller.rb:28:in
call'
lib/gitlab/middleware/read_only.rb:16:in call' lib/gitlab/request_context.rb:18:in
call'
lib/gitlab/metrics/requests_rack_middleware.rb:27:in call' lib/gitlab/middleware/release_env.rb:10:in
call'
Completed 503 Service Unavailable in 182ms (Views: 0.8ms | ActiveRecord: 25.8ms | Elasticsearch: 0.0ms)
Results of GitLab environment info
Expand for output related to GitLab environment info
gitlab@s030anl:~$ sudo gitlab-rake gitlab:env:info
System information System: Ubuntu 16.04 Proxy: no Current User: git Using RVM: no Ruby Version: 2.3.6p384 Gem Version: 2.6.13 Bundler Version:1.13.7 Rake Version: 12.3.0 Redis Version: 3.2.11 Git Version: 2.14.3 Sidekiq Version:5.0.5 Go Version: unknown
GitLab information Version: 10.7.1-ee Revision: d209ff1 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql DB Version: 9.6.8 URL: http://git.actia.se HTTP Clone URL: http://git.actia.se/some-group/some-project.git SSH Clone URL: git@git.actia.se:some-group/some-project.git Elasticsearch: no Geo: no Using LDAP: yes Using Omniauth: no
GitLab Shell Version: 7.1.2 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
gitlab@s030anl:~$ sudo gitlab-rake gitlab:env:infoSystem information System: Ubuntu 16.04 Proxy: no Current User: git Using RVM: no Ruby Version: 2.3.6p384 Gem Version: 2.6.13 Bundler Version:1.13.7 Rake Version: 12.3.0 Redis Version: 3.2.11 Git Version: 2.14.3 Sidekiq Version:5.0.5 Go Version: unknown
GitLab information Version: 10.7.1-ee Revision: d209ff1 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql DB Version: 9.6.8 URL: http://git.actia.se HTTP Clone URL: http://git.actia.se/some-group/some-project.git SSH Clone URL: git@git.actia.se:some-group/some-project.git Elasticsearch: no Geo: no Using LDAP: yes Using Omniauth: no
GitLab Shell Version: 7.1.2 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git gitlab@s030anl:~$ sudo gitlab-rake gitlab:check SANITIZE=true Checking GitLab Shell ...
GitLab Shell version >= 7.1.2 ? ... OK (7.1.2) Repo base directory exists? default... yes Repo storage directories are symlinks? default... no Repo paths owned by git:root, or git:git? default... no User id for git: 998. Groupd id for root: 0 Try fixing it: sudo chown -R git:root /var/opt/gitlab/git-data/repositories For more information see: doc/install/installation.md in section "GitLab Shell" Please fix the error above and rerun the checks. Repo paths access is drwxrws---? default... yes hooks directories in repos are links: ... 41/2 ... ok 41/3 ... ok 41/4 ... ok 42/5 ... ok 42/6 ... ok 42/7 ... ok 42/8 ... ok 42/9 ... ok 42/10 ... ok 42/11 ... ok 42/12 ... ok 42/13 ... ok 42/14 ... ok 42/15 ... ok 42/16 ... repository is empty 42/17 ... ok 42/18 ... ok 42/19 ... ok 42/20 ... ok 42/21 ... ok 42/22 ... ok 42/23 ... ok 42/24 ... ok 42/25 ... ok 42/26 ... ok 42/27 ... ok 42/28 ... ok 42/29 ... ok 42/30 ... ok 42/31 ... ok 42/32 ... ok 42/33 ... ok 43/36 ... ok 43/37 ... ok 43/38 ... ok 43/39 ... ok 44/40 ... ok 44/42 ... ok 44/43 ... ok 45/44 ... ok 66/45 ... ok 66/46 ... ok 66/47 ... ok 66/48 ... ok 67/49 ... ok 68/50 ... ok 68/51 ... ok 72/52 ... ok 72/53 ... ok 72/54 ... ok 72/55 ... ok 72/56 ... ok 72/57 ... ok 72/58 ... ok 72/59 ... ok 72/60 ... ok 72/61 ... ok 72/62 ... ok 72/63 ... ok 72/64 ... ok 72/65 ... ok 72/66 ... ok 72/67 ... ok 72/68 ... ok 72/69 ... ok 72/70 ... ok 72/71 ... ok 72/72 ... ok 72/73 ... ok 72/74 ... ok 72/75 ... ok 72/76 ... ok 73/77 ... ok 73/78 ... ok 74/79 ... ok 74/80 ... ok 74/81 ... ok 74/82 ... ok 74/83 ... ok 74/84 ... ok 74/85 ... ok 74/86 ... ok 74/88 ... ok 74/89 ... ok 74/90 ... ok 74/91 ... ok 74/92 ... ok 74/93 ... ok 74/94 ... ok 20/95 ... ok 76/96 ... ok 76/97 ... ok 77/98 ... ok 77/99 ... ok 77/100 ... ok 77/101 ... ok 77/102 ... ok 77/103 ... ok 77/104 ... ok 77/105 ... ok 77/106 ... ok 77/107 ... ok 77/108 ... ok 77/109 ... ok 77/110 ... ok 77/111 ... ok 77/112 ... ok 77/113 ... repository is empty 77/114 ... ok 77/115 ... ok 77/116 ... ok 78/117 ... ok 78/118 ... ok 78/119 ... ok 78/120 ... ok 78/121 ... ok 79/122 ... ok 79/123 ... ok 79/124 ... ok 79/125 ... ok 79/126 ... ok 79/127 ... ok 79/128 ... ok 79/129 ... ok 79/130 ... ok 79/131 ... ok 79/132 ... ok 80/134 ... ok 81/135 ... ok 81/136 ... ok 81/137 ... ok 81/138 ... ok 81/139 ... ok 44/141 ... ok 83/142 ... ok 83/144 ... ok 83/145 ... ok 87/146 ... ok 43/147 ... ok 43/148 ... ok 43/149 ... ok 43/151 ... ok 43/152 ... ok 44/153 ... ok 92/154 ... repository is empty 39/155 ... ok 39/156 ... ok 20/157 ... ok 76/158 ... ok 94/159 ... ok 94/160 ... ok 94/161 ... ok 43/162 ... ok 43/163 ... ok 97/164 ... ok 39/165 ... ok 72/166 ... ok 106/167 ... ok 20/168 ... ok 39/169 ... ok 20/170 ... ok 39/172 ... ok 81/173 ... ok 2/174 ... repository is empty 2/175 ... ok 20/176 ... ok 108/180 ... ok 108/181 ... ok 108/182 ... ok 108/183 ... ok 108/184 ... ok 108/185 ... ok 108/186 ... ok 108/187 ... ok 108/188 ... ok 41/189 ... ok 94/190 ... ok 94/191 ... ok 94/193 ... ok 94/194 ... ok 110/195 ... repository is empty 110/196 ... ok 20/197 ... ok 39/198 ... ok 39/199 ... ok 108/200 ... ok 108/201 ... ok 115/202 ... ok 72/203 ... ok 107/204 ... ok 39/205 ... ok 39/206 ... ok 20/207 ... ok 108/208 ... ok 108/209 ... ok 108/210 ... ok 108/211 ... ok 108/212 ... ok 107/213 ... ok 107/214 ... ok 72/215 ... ok 72/216 ... ok 77/217 ... ok 39/218 ... ok 110/220 ... ok 112/221 ... ok 110/222 ... ok 110/223 ... ok 110/224 ... ok 108/228 ... ok 108/229 ... ok 108/230 ... ok 110/231 ... ok 41/232 ... ok 110/233 ... ok 108/234 ... ok 115/235 ... repository is empty 20/236 ... ok 108/237 ... ok 108/238 ... ok 108/239 ... ok 108/240 ... ok 108/241 ... ok 108/242 ... ok 108/243 ... ok 112/244 ... ok 41/245 ... ok 20/246 ... ok 20/247 ... ok 1/248 ... ok 138/249 ... ok 119/250 ... ok 108/251 ... ok 112/253 ... ok 3/255 ... ok 142/256 ... ok 39/257 ... ok 72/258 ... ok 39/259 ... ok 146/260 ... ok 25/261 ... ok 142/262 ... ok 25/263 ... ok 108/264 ... ok 142/265 ... ok 25/266 ... ok 20/267 ... ok 158/269 ... ok 158/270 ... ok 11/271 ... ok 110/272 ... ok 76/273 ... ok 158/274 ... ok 165/275 ... ok 165/276 ... ok 25/277 ... ok 165/278 ... ok 165/279 ... ok 165/280 ... ok 165/281 ... ok 165/282 ... ok 165/283 ... ok 165/284 ... ok 108/285 ... ok 158/286 ... ok 20/287 ... ok 110/288 ... ok 158/289 ... repository is empty 76/290 ... repository is empty Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Reply by email is disabled in config/gitlab.yml Checking LDAP ...
Server: ldapmain LDAP authentication... Success LDAP users with access to your GitLab server (only showing the first 100 results)
***removed for public issues***
Checking LDAP ... Finished
Checking GitLab ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... no Try fixing it: sudo chown -R git /var/opt/gitlab/gitlab-rails/uploads sudo find /var/opt/gitlab/gitlab-rails/uploads -type f -exec chmod 0644 {} ; sudo find /var/opt/gitlab/gitlab-rails/uploads -type d -not -path /var/opt/gitlab/gitlab-rails/uploads -exec chmod 0700 {} ; For more information see: doc/install/installation.md in section "GitLab" Please fix the error above and rerun the checks. Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 41/2 ... yes 41/3 ... yes 41/4 ... yes 42/5 ... yes 42/6 ... yes 42/7 ... yes 42/8 ... yes 42/9 ... yes 42/10 ... yes 42/11 ... yes 42/12 ... yes 42/13 ... yes 42/14 ... yes 42/15 ... yes 42/16 ... yes 42/17 ... yes 42/18 ... yes 42/19 ... yes 42/20 ... yes 42/21 ... yes 42/22 ... yes 42/23 ... yes 42/24 ... yes 42/25 ... yes 42/26 ... yes 42/27 ... yes 42/28 ... yes 42/29 ... yes 42/30 ... yes 42/31 ... yes 42/32 ... yes 42/33 ... yes 43/36 ... yes 43/37 ... yes 43/38 ... yes 43/39 ... yes 44/40 ... yes 44/42 ... yes 44/43 ... yes 45/44 ... yes 66/45 ... yes 66/46 ... yes 66/47 ... yes 66/48 ... yes 67/49 ... yes 68/50 ... yes 68/51 ... yes 72/52 ... yes 72/53 ... yes 72/54 ... yes 72/55 ... yes 72/56 ... yes 72/57 ... yes 72/58 ... yes 72/59 ... yes 72/60 ... yes 72/61 ... yes 72/62 ... yes 72/63 ... yes 72/64 ... yes 72/65 ... yes 72/66 ... yes 72/67 ... yes 72/68 ... yes 72/69 ... yes 72/70 ... yes 72/71 ... yes 72/72 ... yes 72/73 ... yes 72/74 ... yes 72/75 ... yes 72/76 ... yes 73/77 ... yes 73/78 ... yes 74/79 ... yes 74/80 ... yes 74/81 ... yes 74/82 ... yes 74/83 ... yes 74/84 ... yes 74/85 ... yes 74/86 ... yes 74/88 ... yes 74/89 ... yes 74/90 ... yes 74/91 ... yes 74/92 ... yes 74/93 ... yes 74/94 ... yes 20/95 ... yes 76/96 ... yes 76/97 ... yes 77/98 ... yes 77/99 ... yes 77/100 ... yes 77/101 ... yes 77/102 ... yes 77/103 ... yes 77/104 ... yes 77/105 ... yes 77/106 ... yes 77/107 ... yes 77/108 ... yes 77/109 ... yes 77/110 ... yes 77/111 ... yes 77/112 ... yes 77/113 ... yes 77/114 ... yes 77/115 ... yes 77/116 ... yes 78/117 ... yes 78/118 ... yes 78/119 ... yes 78/120 ... yes 78/121 ... yes 79/122 ... yes 79/123 ... yes 79/124 ... yes 79/125 ... yes 79/126 ... yes 79/127 ... yes 79/128 ... yes 79/129 ... yes 79/130 ... yes 79/131 ... yes 79/132 ... yes 80/134 ... yes 81/135 ... yes 81/136 ... yes 81/137 ... yes 81/138 ... yes 81/139 ... yes 44/141 ... yes 83/142 ... yes 83/144 ... yes 83/145 ... yes 87/146 ... yes 43/147 ... yes 43/148 ... yes 43/149 ... yes 43/151 ... yes 43/152 ... yes 44/153 ... yes 92/154 ... yes 39/155 ... yes 39/156 ... yes 20/157 ... yes 76/158 ... yes 94/159 ... yes 94/160 ... yes 94/161 ... yes 43/162 ... yes 43/163 ... yes 97/164 ... yes 39/165 ... yes 72/166 ... yes 106/167 ... yes 20/168 ... yes 39/169 ... yes 20/170 ... yes 39/172 ... yes 81/173 ... yes 2/174 ... yes 2/175 ... yes 20/176 ... yes 108/180 ... yes 108/181 ... yes 108/182 ... yes 108/183 ... yes 108/184 ... yes 108/185 ... yes 108/186 ... yes 108/187 ... yes 108/188 ... yes 41/189 ... yes 94/190 ... yes 94/191 ... yes 94/193 ... yes 94/194 ... yes 110/195 ... yes 110/196 ... yes 20/197 ... yes 39/198 ... yes 39/199 ... yes 108/200 ... yes 108/201 ... yes 115/202 ... yes 72/203 ... yes 107/204 ... yes 39/205 ... yes 39/206 ... yes 20/207 ... yes 108/208 ... yes 108/209 ... yes 108/210 ... yes 108/211 ... yes 108/212 ... yes 107/213 ... yes 107/214 ... yes 72/215 ... yes 72/216 ... yes 77/217 ... yes 39/218 ... yes 110/220 ... yes 112/221 ... yes 110/222 ... yes 110/223 ... yes 110/224 ... yes 108/228 ... yes 108/229 ... yes 108/230 ... yes 110/231 ... yes 41/232 ... yes 110/233 ... yes 108/234 ... yes 115/235 ... yes 20/236 ... yes 108/237 ... yes 108/238 ... yes 108/239 ... yes 108/240 ... yes 108/241 ... yes 108/242 ... yes 108/243 ... yes 112/244 ... yes 41/245 ... yes 20/246 ... yes 20/247 ... yes 1/248 ... yes 138/249 ... yes 119/250 ... yes 108/251 ... yes 112/253 ... yes 3/255 ... yes 142/256 ... yes 39/257 ... yes 72/258 ... yes 39/259 ... yes 146/260 ... yes 25/261 ... yes 142/262 ... yes 25/263 ... yes 108/264 ... yes 142/265 ... yes 25/266 ... yes 20/267 ... yes 158/269 ... yes 158/270 ... yes 11/271 ... yes 110/272 ... yes 76/273 ... yes 158/274 ... yes 165/275 ... yes 165/276 ... yes 25/277 ... yes 165/278 ... yes 165/279 ... yes 165/280 ... yes 165/281 ... yes 165/282 ... yes 165/283 ... yes 165/284 ... yes 108/285 ... yes 158/286 ... yes 20/287 ... yes 110/288 ... yes 158/289 ... yes 76/290 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.3.5 ? ... yes (2.3.6) Git version >= 2.9.5 ? ... yes (2.14.3) Git user has default SSH configuration? ... yes Active users: ... 88 Elasticsearch version 5.1 - 5.5? ... skipped (elasticsearch is disabled)
Checking GitLab ... Finished
Possible fixes
(If you can, link to the line of code that might be responsible for the problem)