Commit list returns 404 on branches whose names contain forward slashes
Summary
With the upgrade to GitLab 11.9 the Haskell installation (#55039 (moved)) began blowing up in various ways when faced with branches whose names contain forward slashes:
- Merge requests whose source branch name contains a forward slash suggested that the source branch did not exist ("source branch does not exist. Please restore it or use a different source branch") despite the fact that the pages list most certainly did exist
- Viewing the commits list (e.g. https://gitlab.haskell.org/ghc/ghc/commits/wip/lint) of a branch whose name contains a forward slash returned a 404
Steps to reproduce
Push a branch whose name contains a forward slash (e.g. wip/lint
).
Example Project
Unfortunately I have been unable to reproduce this on gitlab.com.
What is the current bug behavior?
Various things break, as described above.
What is the expected correct behavior?
Things don't blow up.
Relevant logs and/or screenshots
See "possible fixes" section below.
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
System information System: Proxy: no Current User: root Using RVM: no Ruby Version: unknown Gem Version: Bundler Version:1.17.2 Rake Version: 12.3.2 Redis Version: unknown Git Version: 2.19.2 Sidekiq Version:5.2.5 Go Version: unknownGitLab information Version: 11.9.0-ee Revision: Unknown Directory: /nix/store/wlxwq8g78cmiy0qca6zqa089qjj91940-gitlab-ee-11.9.0/share/gitlab DB Adapter: postgresql DB Version: 9.6.11 URL: https://gitlab.haskell.org HTTP Clone URL: https://gitlab.haskell.org/some-group/some-project.git SSH Clone URL: git@gitlab.haskell.org:some-group/some-project.git Elasticsearch: no Geo: no Using LDAP: no Using Omniauth: yes Omniauth Providers: github
GitLab Shell Version: 8.7.1 Repository storage paths:
- default: /var/gitlab/state/repositories GitLab Shell path: /nix/store/yfy2wr01mv6j5g86as6h0jdx1ggjcnqp-gitlab-shell-8.7.1 Git: git
Results of GitLab application Check
Expand for output related to the GitLab application check
Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 8.7.1 ? ... OK (8.7.1) Running /nix/store/yfy2wr01mv6j5g86as6h0jdx1ggjcnqp-gitlab-shell-8.7.1/bin/check Check GitLab API access: OK
Redis available via internal API: OKAccess to /var/gitlab/state/home/.ssh/authorized_keys: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... no Try fixing it: sudo -u git -H RAILS_ENV=production bin/background_jobs start For more information see: doc/install/installation.md in section "Install Init Script" see log/sidekiq.log for possible errors Please fix the error above and rerun the checks.
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... no Try fixing it: sudo chmod 700 /var/gitlab/state/uploads For more information see: doc/install/installation.md in section "GitLab" Please fix the error above and rerun the checks. Uploads directory tmp has correct permissions? ... no Try fixing it: sudo chown -R git /var/gitlab/state/uploads sudo find /var/gitlab/state/uploads -type f -exec chmod 0644 {} ; sudo find /var/gitlab/state/uploads -type d -not -path /var/gitlab/state/uploads -exec chmod 0700 {} ; For more information see: doc/install/installation.md in section "GitLab" Please fix the error above and rerun the checks. Init script exists? ... no Try fixing it: Install the init script For more information see: doc/install/installation.md in section "Install Init Script" Please fix the error above and rerun the checks. Init script up-to-date? ... can't check because of previous errors Projects have namespace: ... Glasgow Haskell Compiler / GHC ... yes Glasgow Haskell Compiler / libffi-tarballs ... yes Glasgow Haskell Compiler / gmp-tarballs ... yes Glasgow Haskell Compiler / arcanist-external-json-linter ... yes Glasgow Haskell Compiler / hsc2hs ... yes Glasgow Haskell Compiler / haddock ... yes Glasgow Haskell Compiler / nofib ... yes Glasgow Haskell Compiler / Packages / array ... yes Glasgow Haskell Compiler / Packages / binary ... yes Glasgow Haskell Compiler / Packages / bytestring ... yes Glasgow Haskell Compiler / Packages / Cabal ... yes Glasgow Haskell Compiler / Packages / containers ... yes Glasgow Haskell Compiler / Packages / deepseq ... yes Glasgow Haskell Compiler / Packages / directory ... yes Glasgow Haskell Compiler / Packages / filepath ... yes Glasgow Haskell Compiler / Packages / haskeline ... yes Glasgow Haskell Compiler / Packages / hpc ... yes Glasgow Haskell Compiler / Packages / mtl ... yes Glasgow Haskell Compiler / Packages / parsec ... yes Glasgow Haskell Compiler / Packages / pretty ... yes Glasgow Haskell Compiler / Packages / process ... yes Glasgow Haskell Compiler / Packages / terminfo ... yes Glasgow Haskell Compiler / Packages / text ... yes Glasgow Haskell Compiler / Packages / time ... yes Glasgow Haskell Compiler / Packages / transformers ... yes Glasgow Haskell Compiler / Packages / unix ... yes Glasgow Haskell Compiler / Packages / Win32 ... yes Glasgow Haskell Compiler / Packages / xhtml ... yes Glasgow Haskell Compiler / Packages / parallel ... yes Glasgow Haskell Compiler / Packages / stm ... yes Ben Gamari / Glasgow Haskell Compiler ... yes Haskell / array ... yes Mr Kerckhove / Glasgow Haskell Compiler ... yes Gabor Greif / Glasgow Haskell Compiler ... yes Peter Trommler / Glasgow Haskell Compiler ... yes Zejun Wu / Glasgow Haskell Compiler ... yes Roland Senn / Glasgow Haskell Compiler ... yes Matthew Pickering / Glasgow Haskell Compiler ... yes David Eichmann / Glasgow Haskell Compiler ... yes Carter Schonwald / Glasgow Haskell Compiler ... yes Alec Theriault / Glasgow Haskell Compiler ... yes Dmitry Ivanov / Glasgow Haskell Compiler ... yes Dejan Mijić / Glasgow Haskell Compiler ... yes Adam Sandberg Eriksson / Glasgow Haskell Compiler ... yes Alexander Biehl / Glasgow Haskell Compiler ... yes Eric Seidel / Glasgow Haskell Compiler ... yes Sergei Trofimovich / Glasgow Haskell Compiler ... yes Sven Tennie / Glasgow Haskell Compiler ... yes Thi H / Glasgow Haskell Compiler ... yes Glasgow Haskell Compiler / Hadrian ... yes Piyush P Kurur / Glasgow Haskell Compiler ... yes cnd / Shake.it.off ... yes mrchurrisky / haskell_projects ... yes Artem Pelenitsyn / Glasgow Haskell Compiler ... yes Andreas Klebinger / Glasgow Haskell Compiler ... yes Geoffrey Huntley / Glasgow Haskell Compiler ... yes Shayne Fletcher / Glasgow Haskell Compiler ... yes Francesco Gazzetta / Glasgow Haskell Compiler ... yes Ara Adkins / Glasgow Haskell Compiler ... yes Iavor S. Diatchki / Glasgow Haskell Compiler ... yes Ömer Sinan Ağacan / Glasgow Haskell Compiler ... yes Alan Mock / Glasgow Haskell Compiler ... yes Susan Potter / Glasgow Haskell Compiler ... yes Sebastian Graf / nofib ... yes Sasa Bogicevic / Glasgow Haskell Compiler ... yes Julian Leviston / Glasgow Haskell Compiler ... yes Simon Hafner / Glasgow Haskell Compiler ... yes Richard Eisenberg / GHC ... yes Richard Eisenberg / haddock ... yes Takenobu Tani / Glasgow Haskell Compiler ... yes Glasgow Haskell Compiler / head.hackage ... yes Michal Terepeta / Glasgow Haskell Compiler ... yes Andrew Martin / Glasgow Haskell Compiler ... yes Sylvain Henry / Glasgow Haskell Compiler ... yes scottgw / Glasgow Haskell Compiler ... yes Simon Jakobi / Glasgow Haskell Compiler ... yes Moritz Angermann / Glasgow Haskell Compiler ... yes Simon Jakobi / haddock ... yes Ryan Scott / Glasgow Haskell Compiler ... yes Ryan Scott / haddock ... yes Herbert Valerio Riedel / GHC ... yes howtonotwin / GHC ... yes chessai / GHC ... yes Matt Renaud / GHC ... yes Ben Price / GHC ... yes Steve Hart / GHC ... yes Glasgow Haskell Compiler / Packages / old-locale ... yes Glasgow Haskell Compiler / Packages / dph ... yes Glasgow Haskell Compiler / Packages / old-time ... yes Glasgow Haskell Compiler / Packages / haskell2010 ... yes Glasgow Haskell Compiler / Packages / haskell98 ... yes Glasgow Haskell Compiler / Packages / base ... yes Glasgow Haskell Compiler / Packages / integer-gmp ... yes Glasgow Haskell Compiler / Packages / ghc-prim ... yes Glasgow Haskell Compiler / Packages / template-haskell ... yes Glasgow Haskell Compiler / Packages / integer-simple ... yes Glasgow Haskell Compiler / ghc-tarballs ... yes Glasgow Haskell Compiler / Packages / utf8-string ... yes Glasgow Haskell Compiler / Packages / hoopl ... yes Glasgow Haskell Compiler / Packages / primitive ... yes Glasgow Haskell Compiler / Packages / random ... yes Glasgow Haskell Compiler / GHC Testsuite ... yes Glasgow Haskell Compiler / Packages / vector ... yes Glasgow Haskell Compiler / Packages / extensible-exceptions ... yes Tao He / GHC ... yes kgardas / GHC ... yes David Eichmann / GHC-Perf-Notes-Test ... yes Glasgow Haskell Compiler / GHC Performance Notes ... yes Abhiroop Sarkar / nofib ... yes Chaitanya Koparkar / hpc ... yes Chaitanya Koparkar / GHC ... yes Ben Gamari / ghc-utils ... yes Alp Mestanogullari / GHC ... yes Samuel Holland / GHC ... yes Kazuki Okamoto / GHC ... yes P.C. Shyamshankar / GHC ... yes John Ky / GHC ... yes Gabor Greif / nofib ... yes toon nolten / GHC ... yes Alex Dudarenko / GHC ... yes Tamar Christina / GHC ... yes Edward Z. Yang / GHC ... yes tweag / GHC ... yes Langston Barrett / GHC ... yes Neil Mitchell / GHC ... yes Alexandre R. Baldé / GHC ... yes Artem Pyanykh / GHC ... yes mniip / GHC ... yes My Nguyen / GHC ... yes My Nguyen / haddock ... yes Alexandre Esteves / GHC ... yes crockeea / GHC ... yes Niklas Hambüchen / GHC ... yes Jan van Brügge / GHC ... yes Vaibhav Sagar / GHC ... yes Oleg Grenrus / nofib ... yes Oleg Grenrus / GHC ... yes Glasgow Haskell Compiler / ci-images ... yes Phil de Joux / GHC ... yes Ben Gamari / test ... yes Rinat Striungis / GHC ... yes Pluralia / GHC ... yes Trịnh Tuấn Phương / GHC ... yes Sophie Taylor / GHC ... yes Ben Gamari / gitlab-migration ... yes Glasgow Haskell Compiler / homepage ... yes Ben Gamari / trac-to-remarkup ... yes Glasgow Haskell Compiler / marge-bot ... yes Michael Peyton Jones / GHC ... yes mgsloan / GHC ... yes Henry Cohen / GHC ... yes Travis Whitaker / GHC ... yes Radosław Rowicki / GHC ... yes Simon Marlow / GHC ... yes dxmgnwma / Testo Drive 365 ... yes Jens Petersen / homepage ... yes Andreas Klebinger / nofib ... yes Takenobu Tani / GHC wiki tracking ... yes dero hereri / health ... yes Alka Tone Keto Reviews / Alka Tone Keto Reviews ... yes Alka Tone Keto Reviews / Alka Tone Keto Shark Tank ... yes Redis version >= 2.8.0? ... no Try fixing it: Update your redis server to a version >= 2.8.0 For more information see: gitlab-public-wiki/wiki/Trouble-Shooting-Guide in section sidekiq Please fix the error above and rerun the checks. Ruby version >= 2.3.5 ? ... yes (2.5.3) Git version >= 2.18.0 ? ... yes (2.19.2) Git user has default SSH configuration? ... yes Active users: ... 3555 Elasticsearch version 5.6 - 6.x? ... skipped (elasticsearch is disabled)
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished
Possible fixes
I spent quite some time tracing this down. I managed to come up with a workaround but still can't claim to understand the root cause of the issue. I'll try to summarize my findings here.
I started by looking at the issue of the commit list page of the branch wip/lint
returning a 404. It turned out that this was due to ExtractsPath#extract_ref
incorrectly splitting the query into a ref of wip
and a path of lint
. Inspecting the implementation of this function suggested that this was due to the wip/lint
ref not being present in the list returned by ExtractPath#ref_names
.
I then turned my attention to working out why the ref was not present. I inspected gitaly's branch ref name query implementation (listRefNames
in gitaly/internal/service/ref/refnames.go
) but found it to be correct. However, I then noticed that this code wasn't being hit when servicing the failing request.
In trying to work out why this was the case I stumbled upon some caching logic for Repository
model's branch_names
list. It just so happens that disabling this caching logic avoided the issue:
diff --git a/app/models/repository.rb b/app/models/repository.rb
index ff355295862..bba51925c5f 100644
--- a/app/models/repository.rb
+++ b/app/models/repository.rb
@@ -37,7 +37,7 @@ class Repository
# stores its data in the `commit_count` cache key.
CACHED_METHODS = %i(size commit_count rendered_readme readme_path contribution_guide
changelog license_blob license_key gitignore
- gitlab_ci_yml branch_names tag_names branch_count
+ gitlab_ci_yml tag_names branch_count
tag_count avatar exists? root_ref has_visible_content?
issue_template_names merge_request_template_names xcode_project?).freeze
@@ -571,7 +571,7 @@ class Repository
end
delegate :branch_names, to: :raw_repository
- cache_method :branch_names, fallback: []
+ #cache_method :branch_names, fallback: []
delegate :tag_names, to: :raw_repository
cache_method :tag_names, fallback: []
I have no explanation for why this help but reverting this patch causes the issue to recur so there is certainly something fishy in this caching logic.