URGENT Cache invalidation issue in 8.14.x: `Repository#has_visible_content?` erroneously gets set to `false`
ZD: https://gitlab.zendesk.com/agent/tickets/51180
A customer upgraded to GitLab 8.14.0, and then noticed very odd behavior:
- Project repositories would go "empty"
- Full repository backups would not actually run, since the project appeared "empty"
- Merge requests would go empty with commits
The state of the repo was temporarily alleviated with a gitlab-rake cache:clear
, but the problem came up shortly after.
When we drilled into one of the repos, we checked the Rails console:
-
Repository#exists?
returnedtrue
-
Repository#has_visible_content?
returnedfalse
-
Repository#branches
returned many branches -
Repository#branch_count
was 161
These lines worry me: https://gitlab.com/gitlab-org/gitlab-ce/blob/v8.14.0/app/models/repository.rb#L1163-1166
If we happen to hit some sort of error, the fallback is to return the branch_count
of 0, which causes the branch count to be 0 and sets the has_visible_content?
flag to false
. This won't change until someone pushes a new branch or removes an old one.
They are using a Docker container to run GitLab and mount volumes from there. We were able to verify the image was able to see the repository path.
/cc: @yorickpeterse