URGENT Cache invalidation issue in 8.14.x: `Repository#has_visible_content?` erroneously gets set to `false`

ZD: https://gitlab.zendesk.com/agent/tickets/51180

A customer upgraded to GitLab 8.14.0, and then noticed very odd behavior:

• Project repositories would go "empty"
• Full repository backups would not actually run, since the project appeared "empty"
• Merge requests would go empty with commits

The state of the repo was temporarily alleviated with a gitlab-rake cache:clear, but the problem came up shortly after.

When we drilled into one of the repos, we checked the Rails console:

• Repository#exists? returned true
• Repository#has_visible_content? returned false
• Repository#branches returned many branches
• Repository#branch_count was 161

These lines worry me: https://gitlab.com/gitlab-org/gitlab-ce/blob/v8.14.0/app/models/repository.rb#L1163-1166

If we happen to hit some sort of error, the fallback is to return the branch_count of 0, which causes the branch count to be 0 and sets the has_visible_content? flag to false. This won't change until someone pushes a new branch or removes an old one.

They are using a Docker container to run GitLab and mount volumes from there. We were able to verify the image was able to see the repository path.

/cc: @yorickpeterse