invalid sha1 pointer in environments/deployments
Summary
We weekly receive an email about failed repository checks. The repocheck.log showed:
# Logfile created on 2019-06-06 08:29:28 +0000 by logger.rb/61378
E,[2019-06-06T08:29:28.798971 #78] ERROR--: Could not fsck repository: error: refs/environments/eu-central-1-brok-5gib7b/deployments/3070: invalid sha1 pointer 00000000000000000000000000000000000000
error: refs/environments/eu-central-1-brok-7th3ed/deployments/3071: invalid sha1 pointer 00000000000000000000000000000000000000
error: refs/environments/eu-central-1-brok-fqtsvc/deployments/3072: invalid sha1 pointer 00000000000000000000000000000000000000
The three deployments (3070, 3071, 3072) are all the same type of job on the same commit, only for different environments. The commit was a merge commit from gitlab merge request.
These ref files are not included on a simple clone, so I can't try to fsck or repair this manually on my local machine. A fsck of the normal repo only shows three dangling blobs and two dangling trees.
Currently the repocheck.log is empty, but https://redacted-gitlab-url.net/admin/projects?last_repository_check_failed=1 still shows the broken repo.
Steps to reproduce
Unfortunately I have no idea, how to reproduce this behaviour. If you need more information, I am happy to provide them.
What is the current bug behavior?
There is an invalid sha1 ref in a file generated by gitlab.
What is the expected correct behavior?
gitlab should generate only valid files into repos.
Related issue
https://gitlab.com/gitlab-org/gitlab-ce/issues/44431 (same error message, but this time it's in environments, not in keep-around)
User Questions
- How to fix this error?
- Can I do a pull/clone, which includes these files?
- (How) can I manually trigger a recheck to verify the problem has been solved?
- How is there a discrepancy between repo list and repocheck.log?
Output of checks
Results of GitLab environment info
System information
System: Ubuntu 16.04
Current User: git
Using RVM: no
Ruby Version: 2.5.5p157
Gem Version: 2.7.6.2
Bundler Version:1.17.3
Rake Version: 12.3.2
Redis Version: 3.0.6
Git Version: 2.21.0
Sidekiq Version:5.2.5
Go Version: unknown
GitLab information
Version: 11.9.8
Revision: Unknown
Directory: /home/git/gitlab
DB Adapter: postgresql
URL: https://-redacted-
HTTP Clone URL: https://-redacted-/some-group/some-project.git
SSH Clone URL: git@-redacted-:some-group/some-project.git
Using LDAP: no
Using Omniauth: yes
Omniauth Providers: openid_connect
GitLab Shell
Version: 8.7.1
Repository storage paths:
- default: /home/git/data/repositories
GitLab Shell path: /home/git/gitlab-shell
Git: /usr/bin/git
(Docker-Image with custom additions. Running on Kubernetes (no helm))