Geo creates larger repos in secondary
While on a call with a customer we saw that the Geo secondary node was returning a 502. When we checked the primary instance Sidekiq had about 940k background jobs in the
process_commitqueue, about 3k in
system_hookand about 40 in
geo_repository_update. Most (95%) of these jobs where regarding the same project. The
system_hook and some other
process_commit jobs where failing and reporting
FIPS integrity verification test failed.
This was about 10hrs after the project was initially pushed to primary. The project is around 4GB and the secondary node ran out of disk space because the same project had taken up around 187GB already.
Steps to reproduce
It might be specific to this case but all that was required to reproduced was to push the same project to primary. This behavior was reproduced on a clean secondary node.
Secondary node should reflect the same project as primary.
Secondary node is stuck
Relevant logs and/or screenshots
Results of GitLab application Check
I don't have this available but did check status on our call and instance is healthy.
Results of GitLab environment info
@brodock mentioned this on a previous conversation
Perhaps it's not pruning or requires GC run?
Slack conversation https://gitlab.slack.com/archives/support/p1482781613003455