Growing pack-objects cluster path on Geo Secondary Gitaly node
Support Request for the Gitaly Team
The goal is to keep these requests public. However, if customer information is required to the support request, please be sure to mark this issue as confidential.
This request template is part of Gitaly Team's intake process.
Customer Information
Salesforce Link: https://gitlab.my.salesforce.com/00161000004yLEyAAM
Zendesk Ticket: https://gitlab.zendesk.com/agent/tickets/449006
Installation Size: Large
Architecture Information:
Slack Channel:
Additional Information:
Support Request
Severity
This would probably be s4 if it wasn't for the alarming size of the one object pack.
Problem Description
One repo's pack directory is consuming 1.4T of space on disk. This only appears to be happening on one Gitaly node on the Geo secondary instance
-bash-4.2$ sudo sh -c 'du -sh /data/git-data/repositories/*'
1.4T /data/git-data/repositories/@cluster
0 /data/git-data/repositories/@geo-temporary
20K /data/git-data/repositories/+gitaly
9.0G /data/git-data/repositories/@hashed
1.2G /data/git-data/repositories/@pools
-bash-4.2$ sudo du -sh /data/git-data/repositories/@cluster/repositories/e1/c9/466686/objects/pack
1.4T /data/git-data/repositories/@cluster/repositories/e1/c9/466686/objects/pack
Troubleshooting Performed
What specifically do you need from the Gitaly team
Help understanding and communicating whether this is normal. If this is abnormal and there is something we can do to mitigate the problem then that would be helpful as well.
Author Checklist
-
Customer information provided -
Severity realistically set -
Clearly articulated what is needed from the Gitaly team to support your request by filling out the What specifically do you need from the Gitaly team
/cc @mjwood @andrashorvath @jcaigitlab @john.mcdonnell @gerardo