Investigate 2021-04-13 file-cny-01 apdex degradatation
In production#4202 (closed) we disabled the CI_PRE_CLONE_SCRIPT for gitlab-org/gitlab and we saw the Gitaly RPC apdex for file-cny-01 degrade. Later in the day we re-enabled the CI_PRE_CLONE_SCRIPT.
In this issue we will do a time-limited investigation into what went wrong there.
- host stats for 2021-04-13
- gitaly host stats for 2021-04-13
- #954 (closed) has useful observations too
At some point we will probably turn the script off again temporarily, so we can gather data, but I think we should first think about what data we want.
The motivating problem of the pack-objects cache was a bottleneck we observed where the Gitaly server was unable to generate the git fetch response data. That bottleneck now appears to be gone, but now we appear to hit a bottleneck in getting the data off of the Gitaly server.