Expire project caches once per push instead of once per ref
Previously ProjectCacheWorker
would be scheduled once per ref, which
would generate unnecessary I/O and load on Sidekiq, especially if many
tags or branches were pushed at once. ProjectCacheWorker
would expire
three items:
- Repository size: This only needs to be updated once per push.
- Commit count: This only needs to be updated if the default branch is updated.
- Project method caches: This only needs to be updated if the default branch changes, but only if certain files change (e.g. README, CHANGELOG, etc.).
Because the third item requires looking at the actual changes in the
commit deltas, we schedule one ProjectCacheWorker
to handle the first
two cases, and schedule a separate ProjectCacheWorker
for the third
case if it is needed. As a result, this brings down the number of
ProjectCacheWorker
jobs from N to 2.
Edited by Stan Hu