Skip to content

Geo: Reduce frequency of redownload attempts

Michael Kozono requested to merge mk/make-repo-sync-retries-more-resilient into master

What does this MR do and why?

  • Instead of only attempting to retry syncs with the "redownload" flow after the threshold is reached, now we alternate between the "normal" and "redownload" flow for each retry
  • Increase the threshold for using the "redownload" flow from 5 to 10 retries

This increases the resilience of retries because:

  1. Sometimes "redownload" doesn't work but a "normal" sync does: #324208 (closed)
  2. The "redownload" logic is not totally compatible with Gitaly Cluster: #332151 (closed)

How to set up and validate locally

For example to test project resync:

  1. Checkout this branch in your secondary GDK/gitlab repo
  2. In Rails console, do registry = Geo::ProjectRegistry.first
  3. registry.update!(repository_retry_count: 7, resync_repository: true)
  4. In another tab: tail -f log/geo.log
  5. Resync the project Git repo: Geo::RepositorySyncService.new(registry.project).execute
  6. Notice that Redownloading repository is not output (whereas it is output without this change)

Similar can be done to test different retry counts, or project-wikis, or designs, or snippets, or group wikis.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Michael Kozono

Merge request reports