Skip to content

Increase minimum mirror update interval from 15 to 30 minutes

Douwe Maan requested to merge dm-mirror-min-delay-30-minutes into master

Per the data in https://gitlab.com/gitlab-org/gitlab-ee/issues/5258, GitLab.com is currently able to update all 19k mirrors in 23 minutes. However, the minimum update interval is currently set at 15 minutes, which means that the vast majority of mirrors "want" another update 15 minutes after an update finishes. Since we are only able to actually update 2/3s of all projects in that time, about 1 in every 3 mirrors are constantly overdue an update, with a next_execution_ts <= NOW() (graph):

Screen_Shot_2018-03-13_at_17.25.45

With CI/CD for external projects (https://gitlab.com/gitlab-org/gitlab-ee/issues/4839) coming in %10.6, GitHub will notify us using a webhook every time it receives a push, so that we can update the mirror, which we do by setting the affected project's next_execution_ts to NOW(). Right now, however, we wouldn't actually get to that project until we make it through the list of "overdue" projects over the next 8 minutes, which is way too long to wait for a GitLab CI pipeline to start when someone pushes to GitHub, or submits a pull request there.

I have a plan to dynamically determine mirror update interval based on total number of mirrors, average update time, and available concurrency in https://gitlab.com/gitlab-org/gitlab-ee/issues/5258 (%10.7 or %10.8), but as a stop-gap solution that has a higher chance of getting picked into %10.6, I think we should manually increase the minimum mirror update interval from 15 to 30 minutes. This is comfortably higher than 23, and leaves some room for the sudden jump in number of mirrors that may come when https://gitlab.com/gitlab-org/gitlab-ee/issues/4839 is announced once %10.6 is released.

If you, dear reviewer, think this makes sense, I will create a %10.6 exception request issue with the same reasoning once this is merged.

Edited by Douwe Maan

Merge request reports