Limit the number of project loaded by UpdateAllMirrorsWorker
Summary
We load all projects in UpdateAllMirrorsWorker#schedule_mirrors!
and then we select all of them that have Project#mirror?
return true.
After enabling the feature availability check, a lot of projects piled up at the beginning of the result needed to be skipped from the Rails side, leading to a huge delay in the processing of licensed projects.
I'm thinking of two solutions for this:
Update the query to include the plan check:
left_outer_joins(namespace: :plan).where("(projects.visibility_level = 20 OR plans.name IN ('early_adopter', 'bronze', 'silver', 'gold'))")
This doesn't work: #196862 (closed)
- Instead of lazy loading the projects we could load them all and
mark_as_failed
the ones without a license. After15
retries they will be dropped from the query.
Edited by 🤖 GitLab Bot 🤖