Ensure that sharding_key_id on project runner points to valid project
We recently took 2 actions to ensure that sharding_key_id
on project runners is valid, so that runners are correctly moved by OrgMover in the future:
-
added a worker to recalculate
sharding_key_id
when the project pointed to byci_runners.sharding_key_id
is deleted. -
added a batched migration that iterates project runners missing a matching
ci_runners_projects
record but do have a fallbackci_runner_projects
record, to assign theproject_id
of the oldestci_runner_projects
record for that runner id.
Unfortunately, some other scenarios remain that could lead to a sharding_key_id
that points to a value that is not represented in a ci_runner_projects
join record, and this is visible through the slow growth of records in this situation:
-
!180264 (merged) Ci::Runner#assign_to: this condition does not seem correct, we want to reassign sharding_key_id
from the moment that it doesn't point to a valid project, even ifrunner_projects
is not empty. -
!180368 (merged) Ci::Runners::UnassignRunnerService#execute: if we're unassigning from a project, we need to check if that project is the one currently referenced in owner
, and calculate a fallback in that case. -
Ci::Runners::SetRunnerAssociatedProjectsService#set_associated_projects: This one is not a problem, because the service never unassigns the owner project. -
!180374 (merged) DELETE projects/:id/runners/:runner_id: Same here, we need to check if that project is the one currently referenced in owner
, and calculate a fallback in that case. -
!180417 (merged) When a runner changes sharding_key_id
for some reason, the change must be propagated toci_runner_machines
andci_runner_taggings
Edited by Pedro Pombeiro - OOO from Oct 13-24