Convert to one batched migrations execution worker per database
What does this MR do and why?
While working on Convert batched migrations execution workers to... (!104027 - merged) I realized that, because of the way the redis key for LimitedCapacity::JobTracker
is generated, max_running_jobs
will be shared between migrations for main
and ci
databases, which is not what we want.
This MR introduce two new classes, MainExecutionWorker
and CiExecutionWorker
, that include the shared code form the ExecutionWorker
module. This is the same approach we use for the scheduling cron workers, for example CiDatabaseWorker
. This way when we convert them to limited capacity workers,running migrations for each database will be tracked in a separate redis set, and independent from each other.
I was hoping we can keep having just one class, and generate the redis key dynamically, but it's not easy, as the job_tracker
instance is generated in LimitedCapacity::Worker#perform
which is before any code from the workers is called.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #372316 (closed)