Draft: [Do not merge!] Spike batch updates on job artifacts belonging to a pipeline
What does this MR do and why?
This is a spike to assess a way to improve the update queries in the existing Ci::UnlockArtifactsService
.
The update_job_artifacts query in Ci::UnlockArtifactsService
was identified to have the potential to generate unbounded WAL.
This MR describes a way to break the workload into smaller batch of updates on a single pipeline that can then be rate-controlled.
Related: #379089 (closed)
To illustrate this conceptually, here is an overview of the desired process:
The existing behaviour is shown in black, where every time Ci::UnlockArtifactsService
is called, it queries for all pipeline IDs that need to be unlocked, then it unlocks the job & pipeline artifacts in all of these pipelien IDs.
This MR introduces the feature flagged part in green, where Ci::UnlockArtifactsService
would enqueue the list of pipeline IDs into a database-backed queue Ci::UnlockPipelineRequest
. A new worker Ci::UnlockPipelineAndArtifactsWorker
will pick a single pipeline off the queue in order to unlock the pipeline and its associated artifacts. This worker uses LimitedCapacity::Worker
to give us the ability to control the rate at which the pipelines are unlocked.
With this in place, we can then resume fixing unlocking pipelines in failed & blocked states. With the queue in place, the fix to these issues can directly insert the pipeline IDs into Ci::UnlockPipelineRequest
and reuse the same unlocking process for the artifacts (shown in blue).
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.