Skip to content

Enable performant artifact expiration by default

drew stachon requested to merge enable-performant-sm-artifact-expiration into master

What does this MR do and why?

This MR switches three feature flags that, together, fully enable our new expired artifact removal process for self managed customers. There are three switches being flipped at once here:

  1. ci_update_unlocked_job_artifacts to start writing values (locked/unlocked, as opposed to unknown) to the ci_job_artifacts table 'locked' column
  2. ci_destroy_unlocked_job_artifacts to start removing expired artifacts based on the ci_job_artifacts table 'locked' column instead of `ci_pipelines.locked'

This will increase expiration throughput of newly created artifacts. If a GitLab installation has a pending backlog of artifacts it has not been able to remove, they will remain expired and unremoved. Due to that, we'll also enable

  1. ci_job_artifacts_backlog_work to let the Ci::UpdateLockedUnknownArtifactsWorker start processing the backlog of unremoved expired artifacts as a separate queue. Right now I don't believe we will need to enable the higher (50,000/7min) throughput limit, since the lower limit (10,000/7min) will be running until we remove this work in %16.0.

Didn't we already do this?

We did for SaaS, and explained our steps and risk mitigation strategy in excruciating detail in epic &7311. Now that we've used this rollout strategy successfully on gitlab.com, I'm confident in using the same strategy for self-managed installations.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports