Skip to content

Add locked column to Ci::JobArtifact

Matt Kasa requested to merge 322817-clean-up-old-expired-job-artifacts-2 into master

What does this MR do?

This will allow us to query ci_job_artifacts for unlocked and expired artifacts without needing to join ci_builds and ci_pipelines in order to get this information.

This is part of the work in #327281 (closed) and #322817 (closed) to resolve the issue we have where expired and unlocked job artifacts are not being cleaned up.

Remaining steps after this MR

  • Verify scheduled index has been created on gitlab.com.
  • Add the index for self-managed using add_concurrent_index.
  • Open an MR to propagate the value from Ci::Pipeline to Ci::JobArtifact for new pipelines and artifacts.
  • Update DestroyAllExpiredService to use the new column and index.
  • Backfill the column.

Database

Migrate up

== 20210826110839 PrepareIndexesForCiJobArtifactsExpireAtUnlocked: migrating ==
== 20210826110839 PrepareIndexesForCiJobArtifactsExpireAtUnlocked: migrated (0.0035s) 

== 20210826120834 AddLockedToCiJobArtifacts: migrating ========================
-- add_column(:ci_job_artifacts, :locked, :smallint, {:default=>2})
   -> 0.0022s
== 20210826120834 AddLockedToCiJobArtifacts: migrated (0.0239s) ===============

Migrate down

== 20210826120834 AddLockedToCiJobArtifacts: reverting ========================
-- remove_column(:ci_job_artifacts, :locked, :smallint, {:default=>2})
   -> 0.0015s
== 20210826120834 AddLockedToCiJobArtifacts: reverted (0.0243s) ===============

== 20210826110839 PrepareIndexesForCiJobArtifactsExpireAtUnlocked: reverting ==
== 20210826110839 PrepareIndexesForCiJobArtifactsExpireAtUnlocked: reverted (0.0018s)

Index to be added asynchronously over a weekend

CREATE INDEX CONCURRENTLY ci_job_artifacts_expire_at_unlocked_idx ON ci_job_artifacts (expire_at) WHERE locked = 0;

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Related to #327281 (closed)

Edited by Matt Kasa

Merge request reports