Skip to content

Add new column to build_artifacts_size_refresh table

Ref: #362640 (closed)

What does this MR do and why?

This MR does 2 things:

  • Adds a new last_job_artifact_id_on_refresh_start column to BuildArtifactsSizeRefresh
  • Persists the data when a new BuildArtifactsSizeRefresh is being scheduled

We will also introduce a new index in !88427 (merged) to make sure this query performs well on production.

We don't need to wait for the index creation as we are not using this query yet. It will be used when we will run our script to recalculate all artifacts' size in #332994 (closed).

Why are we doing this?

To prevent a typebug from happening as pointed out by @fabiopitino when an object gets saved and created_at is filled in before the id.

This will be used to recalculate job artifacts size which is currently broken for some projects.

SQL before

SELECT ci_job_artifacts.id, ci_job_artifacts.size
FROM ci_job_artifacts
WHERE ci_job_artifacts.project_id = 278964
AND ci_job_artifacts.created_at <= '2022-05-08 13:09:28.796704'
AND ci_job_artifacts.id > 29
ORDER BY ci_job_artifacts.created_at ASC, ci_job_artifacts.id ASC
LIMIT 1000;

SQL after with new index

  • 11.986 ms
SELECT "ci_job_artifacts"."id", "ci_job_artifacts"."size"
FROM "ci_job_artifacts"
WHERE "ci_job_artifacts"."project_id" = 278964
AND "ci_job_artifacts"."id" > 29
AND "ci_job_artifacts"."id" <= 5000
ORDER BY "ci_job_artifacts"."id" ASC
LIMIT 1000;
 Limit  (cost=0.58..119.04 rows=82 width=16) (actual time=0.059..0.090 rows=8 loops=1)
   ->  Index Scan using index_ci_job_artifacts_on_project_id_and_id on ci_job_artifacts  (cost=0.58..119.04 rows=82 width=16) (actual time=0.057..0.086 rows=8 loops=1)
         Index Cond: ((project_id = 278964) AND (id > 29) AND (id <= 5000))
 Planning Time: 0.477 ms
 Execution Time: 11.986 ms
(5 rows)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Max Orefice

Merge request reports