Skip to content

Migrate old traces to artifacts

What does this MR do?

Concept


Today, we have four traces.

Old to New

  1. ci_builds.trace (In Database)
  2. Gitlab::Ci::Trace#deprecated_path (with project.ci_id) e.g. builds/2018_01/10/12/1021.log
  3. Gitlab::Ci::Trace#default_path (without project.ci_id) e.g. builds/2018_01/12/1021.log
  4. JobArtifactUploader#default_path (Same with artifacts) e.g. shared/artifacts/4a/44/4a44dc15364204a80fe80e9039455cc1608281820fe2b24f1e5233ade6af1dd5/2017_12_14/938/1/trace.log

We move 1, 2 and 3 to 4.


(Job_count/BATCH_SIZE) * delay = Estimated timing (i.e. The completion of the last queue)
e.g. (20000000 / 10000) * 2 min = 4000 min ≒ 2.7 days 
  • Note: When adding background migrations you must make sure they are announced in the monthly release post along with an estimate of how long it will take to complete the migrations. https://docs.gitlab.com/ce/development/background_migrations.html#cleaning-up
  • Target jobs should be completed, otherwise live-trace will be considered as un-migrated data (This will be artifacts when it's done)

Are there points in the code the reviewer needs to double check?

Why was this MR needed?

Screenshots (if relevant)

Does this MR meet the acceptance criteria?

What are the relevant issue numbers?

close https://gitlab.com/gitlab-org/gitlab-ee/issues/4639 close https://gitlab.com/gitlab-org/gitlab-ce/issues/34317

Edited by Shinya Maeda

Merge request reports