Skip to content

Create BBM to migrate ci_builds_metadata into new destinations

This issue includes also the BBM info from #555430 (closed).

Problem

While new data is being stored and queried from ci_builds and ci_job_prototypes we still have existing data stored in ci_builds_metadata that requires to be migrated.

Until we fully migrate this data into their new home in ci_builds and ci_job_prototypes respectively, we cannot delete code in Ci::Metadatable that references Ci::BuildMetadata model.

Sub-problem 2 (artifacts:expose_as)

options => artifacts:expose_as is needed long term for the MR widgets exposing artifacts and it's used when has_exposed_artfacts: true

This should be considered intrinsic data because it's used in MR widgets to display artifacts.

With #549079 (closed) we will have started writing intrinsic artifacts data to the columns exposed_as and exposed_paths in p_ci_job_artifacts. Also the logic to read from either the old or new data sources should be in place.

Proposal

Migrate all the data away from ci_builds_metadata and into ci_job_prototypes, ci_builds and the new association table for job<->environments.

You will need to work closely with the Database Team (e.g. @morefice).

Sub-proposal 2 (artifacts:expose_as)

In this issue we now migrate the existing intrinsic artifacts data from ci_build_metadata (or ci_job_definitions, depending on the timing) to p_ci_job_artifacts.

After this is migration is complete, we can update the code to only read from the new source.

Note: We should consider running pg_repack on p_ci_job_artifacts after the migration. See the usage stats of "exposed artifacts" here: #550223 (comment 2604368410).

Sub-proposal 3 (execution_config and tag_list)

We need to include execution_config and tag_list in the ci_job_definitions migration when doing a pass over ci_builds.

Edited by 🤖 GitLab Bot 🤖