Copy deployment intrinsic data outside options

Problem

In #545659 (comment 2561128828) we discussed that some deployments data currently stored in ci_builds_metadata.config_options needs to be stored in a different/dedicated table that groupenvironments maintains.

The data is action, deployment_tier and kubernetes.namespace : #545659 (comment 2557799464).

In an initial phase this data will be automatically migrated into p_ci_job_prototypes as part of CI storage optimization - Phase 2: write to new... (&18268) so that we can stop writing to p_ci_builds_metadata. However, while this data can contribute to the deduplication of job prototypes, the p_ci_job_prototypes table should only contain processing data that can potentially be removed when pipelines are archived.

Proposal

Store this data into a separate table, referencing back to ci_builds

  1. create new table
  2. ensure new job records will persist the copy of the data into the new table
  3. create BBM to move data from options in p_ci_job_prototypes and ci_build_metadata into the new table - moved to #573186 (closed).

Note: we can keep the data also stored in p_ci_job_prototypes.config but it should be also persisted in the new table and used as SSoT from there, not from options since this data can disappear in the future.

⚠️ this issue needs to be coordinated with Create BBM to migrate `ci_builds_metadata` into... (#552069) • Marius Bobin • 18.6 • On track and other issues in the same epic ⚠️

Implementation details

To avoid code referencing the data from options we could implement a pattern in Ci::JobPrototypes that allows to store data but later to block reads, so that we can ensure that data such as deployment_tier is fetched using the new table. If fetched through options it should always return nil even though the data exists there.

  • Initially we want to write and read from options until this issue is implemented and we have a dedicated table.
  • After we migrate/copy the data from options (ci_job_prototypes.config['options']) into the new table, we could enforce the pattern above where reads from options for specific keys will return nil.
Edited by Tiger Watson