Create CI Job Definition relations when pipeline creation
What does this MR do and why?
!201445 (merged) needs to be merged first
This MR implements the foundation for saving CI Job Definitions during pipeline creation as part of the broader CI data normalization initiative.
Quick summary:
- Ci::JobDefinition model to store immutable, deduplicated job configuration data
- Ci::JobDefinitionInstance model to link jobs to their definitions
- Jobs with identical configuration share the same JobDefinition record
- Changes are behind the
write_to_new_ci_destinations
feature flag
References
- Previous MR: !201445 (merged)
- Related to Add logic to populate new destinations when pip... (#551860 - closed)
- Part of Reduce growth rate of ci_builds_metadata partit... (&13886)
How to set up and validate locally
-
Enable the feature flag:
Feature.enable(:write_to_new_ci_destinations)
-
Create a pipeline with jobs that have different configurations:
test1: script: echo "test 1" variables: VAR1: value1 test2: script: echo "test 2" interruptible: true test3: # Same as test1 script: echo "test 1" variables: VAR1: value1
-
Verify job definitions are created:
pipeline = Ci::Pipeline.last pipeline.processables.each do |job| puts "#{job.name}: JobDefinition ID #{job.job_definition&.id}" end; nil
-
Expected:
test1
andtest3
should share the sameJobDefinition
IDtest1: JobDefinition ID 4 test2: JobDefinition ID 5 test3: JobDefinition ID 4
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Edited by Furkan Ayhan