Introduce PipelineVariableItem virtual model to represent variables from object storage
What does this MR do and why?
Context
Currently pipeline variables take up a lot of space in the database table p_ci_pipeline_variables. Our goal in the epic gitlab-org#19989 is to offload this data to object storage. To do this, we will leverage our existing pipeline artifacts framework to store the pipeline variables as JSON in an artifact.
We have an approved POC MR (!218525 (diffs)) that contains the overall approach for this objective.
This MR
This MR is the first step of the "write to object storage" component, based on the POC MR.
Specifically, it introduces a new Ci::PipelineVariableItem class that is meant to serve as a virtual model for pipeline variables handled outside of the DB-storage context. As such, it mirrors many of the behaviours of the existing Ci::PipelineVariable model (including the Ci::HasVariable module).
This new class is intended to preserve the validation, attributes, and other essential behaviours from Ci::PipelineVariable to support transforming and validating the data before we store it as a pipeline artifact in a new builder class (to be implemented in the next MR).
This MR also includes a couple minor changes:
- Replaces the hardcoded enum values for
variable_typewith a new constantEnums::Ci::Variable::TYPES. - Updates
Ci::Variables::Collection::Itemto support the newCi::PipelineVariableItemclass. This will come into play in a later MR when variables are read and fabricated into variables collection items.
References
- This MR resolves the task: Introduce PipelineVariableItem class to represe... (#586933 - closed)
- Next task: Introduce PipelineVariablesArtifactBuilder class (#586934 - closed)
- Parent issue: Store pipeline variables as encrypted pipeline_... (#580107 - closed)
- POC MR: Draft: POC Write and read pipeline variables fr... (!218525 - closed)
How to set up and validate locally
- In the Rails console, you can create a new instance of
Ci::PipelineVariableTypeand observe that the attributes/methods output as expected.
pipeline = Ci::Pipeline.last
variable = Ci::PipelineVariableItem.new(pipeline: pipeline, key: 'MY_KEY', value: 'my_value')
variable.attributes
variable.to_hash_variable
variable.to_global_id
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Related to #586933 (closed)