Implement Stop writing pipeline variables to DB behind ci_stop_writing_to_pipeline_variables feature flag
What does this MR do and why?
Background
Currently pipeline variables take up a lot of space in the database table p_ci_pipeline_variables. Our goal in the epic gitlab-org#19989 is to offload this data to object storage by leveraging our existing pipeline artifacts framework to store pipeline variables as encrypted JSON artifacts.
We have completed the following phases:
-
Dual-write (
ci_write_pipeline_variables_artifact): Pipeline variables are written to both DB and object storage. This has been rolled out to 100%, FF cleaned up. -
Reads from artifact (
ci_read_pipeline_variables_from_artifact):Ci::Pipeline#variablesreads from the artifact with DB fallback. This has been rolled out to 100%, FF cleaned up.
This MR
This MR introduces the next phase: stop writing pipeline variables to the DB behind the ci_stop_writing_to_pipeline_variables feature flag.
When the FF is enabled:
-
Ci::Pipelinerejects nested attributes for variables viareject_variables_attributes?, preventing DB writes during pipeline creation -
Chain::Build::Associations#assign_pipeline_variablesskips settingvariables_attributeson the pipeline, but still builds the encrypted artifact via PipelineVariablesArtifactBuilder -
Ci::PipelineVariable#readonly?prevents mutations after creation to keep DB and artifact data in sync
When the FF is disabled (default for now):
- Existing dual-write behaviour continues unchanged
Changes
-
app/models/ci/pipeline.rb: Addedreject_variables_attributes?method; updatedaccepts_nested_attributes_forto use it -
app/models/ci/pipeline_variable.rb: Addedreadonly?method to prevent post-creation mutations -
lib/gitlab/ci/pipeline/chain/build/associations.rb: Wrappedvariables_attributesassignment with FF check -
config/feature_flags/gitlab_com_derisk/ci_stop_writing_to_pipeline_variables.yml: New FF definition - Spec coverage for all changes
References
- Part of epic: gitlab-org#19989
- Issue: #580848 (closed)
- Writes FF cleanup: !224565 (merged)
- Reads FF cleanup: !226146 (merged)
Related to #580848 (closed)