Skip to content

[Feature flag] Enable evaluation of variables inside variables

What

Remove the :variable_inside_variable feature flag introduced in gitlab-runner#26345 (closed)...

Owners

  • Team: Verify:Runner
  • Most appropriate slack channel to reach out to: #g_runner
  • Best individual to reach out to: @pedropombeiro

Expectations

What are we expecting to happen?

  • If the :variable_inside_variable FF is disabled, absolutely nothing should change compared to %13.10
  • If it is enabled, then variables that are evaluated to be sent to the Runner as part of a JobResponse should have values expanded. I.e. if VAR_A="world" and VAR_B="Hello ${VAR_A}!", then the runner will see VAR_B as "Hello world!". If VAR_A is changed to $VAR_B, a cyclic dependency will be detected and, at a low level, unexpanded variables will be returned (as if the FF were disabled). At a higher level, the error will be displayed in the UI when a build is being started.

What might happen if this goes wrong?

In case of a logic failure, the user might see:

  • an unhandled TSort::Cyclic exception, or a missing method exception, as if we were treating a hash as a Gitlab::Ci::Variables::Collection::Item object.
  • wrong variable values suddenly appear (e.g. gitlab-runner#1809 (comment 679726314))

What can we monitor to detect problems with this?

Unfortunately, I'm not aware of any automated way to verify that the deployment is going smoothly other than be on the lookout for reports from users regarding sudden failing builds.

Beta groups/projects

If applicable, any groups/projects that are happy to have this feature turned on early. Some organizations may wish to test big changes they are interested in with a small subset of users ahead of time for example.

  • gitlab-org/gitlab project
  • gitlab-org/gitlab-runner project
  • tmobile/templates_projects/helloworld-1 project
  • vistaprint-org/design-technology/ds/innovation/pipeline-test project

Roll Out Steps

  • Confirm that QA tests pass with the feature flag enabled (if you're unsure how, contact the relevant stable counterpart in the Quality department)
  • Enable on staging (/chatops run feature set variable_inside_variable true --staging)
  • Test on staging
  • Ensure that documentation has been updated
  • Enable on GitLab.com for individual groups/projects listed above and verify behaviour (/chatops run feature set --project=gitlab-org/gitlab variable_inside_variable true)
  • Coordinate a time to enable the flag with the SRE oncall and release managers
    • In #production mention @sre-oncall and @release-managers. Once an SRE on call and Release Manager on call confirm, you can proceed with the rollout
  • Announce on the issue an estimated time this will be enabled on GitLab.com
  • Enable on GitLab.com by running chatops command in #production (/chatops run feature set variable_inside_variable true)
  • Cross post chatops Slack command to #support_gitlab-com (more guidance when this is necessary in the dev docs) and in your team channel
  • Announce on the issue that the flag has been enabled
  • Remove feature flag and add changelog entry
  • After the flag removal is deployed, clean up the feature flag by running chatops command in #production channel

Rollback Steps

  • This feature can be disabled by running the following Chatops command:

NOTE: We should lean towards isolating customer issues and avoid going all out and disabling this for all customers since quite a few of them might have already modified their pipelines to take advantage of the functionality.

/chatops run feature set variable_inside_variable false
Edited by Pedro Pombeiro