Support Preparedness 17.11 - Change - GitLab Runner - Default FF_USE_NATIVE_STEPS to true

Summary

The purpose of this issue is to:

  • Alert support folks of an upcoming change in Gitlab Runner that has the potential to cause CI job failures.

What is happening

Starting with GitLab Runner release %17.11 , we will change the feature flag FF_USE_NATIVE_STEPS to default to true, which will enable the corresponding feature. This change will apply initially only to self-hosted runners since we typically deploy hosted/instance runners on a different schedule.

The impact of the change is that steps will now run natively in the build container specified by the image: setting using the gRPC API, rather than from a separate container maintained by GitLab. This introduces a dependency where the build container needs to trust the certificate used by the server hosting the step. And there may be other dependencies for some edge cases that have not yet been identified.

Affected Runners

This change only affects CI jobs that satisfy both of these conditions:

Both of these conditions must be true for a job to be potentially impacted. In practice this will likely only affect early adopters of steps.

Related Issues/MRs/Epics

Status / What actions have been taken so far

None

Timeline / Important Dates

%17.11 will be released April 17, but any issues will likely show up sometime later (when folks upgrade their private runners, and when we deploy hosted/instance runners).

What impact will this have on users?

This may cause CI jobs to fail.

What this may look like for Support

Anticipated Support Impact: Low/Unknown

What errors or messages users may report: Some of the currently known issues are listed in https://docs.gitlab.com/runner/executors/docker/#known-issues and https://docs.gitlab.com/ci/steps/#troubleshooting, but there may be other failure modes we have not seen yet.

What workarounds/solutions are available?

The feature is controlled by a feature flag and can be disabled in both the CI and runner configurations.

CI Config

The feature can be disabled by developers in the CI configuration, either globally or for individual jobs by setting the variable FF_USE_NATIVE_STEPS to false:

globally

variables:
    FF_USE_NATIVE_STEPS: false

or for an individual job.

my_job:
	variables:
    	FF_USE_NATIVE_STEPS: false

Runner Config

The feature can be also be disabled entirely for a runner in the runner configuration in two ways:

  1. By setting FF_USE_NATIVE_STEPS to false in the environment section:
[[runners]]
  ...
  environment = ["FF_USE_NATIVE_STEPS=false"]

Note that CI variables take precedence over runner environment setting, so if FF_USE_NATIVE_STEPS so explicitly set to true in the CI config, it will override the value in the runner environment config.

  1. By setting FF_USE_NATIVE_STEPS to false in the runners.feature_flags section:
[[runners]]
  ...
  [runners.feature_flags]
    FF_USE_NATIVE_STEPS = false

Note that unlike the environment approach, this approach cannot be overridden by setting FF_USE_NATIVE_STEPS in the CI configuration, and is thus a safer approach if users want to definitively and absolutely disable this feature.

Do users need to be contacted?

  • No

DRIs/Contacts for questions and approvals for communications/action items

Communication to Support team

  1. Announced to team in
    1. #support_gitlab-com
Edited by Justin Farmiloe