Pull helper image from GitLab.com registry

Steve Xuereb - Out of Office back 2026-01-05requested to merge
27196-pull-image-from-gitlab-registry into master

What does this MR do?

Pull helper image from GitLab.com registry behind the FF_GITLAB_REGISTRY_HELPER_IMAGE feature flag

Why was this MR needed?

In !2540 (merged) the helper images are also published on registry.gitlab.com because of the new DockerHub limits.

By default, Docker and Kubernetes runners are pulling images from DockerHub, when FF_GITLAB_REGISTRY_HELPER_IMAGE feature flag is turned on, it will pull the helper image from registry.gitlab.com. This prevents users from reaching the DockerHub limits when just running a build.

What's the best way to test this MR?

Docker Linux

  1. Have the following .gitlab-ci.yml

    .gitlab-ci.yml 
    job:
  stage: test
 script:
 - sleep 300

  2. Make sure you don't have gitlab/gitlab-runner-helper:x86_64-latest locally, if you do run docker rmi gitlab/gitlab-runner-helper:x86_64-latest

  3. Start the runner for example with go run main.go run -c config.toml

    example config.toml 
    [[runners]]
  name = "docker"
  url = "http://192.168.1.79:3000/"
  token = "xxxx"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "alpine:3.12"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache"]
    pull_policy = "if-not-present"
    shm_size = 0

  4. Run the job

  5. You should see the following inside the job trace

  6. Make sure you don't have registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-latest locally, if you do run docker rmi registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-latest

  7. Run the same job, but this time set FF_GITLAB_REGISTRY_HELPER_IMAGE to true either in the config.toml or when you start the pipeline. More information on how to enable a feature flag

  8. You should see the following inside the job trace

Docker Windows

  1. Have the following .gitlab-ci.yml

    .gitlab-ci.yml 
    job:
  stage: test
 script:
 - sleep 300

  2. Start windows dev environment vagrant up

  3. Run vagrant ssh or vagrant rdp to log into the Windows machine

  4. Make sure you don't have gitlab/gitlab-runner-helper:x86_64-latest-servercore1809 locally, if you do run docker rmi gitlab/gitlab-runner-helper:x86_64-latest-servercore1809

  5. Start the runner for example with go run .\main.go run -c .\config.toml

    example config.toml 
    [[runners]]
  name = "docker-windows"
  url = "http://192.168.190.160:3000/"
  token = "xxxx"
  executor = "docker-windows"
  [runners.docker]
    tls_verify = false
    image = "mcr.microsoft.com/windows/servercore:1809-amd64"
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["C:\\cache"]

  6. Run the job

  7. You should see the following in the job trace

  8. Run the same job, but this time set FF_GITLAB_REGISTRY_HELPER_IMAGE to true either in the config.toml or when you start the pipeline. More information on how to enable a feature flag

  9. You should see the following in the job trace

Kubernetes

    1. Have the following .gitlab-ci.yml
    .gitlab-ci.yml 
    job:
  stage: test
 script:
 - sleep 300

  2. Start minikube minikube start

  3. Run GitLab Runner for example go run main.go run -c config.toml

    config.toml 
    [[runners]]
  name = "kubernetes"
  url = "http://192.168.1.79:3000/"
  token = "xxxxxxx"
  executor = "kubernetes"
  [runners.kubernetes]
    bearer_token_overwrite_allowed = false
    image = "alpine:3.12"

  4. When the job is running, run the following command to check which help image was pulled

    1. Find out pod running job from job trace Running on runner-mkp1g3gh-project-20-concurrent-0mxspm via steve-mbp-gitlab.local...

    2. kubectl get po/runner-mkp1g3gh-project-20-concurrent-0mxspm -o json | jq '.spec.containers | keys[] as $k | "\\(.[$k] | .image)"'

      "alpine:3.12"
"gitlab/gitlab-runner-helper:x86_64-latest"

  5. Run the same job, but this time set FF_GITLAB_REGISTRY_HELPER_IMAGE to true either in the config.toml or when you start the pipeline. More information on how to enable a feature flag

  6. When the job is running, run the following command to check which help image was pulled

    1. Find out pod running job from job trace Running on runner-mkp1g3gh-project-20-concurrent-1hsw7h via steve-mbp-gitlab.local...

    2. kubectl get po/runner-mkp1g3gh-project-20-concurrent-0mxspm -o json | jq '.spec.containers | keys[] as $k | "\\(.[$k] | .image)"'

      "alpine:3.12"
"registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-latest"

Test local import for Docker executor

  1. Make sure you don't have the image locally with the name of gitlab/gitlab-runner-helper: docker images -a | grep "gitlab-runner-helper" | awk '{print $3}' | xargs docker rmi

  2. Build image archive: make helper-dockerarchive TAR_XZ="out/helper-images/prebuilt-x86_64.tar.xz"

  3. Run test that will import the image: go test -ldflags (make print_ldflags) -run TestDockerCommandMultistepBuild -v ./executors/docker/

  4. Check that the image is now imported by running docker images | grep "gitlab-runner-helper". The Created Dated should be in seconds/minutes since it was just created when you ran the test.

    gitlab/gitlab-runner-helper                                          x86_64-34026ee0     2e0a58807cb2        47 seconds ago      64.4MB

What are the relevant issue numbers?

Phase 2 #27196 (closed)

Edited by Steve Xuereb - Out of Office back 2026-01-05

Merge request reports