GitLab Runner using cached images even when disallowed
Summary
The GitLab Runner can use images cached on it even when this is disallowed by setting allowed_pull_policies = ["always"]
. This can provide everyone using the GitLab Runner with access to potentially sensitive code stored inside the image.
Steps to reproduce
The git repository structure is not relevant. All you need is an arbitrary job with pull_policy: [if-not-present, always]
and running it on GitLab Runner with the default settings for allowed_pull_policies
(i.e., allowed_pull_policies = ["always"]
).
.gitlab-ci.yml
stages:
- test
test:
stage: test
image:
name: python:3.12-slim
pull_policy: [if-not-present, always]
script:
- echo "Running tests ..."
Actual behavior
Assuming that the relevant image is not cached on the GitLab runner, the image is simply pulled when the job runs for the first time. In subsequent runs, the image is not pulled, but the cached version is used instead.
If I instead use pull_policy: if-not-present
for the job, I get the following error, as expected:
ERROR: Job failed: invalid pull policy for image "python:3.12-slim": pull_policy ([if-not-present])
defined in GitLab pipeline config is not one of the allowed_pull_policies ([always])
It appears that the GitLab Runner first checks that the set of pull policies specified in pull_policy
contains at least one policy that is allowed, but then executes the policies in the specified order without regard to what is and is not allowed.
Expected behavior
The image should be pulled even after the first run because the if-not-present policy is not allowed on the GitLab runner.
Relevant logs and/or screenshots
The following is the initial part of the log from running the job from Steps to reproduce.
job log
Running with gitlab-runner 17.10.0 (67b2b2db)
on docker-runner-1 t1_hERzWK, system ID: s_687ab8a707ba
Preparing the "docker" executor
Using Docker executor with image python:3.12-slim ...
Using locally found image version due to "if-not-present" pull policy
Environment description
Private GitLab instance using the official Docker image gitlab/gitlab-ce:17.10.0-ce.0.
config.toml contents
concurrent = 1
check_interval = 0
connection_max_age = "15m0s"
shutdown_timeout = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "docker-runner-1"
url = <redacted>
id = 11
token = <redacted>
token_obtained_at = 2025-03-23T23:43:19Z
token_expires_at = 0001-01-01T00:00:00Z
executor = "docker"
[runners.cache]
MaxUploadedArchiveSize = 0
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "docker"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/runner/services/docker", "/cache"]
shm_size = 0
network_mtu = 0
Used GitLab Runner version
Running with gitlab-runner 17.10.0 (67b2b2db)
on docker-runner-1 t1_hERzWK, system ID: s_687ab8a707ba
Preparing the "docker" executor
Using Docker executor with image python:3.12-slim ...
Possible fixes
Unknown.