Skip to content

Fix permissions of docker volumes created by Runner

What does this MR do?

Revert "Remove cache-init command"

This reverts commit dbd6885c.

The cache-init command was removed because we started mounting volumes directly instead of using cache containers, which have rw by default (read/write). However this doesn't take into account unix permissions meaning if the user isn't part of the root group they can't write to the directory so this command needs to be re-introduced as there doesn't seem to be a way in Docker to specify permissions when mounting volumes.

Extract container wait in separate pkg

The logic on what the Runner defines as running container/finished container can be used in multiple places. Extract it into its own pkg so other pkgs can reuse this logic.

Fix permissions for docker volumes created by Runner

By default when you mount a Docker volume to a container it gets the 755 permissions, meaning that anybody can read from it but users who aren't root, can't write to it. Before !1989 (merged) be used to run cache-init on the directory before the volume was created, this is no longer the case. This breaks a lot of users script that used to write to the volumes that were mounted, and is a fairly big regression.

Create a new pkg permission that handles updating permissions to the volumes that we create, and make it more explicit that we are updating the permissions of the volumes, as before it wasn't very clear that it was being done. The permissions is implemented both for Linux and Windows, currently the windows one is noop because the goal of this commit is to fix the regression we never supported this on Windows and will be tackled in #25480.

Add an integration test so that we make sure we never have this issue anymore.

Why was this MR needed?

We can see that the permissions changed for the build directory:

v12.10.x
/ # ls -la
total 72
drwxr-xr-x    1 root     root          4096 Apr 27 09:43 .
drwxr-xr-x    1 root     root          4096 Apr 27 09:43 ..
-rwxr-xr-x    1 root     root             0 Apr 27 09:43 .dockerenv
drwxr-xr-x    2 root     root          4096 Apr 23 06:25 bin
drwxr-xr-x    3 root     root          4096 Apr 27 09:42 builds                         <--- Only root can write to it.
drwxr-xr-x    2 root     root          4096 Apr 27 09:43 cache                          <--- Only root can write to it.
drwxr-xr-x   10 root     root          3260 Apr 27 09:43 dev
drwxr-xr-x    1 root     root          4096 Apr 27 09:43 etc
drwxr-xr-x    2 root     root          4096 Apr 23 06:25 home
drwxr-xr-x    5 root     root          4096 Apr 23 06:25 lib
drwxr-xr-x    5 root     root          4096 Apr 23 06:25 media
drwxr-xr-x    2 root     root          4096 Apr 23 06:25 mnt
drwxr-xr-x    2 root     root          4096 Apr 23 06:25 opt
dr-xr-xr-x  184 root     root             0 Apr 27 09:43 proc
drwx------    1 root     root          4096 Apr 27 09:45 root
drwxr-xr-x    2 root     root          4096 Apr 23 06:25 run
drwxr-xr-x    2 root     root          4096 Apr 23 06:25 sbin
drwxr-xr-x    2 root     root          4096 Apr 23 06:25 srv
dr-xr-xr-x   13 root     root             0 Apr 27 09:43 sys
drwxrwxrwt    2 root     root          4096 Apr 23 06:25 tmp
drwxr-xr-x    7 root     root          4096 Apr 23 06:25 usr
drwxr-xr-x   12 root     root          4096 Apr 23 06:25 var
v12.9.0
/ # ls -la
total 72
drwxr-xr-x    1 root     root          4096 Apr 27 09:48 .
drwxr-xr-x    1 root     root          4096 Apr 27 09:48 ..
-rwxr-xr-x    1 root     root             0 Apr 27 09:48 .dockerenv
drwxr-xr-x    2 root     root          4096 Apr 23 06:25 bin
drwxrwxrwx    3 root     root          4096 Apr 27 09:48 builds                             <--- Everything can read/write to it
drwxrwxrwx    2 root     root          4096 Apr 27 09:48 cache                              <--- Only root can write to it.
drwxr-xr-x   10 root     root          3260 Apr 27 09:48 dev
drwxr-xr-x    1 root     root          4096 Apr 27 09:48 etc
drwxr-xr-x    2 root     root          4096 Apr 23 06:25 home
drwxr-xr-x    5 root     root          4096 Apr 23 06:25 lib
drwxr-xr-x    5 root     root          4096 Apr 23 06:25 media
drwxr-xr-x    2 root     root          4096 Apr 23 06:25 mnt
drwxr-xr-x    2 root     root          4096 Apr 23 06:25 opt
dr-xr-xr-x  183 root     root             0 Apr 27 09:48 proc
drwx------    1 root     root          4096 Apr 27 09:49 root
drwxr-xr-x    2 root     root          4096 Apr 23 06:25 run
drwxr-xr-x    2 root     root          4096 Apr 23 06:25 sbin
drwxr-xr-x    2 root     root          4096 Apr 23 06:25 srv
dr-xr-xr-x   13 root     root             0 Apr 27 09:48 sys
drwxrwxrwt    2 root     root          4096 Apr 23 06:25 tmp
drwxr-xr-x    7 root     root          4096 Apr 23 06:25 usr
drwxr-xr-x   12 root     root          4096 Apr 23 06:25 var

Looking both at ContainerCreate, and VolumeCreate it doesn't seem like we can set permissions/owner on the mounted file and seems like it's a limitation of Docker itself. As pointed out :rw doesn't have to do anything with permissions.

Testing

Users confirming that everything is working:

  1. #25440 (comment 334652133)
  2. #25440 (comment 335787609)

Linux

.gitlab-ci.yml
job:
  image: registry.gitlab.com/gitlab-org/ci-cd/tests/gitlab-test/nonroot:2e02d8b1
  stage: test
  script:
  - echo "test" > /test/test.txt
config.toml
[[runners]]
  name = "docker"
  url = "http://192.168.190.160:3000"
  token = "fL_5iHR7khbpFE41AFMm"
  executor = "docker"
  [runners.docker]
    tls_verify = false
    image = "registry.gitlab.com/gitlab-org/ci-cd/tests/gitlab-test/nonroot:2e02d8b1"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache", "/test"]
    shm_size = 0
    pull_policy = "if-not-present"
    helper_image = "gitlab/gitlab-runner-helper:x86_64-4c96e5ad"

Running the jobs on different version of the runners, running docker volume prune and deleting any other volumes that are used to make sure a fresh set of volume is created everytime.

Windows

On windows this is not yet supported, it never worked 👉 #25480

Does this MR meet the acceptance criteria?

  • Documentation created/updated
  • Added tests for this feature/bug
  • In case of conflicts with master - branch was rebased

What are the relevant issue numbers?

Closes #25440 (closed)

Edited by Yorick Peterse

Merge request reports