Support non-root Dockerfiles in SAST integration-test
Problem to solve
Recently, the Go and C downstream tests in Semgrep were replaced by integration-test. This is part of a larger effort to reduce reliance on downstream QA tests across groupstatic analysis (see #336821 (closed) for more context).
During this work, we found that tests against the FIPS variant of the Semgrep image were failing due to permissions issues. In order to work iteratively, it was decided to exclude the FIPS tests for C and Go in the initial MR.
We're running into permissions issues due to the way that integration-test
executes, and because FIPS images run under the gitlab
user instead of root
:
- The
integration-test
runner is executed as a Docker container. The current working directory of your computer is bind-mounted into it. The container runs under theroot
user. - For each test project (C, Go, etc.) the
integration-test
container uses Docker-in-Docker to run the analyser. It bind mounts the test project that was bind-mounted from your computer, into the analyser at/app
so it can be scanned. - The analyser executes (under the
gitlab
user) and attempts to write thegl-sast-report.json
artefact into the test project. This action fails because the directory is not owned bygitlab
(because it was mounted by theintegration-test
container) which thegitlab
user does not have permissions to write into.
Example
As an example, this is the flow when running integration-test
against the Semgrep FIPS image locally.
- I checkout and build the integration-test Dockerfile locally.
git clone git@gitlab.com:gitlab-org/security-products/analyzers/integration-test.git
cd integration-test
docker build -t integration-test-jliu .
- I checkout and build Semgrep.
git clone git@gitlab.com:gitlab-org/security-products/analyzers/semgrep.git
cd semgrep
docker build -t semgrep:jliu-fips-integration-test .
- I run the integration-test runner. Two bind-mounts are created, one to pass through the entirety of the
semgrep
checkout, and another to pass through my Docker daemon socket.
cd semgrep
docker run -it --rm -v "$PWD:$PWD" -w "$PWD" \
-e TMP_IMAGE=semgrep:jliu-fips-integration-test \
-v /var/run/docker.sock:/var/run/docker.sock \
integration-test-jliu rspec
-
Internally, the
integration-test
runner executes the analyser against the test project fixture. In this case it's testing the C project.
Notice that it bind-mounts a subdirectory of my semgrep
checkout into /app
of the analyser container. This is the -v /Users/james/code/gl/security-products/analyzers/semgrep/tmp/test-46070/c/running-image-with-test-project-with-c-behaves-like-successful-scan-creates-a-report:/app
param.
docker run -t --rm -v /Users/james/code/gl/security-products/analyzers/semgrep/tmp/test-46070/c/running-image-with-test-project-with-c-behaves-like-successful-scan-creates-a-report:/app -w /app --env ANALYZER_INDENT_REPORT="true" --env CI_PROJECT_DIR="/app" semgrep:jliu-fips-integration-test
- The analyser runs the scan against the C test project at
/app
and attempts to write thegl-sast-report.json
artefact into/app
. This action fails because the analyser is running under thegitlab
user, but/app
is owned by someone else.
Proposal
I think there are two options:
- Standardise on non-root users for groupstatic analysis analysers, and adjust the integration-test Dockerfile to use the same user. This requires some coordination, but there has been some prior discussions on using non-root users.
- Adjust the options passed to the Docker command that spins up the analyser to forcefully use the
root
user. I tested this locally and it works.
Another option is to override the Docker user in the CI config. When running the non-FIPS integration-test, specify no user, but when running the FIPS integration-test, override to use the gitlab
user. Unfortunately it isn't possible to specify an alternative Docker user for GitLab CI jobs.
Addendum
I initially thought the /app
directory was owned by root
, but it's more peculiar than that. If I run integration-test
interactively:
docker run -it --rm -v "$PWD:$PWD" -w "$PWD" \
-e TMP_IMAGE=semgrep:jliu-fips-integration-test \
-v /var/run/docker.sock:/var/run/docker.sock \
integration-test-jliu /bin/ash
and then check the permissions on the $PWD
bind mount:
/Users/james/code/gl/security-products/analyzers/semgrep # ls -la
total 228
drwxr-xr-x 1 501 dialout 1056 Nov 2 00:02 .
dialout
???
If I then run the analyser container within the integration-test
container:
/Users/james/code/gl/security-products/analyzers/semgrep # docker run -it --rm -v /Users/james/code/gl/security-products/analyzers/semgrep/tmp/test-46070/c/running-image-with-test-project-wit
h-c-behaves-like-successful-scan-creates-a-report:/app semgrep:jliu-fips-integration-test /bin/bash
and check the permissions:
[gitlab@0966ddd4af4d /]$ ls -la /app
total 220
drwxr-xr-x 1 501 games 224 Nov 2 03:53 .
drwxr-xr-x 1 root root 4096 Nov 2 04:25 ..
-rw-r--r-- 1 501 games 179 Nov 2 03:53 Makefile
-rw------- 1 501 games 3411 Nov 2 03:53 gl-sast-report.json
-rw-r--r-- 1 501 games 72 Nov 2 03:53 hello.c
-rw-r--r-- 1 501 games 198484 Nov 2 03:53 semgrep.sarif
drwxr-xr-x 1 501 games 96 Nov 2 03:53 subdir
what the heck is games
?