fix: Use scripts/start.sh for proper handling of non-zero exitcodes
What does this MR do?
Depends on gitlab-org/security-products/post-analyzers/scripts!5 (merged) to get passing pipeline
Fixes bug where non-zero exitcodes were not properly returned, resulting in invalid configuration not failing.
This is ultimately attributed to the use of the older/incorrect start.sh packaged with TC. We should deprecate this script for the more sophisticated https://gitlab.com/gitlab-org/secure/tools/analyzer-scripts/ version.
Testcase
This is a bit complex locally but can be done like so:
Build scripts container
❯ pwd
/Users/theoretick/code/gl/security-products/post-analyzers/scripts
❯ analyzer-build
tag: scripts:run-vet-regardless-of-tracking-calculator
[+] Building 0.2s (7/7) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 84B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/alpine:3.14 0.0s
=> [1/2] FROM docker.io/library/alpine:3.14 0.0s
=> [internal] load build context 0.1s
=> => transferring context: 1.15kB 0.0s
=> CACHED [2/2] COPY /start.sh /start.sh 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:2c157b150c61623c1bf02f61fe7a389907eb8ba4dae05d9ec09f9808deabfb67 0.0s
=> => naming to docker.io/library/scripts:run-vet-regardless-of-tracking-calculator 0.0sBuild semgrep container
❯ pwd
/Users/theoretick/code/gl/security-products/analyzers/semgrep
❯ git diff Dockerfile
diff --git a/Dockerfile b/Dockerfile
index 0eeb80c..f6a991a 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -2,7 +2,8 @@ ARG SCANNER_VERSION=0.69.1
ARG POST_ANALYZER_SCRIPTS_VERSION=0.0.3
ARG TRACKING_CALCULATOR_VERSION=2.2.3
-FROM registry.gitlab.com/security-products/post-analyzers/scripts:${POST_ANALYZER_SCRIPTS_VERSION} AS scripts
+# FROM registry.gitlab.com/security-products/post-analyzers/scripts:${POST_ANALYZER_SCRIPTS_VERSION} AS scripts
+FROM scripts:run-vet-regardless-of-tracking-calculator AS scripts
FROM registry.gitlab.com/security-products/post-analyzers/tracking-calculator:${TRACKING_CALCULATOR_VERSION} AS tracking
FROM golang:1.15-alpine AS build
❯ analyzer-build
tag: semgrep:use-start-script
[+] Building 1.7s (23/23) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.72kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/python:3.9-alpine 1.2s
=> [internal] load metadata for docker.io/library/golang:1.15-alpine 0.0s
=> [internal] load metadata for registry.gitlab.com/security-products/post-analyzers/tracking-calculator:2.2.3 0.6s
=> [internal] load metadata for docker.io/library/scripts:run-vet-regardless-of-tracking-calculator 0.0s
=> [stage-3 1/8] FROM docker.io/library/python:3.9-alpine@sha256:eb1b2038f12c8916be54329319a625de2dfec4266b718efdb798cc149b342a2f 0.0s
=> [build 1/5] FROM docker.io/library/golang:1.15-alpine 0.0s
=> [tracking 1/1] FROM registry.gitlab.com/security-products/post-analyzers/tracking-calculator:2.2.3@sha256:62cc014aa3848db552e950209accfef17896b186c6731697622ba88ba0a34147 0.0s
=> [scripts 1/1] FROM docker.io/library/scripts:run-vet-regardless-of-tracking-calculator 0.0s
=> [internal] load build context 0.4s
=> => transferring context: 8.80MB 0.4s
=> CACHED [build 2/5] WORKDIR /go/src/buildapp 0.0s
=> CACHED [build 3/5] COPY . . 0.0s
=> CACHED [build 4/5] RUN CHANGELOG_VERSION=$(grep -m 1 '^## v.*$' "CHANGELOG.md" | sed 's/## v//') && PATH_TO_MODULE=`go list -m` && go build -ldflags="-X ' 0.0s
=> CACHED [build 5/5] RUN addgroup -g 1000 semgrep && adduser -u 1000 -D -h /home/semgrep -G semgrep semgrep && touch /ca-cert-additional-gitlab-bundle.pem && ch 0.0s
=> CACHED [stage-3 2/8] COPY --from=build /analyzer-semgrep /analyzer-binary 0.0s
=> CACHED [stage-3 3/8] COPY --from=build /ca-cert-additional-gitlab-bundle.pem /etc/ssl/certs/ca-cert-additional-gitlab-bundle.pem 0.0s
=> CACHED [stage-3 4/8] COPY rules /rules 0.0s
=> CACHED [stage-3 5/8] RUN mkdir /.cache && chmod -R g+rw /.cache 0.0s
=> CACHED [stage-3 6/8] RUN apk add --no-cache git && pip install semgrep==0.69.1 0.0s
=> CACHED [stage-3 7/8] COPY --from=tracking /analyzer-tracking /analyzer-tracking 0.0s
=> CACHED [stage-3 8/8] COPY --from=scripts /start.sh /analyzer 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:21705d84684fa68c2f6cd9cdc120c5f79bf014e2136159b1a24c44109bece973 0.0s
=> => naming to docker.io/library/semgrep:use-start-script 0.0sTest against python-pip-flask-custom-rulesets-with-passthrough
❯ cd -
~/code/gl/security-products/analyzers/semgrep/test/fixtures/python-pip-flask-custom-rulesets-with-passthrough
❯ docker run --rm --volume "$PWD":/tmp/app --env GITLAB_FEATURES="sast_custom_rulesets,vulnerability_finding_signatures" --env CI_PROJECT_DIR="/tmp/app" semgrep:use-start-script
[INFO] [Semgrep] [2021-10-25T21:22:55Z] ▶ GitLab Semgrep analyzer v2.13.4
[INFO] [Semgrep] [2021-10-25T21:22:55Z] ▶ Detecting project
[INFO] [Semgrep] [2021-10-25T21:22:55Z] ▶ Found relevant files in project, analyzing entire repository
[INFO] [Semgrep] [2021-10-25T21:22:55Z] ▶ Running analyzer
[FATA] [Semgrep] [2021-10-25T21:22:55Z] ▶ stat /tmp/app/semgrep_rulesssss: no such file or directory
❯ echo $?
1What are the relevant issue numbers?
gitlab-org/gitlab#342434 (closed)
Does this MR meet the acceptance criteria?
-
Changelog entry added -
Documentation created/updated for GitLab EE, if necessary -
Documentation created/updated for this project, if necessary -
Documentation reviewed by technical writer or follow-up review issue created -
Tests added for this feature/bug -
Job definition updated, if necessary -
Conforms to the code review guidelines -
Conforms to the Go guidelines -
Security reports checked/validated by reviewer
Edited by Zach Rice