Skip to content

Unexpected failures on semgrep analyzer

Summary

semgrep is now returning proper exit codes correctly which is causing a spike in reported failure rates. This is to be expected but we appear to be failing in certain unclear cases.

Most recently, semgrep is returning a exitcode 3 on https://cdn.jsdelivr.net/npm/chart.js@3.5.1/dist/chart.min.js which is causing a failure. I would guess this is due to the file being considered unparseable due to either LoC size or some other constraint.

In other cases, semgrep is returning exitcode 2 when rule evaluations timeout, see default --timeout of 30sec

In other cases, semgrep is returning exitcode 2 when ignore annotations to not match rules that are currently loaded. See https://gitlab.com/chris-semgrep/semgrep-exit-status-2

Steps to reproduce

  1. Download https://cdn.jsdelivr.net/npm/chart.js@3.5.1/dist/chart.min.js
  2. Run docker run -it --rm --volume "$PWD":/tmp/app --env SECURE_LOG_LEVEL=debug --env CI_PROJECT_DIR="/tmp/app" registry.gitlab.com/gitlab-org/security-products/analyzers/semgrep:2 /analyzer analyze /tmp/app
  3. Observe exit code

Example Project

What is the current bug behavior?

Semgrep fails with exitcode 3 for unknown error

What is the expected correct behavior?

Semgrep doesn't fail

Relevant logs and/or screenshots

Output of checks

This bug happens on GitLab.com

Possible fixes

Either rescue this specific exit code (if semgrep is best-effort) or file upstream on failing file

Workaround

Pin SAST job to v2.13.1:

include:
  - template: Security/SAST.gitlab-ci.yml

semgrep-sast:
  variables:
    SAST_ANALYZER_IMAGE_TAG: "2.13.1"
Edited by Lucas Charles