Enable GSS analyzer via latest Secret Detection template

Craig Smithrequested to merge
craigmsmith-add-gss-to-latest-sd-template into master

What does this MR do?

Extends the latest Secret Detection template so the new GSS analyzer (gitlab-advanced-secrets) can be enabled directly from the template.

Setting SECRET_DETECTION_ENABLE_GSS to "true" makes the existing secret_detection job swap its image to the GSS analyzer instead of the default secrets analyzer:

include:
  - template: Jobs/Secret-Detection.latest.gitlab-ci.yml

secret_detection:
  variables:
    SECRET_DETECTION_ENABLE_GSS: "true"

The analyzer is still in beta, so:

  • It is disabled by default and only runs when a user explicitly opts in by setting SECRET_DETECTION_ENABLE_GSS to "true"; existing pipelines are unaffected.
  • It is only added to the latest template, not the stable one.
  • It is pinned to the rolling edge tag (no stable version is published yet).
  • It runs as the same secret_detection job, which guarantees it never runs in parallel with the existing secrets analyzer (avoiding unexpected finding deduplication).
  • FIPS projects opt into the FIPS-enabled image the same way as the default analyzer, by setting SECRET_DETECTION_IMAGE_SUFFIX to "-fips". This is appended to the GSS image so the job resolves to gitlab-advanced-secrets:edge-fips. Note: a FIPS-enabled GSS image is not published yet, so FIPS opt-in fails to pull the image until it is; the job will then work automatically with no template change.

Generic secret detection

The template also turns generic secret detection on for projects that enable GSS, by defaulting SECRET_DETECTION_GSS_ENABLE_GENERIC_SECRETS to "true". Generic detection is one of the main reasons to adopt GSS, so opting into GSS should not also require knowing about and setting a second variable.

Projects that want only the vendor-specific rules opt out:

secret_detection:
  variables:
    SECRET_DETECTION_ENABLE_GSS: "true"
    SECRET_DETECTION_GSS_ENABLE_GENERIC_SECRETS: "false"

The default secrets analyzer ignores this variable, so projects that have not enabled GSS are unaffected. Generic findings stay limited to high confidence: the analyzer passes --min-confidence high whenever generic secret detection is on, and that behaviour is unchanged here.

How to test

Because the include: - template: form always pulls the template bundled with the running GitLab version, test this branch's version with a remote include pointing at the raw file on this branch.

  1. Create a test project and commit a file containing a leaked secret (for example a fake AWS key such as AKIAIOSFODNN7EXAMPLE paired with a secret access key) so there is something to detect. Also commit a file with a generic secret so there is something for generic detection to find; the analyzer's qa/fixtures/generic-secrets/app_config.py fixture works for this.

  2. Add this .gitlab-ci.yml to the project:

    include:
  - remote: 'https://gitlab.com/gitlab-org/gitlab/-/raw/craigmsmith-add-gss-to-latest-sd-template/lib/gitlab/ci/templates/Jobs/Secret-Detection.latest.gitlab-ci.yml'

secret_detection:
  variables:
    SECRET_DETECTION_ENABLE_GSS: "true"

  3. Run a pipeline and open the secret_detection job. Confirm:

    • The job image is registry.gitlab.com/security-products/gitlab-advanced-secrets:edge.
    • The leaked secret is reported in the secret detection results.
    • The generic secret is also reported, without setting SECRET_DETECTION_GSS_ENABLE_GENERIC_SECRETS. The scanner is invoked with --generic-secrets --min-confidence high; add SECURE_LOG_LEVEL: "debug" to the job variables to see the command in the job log.

  4. Add SECRET_DETECTION_GSS_ENABLE_GENERIC_SECRETS: "false" to the job variables and re-run. Confirm the generic secret is no longer reported and the AWS key still is.

  5. Remove the SECRET_DETECTION_ENABLE_GSS override and re-run. Confirm the job falls back to the default registry.gitlab.com/security-products/secrets:7 image.

FIPS

To verify the FIPS wiring, add the -fips suffix override alongside the GSS opt-in:

include:
  - remote: 'https://gitlab.com/gitlab-org/gitlab/-/raw/craigmsmith-add-gss-to-latest-sd-template/lib/gitlab/ci/templates/Jobs/Secret-Detection.latest.gitlab-ci.yml'

secret_detection:
  variables:
    SECRET_DETECTION_ENABLE_GSS: "true"
    SECRET_DETECTION_IMAGE_SUFFIX: "-fips"

Confirm the job image resolves to registry.gitlab.com/security-products/gitlab-advanced-secrets:edge-fips. Because that image is not published yet, the job currently fails at image pull — this is expected, and it will start working automatically once the analyzer's deploy pipeline ships the -fips variant (the same way the default analyzer ships both secrets:7 and secrets:7-fips).

This wiring is also covered by the template spec (ee/spec/lib/gitlab/ci/templates/secret_detection_latest_gitlab_ci_yaml_spec.rb), which asserts the resolved image is gitlab-advanced-secrets:edge-fips when the suffix is set.

References

Notes

  • Documentation update (the third implementation-plan item) is intentionally out of scope here and will be handled separately. That includes documenting that SECRET_DETECTION_GSS_ENABLE_GENERIC_SECRETS now defaults to "true" and how to opt out, which is tracked as the third checklist item of https://gitlab.com/gitlab-org/gitlab/-/work_items/612351+s.
  • The corresponding secret-detection CI/CD Component is intentionally not updated here. It has no stable/latest split and GSS is still beta (pinned to edge), so the component will be updated when GSS is promoted to the stable template.
Edited by Craig Smith

Merge request reports