MVC-backend: Perform secret detection on full history of the repository
Problem to solve
Secret Detection will check the latest version of the repo (or latest commit) as implemented in https://gitlab.com/gitlab-org/gitlab-ee/issues/6719.
Users may want to check secrets in the entire history of their repositories. Even if we run checks on each pipeline, there are cases where this is not enough (ci skip
, job disabled, etc).
Users may also want to check just the list of commits of a specific merge request, to ensure they are not introducing any secret with their changes.
Target audience
- Delaney, Development Team Lead, https://design.gitlab.com/research/personas#persona-delaney
- Sasha, Software Developer, https://design.gitlab.com/research/personas#persona-sasha
- Sidney, Systems Administrator, https://design.gitlab.com/research/personas#persona-sidney
- Sam, Security Analyst, https://design.gitlab.com/research/personas#persona-sam
JTBD: When protecting my project against secret leaks, I want to know if there are secrets in my commit history so that I can be assured my companies keys are safe from bad actors.
Proposal
Allow users to set the secret detection mode via an environment variable.
- A full scan of the repo commit history through a custom variable:
SECRET_FULL_HISTORY
. - Decouple SAST and secret detection analyzers.
- Display secret detection results are in the following issues:
- MVC-frontend Secret detection: show scan result #204982 (closed) (Solution validation)
- MVC-frontend Secret detection: add secrets scan to configuration page #204987 (closed) (Design)
- MVC-frontend Secret detection: show user status(empty/failed/not configure) #13398 (closed) (Design)
Implementation Checklist :
-
Update gitleaks and trufflehog to use latest their versions #12948 (closed) -
Create a secrets template in https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates/Security. To start off this would be a single job to analyze full git histories. -
Add logic in the secrets analyzer to skip the flattening of the repo and proceed with a history scan if a ci/cd var is set, SECRET_FULL_HISTORY
. -
Add logic to support secret scans for commit ranges based on SECRET_COMMIT_FROM
andSECRET_COMMIT_TO
h
Edited by Zach Rice