Discovery: Perform secret detection on full history of the repository
Problem to solve
Secret Detection will check the latest version of the repo (or latest commit) as implemented in https://gitlab.com/gitlab-org/gitlab-ee/issues/6719.
Users may want to check secrets in the entire history of their repositories. Even if we run checks on each pipeline, there are cases where this is not enough (ci skip, job disabled, etc).
Users may also want to check just the list of commits of a specific merge request, to ensure they are not introducing any secret with their changes.
Target audience
- Delaney, Development Team Lead, https://design.gitlab.com/research/personas#persona-delaney
- Sasha, Software Developer, https://design.gitlab.com/research/personas#persona-sasha
- Sidney, Systems Administrator, https://design.gitlab.com/research/personas#persona-sidney
- Sam, Security Analyst, https://design.gitlab.com/research/personas#persona-sam
JTBD: When protecting my project against secret leaks, I want to know if there are secrets in my commit history, so that I can be assured my companies keys are safe from bad actors.
Proposal
Allow users to set the secret detection mode via an environment variable.
- Full scan of the repo commit history through a custom variable:
SECRET_FULL_HISTORY. - Decouple SAST and secret detection analyzers.
- Display secret detection results in a separate section from vulnerabilities.
What does success look like, and how can we measure that?
Number of executions with this variable set.
What is the type of buyer?
Executives
Design:
MVC
Scope:
This will begin at the project level for now and we will create another issue to add this feature at the group level in future iterations. Areas of the application that will have UI changes:
- Project level dashboard
- Pipeline page
- MR page
Designs
Project Dashboard
| Project dashboard | Project dashboard w/alert |
|---|---|
 |
 |
| IF Full history scan has been run. | IF Full history scan has not been run |
| + Tabs added to top of filter section | + Alert added to top of page |
| + Counters added to the badges | |
| - Page title is removed |
Alert language
Full history secret scan not detected
Look for past instances where secrets may have unintentionally revealed in your project. Learn more about running a full history scan here.
This would link to documentation about detailing how to run a pipeline with the custom variable. | Project dashboard - secrets tab |
|---|
 |
| - Filtering is removed |
| - Confidence level is removed |
| / Interactions remain the same as today |
No other changes to the dashboard other than adding the tab and removing the ability to filter and confidence labels. The decision to remove these features was based on the low value they provide to the user. Today there isn't a way to filter the secrets list in a meaningful way. With confidence, I choose to remove it since all confidence levels from gitleaks are unknown thus not helpful to the user. A secret is a secret and it should be looked into regardless of the confidence we have in our detection.
Pipeline & MR
Minor changes to these areas are anticipated due to the direction of adding a new parser.
| MR | Pipeline report |
|---|---|
 |
 |
| + Secret section in MR security widget | + Secret section in pipeline security tab |
| - Confidence level is removed | - Confidence level is removed |
| / Section works the same as the others today | / Section works the same as the others today |
| / Interacting with these types of vulnerabilities remains the same as today | / Interacting with these types of vulnerabilities remains the same as today |
Discovery Status:
-
Understand problem and ask -
Familarize with Secret Detection and how it's handled in the industry -
Wireframe a solution -
Gather feedback on solution -
Finalize experience and designs -
Update or add to the design system (if necessary)
-
-
Propose final designs with descriptions, mocks, videos, prototypes as necessary -
Break proposal down into an MVC candidate -
Create follow-up issues to fill the gap between the final proposal and MVC.
Decisions:
- Start at the project level for the MVC.
- Secret detection will be decoupled from SAST
- Custom variable
SECRET_FULL_HISTORYwill be used to scan the full commit history of the project through a manual pipeline. - Secrets will get a dedicated section in the Project dashboard, MR, and pipeline sections.