Establish process for new patterns and pattern improvements
Overview
The overall outcomes of this issue will be to have our process for making changes to our secret detection rules documented, as well as verifying the set of rules that we want to ship with SPP are in sync with what we have tagged as "gitlab_blocking" in the secrets analyzer. The process documentation will include guidance on adding rules, updating rules, and tagging rules as "gitlab_blocking". The process documentation should also include any limitations or reasoning for not doing something, such as certain automations.
Proposal
-
Brainstorm and establish criteria for how to measure a "well-defined pattern" based on the definition provided in the blueprint. This currently roughly equates to
gitlab_blocking
from the Tags section of thesecrets
analyzer README. -
Once the criteria has been established, determine how we evaluate existing patterns against it. This will likely be similar to what we're doing with sast-rules via BAP.
-
Create baseline of SPP rules based on output of above process.
Questions to answer
- When introducing a new pattern, what is the process we should follow to determine it's classification and tag it appropriately?
- Think from the perspective of someone introducing a pattern via a community contribution. What checks would we want in place to ensure the level of quality is enforced for well-defined patterns and tagging it as "gitlab_blocking"
- How can we measure improvements to patterns? For example we have a handful of pattern improvements suggested by VR, what data should we use to validate that it is improving the pattern?
- What test gaps exist currently that we need to address? How can we continuously evaluate/benchmark each pattern so we're not introducing regressions?
Issues/MRs to evaluate
When the initial process has been established, we have a number of Issues/MRs we can use the process established in this Issue to evaluate. The below items are not going to be completed with this issue, but will have better guidance once it is completed.
- Draft: Fix broken regular expressions (gitlab-org/security-products/analyzers/secrets!306) • Adam Cohen
- Reduce FP in secret detection by adding word bo... (#424562) • Unassigned • Backlog
- Implement regex improvements for SPP previously... (#466071) • Unassigned • Backlog
- Update Password In URL rule (gitlab-org/security-products/analyzers/secrets!295) • Ben Benhemo • 17.0
Implementation Plan
-
Determine criteria for accepting pattern improvements for existing patterns in the gitleaks.toml of the secrets
analyzer -
Determine criteria for accepting a new pattern for the gitleaks.toml of the secrets
analyzer -
Determine criteria for applying the gitlab_blocking
tag - Apply established criteria to the existing patterns in the gitleaks.toml of the
secrets
analyzer to establish baseline - Determine process (possibly manual, or semi-manual, to start) to move
gitlab_blocking
patterns from thesecrets
analyzer gitleaks.toml file to the gitleaks.toml of the secret push protection gem's gitleaks.toml file - Iterate on the criteria and process as we adopt new rules and rule improvements (to happen continually while doing the above items)
Items with
Refinement Progress
If a checkbox is not relevant for the issue, please remove it.
-
This issue describes a problem to solve, or a task to complete, and it's confirmed. -
This issue describes a proposal or an implementation plan that outlines a way to solve the problem or complete the task. -
This issue requires assistance or support from other groups, and it's indicated in the issue description. -
This issue could affect application security or performance, and the concern is explained in the issue description. -
This issue is the smallest iteration possible and doesn't require further break down. -
This issue has weight set - based on how many tasks or merge requests are required - and needs weight label is removed. -
This issue is labeled correctly. -
This issue is reviewed by another team member to confirm strategy and estimate. -
Finally, add workflowready for development label to this issue.