17.10 Planning Issue - Secret Detection
🔒 Secure, Secret Detection - Milestone Planning
This is a planning issue for Category:Secret Detection which is maintained by groupsecret detection.
See the group handbook page for more about this issue and how it fits into group workflows.
Milestone Key Dates
- Start Date: 2025-02-15
- Code Freeze: 2025-03-14
- Release Date: 2025-03-20
Narrative
Secret Detection in Job Artifacts
In 17.10 we will be kicking off refinement for secret detection in job artifacts. Today, Secret Detection protects users from introducing new secrets into codebases by running in an opt-in MR pipeline. However, there are many other places where secrets could exist. We are expanding secret detection to CI/CD Jobs Artifacts.
In this milestone we will work in refining which Job Artifacts types and files we will scan for secrets and conduct user research that aims to better understand secrets leaked outside of code. We will transition the new, secret detection service (SDS) from passive to active mode. This is a dependency for expanding secret detection to other objects and components.
Validity Checks for SD findings
In the previous milestone we finalized requirements for this feature, conducted a POC which proved out how we plan to implement this, and kicked off development on this effort by creating a new Runway service called Secret Detection Response Service (SDRS).
This milestone will be heavily focused on implementation for the Experiment iteration. We're wrapping up a discussion that will help us choose an approach (post-analyzer vs. SideKiq job) to how we trigger validity checks.
I would consider making progress against this feature to be one of our top priorities this milestone due to the delivery target we've established, Experiment by end of Q1.
Transition SDS from Passive to Active Mode
In the previous milestone, we fixed all remaining issues in order to get traffic flowing (at the same rate as the SDS scanning gem) to the SDS. We started collecting observations, metrics, and discussed whether or not the traffic we're receiving is enough for us to make a call on moving to the next phase.
In %17.10, we'll continue that analysis with a goal of moving to using SDS as a primary SD scanner, with the gem as a fallback.
Enable SPP for all GitLab-owned projects
In the previous milestone we kicked off a migration/rename to hopefully remove all traces of "pre-receive secret detection" from our codebase and DB. We are investigating an issue that resulted from this change, but have a fix in place which should be deployed early in %17.10.
This milestone's focus should be on wrapping up the work for scanning only set of changes instead of entire files. This is the last remaining prerequisite before we can start enabling SPP for GitLab-owned projects at scale.
Priorities
Key items to deliver
This section lists items that should be ready to deliver (or at least to move forward). Many of these items should be defined as ~Deliverable items, assuming they are feasible to deliver in the milestone.
---
display: table
fields: title, assignee, labels("workflow::*"), labels("Deliverable"),state, milestone
---
label = "group::secret detection" AND label = "type::feature" AND milestone = "17.10" AND assignee!="abellucci" AND assignee != "phillipwells" AND label != "workflow::planning breakdown" AND label != "workflow::refinement" AND label != "workflow::problem validation"---
display: table
fields: title, assignee, labels("workflow::*"), labels("Deliverable"), state, milestone
---
label = "group::secret detection" AND label = "type::maintenance" AND milestone = "17.10" AND assignee!="abellucci" AND assignee != "phillipwells" AND label != "workflow::planning breakdown" AND label != "workflow::refinement" AND label != "workflow::problem validation"---
display: table
fields: title, assignee, labels("priority::*"), labels("severity::*"), labels("workflow::*"), labels("Deliverable"), state, milestone
---
label = "group::secret detection" AND label = "type::bug" AND milestone = "17.10" AND assignee!="abellucci" AND assignee != "phillipwells" AND label != "workflow::planning breakdown" AND label != "workflow::refinement" AND label != "workflow::problem validation"Looking forward
This section lists items that are in earlier stages of planning. Refining them is an important part of this milestone because it sets us up to work on them in the following milestones. Primary areas of responsibility are listed, but everyone can contribute!
This is almost certainly more than we can take on. Items that are marked as Deliverable are expected to be workflowready for development by the end of the milestone.
---
display: table
fields: title, assignee, labels("workflow::*"), labels("Deliverable"), state, milestone
---
label = "group::secret detection" AND label in ("workflow::planning breakdown","workflow::refinement","workflow::problem validation") AND milestone = "17.10" AND assignee!="abellucci"Please suggest others or add them directly.
Product and UX
This section includes other Product and UX context that may not fit into the Looking forward section above.
Product Manager: @abellucci
---
display: table
fields: title, assignee, state, milestone
---
label = "group::secret detection" AND label != "Planning Issue" AND milestone = "17.10" AND assignee = "abellucci"Documentation
This section includes group inputs and the plan for Technical Writing in the milestone.
Technical Writing stable counterpart: @phillipwells
---
display: table
fields: title, assignee, state, milestone
---
label = "group::secret detection" AND milestone = "17.10" AND assignee = "phillipwells"