Secure:Secret Detection – Refinement Process – Proposal and Feedback
Overview
As we approach the split of groupstatic analysis into two separate groups:
-
Secure::Secret Detection
- Secret Detection & Code Quality -
Secure::Static Analysis
- SAST & IAC Scanning
It seems to be a good time to review the refinement process that will be used moving forward for the Secure::Secret Detection
team.
This issue aims to provide a proposal for an improved refinement process, and receive any feedback on the proposed process.
History
In groupstatic analysis, two processes existed for refinement. Let's go over them briefly:
1️⃣ MoSCoW Process
This asynchronous process aimed to triage the backlog mostly, in order to eliminate stale issues and prioritize existing ones.
The process was supposed to have a regular cadence (runs for 1 week, every milestone), but was often missed or skipped (i.e. the latest time the process took place was in %16.3, and looking at earlier ones, it didn't seem like it happened consistently).
Please see this %15.3 for an example.
2️⃣ Refinement during Software Delivery workflow
As part of the product development workflow, the software delivery process involved an experimental refinement phase, in which engineers where expected to:
- Add an implementation plan.
- Determine the relative size of the issue and apply it as weight.
This however, was not strictly followed, and often times the planning issues (see example) that would supposedly determine the issues to be refined in a specific milestone would also be abandoned (see examples: 1, 2).
This usually resulted in refinement taking place only in isolated instances, and not as a team effort.
Proposal
This issue proposes we improve the refinement process, by making it a team practice that involves all team members, to ensure everyone has the chance to refine issues, and so that they also get the know-how in the process. It also proposes using a different format in which:
Updated Format – 2024-04-11
- A planning issue is created with topics/issue to work on next. This proposal isn't concerned with how those issues are identified.
- The refinement process is kicked off when the planning issue is finalized.
- A bot or an automated script assigns a number of issues (e.g. 2-3) randomly to each engineer (similar to SAST monthly updates).
- An engineer is responsible for refining their assigned issues, but could ask for help if needed.
- Engineers would follow a certain checklist to determine if an issue is refined and ready to be picked up.
- The refinement process will be time-boxed (e.g. one week), after which all issues ready for development is picked up.
- When an engineer completes refining an issue, they pass it on to another engineer (a reviewer) for review.
- The reviewer should follow the guidelines outlined in the checklist as much as possible:
- If the reviewer agrees with the engineer, the issue is marked as ready for development.
- If they are in disagreement, they should discuss the reason and find a way forward.
- If a disagreement cannot be resolved, the issue is brought to next team meeting for discussion.
- Pending issues can continue to be refined, and depending on their status may or may not be included in the milestone.
Initial Proposed Format
After a planning issue determines the topics/issues to work on next, the refinement process begins.The refinement process involves assigning a number of issues per milestone to each engineer.The engineer would be responsible for refining their own issues, but could ask for help if needed.They would follow a certain checklist to determine if an issue is refined and ready to be picked up.When all issues are refined, work to deliver and implement issues begin.
The checklist could potentionally look like the following:
**Please copy the list below into the issue you are refining, and check them as you deem appropriate.**
#### Refinement Progress
If a checkbox is not relevant for the issue, please remove it.
- [ ] This issue describes a problem to solve, or a task to complete, and it's confirmed.
- [ ] This issue describes a proposal or an implementation plan that outlines a way to solve the problem or complete the task.
- [ ] This issue requires assistance or support from other groups, and it's indicated in the issue description.
- [ ] This issue could affect application security or performance, and the concern is explained in the issue description.
- [ ] This issue is the smallest iteration possible and doesn't require further break down.
- [ ] This issue has weight set - according to [this list of possible values](https://handbook.gitlab.com/handbook/engineering/development/sec/secure/workflow/#possible-values) - and ~"needs weight" label is removed.
- [ ] This issue has a success criteria defined, and it is outlined in the issue description.
- [ ] This issue is labeled correctly.
- [ ] This issue is reviewed by another team member to confirm proposal/implementation plan and weight.
- [ ] Finally, add ~"workflow::ready for development" label to this issue and unassign yourself.
#### Refinment Review Guidelines
If you're assigned this issue to review its refinement, please follow the guidelines below.
1. Please validate the proposal or the implementation plan described in the issue.
1. Please validate the weight of the issue according to [this list of possible values](https://handbook.gitlab.com/handbook/engineering/development/sec/secure/workflow/#possible-values).
1. If in disagreement, please state your thoughts/reasoning and notify the engineer refining this issue.
1. If the disagreement can't be resolved, please bring this issue to the next team meeting for discussion.
I'm also open for any feedback or suggestions. We could probably create an issue template to be used in refinement issues similar to how the planning issues look like, in which it is clear which issues is assigned to which engineer, and would include the above checklist to make it easy for them to copy it to a to-be-refined issue.
Please note: this resembles a process we tried to follow recently with refining the issues for the Pre-receive SD beta/.com epic, so I thought we could build on that, and improve refinement for the entire team based on how this initiative is perceived.