<!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> *This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.* <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> <!-- The first four sections: "Problem to solve", "Intended users", "User experience goal", and "Proposal", are strongly recommended, while the rest of the sections can be filled out during the problem validation or breakdown phase. However, keep in mind that providing complete and relevant information early helps our product team validate the problem and start working on a solution. --> ### Problem to solve A robust fuzzing job can be a long running task. Additionally, customers may wish to fuzz a target indefinitely until a crash is found. For mature projects, this could be hours or days. Longer duration fuzz testing will find additional bugs and vulnerabilities. In the above scenario, running a fuzzing job in a pipeline does not make sense, since the pipeline will never complete or will timeout after a certain period. ### User experience goal <!-- What is the single user experience workflow this problem addresses? For example, "The user should be able to use the UI/API/.gitlab-ci.yml with GitLab to <perform a specific task>" https://about.gitlab.com/handbook/engineering/ux/ux-research-training/user-story-mapping/ --> Consider how, where, and when to surface results from continuous fuzz testing as compare to results from a pipeline. Since there is no "done" to a continuous fuzz test job, we don't have the normal "end of a pipeline" step to collect and process results. - _Idea_: Consider introducing a new screen where continuous fuzz testing results are collected and then can be manually promoted to vulnerabilities/findings after a user has reviewed them. This would be similar to some of the earlier designs we saw where all fuzz testing results were on their own screen. A goal is that the same approach & UX can be used for both coverage-guided & API fuzzing. ### Proposal <!-- How are we going to solve the problem? Try to include the user journey! https://about.gitlab.com/handbook/journeys/#user-journey --> GitLab should support fuzzing jobs that run outside of a pipeline, and run until one of the following occurs: 1. Indefinitely 1. ~~A crash is found~~ 1. ~~A set amount of time is past~~ Note: Provide a variable to let the users choose the other mode if so desired. When setting up continuous fuzz testing jobs, **require** a user to indicate a specific job in their pipeline to use as the fuzz testing target. This would allow GitLab to not have to guess or build complicated auto-detection logic to find out what to fuzz for and users would likely already have this configured since they were using normal fuzz testing already. - We can revisit this in the future too, but want to simplify this seemingly complex problem. Allow the user to start the continuous job: 1. Manually triggered and specify needed configuration options. Continuous fuzz testing should not block a pipeline job that is started as part of a commit. - This is so that the CI pipeline will actually finish, since continuous fuzz testing wont ever finish ### Further details <!-- Include use cases, benefits, goals, or any other details that will help us understand the problem better. --> For the first iteration of continuous fuzz testing, we want to focus on: 1. Migrating existing functionality GitLab acquired, rather than build from scratch where possible 1. Iterating small with a "[boring solution](https://about.gitlab.com/handbook/values/#boring-solutions)" that we can expand on and improve more later. Questions to answer: - What scenarios should cause a continuous fuzzing job to stop - How can a user stop a long running fuzz job? - When should new builds be fuzzed, if a previous build is being fuzzed indefinitely - How does GitLab save and present results for a job that is still running? - What can we re-use from on-demand scans? ### Permissions and Security <!-- What permissions are required to perform the described actions? Are they consistent with the existing permissions as documented for users, groups, and projects as appropriate? Is the proposed behavior consistent between the UI, API, and other access methods (e.g. email replies)? Consider adding checkboxes and expectations of users with certain levels of membership https://docs.gitlab.com/ee/user/permissions.html * [ ] Add expected impact to members with no access (0) * [ ] Add expected impact to Guest (10) members * [ ] Add expected impact to Reporter (20) members * [ ] Add expected impact to Developer (30) members * [ ] Add expected impact to Maintainer (40) members * [ ] Add expected impact to Owner (50) members --> ### Documentation <!-- See the Feature Change Documentation Workflow https://docs.gitlab.com/ee/development/documentation/workflow.html#for-a-product-change * Add all known Documentation Requirements in this section. See https://docs.gitlab.com/ee/development/documentation/feature-change-workflow.html#documentation-requirements * If this feature requires changing permissions, update the permissions document. See https://docs.gitlab.com/ee/user/permissions.html --> Documentation needs to be updated to describe: * What continuous fuzz testing is & the problems it addresses * Discussion of the differences between continuous fuzzing & normal fuzzing * Configuring continuous fuzz testing * An example project * Screenshots / visuals where applicable ### Availability & Testing <!-- This section needs to be retained and filled in during the workflow planning breakdown phase of this feature proposal, if not earlier. What risks does this change pose to our availability? How might it affect the quality of the product? What additional test coverage or changes to tests will be needed? Will it require cross-browser testing? Please list the test areas (unit, integration and end-to-end) that needs to be added or updated to ensure that this feature will work as intended. Please use the list below as guidance. * Unit test changes * Integration test changes * End-to-end test change See the test engineering planning process and reach out to your counterpart Software Engineer in Test for assistance: https://about.gitlab.com/handbook/engineering/quality/test-engineering/#test-planning --> ### What does success look like, and how can we measure that? <!-- Define both the success metrics and acceptance criteria. Note that success metrics indicate the desired business outcomes, while acceptance criteria indicate when the solution is working correctly. If there is no way to measure success, link to an issue that will implement a way to measure this. --> ### What is the type of buyer? <!-- What is the buyer persona for this feature? See https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/buyer-persona/ In which enterprise tier should this feature go? See https://about.gitlab.com/handbook/product/pricing/#four-tiers --> gitlab~3207279 ### Is this a cross-stage feature? <!-- Communicate if this change will affect multiple Stage Groups or product areas. We recommend always start with the assumption that a feature request will have an impact into another Group. Loop in the most relevant PM and Product Designer from that Group to provide strategic support to help align the Group's broader plan and vision, as well as to avoid UX and technical debt. https://about.gitlab.com/handbook/product/#cross-stage-features --> ### Links / references <!-- Label reminders - you should have one of each of the following labels if you can figure out the correct ones -->