Design: Cancel vulnerabilities detected in a pipeline

Proposal

Running our security scanning tools in our pipelines can sometimes generate a lot of false positives for different reasons (among, but not limited to):

• Analyzers can suffer from bugs
• CI/CD misconfigurations
• A change in the way dependencies might generate temporary files (which would be scanned in turn)

While having a way to mass delete records would bring all the power needed to roll back this kind of situation, it's suffering from too many blockers to be available in the near future.

When thousands of vulnerabilities are generated, bulk actions get amplified and can lead to missing important and critical information.

We thought of an intermediate iteration with @matt_wilson, that will solve our immediate needs without waiting for other features. Generally, these events are occurring as part of a specific pipeline. Subsequent scans would not add the same vulnerabilities over and over again, because they should have the same signature. Therefore, canceling the results of the initial pipeline would remove only the relevant vulnerabilities, considering the root problem has been fixed of course. Not only this solution is very efficient, but it also provides a good UX because we already have a link between vulnerabilities and where they were detected for the first time (in the vulnerability page).

Requirements

• Any pipeline that has previously ran against the default branch and generated vulnerability objects can be rolled back.
• The rollback mechanism needs to be easily accessible/highly visible (to those with adequate permissions). For example, it may make sense to add this as an option on the pipeline security tab.
• When rolling back vulnerabilities created with a particular pipeline, we will store a record of who performed the action and when. We should also store the cancelled vulnerabilities for a fixed period of time (3 months?) before automatically removing them.
• When vulnerabilities are cancelled, we should provide users an option to skip any that have been Dismissed or have associated issues. This way, important historical data is preserved.

Open questions

• Who would have the permission to delete vulnerabilities from a pipeline?
  • Maintainers and above should be the only roles able to do this action. This is not very frequent, so finding a user able to help Security is acceptable.
• Where do report these actions?
  • The Security Dashboard would be a good place to report recent deletes, until we have better security feeds. These dashboards would reflect these cancels anyway, with big spikes when vulnerabilities are created and then deleted, so it makes sense to have a related event close to them visually.
  • Alternatively, the cancelled vulnerabilities could be removed entirely from the dashboards so as not to "pollute" the charts. A separate means of showing what was canceled would be needed.
• Is there any regulatory/compliance issue with the deletion of security data?
  • Cancel pipeline security results will only be transparent if the root cause is fixed, so a change in the code for example would still be reported if present.
  • The feature is only available for Maintainers and above, who will have the responsibility to use it cautiously, like any other feature altering or deleting data