Automatically merge a vulnerability fix when ready
Problem to solve
Auto Remediate could create a MR to update dependencies used in your code in order to fix security warnings. We want to automatically merge this MR into the original branch if the problem has been fixed, and if the application tests are still passing. In this way we can ensure that the change is valuable (fixes the vulnerability) and it is not introducing problems (failed tests).
Further details
Setting the MWPS is not enough, because the pipeline will pass if the tests pass, but it cannot guarantee that the vulnerability has been fixed. We could assume it is because of our knowledge used to create the fix, but checking the security report is something that should be done.
Proposal
When a merge request to autofix a vulnerability is created (see e.g. https://gitlab.com/gitlab-org/gitlab-ee/issues/5656), check if the security report doesn't contain the original vulnerability anymore, and merge if the pipeline is green.
As an alternative, merge if the security report is better than the previous one (no new vulns added, some fixed), and the pipeline is green. In this case, even if all the problems have not been fixed, we are improving the security of the app so it is valuable anyway.
There must be an option in settings for a user to disable this behavior if desired. Default to auto-merging being done.
What does success look like, and how can we measure that?
Number of autofix MR created and merged.