Don't diff reports from different scan configurations
Problem to solve
To determine if an MR introduces and/or fixes vulnerabilities, the DAST report from the MR pipeline is diffed against the DAST report from master. The result is displayed in the security MR widget:
This is problematic if the DAST job on master and the MR job use different configurations. For example, DAST does a baseline scan in the MR pipeline and a full scan on master. The result is that in the security widget on the MR some findings will be marked as resolved because they were found on master but not on the MR. However, the findings have not been resolved but were simply missed by the baseline scan.
I believe running a baseline scan on the MR and a full scan on master is a reasonable approach, as MR pipelines have stricter time constraints, while on master we can run a scheduled pipeline that is less time-contrained. This problem likely extends to other dynamic testing techniques like fuzzing as it has similar constraints.
Intended users
Proposal
Don't diff reports that were produced from different scan configuration against each other. In the special case of DAST, we know that the findings of a baseline scan are a subset of the findings from a full scan; hence we could only diff the findings from passive scan rules.
If we could produce a test case for each active scan finding that checks if the vulnerability is present (think regression tests), we could repeat those tests in MR pipelines to determine if the finding was fixed.