[Spike] Investigate more robust ways of sync approval rules and security reports
Timebox: 5 days
Current scenario
So far approval rules are synced only based on the head pipeline (and base pipeline when comparison is involved).
Recently this feature has been enhanced to no only affect new merge requests but also all existing open merge requests. Therefore whenever policies are changed, new rules are generated for each of the related open MRs which in their turn require a rerun of the head pipeline in order to sync those new rules against the security report of the head pipeline.
Limitations
The current approach considers only the head pipeline which seems to be limited when trying to cover the following use cases:
- Approvals are not required when security jobs are removed from the pipeline.
- Approvals are required when security jobs are included in the pipeline and their conditions are met.
- Although detached pipelines do not contain security jobs, it is not desirable that approvals are not required.
- Whenever policy changes and new approval rules are created, approval rules are expected to be synced against the pipeline even if they are detached.
- When security policies consider only whether or not vulnerabilities are pre-existing on the default branch (for example, in a
detected
orconfirmed
state) then the policy engine still requires the pipeline to fully complete and it still requires security scans to run. Ideally the engine should be intelligent enough to not assume that all vulnerabilities have been resolved if a vulnerability report was never generated (the scans did not run) and ideally the approvals would not be required from the beginning without having to wait for the entire pipeline to complete.
Although (1) and (2) have been stable, an initial attempt to address number (3) has affected number (4), therefore being the motivation for this spike to be created.
Important notes
- As stated in #346843 (comment 1124564373), we need to remember to evaluate findings and enforce policies when all security scans of given type are finished.
Goal
The goal of this spike is to encourage the proposal of a solution robust enough to cover the above cases.