Allow ingesting artifacts from failed security jobs
won't do
On February 28, 2024, we have decided not to do this work. Allow security reports to be read for pipelines... (&12876 - closed) and security reports are behaving as expected.
Some users intentionally fail the pipeline to block the merge request from being merged. This is not a best practice, instead, we encourage users to use security policies so that the right approvers are required to review vulnerabilities for the merge request to merge.
Problem to solve
Some users are purposefully setting security jobs with a failing exit code if any vulnerabilities are detected so their pipeline fails. Some also want vulnerabilities in the failed job to still show up in the MR and pipeline so they can be assessed and any corrective action taken. In the past, it was still possible to ingest findings from these failed jobs, as long as the artifact was produced successfully. However, the behavior was changed to ignore findings from all failed security jobs. The reason is to reduce uncertainty as we cannot tell the difference between job artifacts that are parsable but do not contain all vulnerabilities from a scan versus those that do.
With this change, any users who relied on security jobs failing to block subsequent pipeline actions could not longer do so. The only options now are to leverage other product features.
Purpose
This issue is to gather feedback and solicit input from anyone currently relying on failed security jobs that still output complete report artifacts. It will discuss options to restore same or similar functionality as well as merits or drawbacks.
Possible options
New CI configuration option
We could introduce the ability to configure pipelines so that valid artifacts are used even from security jobs that fail.
Warning on the pipeline security tab
Show a warning message on the "pipeline security tab" that results are included from a failed job. This might mean we would have to revert the behavior to allow artifacts from failed jobs or introduce a new configuration as above.
Documentation or training on in-product alteratives
- Use dependent jobs.
- Set a conditional on the deploy job based on the security report findings. In the deploy job, download the report(s), parse, and exit 1 if any findings exist.
- Use Security approval rules in MRs. This will prevent merging code from feature branches when new vulnerabilities are detected.