Improve Security Policy evaluation for chained MRs with missing target branch pipelines

Summary

Security Policy evaluation and the MR security widget are not functioning correctly for chained Merge Requests (MRs) when the pipeline for the target branch is missing. This issue is related to the broader problem described in #436956 (closed), where the report comparison logic fails when the base commit doesn't have a pipeline on the target branch.

Steps to reproduce

1. Create a new branch (feature-1) from the default branch (master)
2. Add a commit to the feature-1 branch
3. Merge feature-1 into master, but do not delete feature-1
4. Add a new commit to feature-1
5. Create a Merge Request from feature-1 into master
6. Observe that all vulnerabilities are shown as new, and the comparison pipeline for the target branch is None

What is the current bug behavior?

In scenarios involving chained MRs or long-lived branches that merge periodically, the security policy evaluation and MR security widget may incorrectly report all vulnerabilities as new, even when they already exist on the target branch. This occurs because the merge-base commit of the MR doesn't have a pipeline in the context of the target branch.

What is the expected correct behavior?

The security policy evaluation and MR security widget should correctly identify existing vulnerabilities on the target branch, even when the pipeline for the merge-base commit is missing in the target branch context.

Proposed solution

Investigate and implement a solution that allows for accurate security policy evaluation and vulnerability reporting in scenarios with chained MRs or long-lived branches, even when pipelines are missing for certain commits on the target branch.

Related issues

• Report comparison logic does not work when the ... (#436956 - closed)
• MR security widget and approval rules work inco... (#457723 - closed)

Impact

This issue affects users who work with long-lived branches or have workflows involving chained MRs. It can lead to false positives in vulnerability reporting and incorrect security policy evaluations, potentially causing unnecessary work for security teams and delays in the development process.

Workaround

Currently, the recommended workaround is to use merged results pipelines. However, a more robust solution is needed to address this issue comprehensively.