Create violations and bot message for fail-open policies

What does this MR do and why?

This MR changes logic around fail-open policies to persist violations for these cases so that we can highlight skipped policies in the policy bot message. The violations for fail-open policies are created with a different status so that they don't block the MR with policy mergability check.

Changes in this MR:

  • Rename violations completed status to failed
  • Add warn status
  • Refer to failed violations for policy mergability check
  • Persist errored, fail-open violations as warn
  • Display warn violations in bot comment

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Fail-closed (current) Mixed fail-open and fail-closed Fail-open
CleanShot_2024-10-10_at_18.30.13_2x CleanShot_2024-10-11_at_10.13.23_2x CleanShot_2024-10-11_at_10.12.28_2x

How to set up and validate locally

  1. Create a project
  2. Create a new merge request approval policy:
    type: approval_policy
    name: Scans
    description: ''
    enabled: true
    rules:
      - type: scan_finding
        scanners: []
        vulnerabilities_allowed: 0
        severity_levels: []
        vulnerability_states: []
        branch_type: protected
    actions:
      - type: require_approval
        approvals_required: 1
        role_approvers:
          - developer
      - type: send_bot_message
        enabled: true
    fallback_behavior:
      fail: open
  3. Create another merge request approval policy:
    type: approval_policy
    name: Licenses
    description: ''
    enabled: true
    rules:
      - type: license_finding
        match_on_inclusion_license: true
        license_types:
          - MIT License
        license_states:
          - newly_detected
        branch_type: protected
    actions:
      - type: require_approval
        approvals_required: 1
        role_approvers:
          - developer
      - type: send_bot_message
        enabled: true
    fallback_behavior:
      fail: open
  4. To verify that bot comment is created when scanner configuration is missing, create a MR by updating README.md
  5. Verify that bot comment is created and it contains message: The following policies enforced on your project were skipped because they are configured to fail open: Scans, Licenses.
  6. To verify that MR gets blocked with a correct scanner configuration, add .gitlab-ci.yml into the project:
    include:
      - template: Jobs/Dependency-Scanning.gitlab-ci.yml
  7. Add empty requirements.txt file
  8. Create another MR, adding pytest into requirements.txt
  9. Verify that approvals are required and the bot comment contains message: Resolve all violations in the following merge request approval policies: Licenses.

Related to #474853 (closed)

Edited by Martin Cavoj

Merge request reports

Loading