Support `fallback_behavior` for MR approval policies (!147838) · Merge requests · GitLab.org / GitLab

Dominic Bauer requested to merge 451784-be-support-fallback_behavior-for-approval-policies into master Mar 25, 2024

What does this MR do and why?

Introduces the fallback_behavior property to merge request approvals (&10816):

The idea for MVC is quite simple:

we add toggle to YAML mode only,

with the toggle enabled (potentially something like fallback_behavior: { fail: closed }) we will behave as we are right now,

with the toggle disabled (potentially something like fallback_behavior: { fail: open }) we will not require approvals when policy or project is misconfigured,

we use fallback_behavior: { fail: open } format to allow for further extensions in the future, potentially we could have a list of scenarios where we want to have different scenarios or some exceptions, like we had for many previous features

From the description of this Epic we want to support these scenarios:

Required scanners defined in the condition did not run or produce any artifacts.

For scanners that require builds, when a project does not have a build pipeline.

Required number of approvals became higher than available, valid approvers.

When a security policy fails for an unspecified reason.

See &10816 (comment 1797964157)

Database queries

UPDATE
	"approval_merge_request_rules"
SET
	"approvals_required" = 0
WHERE
	"approval_merge_request_rules"."id" IN (121649063, 121649062);

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/27347/commands/85114

SELECT "approval_merge_request_rules".* FROM "approval_merge_request_rules" WHERE "approval_merge_request_rules"."merge_request_id" = 260129972;

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/27374/commands/85192

DELETE FROM "scan_result_policy_violations"
WHERE "scan_result_policy_violations"."merge_request_id" = 260129972
	AND "scan_result_policy_violations"."scan_result_policy_id" = 6686754;

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/27347/commands/85119

SELECT
	"approval_merge_request_rules".*
FROM
	"approval_merge_request_rules"
WHERE
	"approval_merge_request_rules"."merge_request_id" = 260129972
	AND "approval_merge_request_rules"."report_type" = 4;

https://console.postgres.ai/gitlab/gitlab-production-main/sessions/27347/commands/85121

How to set up and validate locally

Create a new project and enable the feature flag:

Feature.enable(:merge_request_approval_policies_fallback_behavior, Project.last)

Navigate to Settings > CI/CD and disable Default to Auto DevOps pipeline.
Add a second Owner member to the project.

Validating `scan_finding` rules

Navigate to Secure > Policies and create the following approval policy:

type: approval_policy
name: Container Scanning Fail Closed
enabled: true
rules:
- type: scan_finding
  scanners:
  - container_scanning
  vulnerabilities_allowed: 0
  severity_levels: []
  vulnerability_states: []
  branch_type: protected
actions:
- type: require_approval
  approvals_required: 1
  role_approvers:
    - owner

Open a new MR that adds a new unrelated file.
Verify the Container Scanning Fail Closed rule requires approval (compare with bot comment).
Navigate to Secure > Policies and create the following approval policy:

type: approval_policy
name: Container Scanning Fail Open
enabled: true
fallback_behavior:
  fail: open
rules:
- type: scan_finding
  scanners:
  - container_scanning
  vulnerabilities_allowed: 0
  severity_levels: []
  vulnerability_states: []
  branch_type: protected
actions:
- type: require_approval
  approvals_required: 1
  role_approvers:
    - owner

Verify that:
- Container Scanning Fail Closed rule requires approval.
- Container Scanning Fail Open rule doesn't require approval.
Commit the following .gitlab-ci.yml to the MR source branch:

container_scanning:
  image: alpine
  stage: test
  allow_failure: true
  artifacts:
    reports:
      container_scanning: gl-container-scanning-report.json
    paths: [gl-container-scanning-report.json]
  script:
    - apk add curl
    - curl -o gl-container-scanning-report.json https://gitlab.com/-/snippets/3690310/raw/main/gl-container-scanning-report-with-findings.json

Verify that both Container Scanning Fail Closed and Container Scanning Fail Open rules require approval.
Navigate to Secure > Policies and edit the existing Container Scanning Fail Open to make it invalid:

type: approval_policy
name: Container Scanning Fail Open
enabled: true
fallback_behavior:
  fail: open
rules:
  - type: scan_finding
    scanners:
      - container_scanning
    vulnerabilities_allowed: 0
    severity_levels: []
    vulnerability_states: []
    branch_type: protected
actions:
  - type: require_approval
-   approvals_required: 1
+   approvals_required: 2
    role_approvers:
      - owner

Verify that:
- Container Scanning Fail Closed rule requires approval.
- Container Scanning Fail Open rule doesn't require approval.
Update the existing .gitlab-ci.yml on the MR source branch so that there are no Container Scanning findings:

container_scanning:
  image: alpine
  stage: test
  allow_failure: true
  artifacts:
    reports:
      container_scanning: gl-container-scanning-report.json
    paths: [gl-container-scanning-report.json]
  script:
    - apk add curl
-   - curl -o gl-container-scanning-report.json https://gitlab.com/-/snippets/3690310/raw/main/gl-container-scanning-report-with-findings.json
+   - curl -o gl-container-scanning-report.json https://gitlab.com/-/snippets/3690310/raw/main/gl-container-scanning-report-without-findings.json

Verify that:
- both Container Scanning Fail Closed and Container Scanning Fail Open rules don't require approval.
- the bot comment notes that all policy violations were resolved.

Validating `license_finding` rules

In the same project, navigate to Secure > Policies and create the following approval policy:

type: approval_policy
name: Deny MIT Fail Closed
enabled: true
rules:
- type: license_finding
  match_on_inclusion: true
  license_types:
  - MIT
  license_states:
  - newly_detected
  branch_type: protected
actions:
- type: require_approval
  approvals_required: 1
  role_approvers:
    - owner

Verify that:
- Only the Deny MIT Fail Closed rule requires approval.
- the bot comment notes the policy violation.
Navigate to Secure > Policies and create the following approval policy:

type: approval_policy
name: Deny MIT Fail Open
enabled: true
fallback_behavior:
  fail: open
rules:
- type: license_finding
  match_on_inclusion: true
  license_types:
  - MIT
  license_states:
  - newly_detected
  branch_type: protected
actions:
- type: require_approval
  approvals_required: 1
  role_approvers:
    - owner

Verify that only the Deny MIT Fail Closed rule requires approval.
Commit the following Gemfile.lock:

GEM
  remote: https://rubygems.org/
  specs:
    rack (3.0.10)

PLATFORMS
  arm64-darwin-22
  ruby

DEPENDENCIES
  rack

BUNDLED WITH
   2.5.4

Update the existing .gitlab-ci.yml and add Dependency Scanning:

+include:
+  - template: Jobs/Dependency-Scanning.gitlab-ci.yml

Verify that both the Deny MIT Fail Closed and Deny MIT Fail Open rules require approval.
Navigate to Secure > Policies and edit the existing Deny MIT Fail Open rule to make it invalid::

type: approval_policy
name: Deny MIT Fail Open
enabled: true
fallback_behavior:
  fail: open
rules:
- type: license_finding
  match_on_inclusion: true
  license_types:
  - MIT
  license_states:
  - newly_detected
  branch_type: protected
actions:
- type: require_approval
- approvals_required: 1
+ approvals_required: 2
  role_approvers:
    - owner

Verify that only the Deny MIT Fail Closed rule requires approval.
Update the existing Gemfile.lock:

GEM
  remote: https://rubygems.org/
  specs:

PLATFORMS
  arm64-darwin-22
  ruby

DEPENDENCIES

BUNDLED WITH
   2.5.4

Verify that:
- No rule requires approval.
- the bot comment notes that all policy violations were resolved.

Validating MRs without pipelines

In the same project, open a new MR that adds a new unrelated file.
Verify that:
- Only the Container Scanning Fail Closed and Deny MIT Fail Closed rules require approval.
- the bot comment notes the policy violation.
Navigate to Secure > Policies and disable both these policies.
Verify that:
- No rule requires approval.
- the bot comment notes that all policy violations were resolved.

Related to #451784 (closed)

Edited Apr 04, 2024 by Dominic Bauer

Admin message

Support `fallback_behavior` for MR approval policies

What does this MR do and why?

Database queries

How to set up and validate locally

Validating scan_finding rules

Validating license_finding rules

Validating MRs without pipelines

Merge request reports

Validating `scan_finding` rules

Validating `license_finding` rules