Allow scan execution policies to create pipelines

What does this MR do and why?

As someone accountable for an organization's security posture, I want to be confident that security scans are running on all of my repositories, even if those repositories do not have CI/CD configured.

Today, scan execution policies only ensure that the security scan configuration is present in pipelines that run. Something still needs to ensure that those pipelines are triggered. To ensure that those pipelines are triggered, new pipelines should be created if a scan execution policy applies to the repository. This should happen even if Auto DevOps is disabled and no .gitlab-ci.yml is present.

We are adding a new config source to create pipelines with the security policy scans if the Auto DevOps is disabled and no .gitlab-ci.yml is available.

Related issue #403723 (closed)

Screenshots or screen recordings

Pipeline with the new security policy template

Pipeline with the .gitlab-ci.yml jobs merged with the security scan defined in the policy.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

1. Enable the feature flag

rails c

Feature.enable(:scan_execution_policy_pipelines)

1. Create a new project with a readme file.
2. Go to /-/settings/ci_cd
3. Click Expand in the Auto DevOps section
4. Disable the option Default to Auto DevOps pipeline
5. Create a new Scan execution policy with the content

type: scan_execution_policy
name: 'Test policy pipeline'
description: ''
enabled: true
rules:
  - type: pipeline
    branches:
      - '*'
actions:
  - scan: container_scanning
    tags: []

6. Click Configure with a merge request
7. Merge the new MR
8. Update the readme file using the web ide and push the changes to the main branch
9. Go to '-/pipelines' page
10. Check if a pipeline was created
11. Check if the pipeline contains the jobs defined in the new template
12. Add a .gitlab-ci.yml file with the content

test1:
  stage: test
  script:
    - echo "Do a test here"

13. Wait for the pipeline execution
14. Check if the pipeline contains the jobs defined by the Scan Execution policy

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

.gitlab/ci/cng/main.gitlab-ci.yml

+---
+default:
+  interruptible: true
+
+stages:
+  - prepare
+  - deploy
+
+include:
+  - local: .gitlab/ci/global.gitlab-ci.yml
+
+review-build-cng-env:
+  image: ${GITLAB_DEPENDENCY_PROXY_ADDRESS}ruby:${RUBY_VERSION}-alpine3.16
+  stage: prepare
+  needs:
+    # We need this job because we need its `cached-assets-hash.txt` artifact, so that we can pass the assets image tag to the downstream CNG pipeline.
+    - pipeline: $PARENT_PIPELINE_ID
+      job: build-assets-image
+  variables:
+    BUILD_ENV: build.env
+  before_script:
+    - source ./scripts/utils.sh
+    - install_gitlab_gem
+  script:
+    - 'ruby -r./scripts/trigger-build.rb -e "puts Trigger.variables_for_env_file(Trigger::CNG.new.variables)" > $BUILD_ENV'
+    - echo "GITLAB_ASSETS_TAG=$(assets_image_tag)" >> $BUILD_ENV
+    - ruby -e 'puts "FULL_RUBY_VERSION=#{RUBY_VERSION}"' >> build.env
+    - cat $BUILD_ENV
+  artifacts:
+    reports:
+      dotenv: $BUILD_ENV
+    paths:
+      - $BUILD_ENV
+    expire_in: 7 days
+    when: always
+
+review-build-cng:
+  stage: prepare
+  inherit:
+    variables: false
+  variables:
+    GITLAB_REF_SLUG: "${GITLAB_REF_SLUG}"
+    # CNG pipeline specific variables
+    GITLAB_VERSION: "${GITLAB_VERSION}"
+    GITLAB_TAG: "${GITLAB_TAG}"
+    GITLAB_ASSETS_TAG: "${GITLAB_ASSETS_TAG}"
+    FORCE_RAILS_IMAGE_BUILDS: "${FORCE_RAILS_IMAGE_BUILDS}"
+    CE_PIPELINE: "${CE_PIPELINE}"  # Based on https://docs.gitlab.com/ee/ci/jobs/job_control.html#check-if-a-variable-exists, `if: '$CE_PIPELINE'` will evaluate to `false` when this variable is empty
+    EE_PIPELINE: "${EE_PIPELINE}"  # Based on https://docs.gitlab.com/ee/ci/jobs/job_control.html#check-if-a-variable-exists, `if: '$EE_PIPELINE'` will evaluate to `false` when this variable is empty
+    GITLAB_ELASTICSEARCH_INDEXER_VERSION: "${GITLAB_ELASTICSEARCH_INDEXER_VERSION}"
+    GITLAB_KAS_VERSION: "${GITLAB_KAS_VERSION}"
+    GITLAB_METRICS_EXPORTER_VERSION: "${GITLAB_METRICS_EXPORTER_VERSION}"
+    GITLAB_PAGES_VERSION: "${GITLAB_PAGES_VERSION}"
+    GITLAB_SHELL_VERSION: "${GITLAB_SHELL_VERSION}"
+    GITLAB_WORKHORSE_VERSION: "${GITLAB_WORKHORSE_VERSION}"
+    GITALY_SERVER_VERSION: "${GITALY_SERVER_VERSION}"
+    RUBY_VERSION: "${FULL_RUBY_VERSION}"
+  trigger:
+    project: gitlab-org/build/CNG-mirror
+    branch: $TRIGGER_BRANCH
+    strategy: depend