Pipeline execution policy foundation

What does this MR do and why?

This is a Pipeline execution policy PoC that uses dry run to evaluate policy pipelines and merge them into the project pipeline. Policy stages that are not defined in main pipeline will be ignored. In the follow-up, we will add reserved stages which will be injected and adding jobs to these stages will be guaranteed to work.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Policy jobs are merged to the project pipeline:

CleanShot_2024-04-19_at_17.40.43

Error handling - when there's an error in the PEP:

How to set up and validate locally

1. Enable the feature flag echo "Feature.enable(:pipeline_execution_policy_type)" | rails c
2. Create two new projects. One for testing and one as the security policy project
3. On the testing projects left sidebar, select Security & Compliance and Policies
4. Select Edit policy project
5. Select your security policy project and Save
6. On you security policy project, create a new file .gitlab/security-policies/policy.yml with content:

   ---
   pipeline_execution_policy:
   - name: test_pipeline_execution
     description: hey
     enabled: true
     content:
       policy test job:
         stage: test
         script:
         - echo "Hello World"

7. Add a simple .gitlab-ci.yml to the project:

   project job:
     stage: test
     script:
     - echo "Project job."

8. Start a pipeline. It should contain the policy test job defined in the pipeline execution policy.

Related to #441252 (closed) and #455261 (closed).

changed milestone to %17.0

added Category:Security Policy Management Deliverable backend devopsgovern featureenhancement groupsecurity policies missed-deliverable missed:16.10 sectionsec typefeature workflowrefinement labels

assigned to @mcavoj

	2 Warnings
	This merge request is quite big (1145 lines changed), please consider splitting it into multiple merge requests.
	featureaddition and featureenhancement merge requests normally have a documentation change. Consider adding a documentation update or confirming the documentation plan with the Technical Writer counterpart. For more information, see: The Handbook page on merge request types. The definition of done documentation.

	1 Message
	CHANGELOG missing: If this merge request needs a changelog entry, add the Changelog trailer to the commit message you want to add to the changelog. If this merge request doesn't need a CHANGELOG entry, feel free to ignore this message.

Reviewer roulette

Category	Reviewer	Maintainer
backend	@panoskanell (UTC+3, 1 hour ahead of author)	@dskim_gitlab (UTC+10, 8 hours ahead of author)
~"Verify"	Reviewer review is optional for ~"Verify"	@lauraXD (UTC+2, same timezone as author)

Please check reviewer's status!

• Reviewer is available!
• Reviewer is unavailable!

Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

If needed, you can retry the danger-review job that generated this comment.

Generated by Danger

changed the description

@furkanayhan @fabiopitino I created a separate MR for the PoC which uses the dry-run approach. A happy path works and the policy jobs are merged onto the project pipeline.

What is not handled is a case when the project pipeline would be filtered out (e.g. no CI setup, or workflow rules refused), but we still want to enforce the policy pipeline. I'm going to explore this next week.

In the meantime, could you please have a look and let me know if this concept is more or less how you imagined? Thanks

requested review from @furkanayhan

added 1 commit

• 48db7f81 - Pipeline execution policy POC using dry run

Compare with previous version

added 198 commits

• 48db7f81...872bae86 - 197 commits from branch master
• d7aec5d2 - Pipeline execution policy POC using dry run

Compare with previous version

added 117 commits

• d7aec5d2...6bdb12b1 - 116 commits from branch master
• 26c2ea5e - Pipeline execution policy POC using dry run

Compare with previous version

changed the description

mentioned in merge request !150334 (merged)

removed review request for @furkanayhan