## Release notes The pipeline execution action extends capabilities of the scan execution policy type to support enforcement of generic CI jobs, scripts, and instructions. In addition to configuring GitLab's security scanners out-of-the-box through scan execution policies, the pipeline execution action enables security and compliance teams to enforce customized GitLab security scanning templates, 3rd party security scanning templates, create custom reporting rules through CI jobs, or create any custom scripts/rules possible through GitLab CI. This feature dynamically evaluates the pipeline execution YAML defined in the policy and compares it against any project the policy is enforced against and merges it based on set of rules, overriding the project CI configuration such that security and compliance can reliably be enforced. As with all GitLab Policies, enforcement can be managed centrally by designated security and compliance team members who create and manage the policies. [Learn how to get started by creating your first scan execution policy](https://docs.gitlab.com/ee/tutorials/scan_execution_policy/)! ## Problem to solve Users need a single solution for enforcing jobs to be run as part of a project pipeline. They want a way to combine the flexibility of [compliance framework pipelines](https://docs.gitlab.com/ee/user/project/settings/index.html#compliance-pipeline-configuration) with the simplicity of [scan execution policies](https://docs.gitlab.com/ee/user/application_security/policies/#scan-execution-policy-schema). There are many cases that could be addressed using custom yaml to define policy rules, but here are a few of the most common we've heard so far: 1. I want to include CI templates using `include` statements and be able to pull in templates such as GitLab's `.latest` security scanner templates. See [CI templates](https://gitlab.com/gitlab-org/gitlab/-/tree/master/lib/gitlab/ci/templates). 2. I want to enforce execution of a 3rd party scanner, so that I can capture additional security results and manage them alongside results from GitLab scanners in the vulnerability report. 3. I want to enforce a custom script/check for compliance. 4. I want to enforce scanners to analyze code stored in GitLab, but call out to a 3rd party CI tool to continue a pipeline. A common case here may be for users migrating to GitLab, but some teams still may be using 3rd party CI tools in the meantime. 5. I want to export results from scanners to a 3rd party tool. 6. I want to block MRs until a UAT in an external tool is completed (quality gate), so that I can ensure the quality of changes to my production code. 7. I want to standardize custom project badges that can quickly communicate to users in a project which security policies are enabled and if pipelines are blocked by a given scan result, so that my security team and developer team can easily understand the security state of a project. 8. I want to create a custom badge in projects to communicate the security grade, so that team members are incentivized to keep their score high. ## Intended users * [Cameron (Compliance Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#cameron-compliance-manager) * [Devon (DevOps Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#devon-devops-engineer) * [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator) * [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst) * [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer) ## Proposal ### User Experience A walkthrough of the proposal can be viewed at https://youtu.be/grg_M1MtiYw?list=PL05JrBw4t0Kq4CHpCTMv3OdquJXm6ggYr&t=6 1. Users will be able to import CI yml file as an action for a scan execution policy. 1. After import, user can still edit the file content, it will be saved in the policy setting, but it won't affect the file itself. 2. Users will be able to link CI yml file as an action for a scan execution policy. 1. After linked a yml file, it will function in the same way that compliance framework pipelines function today by executing the CI defined in the path together with the project's `gitlab-ci.yml` file. 2. Explicitly including the project's CI yml file inside the compliance CI yml file will no longer be required. This will be done automatically by default. 3. Group owners who edit a Compliance Framework label will no longer have the option to define a compliance framework pipeline. Instead they will see a list of Security Policies that use the given Compliance Framework label. Note: Some of this work may be partially done as part of https://gitlab.com/groups/gitlab-org/-/epics/5510+. 4. We will need to define the order of operations for handling custom jobs defined within a security policy and ensure conflicts, such in the case of CI variables, are easily addressed. 5. ~~User should be able to differentiate that given Job (both from Custom YAML and regular scans in Scan Execution Policies) was enforced by Security Policy.~~ Moved to https://gitlab.com/groups/gitlab-org/-/epics/11796+. ### Order of operations 1. Capture the `.gitlab-ci.yml` file from the repository where we have enforced the given policy. 2. Merge the Custom YAML policy with this file overriding all fields. 3. Merge YAML which was created as a result of OOTB policy rules. OOTB policy rules defined in the UI or policy yaml take precedence over any custom YAML. 4. We do not maintain any additional logic to merge variables. 5. Variables should honor variable precedence in https://docs.gitlab.com/ee/ci/variables/#cicd-variable-precedence, otherwise they will override project settings. Once https://gitlab.com/gitlab-org/gitlab/-/issues/424028+ is completed, SEP variables will be treated with highest precedence. 6. Includes should be executed in the order they appear in the custom yaml. ### Requirements Analysis As we compare capabilities between compliance pipelines and the future scan execution policies with custom yaml support, it's important to understand if/how behaviors may change between the two when users migrate, which requirements will be supported in the initial MVC, and which areas we may plan to improve in the future (but consider out of scope for now). We'll capture the core requirements below: <details> <summary> #### Requirements Matrix </summary> <table> <tr> <td> **Criteria** </td> <td> **Basic Description/Context** </td> <td> **Compliance Pipelines** </td> <td> **Scan execution policies (current)** </td> <td> **Scan execution policy with custom yaml (planning)** </td> </tr> <tr> <td> **Management and configuration** </td> <td> </td> <td>Compliance pipeline yaml created within Compliance Framework settings</td> <td>OOTB GitLab Native scanners can be configured, default/custom CI variables can be passed through the policy to the job execution</td> <td>Custom jobs created within Security Policy, or a configuration file can be linked to from the policy. Policies can then be linked to a Compliance Framework</td> </tr> <tr> <td> **Creating and validating the pipeline execution yaml config** </td> <td> </td> <td>Today, compliance pipelines require you to validate the yaml elsewhere before adding to the remote config file. Testing/validation can be more challenging and there is a lack of visibility in the project config of how compliance pipelines impact the configuration.</td> <td> </td> <td> </td> </tr> <tr> <td> **Access and Permissions** </td> <td> </td> <td> Any developer or maintainer must have a `Reporter` role in the project containing the remote CI yaml file, in order to trigger the pipeline. </td> <td> * Developers must have access to SPP to view policies in a project. * Owners must have access to the SPP and the group/sub-group/project </td> <td> Any developer or maintainer must have a `Reporter` role in the project containing the remote CI yaml file, in order to trigger the pipeline. </td> </tr> <tr> <td> **Variable precedence** </td> <td> </td> <td>Variable precedence is not controlled/affected by compliance pipelines.</td> <td>SEP variables share the highest level of precedence as defined in our CI documentation - manually triggered pipelines would override SEP variables, for example</td> <td> SEP variables are alone treated with highest precedence. This is now addressed with[ this release](https://about.gitlab.com/releases/2023/12/21/gitlab-16-7-released/#enforce-variables-in-scan-execution-policies-with-the-highest-precedence). </td> </tr> <tr> <td> **Before and after scripts** </td> <td>Often used for installing dependencies required for a given project, or for generating custom reports after a job.</td> <td> See related [issue](See%20issue.). Before and after scripts must be added to .pre or .post stages or pre-compliance/post-compliance (but the latter are arbitrary and require dev projects to add these stages). This appears also like more of a best practice in compliance pipelines, not sure if you technically cannot use before/after scripts. </td> <td> </td> <td>We likely need some controls to detect before/after scripts, and handle this based on various requirements.</td> </tr> <tr> <td> **Managing pipeline stages** </td> <td> </td> <td> \- Projects must have the stages defined in the pipeline or the compliance pipeline will fail<br>- You can add jobs to a pre or post compliance stage or pre/post stage, but you can't inject stages dynamically" </td> <td>Policy jobs for scans other than DAST scans are created in the test stage of the pipeline. If you modify the default pipeline stages, to remove the test stage, jobs will run in the scan-policies stage instead. This stage is injected into the CI pipeline at evaluation time if it doesn’t exist. If the build stage exists, it is injected just after the build stage. If the build stage does not exist, it is injected at the beginning of the pipeline. DAST scans always run in the dast stage. If this stage does not exist, then a dast stage is injected at the end of the pipeline.</td> <td>SEPs will allow customers to dynamically inject stages into development projects, following certain configuration rules</td> </tr> <tr> <td> **No .gitlab-ci.yml** </td> <td> </td> <td>Requires projects to have a CI configuration or compliance pipeline will attempt to execute and fail</td> <td>Implicitly creates a pipeline to run jobs, but is currently limited to OOTB GitLab security scanners</td> <td> We have https://gitlab.com/gitlab-org/gitlab/-/issues/432050 to ensure this support is added for the pipeline execution action. </td> </tr> <tr> <td> **Include statements** </td> <td>Used more for re-using config, distinct from parent/child or multi-project pipelines</td> <td> </td> <td>N/A</td> <td>Includes should be executed in the order they appear in the pipeline execution action.</td> </tr> <tr> <td> **Merge request pipelines** </td> <td> </td> <td> </td> <td>N/A</td> <td> </td> </tr> <tr> <td> **Merged results pipelines** </td> <td> </td> <td> </td> <td>N/A</td> <td> </td> </tr> <tr> <td> **Scheduled Pipelines** </td> <td> </td> <td>Not currently supported in Compliance pipelines</td> <td>Scheduled pipelines can be configured and enforced for OOTB GitLab scanners</td> <td> \- Not to be included for initial MVC requirements<br>- Additional security reviews and considerations may be required before adding this support, and as compliance pipelines do not offer any scheduled pipeline capability, it's not required for deprecating compliance pipelines </td> </tr> <tr> <td> **Parent/Child Pipelines** </td> <td>Uses "trigger: include:" to include a child pipeline in the yaml</td> <td> </td> <td>N/A</td> <td> </td> </tr> <tr> <td> **Multi-project pipelines (**[**ref**](https://about.gitlab.com/blog/2022/02/22/parent-child-vs-multi-project-pipelines/)**)** </td> <td>Uses "trigger: project:" to trigger a downstream pipeline after this component pipeline completes</td> <td> </td> <td>N/A</td> <td> </td> </tr> <tr> <td> **Needs statements** </td> <td> </td> <td> </td> <td>N/A</td> <td>Needs statements may allow jobs to execute in project pipelines without waiting on compliance jobs. We may need some way to postpone or modify needs to support compliance use cases.</td> </tr> <tr> <td> **.pre/.post stages** </td> <td>Hidden stages in all pipelines</td> <td> It seems there are some limitations here today. See https://gitlab.com/gitlab-org/gitlab/-/issues/412279+. Jobs defined in .pre/.post also cannot run in empty pipelines - https://gitlab.com/gitlab-org/gitlab/-/issues/420339+. </td> <td>N/A</td> <td>? - This use case should likely be tested in SEP to confirm it will be addressed.</td> </tr> <tr> <td> [**Workflow keyword**](https://docs.gitlab.com/ee/ci/yaml/workflow.html)\*\* / rules\*\* </td> <td>Controls when pipelines are created, such as based on a tag or pipeline source type. Appears as "workflow: rules:" in yaml.</td> <td> When defining workflow rules in the compliance yaml, there can be conflicts with the project yaml. A workaround may be to use downstream pipelines to sandbox the compliance jobs, but this can require lots of custom CI logic to determine when/how to enforce the jobs based on project configuration. </td> <td>N/A</td> <td> We may need to compare various compliance vs project CI configs to compare common workflow patterns. Could there be a new pattern to sandbox the security/compliance jobs more reliably? Or, we may have some way of reading various workflow rules and merging them more dynamically. </td> </tr> <tr> <td> [**Custom/remote CI Configuration**](https://docs.gitlab.com/ee/ci/pipelines/settings.html#custom-cicd-configuration-file-examples)\*\* (or "external" config file)\*\* </td> <td>When a project that is being enforced uses a custom/remote config for their own CI, compliance pipelines break</td> <td>Not currently supported in Compliance pipelines</td> <td>N/A</td> <td> </td> </tr> <tr> <td> [**rules:when:always block**](https://docs.gitlab.com/ee/user/group/compliance_frameworks.html#ensure-compliance-jobs-are-always-run) </td> <td> </td> <td>Best practice in Compliance pipelines to use in all jobs to achieve expected behavior (ensure jobs cannot be overriden in some cases, or that they can be in others)</td> <td>N/A</td> <td> </td> </tr> <tr> <td> [**Explicitly set container image**](https://docs.gitlab.com/ee/user/group/compliance_frameworks.html#ensure-compliance-jobs-are-always-run) </td> <td> </td> <td>Best practice in Compliance pipelines is to explicitly set the container image to run the job in. This ensures that your script steps execute in the correct environment.</td> <td>N/A</td> <td> </td> </tr> <tr> <td> [**extends statement**](https://docs.gitlab.com/ee/user/group/compliance_frameworks.html#compliance-jobs-are-overwritten-by-target-repository) </td> <td> </td> <td> Compliance pipelines can be overridden when \`extends\` is used in the compliance pipeline yaml. </td> <td>N/A</td> <td> </td> </tr> <tr> <td> [**Prefill variables in manually triggered pipelines**](https://docs.gitlab.com/ee/user/group/compliance_frameworks.html#prefilled-variables-are-not-shown) </td> <td> </td> <td>Limitation in compliance pipelines today. When using manual job in dev project, manual variables should prefill with those defined in compliance pipeline.</td> <td>This could be a consideration for SEP today, but security policies do not prefill (or enforce) variables for manual pipeline triggers either.</td> <td> </td> </tr> <tr> <td> [**CI Components**](https://docs.gitlab.com/ee/ci/components/) </td> <td>Offers more stable/versioned CI templates -- currently still working towards MVC but it will help to understand where they are heading and the impact our plans today.</td> <td> </td> <td> </td> <td> Created https://gitlab.com/groups/gitlab-org/-/epics/12322+ to start exploring this. Currently of scope for current plans in for the Pipeline Execution Action. </td> </tr> <tr> <td> Allow [**Bot users to execute Pipeline Execution Action CI jobs**](https://gitlab.com/gitlab-org/gitlab/-/issues/404707 "Compliance Framework not found or access denied") </td> <td> </td> <td>Compliance pipelines have a limitation when running a pipeline using a bot user.</td> <td> </td> <td>Our current implementation requires users to be members of the repository containing any remote CI configuration to be able to run/execute the pipeline - this too would include bot users. A script may be created to add any users needed to run the jobs to the project.</td> </tr> <tr> <td> **Support git refs in compliance configuration** </td> <td> \- With git refs, teams could better test and understand the impact of changes compliance pipelines to before rolling them out globally.<br>- This requirement would add the ability to define a git ref on the compliance yaml file - e.g. compliance@some/file:ref </td> <td> Compliance pipelines do not currently support git refs (see [issue](https://gitlab.com/gitlab-org/gitlab/-/issues/378517 "Support git refs in Compliance Pipeline Configuration")). Only one compliance pipelines file can be specified for a compliance framework. </td> <td> </td> <td>Our current experimental implementation supports git refs, allowing users to more easily test branch configuration for policies stored in a lower non-protected branch before enforcing the main branch configuration.</td> </tr> <tr> <td> **Enforce/lock variables for compliance (**[**epic**](https://gitlab.com/groups/gitlab-org/-/epics/11797 "Optional control of all scan execution policy CI variables")**)** </td> <td> Undefined variables currently leave project CI configuration to take precedence, allowing project, group, or instance-inherited variables to take effect. When variables are defined in an SEP, they take precedence over all other CI variable types. </td> <td> </td> <td> </td> <td>This epic will prevent circumvention of policies when variables are left undefined (when necessary for compliance requirements) for scan execution policies, including the pipeline execution action.</td> </tr> <tr> <td> **Store a source value for jobs (**[**epic**](https://gitlab.com/groups/gitlab-org/-/epics/11796 "Store a `source` value for Jobs")**)** </td> <td>This could verify that a build artifacts were executed by a scan execution policy.</td> <td> </td> <td> </td> <td> </td> </tr> <tr> <td> **Understanding the final merged yaml (between compliance and project configs)** </td> <td>GitLab CI at the project configuration level has Visualize and Full Configuration tools, to help developers understand their configuration.</td> <td> Compliance pipelines do not currently affect the visualize and full configuration tabs. Developers will be able to see compliance pipelines running with additional jobs, as the config overwrites the project config and then includes the project jobs, but there is no delineation between project config and compliance config. </td> <td> </td> <td> We should explore how we might represent the final config under the "full configuration" view. This will give project teams more confidence in how their pipeline is running AFTER compliance enforcement. As well as some guidance to help debug any issues. We may want to then further explore better communicating where these jobs come from security policies in the full config and visualize views. </td> </tr> <tr> <td> **Manual job behaviors** </td> <td>Manual jobs can be defined in CI such that they will not automatically trigger in the pipeline and require users to manually trigger the job when ready.</td> <td>Compliance pipelines have no special handling of manual jobs.</td> <td>Policies didn't have any previous manual job options.</td> <td> Manual jobs executed through CI allow for variables to be injected when run and they can be retried. For the Pipeline Execution Policy Action, we might consider blocking injection of variables into manual jobs, enforcing only variables defined in the policy upfront, and block retry. </td> </tr> </table> </details> ### Acceptance Criteria * [ ] Pipeline execution policies should support injecting jobs directly into the GitLab CI yaml of the target projects. * [ ] At minimum, custom yaml jobs should align with [existing CI variable precedence](https://docs.gitlab.com/ee/ci/variables/#cicd-variable-precedence). Ideally, all CI variables defined in a scan execution policy job should execute as highest precedence. Specifically, scan execution job variables should take precedence over project, group, and instance variables, among others. The latter enhancement will be handled in https://gitlab.com/gitlab-org/gitlab/-/issues/424028+. * [ ] Custom scan execution policies should execute custom yaml in projects that do not contain an existing CI configuration, the same as standard scan execution policies work today. * [ ] Must be able to inject jobs into project pipelines in a way that cannot be circumvented and does not require aligning stages. Jobs must be unique, additive, and non-conflicting if the jobs are defined elsewhere (even other scan execution or pipeline execution policies). * [ ] There should be some convention or dynamic handling of stages to allow for enforcement of jobs throughout an existing pipeline config as needed - typically before the pipeline, after the pipeline, or after the build (a security/compliance test stage). I think customers will still want flexibility to enforce jobs for various complex use cases as well, which may be outside of any `reserved` stages. * [ ] Job variables must be enforced and not possible to override (unless they want them to be usable at project level) * [ ] Some orgs leverage compliance pipelines to fully override project pipelines, and they are using downstream pipelines to give any flexibility they can. They want to be confident in their controls, and then to allow flexibility at project level. We may need to give a way for some customers to fully override the project pipelines. * [ ] The jobs enforced in project pipelines should be visible to project teams in some way (we can add more features here post-MVC but need some layer of visibility to start). * [ ] If projects do not have CI at all, the policy should still enforce the security/compliance jobs to run on the project. * [ ] We should have a way to handle workflow rules across projects intuitively. Customers may have teams using two or more different patterns, causing conflicts or gaps in scan execution. As we consider Acceptance Criteria, there may also be use cases that we are not yet considering required unless/until we have more discovery and understanding of the need. We'll capture those below: - Multi-project pipelines - Parent/child pipelines ## Design **Original design issue:** [**gitlab#379123**](#) | Overview of the two actions to execute a code block | Show the import file option in the execute code block section | Steps for importing code block | After importing, show, load the code from the file as editable | Link file option | Error message | |-----------------------------------------------------|---------------------------------------------------------------|--------------------------------|----------------------------------------------------------------|------------------|---------------| |  |  |  |  |  |  | Solution validation completed in https://gitlab.com/gitlab-org/ux-research/-/issues/2287+. ## Permissions and Security This will not change permissions ## Documentation Documentation on both [Security Policies](https://docs.gitlab.com/ee/user/application_security/policies/) and [Compliance Pipelines](https://docs.gitlab.com/ee/user/project/settings/index.html#compliance-pipeline-configuration) will need to be updated ## Availability & Testing ## What does success look like, and how can we measure that? * Positive sentiment/feedback from core customers moving from compliance pipelines. * Support issue volume * Increase in MAU for Security Policies ## What is the type of buyer? ~"GitLab Ultimate" ## Is this a cross-stage feature? This feature will combine the Scan Execution Policy feature with the Compliance Framework Pipeline feature built by ~"group::compliance" in [this Epic](https://gitlab.com/groups/gitlab-org/-/epics/4082 "Improve the concept of compliance frameworks") ## Links / references https://gitlab.com/groups/gitlab-org/-/epics/4425 https://gitlab.com/groups/gitlab-org/-/epics/5510 _This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc._