Evaluate redesign of Compliance Framework pipelines
Summary
We have been seeing an increase in reported problems related to running Compliance Framework pipelines. This issue serves to consolidate all the challenges we are currently aware of. Also included are considerations for future use cases. The goal is to evaluate a redesign of Compliance Framework pipelines which may serve as a more maintainable, holistic solution to all of these challenges.
Current implementation
We essentially include:
the Compliance config file into the pipeline. If the user would also like to run the Project's config file, they must include:
it in the Compliance file.
Challenges
1. Unable to support a project with an external config file or no config file
We currently suggest that customers include a Project's config file in the Compliance like so:
include:
- project: '$CI_PROJECT_PATH'
file: '$CI_CONFIG_PATH'
ref: '$CI_COMMIT_SHA'
However, this breaks when the project has a custom CI/CD configuration file that is external (in another project) or remote.
No config file
For the same reason an external project config file breaks the pipeline,
Applying a compliance framework label to projects with a missing
.gitlab-ci.yml
file will cause compliance pipelines to fail
Epic: &7315
2. CI variables in the Compliance config can be overwritten
Due to the CI variables precedence order, variables that are specified within a Compliance config job are low in the list (6th item) and can be overwritten in several ways including Project and Group-level variables.
Issue: #415870 | Discussion: #393960 (comment 1437362050)
3. Compliance jobs can be overwritten by target repository
extends
Using If you use the extends statement in a compliance pipeline configuration, compliance jobs are overwritten by the target repository job.
Doc link | Related Issue: #337321
Order of includes
Including the "local" project CI file last within the list of templates and files to include enables the compliance framework job to be overridden.
Issue: #414004
stages:
)
4. Conflict with top-level global keywords (e.g. For example, the stages:
keyword cannot be specified both in the Compliance file and the Project config file; the stages:
definition in the Compliance file takes precedence. The Project is unable to use its own independent stages:
definition for its workflow.
Discussion: #393960 (comment 1425465333) (item 2)
5. Must ensure compliance jobs are always run
6. Prefilled variables are not shown when manually starting pipeline
Because of a known issue, compliance pipelines in GitLab 15.3 and later can prevent prefilled variables from appearing when manually starting a pipeline.
7. Need to unify Scan Execution Policies and Compliance Framework
Case study: How Secure and Govern stages rely on Verify architecture
8. Compliance Framework not found or access denied when run by bot user
Issue: #404707 (closed)
9. Compliance pipeline jobs don't run in a specific situation due to GitLab only checking the project's pipeline configuration
Issue: #412279
10. .pre/.post jobs should be optionally allowed in empty pipelines
Issue: #420339
Other Considerations
1. How will it work with CI Events?
Issue: #363384 (comment 1430046183)
2. How will it work with CI Catalog?
3. How will it work with predefined CI variables?
With regards to the parent-child pipeline approach.
4. Future plans for Compliance Frameworks?
Support git refs in Compliance Pipeline Configuration
Issue: #378517
Option to run Compliance pipeline after Project pipeline
Discussion: #393960 (comment 1450331081)