Skip to content

Compliance Pipelines to Pipeline Execution Policy Variable Migration issue

Summary

As part of migrating from Compliance Pipelines to Pipeline Execution Policy, Customers have reported a bug / blocker for migration.

  • When using Compliance Pipeline we are able to define / enforce both Compliance & Project Variables from CI file.

    (in UI before triggering pipeline for a branch)

    image.png

  • But are unable to do so when using Pipeline Execution Policy (Post migration from existing Compliance pipeline CI file)

    image.png

Steps to reproduce

  • Create a Compliance Pipeline with Variables in Compliance Pipeline and Project Pipeline using Compliance Framework
  PROJECT_VAR:
    value: "a"
    options:
      - "a"
      - "b"
      - "c"
  • Apply Compliance Framework over a Project (Where Compliance Pipeline exist)
  • Go To Project > Pipelines > New Pipeline in Project
    • Notice the CI Variables getting auto-populated in New Pipeline UI
  • Migrate the Compliance Pipeline to Pipeline Execution Policy
  • Go To Project > Pipelines > New Pipeline in Project where PEP was implemented with same logic
    • Trigger the Pipeline and notice missing CI Variables as shown in screenshots above

Example Project

What is the current bug behavior?

Missing CI Variables when migrating from Compliance Pipelines to Pipeline Execution Policy

What is the expected correct behavior?

When migrating from Compliance Pipeline to Pipeline Execution Policy the CI Variables should load and the behaviour should remain the same.

Relevant logs and/or screenshots

Added in above Summary

Output of checks

Possible fixes

We would need to implement support for prefilled variables for PEP.

As far as I understand, the prefilled variables are fetched like this:

  • By invoking Ci::ListConfigVariablesService which uses YamlProcessor to parse the CI config
  • It has no context of pipeline execution policies and only parses the project config
  • As this service is invoked via GQL from the PipelineVariablesForm, it wouldn't be performant enough to parse all PEP configs in addition to the project config

I think we could make it work by extending our "PEP config analyzer" which is currently parsing the enforced scanners from PEP configs. We have a BBM planned for %18.0 to backfill data. We can:

  • extend the service now to parse out the prefilled variables and store them in the DB
  • the BBM will take care of backfilling this data for all existing PEPs
  • extend Ci::ListConfigVariablesService to merge PEP prefilled variables from the DB for all PEPs applicable to the project.
Edited by 🤖 GitLab Bot 🤖