Enhancement: Preserve Original Terraform Plan for Apply Stage in Merge Requests

Enhancement: Preserve Original Terraform Plan for Apply Stage in Merge Requests

Current Behavior

When executing the apply step after a merge request, the current OpenTofu component implementation creates a new plan instead of using the originally approved plan from the MR pipeline. This creates a potential security risk as infrastructure changes could differ from what was initially approved if API changes or other updates occur between approval and merge.

Proposed Enhancement

Implement functionality to access and use the original plan from the merge request in the apply step. This ensures that what gets deployed exactly matches what was reviewed and approved by the platform team.

Use Case

As a platform team managing infrastructure as code:

  • We want to ensure that the exact infrastructure changes reviewed and approved in a merge request are the ones being applied
  • We need to prevent potential discrepancies between approved plans and actual deployments
  • We require a clear audit trail showing that deployed changes match approved plans

Expected Behavior

  1. When a merge request is created, the pipeline generates a Terraform plan
  2. Platform team reviews and approves the plan
  3. Upon merge, the apply step uses the exact same plan that was reviewed and approved
  4. If there are any state changes between approval and merge, the pipeline should fail rather than generate a new plan

Benefits

  • Security: Ensures only reviewed and approved infrastructure changes are deployed
  • Compliance: Provides clear audit trail between approved changes and actual deployments
  • Consistency: Guarantees infrastructure deployments match approved plans
  • Workflow: Enhances approval process reliability

Technical Considerations

  • Need to implement plan caching/storage between pipeline stages
  • Ensure proper handling of Terraform state files
  • Consider implementation of plan comparison mechanisms
  • Handle cases where state has changed between approval and merge

Additional Context

This enhancement would benefit any organization using GitLab for infrastructure as code management, especially those with strict change control requirements.

Implementation Plan

This is an outline of the implementation of a first possible iteration. The goal is to generally implement the feature inside this component without changing GitLab itself.

We need the following:

In the MR pipeline:

  • In a MR pipeline the plan job must be configured to store the plan as an artifact. It already does that, but lets make sure it's in a reasonable format and we'd actually have access to it.
    • About the access: by default we use artifacts:access: none to prevent general API access and only allow runners (CI_JOB_TOKEN) to use it. I'm not sure if a CI Job Token from another pipeline in the same project would be accepted. We need to verify.
  • The artifacts needs to "survive" the merge. I think it does.

In the default branch pipeline:

  • The apply job doesn't need to change
  • We can introduce a new job template to fetch the plan artifact from the last MR pipeline of the commit at HEAD.
  • We can possible use this REST API to get the associated MR for a commit - there may be a better one, we can check. Good thing is, that we have at least one that would be available
  • We can use the Merge Request APIs to fetch the latest pipeline
  • Use the Job Artifacts APIs to download the artifact and store is as artifact of our own job for apply to consume.

A few considerations:

  • We may fetch the artifact by the "branch", see here.
    • this would imply that we can't auto-delete the branch after merge.
  • If merge trains are enabled, they must not plan otherwise the reviewed plan might be obsolete
Edited by Timo Furrer