Draft: Protected Terraform States: POC — API enforcement spike

Gerardo Navarrorequested to merge
gitlab-community/gitlab-org/gitlab:590956-gerardo-navarro-protected-terraform-state-poc into master

What does this MR do and why?

This is a proof-of-concept (POC) for the Protected Terraform States epic. It validates the core approach before splitting into proper implementation MRs.

Problem

Today, any maintainer can run terraform apply from any branch — including unreviewed feature branches — or from their laptop via a personal access token. There is no way to restrict Terraform state writes to protected branches only.

Where we want to go

A protected flag per Terraform state. When protected:

  • Writes (apply/lock/unlock/delete) are only allowed from CI/CD pipelines on protected branches
  • Reads (plan) remain unrestricted
  • Local execution via PAT is blocked with HTTP 403

This follows the same pattern as protected CI variables: a simple boolean + branch check.

What this POC does NOT include

This is a validation spike — not intended to be merged. The following will be addressed in follow-up MRs:

  • Tests, feature flag, EE/Premium gating
  • GraphQL API and frontend changes
  • Data migration for existing states
  • Auto-protect on creation, usage instrumentation, documentation

🛠️ with ❤️ at Siemens

References

Demo

The following demonstrates the POC end-to-end on a local GDK instance using OpenTofu with mara.maintainer (PAT-based access, no CI context).

Summary

Step Action State Result
1 tofu apply Unprotected ✅ Succeeded
2 Protect via Rails console — protected=true
3 tofu apply Protected ❌ Rejected (403 on lock)
4 tofu plan Protected ✅ Succeeded (reads allowed)
5 Unprotect, then tofu apply Unprotected ✅ Succeeded

Console logs

Step 1: tofu apply — unprotected state ✅ 
$ source .env && TF_VAR_trigger_value="10" tofu apply -auto-approve

Acquiring state lock. This may take a few moments...
random_id.demo: Refreshing state... [id=sAZLFu-2Hyg]
null_resource.demo: Refreshing state... [id=1787930924289812686]

Plan: 2 to add, 0 to change, 2 to destroy.

null_resource.demo: Destroying... [id=1787930924289812686]
null_resource.demo: Destruction complete after 0s
random_id.demo: Destroying... [id=sAZLFu-2Hyg]
random_id.demo: Destruction complete after 0s
random_id.demo: Creating...
random_id.demo: Creation complete after 1s [id=9bU9ao3zLII]
null_resource.demo: Creating...
null_resource.demo (local-exec): Terraform apply succeeded. Random ID: f5b53d6a8df32c82
null_resource.demo: Creation complete after 0s [id=4907906850119033857]
Releasing state lock. This may take a few moments...

Apply complete! Resources: 2 added, 0 changed, 2 destroyed.
Step 2: Protect the state 
$ bundle exec rails runner /tmp/protect_on.rb

State 'default' is now PROTECTED: protected=true
Step 3: tofu apply — PROTECTED state ❌ (rejected) 
$ source .env && TF_VAR_trigger_value="11" tofu apply -auto-approve

Acquiring state lock. This may take a few moments...
╷
│ Error: Error acquiring the state lock
│
│ Error message: HTTP remote state endpoint invalid auth
│
│ OpenTofu acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
╵

The lock request receives HTTP 403 from the API. OpenTofu surfaces this as "invalid auth".

Step 4: tofu plan — PROTECTED state ✅ (reads still work) 
$ source .env && TF_VAR_trigger_value="11" tofu plan -lock=false

random_id.demo: Refreshing state... [id=9bU9ao3zLII]
null_resource.demo: Refreshing state... [id=4907906850119033857]

Plan: 2 to add, 0 to change, 2 to destroy.

Reads (GET /state/:name) are not blocked — tofu plan works even when the state is protected.

Step 5: Unprotect, then tofu apply ✅ (works again) 
$ bundle exec rails runner /tmp/protect_off.rb
State 'default' is now UNPROTECTED: protected=false

$ source .env && TF_VAR_trigger_value="11" tofu apply -auto-approve

Acquiring state lock. This may take a few moments...

Plan: 2 to add, 0 to change, 2 to destroy.

random_id.demo: Creating...
random_id.demo: Creation complete after 0s [id=E-MY8MsFxGI]
null_resource.demo: Creating...
null_resource.demo (local-exec): Terraform apply succeeded. Random ID: 13e318f0cb05c462
null_resource.demo: Creation complete after 0s [id=3073973673400573968]

Apply complete! Resources: 2 added, 0 changed, 2 destroyed.

How to set up and validate locally

  1. Switch to this branch and install dependencies 
    git checkout 590956-gerardo-navarro-protected-terraform-state-poc
bundle install
bundle exec rails db:migrate
  2. Seed the mara.maintainer user (if not already present) 
    bundle exec rake db:seed_fu FILTER=mara
  3. Create the test group and project 
    bundle exec rails runner .claude/workspace/csc-3497-protected-terraform-states/setup_terraform_test_project.rb
    Note the PROJECT_ID from the output.
  4. Install OpenTofu and initialize 
    cd .claude/workspace/csc-3497-protected-terraform-states/demo-terraform-project
cp .env.example .env
# Edit .env — set PROJECT_ID from step 3
source .env
tofu init
  5. Run tofu apply -auto-approve — should succeed
  6. Protect the state 
    # bundle exec rails console
Terraform::State.find_by(project_id: <PROJECT_ID>, name: 'default').update!(protected: true)
  7. Run TF_VAR_trigger_value="2" tofu apply -auto-approve — should fail with 403
  8. Run tofu plan -lock=false — should still work (reads allowed)
  9. Unprotect and apply again to confirm it works

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

MR Checklist (@gerardo-navarro)
  • This is a POC only — not intended for merge to master
  • Follow-up MRs planned for: DB migration, feature flag, EE service, API enforcement, GraphQL, frontend, data migration, instrumentation, docs
Edited by Gerardo Navarro

Merge request reports