Skip to content

Add FF around terraform state name validation

What does this MR do and why?

On !105674 (merged), we've fixed a bug where terraform state names couldn't be managed with dots (.) if people were using state locks, which seems like it was caused by a limitation of grape. So when terraform was reaching our api/v4/projects/:id/state/:name/lock endpoint, we would get 404s if the name had a . in it.

We fixed it by overriding our requirement option on that endpoint namespace. Although, there were some users who were creating terraform states without locking. This can be done by using -lock=false on your terraform commands. In those cases, users were able to create the states, but the state name got stripped from its dot. So a foo.bar state name got saved as foo. Still, users were able to create them. For those users, once we merged the fix, they started creating new states with the dot included. So foo.bar became possible. This means that their terraform state management broke, as their old states are tracked in foo, not in foo.bar.

We're ok about keeping this behaviour in GitLab.com, and we'll try to communicate to users how they should act, which is by renaming their state names back to whatever they user before the dots. But to avoid more support overload, we want to disable this new fix to self-managed. So that we can properly warn self-managed in time, then deprecate the old behaviour in %16.0. To achieve this, we're bringing the behaviour back by returning 404 on the lock state endpoint in case the users tries to use it with a state name including a dot.

Related Issue

#385564 (closed)

How to set up and validate locally

With the FF false (old behaviour)

  1. Feature.disable(:allow_dots_on_tf_state_names)
  2. Created a project and try to deploy with a . in the state name:
# backend.tf

terraform {
  backend "http" {}
}
# main.tf

resource "local_file" "foo" {
  filename             = "${path.module}/foo.bar"
  content              = "bar!"
  directory_permission = "0755"
  file_permission      = "0755"
}
# .gitlab-ci.yml

include:
  - template: Terraform/Base.gitlab-ci.yml

variables: 
  TF_STATE_NAME: foo.tfstate

stages:
  - deploy

deploy:
  extends: .terraform:deploy
  environment:
    name: $TF_STATE_NAME
  script:
    - gitlab-terraform init
    - gitlab-terraform plan
    - gitlab-terraform apply
  rules: 
    - when: always
  1. Verify your job fails with:
Terraform has been successfully initialized!

│ Error: Error acquiring the state lock

│ Error message: Unexpected HTTP response code 404

│ Terraform acquires a state lock to protect the state from being written
│ by multiple users at the same time. Please resolve the issue above and try
│ again. For most commands, you can disable locking with the "-lock=false"
│ flag, but this is not recommended.
  1. Verify no state got created in the database. Terraform::State.count should be zero.
  2. Add -lock=false to all of gitlab-terraform commands. So same as above but replaced the commands:
...
  script:
    - gitlab-terraform init -lock=false
    - gitlab-terraform plan -lock=false
    - gitlab-terraform apply -lock=false
  1. Verify that the job succeeded and that the state name is created stripped from the name. Terraform::State.last.name.
  2. Turn the FF on: Feature.enable(:allow_dots_on_tf_state_names)
  3. Redo the above, perhaps with a new TF_STATE_NAME name.
  4. Verify that state name is created with dot independently from the lock endpoint.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Timo Furrer

Merge request reports