Exception for late deprecation notice: Enable encrypted state support by breaking Terraform state files stale since October 2020
Breaking Change Exception Request
Executive summary
We are requesting an exception to add a breaking change to 17.0. This breaking change would affect the Terraform state files that were not updated since October 2020. On gitlab.com, this is ~0.3% of the Terraform state files. In October 2020, we added state version support to the Terraform state storage. Versioned state files are not affected by this change.
We would not break the users' infrastructure, just their representation in the Terraform state.
This breaking change would enable support for encrypted state files for OpenTofu users, and would be the preferred workaround for some pipeline artifact visibility setting limitations.
More details on the visiblity setting limitations
Currently, the related documentation writes:Like any other job artifact, Terraform plan data is viewable by anyone with the Guest role on the repository. Neither Terraform nor GitLab encrypts the plan file by default. If your Terraform plan.json or plan.cache files include sensitive data like passwords, access tokens, or certificates, you should encrypt the plan output or modify the project visibility settings. You should also disable public pipelines and set the artifact’s public flag to false (public: false). This setting ensures artifacts are accessible only to GitLab administrators and project members with at least the Reporter role.
Enabling encrypted state file support would allow for more secure IaC pipelines as shown in the quotation.
Impact assessment
How many customers are impacted?
On gitlab.com around 0.3% of the Terraform state files were not updated since October 2020. As the Terraform state storage feature got added to GitLab in May 2020, these are likely test use cases. We tried to look at the projects that store these state files, there are 386 distinct projects and most of them did not run any pipelines in the past years.
We expect similar or lower number of self-managed installations given the test-use-case assumption behind the 0.3%.
Can we get the same outcome without a breaking-change?
No.
Can the breaking-change wait till the next major release, or the next scheduled upgrade stop?
We can wait with the breaking change until %18.0, but it adds serious technical debt to the related codebase.
What is the alternative for customers to do the same job the change will break?
Any Terraform/OpenTofu run that wants to modify the state would break.
We would not break the users' infrastructure, just their representation in a Terraform state. The user could start a new Terraform state and import their infrastructure there.
How difficult is it for customers to migrate to the alternative? Is there a migration plan?
Affected users would require to take several manual steps, and a solid understanding of Terraform/OpenTofu.
We can not automatically migrate the affected users.
Communication plan
- Add deprecation notice to 16.11 release post
- Add a removal notice to 17.0 release post
Tasks
-
Obtain approval from the VP of Development -
Obtain approval from the VP of Product Management -
Obtain approval from the VP of Customer Support -
Obtain approval from the CPO or CTO -
Notify Support and Customer Success so they can share information with relevant customers.
Approvals
-
VP / Sr Dir of Development
- @timzallmann -
VP of Product Management
- @mflouton -
VP of Customer Support
- @jscarborough -
CPO
-@david
-
CTO
-@sabrinafarmer