Upgrade to the new `terraform-plugin-framework`
Apparently the currently in public beta terraform-plugin-framework
will release a first major v1.0.0
Version next month (2022-12).
Background
The GitLab Terraform Provider
is based on the terraform-plugin-sdk v2
. That SDK is used for everything Terraform related in the provider, that is:
- serving the provider plugin (it's a separate process communicating via gRPC with Terraform)
- declaring the provider plugin using a provider Schema
- declaring all the resources and data sources using resource Schemas
- communicating with Terraform using the Plugin Protocol v5
- registering all the CRUD function which implement the actual GitLab business logic
- acceptance testing framework
Problem
The terraform-plugin-sdk
(using the v5 Protocol) has limitations which we've ran into multiple times in our own resources and data sources. Some of the more important ones:
- not easy or even impossible (in most cases) to implemented nested schemas in a resource or data source
- not good enough support for complex types
- some attribute definitions have weird behavior for historical reasons
- unable to detect if a value was set in config, in the plan or in the state
these are just a few, but a good list is here.
Solution: Migration
Now that the terraform-plugin-framework
will get its first stable release we could start the migration. So, what does that mean?
Unfortunately, the compatibility between the SDK and the framework is not that big -> depending on the resource / data source that may require significant effort - especially when we talk about improving of the current shortcomings of those. (To get a good idea of what that means you can look at the difference of the resource implementations of the scaffold provider: terraform-provider-scaffolding
vs. terraform-provider-scaffolding-framework
).
In order to not do one huge big-bang migration we can rely on the following plan to slowly transition towards a provider fully implemented using the new framework:
- Upgrade the current provider to use the Terraform Protocol v6 instead of v5 (the framework uses v6)
- I've already played around with this in !1336 (merged)
- Impact: only Terraform v1.0 > is supported (I need to double check that though)
- Implement a muxing server so that we can use multiple providers: one using the upgraded SDK server and one using the new framework server.
- We can use
terraform-plugin-mux
to implement that, which I already started played around with in !1336 (merged) - Impact: we can slowly migrate resource by resource, data source by data source to the new framework provider
- We can use
- Start implementing new resources / data sources using the new framework provider and migrate already existing SDK resources / data sources to the new framework one-by-one.
- Profit
🎉
How to ensure a smooth migration for our users?
When we migrate a resource or data source from the SDK to the framework we can ensure that it doesn't break the users config or state by implementing an acceptance test as described here.
%v4.0.0 of the provider?
How does that relate to%v4.0.0 is a not yet really well defined milestone for a future release which intentionally breaks backwards-compatibility (we haven't done that for as long as I'm on the project ...). That is required to solve UX issues in the schemas and often not-well defined resource id
s. There are quite some issues already in that milestone which should give an overview of the details.
My proposal would be to start the migration to the new framework independent of the %v4.0.0 release. However, I suggest that when we plan to do %v4.0.0 we also migrate those breaking resources / data sources to the framework. Preferably, we collect all the resources / data sources that benefit from such a breaking change and do them "all at once".
I also think it makes sense to first gain more experience in the new framework in terms of possibilities of the schema definitions and only then start with %v4.0.0 related changes.
Resources
- Documentation: Migrating from SDK
- Documentation: Combining and Translating Providers
- Documentation:
terraform-plugin-framework
- Framework Scaffolding Provider: Source Code
- Forum for questions: HashiCorp Discuss