feat: speedup Terraform steps by validating root modules only
Part of https://gitlab.com/gitlab-com/gl-infra/gitlab-dedicated/team/-/issues/3157
This reduces the time taken to terraform validate a repository by skipping all non-root modules.
For GitLab Dedicated: this has brought the Terraform validate step down from 8:24 to 1:59.
It also allows us to remove "fake" provider blocks in child modules, which reduces the warning noise of Terraform substantially. cc @marin
BREAKING CHANGE: from this point forward, only root modules are validated and have tflint run on them.
Other modules are validated by being called from a root module. Any unused modules will not be validated. Root modules are signalled by the fact that they have
a .terraform.lock.hcl
file in the same directory.
terraform validate
on...
Determining which Terraform Modules to Run terraform validate
will recursively valid all child modules for a given root module.
This means that there is no need to validate the child modules if the root module has been validated.
Unfortunately there is no canonical way to determine if a module is a root module in Terraform or not, so that the task can selectively run on root modules only.
As a proxy, this task uses the presence of .terraform.lock.hcl
to signal that a module is a root module.
This has some implications for users of this task:
- Be sure to check the
.terraform.lock.hcl
files in for your root modules. - Only add
.terraform.lock.hcl
for your root modules. If a module is not a root module, but has a checked in.terraform.lock.hcl
file, it's recommended that you remove it from git. - Any module that exists in the repo,
but does not have a
.terraform.lock.hcl
file and is not connected to the root module, either directly or indirectly, will not be linted.
Running terraform validate
on root modules only is much more efficient and results in a faster builder,
for relatively little cost.
tflint
on...
Determining which Terraform Modules to Run This task will run tflint
with the --module
option.
This will recurse through all child modules to ensure that the tflint
rules also apply to them.
Unfortunately there is no canonical way to determine if a module is a root module in Terraform or not, so that the task can selectively run on root modules only.
As a proxy, this task uses the presence of .terraform.lock.hcl
to signal that a module is a root module.
This has some implications for users of this task:
- Be sure to check the
.terraform.lock.hcl
files in for your root modules. - Only add
.terraform.lock.hcl
for your root modules. If a module is not a root module, but has a checked in.terraform.lock.hcl
file, it's recommended that you remove it from git. - Any module that exists in the repo,
but does not have a
.terraform.lock.hcl
file and is not connected to the root module, either directly or indirectly, will not be linted.
Running tflint
on root modules only is much more efficient and results in a faster builder,
for relatively little cost.