Skip to content

feat: speedup Terraform steps by validating root modules only

Andrew Newdigate requested to merge tf-root-modules-only into main

Part of https://gitlab.com/gitlab-com/gl-infra/gitlab-dedicated/team/-/issues/3157

This reduces the time taken to terraform validate a repository by skipping all non-root modules.

cc @mhuseinbasic

For GitLab Dedicated: this has brought the Terraform validate step down from 8:24 to 1:59.

It also allows us to remove "fake" provider blocks in child modules, which reduces the warning noise of Terraform substantially. cc @marin

BREAKING CHANGE: from this point forward, only root modules are validated and have tflint run on them.

Other modules are validated by being called from a root module. Any unused modules will not be validated. Root modules are signalled by the fact that they have a .terraform.lock.hcl file in the same directory.

Determining which Terraform Modules to Run terraform validate on...

terraform validate will recursively valid all child modules for a given root module.

This means that there is no need to validate the child modules if the root module has been validated.

Unfortunately there is no canonical way to determine if a module is a root module in Terraform or not, so that the task can selectively run on root modules only.

As a proxy, this task uses the presence of .terraform.lock.hcl to signal that a module is a root module.

This has some implications for users of this task:

  1. Be sure to check the .terraform.lock.hcl files in for your root modules.
  2. Only add .terraform.lock.hcl for your root modules. If a module is not a root module, but has a checked in .terraform.lock.hcl file, it's recommended that you remove it from git.
  3. Any module that exists in the repo, but does not have a .terraform.lock.hcl file and is not connected to the root module, either directly or indirectly, will not be linted.

Running terraform validate on root modules only is much more efficient and results in a faster builder, for relatively little cost.

Determining which Terraform Modules to Run tflint on...

This task will run tflint with the --module option. This will recurse through all child modules to ensure that the tflint rules also apply to them.

Unfortunately there is no canonical way to determine if a module is a root module in Terraform or not, so that the task can selectively run on root modules only.

As a proxy, this task uses the presence of .terraform.lock.hcl to signal that a module is a root module.

This has some implications for users of this task:

  1. Be sure to check the .terraform.lock.hcl files in for your root modules.
  2. Only add .terraform.lock.hcl for your root modules. If a module is not a root module, but has a checked in .terraform.lock.hcl file, it's recommended that you remove it from git.
  3. Any module that exists in the repo, but does not have a .terraform.lock.hcl file and is not connected to the root module, either directly or indirectly, will not be linted.

Running tflint on root modules only is much more efficient and results in a faster builder, for relatively little cost.

Edited by Andrew Newdigate

Merge request reports