Cache issues when module versions are updated
Tofu jobs using this component will fail with "Error: Module version requirements have changed" when a module dependency version is changed until the runner caches are cleared for the project.
The implicit tofu init enabled by default doesn't handle the module version bump, and there is no way to invalidate the project runner caches when a module version changes, since that information is not recorded in the lockfile so use_lockfile_for_cache_key does not work.
As an example, here's where I am experiencing this issue. The GitLab CI YAML looks like:
workflow:
rules:
- if: $CI_MERGE_REQUEST_ID
variables:
ASSUMED_ROLE_NAME: "${CI_ROLE_BASE_NAME}-readonly"
- if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
variables:
ASSUMED_ROLE_NAME: "${CI_ROLE_BASE_NAME}-admin"
- if: $CI_COMMIT_TAG
variables:
ASSUMED_ROLE_NAME: "${CI_ROLE_BASE_NAME}-admin"
stages:
- lint
- validate
- build_artifacts
- build
- test
- deploy
- cleanup
variables:
OPENTOFU_VERSION: "1.10.6"
GITLAB_TOFU_INIT_FLAGS: "-lockfile=readonly" # ensure use of locked dependencies
AWS_ACCOUNT_ID: "12345"
AWS_REGION: "eu-central-1"
CI_ROLE_BASE_NAME: "gitlab_ci_role"
TF_VAR_image_tag: $CI_COMMIT_SHORT_SHA
include:
- component: gitlab.com/components/opentofu/job-templates@3.5
inputs:
opentofu_version: $OPENTOFU_VERSION
auto_define_backend: true
auto_encryption: true
auto_encryption_passphrase: $TOFU_STATE_ENCRYPTION_PASSPHRASE
enable_id_tokens: true
id_tokens_setup_script: .gitlab/ci/setup-id-tokens.sh
post_mr_plan_comment: true
.gitlab-tofu:id_tokens:
id_tokens:
GITLAB_OIDC_TOKEN:
aud: https://gitlab.com
# monorepo with several apps, simplified here
Tofu Validate:
extends: [.opentofu:validate]
parallel:
matrix:
- GITLAB_TOFU_ROOT_DIR: "apps/example/deploy"
GITLAB_TOFU_STATE_NAME: "example"
rules:
- if: $GITLAB_TOFU_STATE_NAME == "example"
changes:
- apps/example/deploy/**/*
# other jobs omitted but they should be irrelevant to the validate job failing
apps/example/deploy/main.tf looks like:
module "example_scheduler" {
source = "gitlab.com/org/scheduled-ecs-task/aws"
version = "1.0.0"
...
}
The job output on a MR pipeline where I bumped version from 0.1.0 to 1.0.0 looks like:
$ test -f ".gitlab/ci/setup-id-tokens.sh" && . .gitlab/ci/setup-id-tokens.sh
{
"UserId": "...",
"Account": "12345",
"Arn": "arn:aws:sts::12345:assumed-role/role-readonly/GitLabRunner-123-456"
}
$ gitlab-tofu validate
gitlab-tofu: automatically defining the HTTP backend in __gitlab-opentofu-backend.tf
Error refreshing state: Unsupported state file format: This state file is encrypted and can not be read without an encryption configuration
╷
│ Error: Module version requirements have changed
│
│ on main.tf line 42, in module "example_scheduler":
│ 42: source = "gitlab.com/org/scheduled-ecs-task/aws"
│
│ The version requirements have changed since this module was installed and
│ the installed version (0.1.0) is no longer acceptable. Run "tofu init" to
│ install all modules required by this configuration.
╵
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: exit code 1
After clearing cache:
$ gitlab-tofu validate
gitlab-tofu: automatically defining the HTTP backend in __gitlab-opentofu-backend.tf
Initializing modules...
Downloading gitlab.com/org/scheduled-ecs-task/aws 1.0.0 for example_scheduler...
- example_scheduler in .terraform/modules/example_scheduler
Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Installing hashicorp/aws v6.14.1...
- Installed hashicorp/aws v6.14.1 (signed, key ID 0C0AF313E5FD9F80)
Providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://opentofu.org/docs/cli/plugins/signing/
OpenTofu has been successfully initialized!
Success! The configuration is valid.
Edited by Genevieve Mendoza