Skip to content

Cache issues when module versions are updated

Tofu jobs using this component will fail with "Error: Module version requirements have changed" when a module dependency version is changed until the runner caches are cleared for the project.

The implicit tofu init enabled by default doesn't handle the module version bump, and there is no way to invalidate the project runner caches when a module version changes, since that information is not recorded in the lockfile so use_lockfile_for_cache_key does not work.

As an example, here's where I am experiencing this issue. The GitLab CI YAML looks like:

workflow:
  rules:
    - if: $CI_MERGE_REQUEST_ID
      variables:
        ASSUMED_ROLE_NAME: "${CI_ROLE_BASE_NAME}-readonly"
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      variables:
        ASSUMED_ROLE_NAME: "${CI_ROLE_BASE_NAME}-admin"
    - if: $CI_COMMIT_TAG
      variables:
        ASSUMED_ROLE_NAME: "${CI_ROLE_BASE_NAME}-admin"

stages:
  - lint
  - validate
  - build_artifacts
  - build
  - test
  - deploy
  - cleanup

variables:
  OPENTOFU_VERSION: "1.10.6"
  GITLAB_TOFU_INIT_FLAGS: "-lockfile=readonly" # ensure use of locked dependencies
  AWS_ACCOUNT_ID: "12345"
  AWS_REGION: "eu-central-1"
  CI_ROLE_BASE_NAME: "gitlab_ci_role"
  TF_VAR_image_tag: $CI_COMMIT_SHORT_SHA

include:
  - component: gitlab.com/components/opentofu/job-templates@3.5
    inputs:
      opentofu_version: $OPENTOFU_VERSION
      auto_define_backend: true
      auto_encryption: true
      auto_encryption_passphrase: $TOFU_STATE_ENCRYPTION_PASSPHRASE
      enable_id_tokens: true
      id_tokens_setup_script: .gitlab/ci/setup-id-tokens.sh
      post_mr_plan_comment: true


.gitlab-tofu:id_tokens:
  id_tokens:
    GITLAB_OIDC_TOKEN:
      aud: https://gitlab.com

# monorepo with several apps, simplified here
Tofu Validate:
  extends: [.opentofu:validate]
  parallel:
    matrix:
      - GITLAB_TOFU_ROOT_DIR: "apps/example/deploy"
        GITLAB_TOFU_STATE_NAME: "example"
  rules:
    - if: $GITLAB_TOFU_STATE_NAME == "example"
      changes:
        - apps/example/deploy/**/*

# other jobs omitted but they should be irrelevant to the validate job failing

apps/example/deploy/main.tf looks like:

module "example_scheduler" {
  source  = "gitlab.com/org/scheduled-ecs-task/aws"
  version = "1.0.0"
  ...
}

The job output on a MR pipeline where I bumped version from 0.1.0 to 1.0.0 looks like:

$ test -f ".gitlab/ci/setup-id-tokens.sh" && . .gitlab/ci/setup-id-tokens.sh
{
    "UserId": "...",
    "Account": "12345",
    "Arn": "arn:aws:sts::12345:assumed-role/role-readonly/GitLabRunner-123-456"
}
$ gitlab-tofu validate
gitlab-tofu: automatically defining the HTTP backend in __gitlab-opentofu-backend.tf
Error refreshing state: Unsupported state file format: This state file is encrypted and can not be read without an encryption configuration

│ Error: Module version requirements have changed

│   on main.tf line 42, in module "example_scheduler":
│   42:   source  = "gitlab.com/org/scheduled-ecs-task/aws"

│ The version requirements have changed since this module was installed and
│ the installed version (0.1.0) is no longer acceptable. Run "tofu init" to
│ install all modules required by this configuration.

Cleaning up project directory and file based variables 00:01
ERROR: Job failed: exit code 1

After clearing cache:

$ gitlab-tofu validate
gitlab-tofu: automatically defining the HTTP backend in __gitlab-opentofu-backend.tf
Initializing modules...
Downloading gitlab.com/org/scheduled-ecs-task/aws 1.0.0 for example_scheduler...
- example_scheduler in .terraform/modules/example_scheduler
Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Installing hashicorp/aws v6.14.1...
- Installed hashicorp/aws v6.14.1 (signed, key ID 0C0AF313E5FD9F80)
Providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://opentofu.org/docs/cli/plugins/signing/
OpenTofu has been successfully initialized!
Success! The configuration is valid.
Edited by Genevieve Mendoza