Skip to content
Snippets Groups Projects
Verified Commit 9fcafc61 authored by Timo Furrer's avatar Timo Furrer :juggling:
Browse files

Add new `Terraform-Module.gitlab-ci.yml` CI/CD template

This change set introduces a new CI/CD template named
`Terraform-Module.gitlab-ci.yml` which can be used to easily deploy a
Terraform Module to the GitLab Terraform Registry.

Prior to this change the documentation contained a pipeline snippet
which could be used to achieve the same. This is a little cumbersome for
users though and it would be nice if that snippet would just exist as a
template - that's what this MR does ;)

The template comes as a *pipeline* and *job* template which makes it
easy to adapt to more complex use-cases like deploying multiple
Terraform Modules from the same pipeline.

This change set also contains the relevant new template metrics and
specs for the new templates.

Changelog: added
MR: !110493
parent c8098e8c
No related branches found
No related tags found
No related merge requests found
This commit is part of merge request !110493. Comments created here will be created in the context of that merge request.
......@@ -107,6 +107,38 @@ Where `<namespace>` is the [namespace](../../../user/namespace/index.md) of the
## Publish a Terraform module by using CI/CD
> CI/CD template [introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/110493) in GitLab 15.9.
### Use a CI/CD template (recommended)
You can use the [`Terraform-Module.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform-Module.gitlab-ci.yml)
or the advanced [`Terraform/Module-Base.gitlab-ci.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform/Module-Base.gitlab-ci.yml)
CI/CD template to publish a Terraform module to the GitLab Terraform Registry:
```yaml
include:
template: Terraform-Module.gitlab-ci.yml
```
The pipeline contains the following jobs:
- `fmt` - Validate the formatting of the Terraform module.
- `kics-iac-sast` - Test the Terraform module for security issues.
- `deploy` - For tag pipelines only. Deploy the Terraform module to the GitLab Terraform Registry.
#### Pipeline variables
You can configure the pipeline with the following variables:
| Variable | Default | Description |
|----------------------------|----------------------|-------------------------------------------------------------------------------------------------|
| `TERRAFORM_MODULE_DIR` | `${CI_PROJECT_DIR}` | The relative path to the root directory of the Terraform project. |
| `TERRAFORM_MODULE_NAME` | `${CI_PROJECT_NAME}` | The name of your Terraform module. Must not contain any spaces or underscores. |
| `TERRAFORM_MODULE_SYSTEM` | `local` | The system or provider of your Terraform module targets. For example, `local`, `aws`, `google`. |
| `TERRAFORM_MODULE_VERSION` | `${CI_COMMIT_TAG}` | The Terraform module version. You should follow the semantic versioning specification. |
### Deploy manually via CI/CD
To work with Terraform modules in [GitLab CI/CD](../../../ci/index.md), you can use
`CI_JOB_TOKEN` in place of the personal access token in your commands.
......@@ -114,21 +146,21 @@ For example, this job uploads a new module for the `local` [system provider](htt
```yaml
stages:
- upload
- deploy
upload:
stage: upload
stage: deploy
image: curlimages/curl:latest
variables:
TERRAFORM_MODULE_DIR: ${CI_PROJECT_DIR} # The path to your Terraform module
TERRAFORM_MODULE_NAME: ${CI_PROJECT_NAME} # The name of your Terraform module
TERRAFORM_MODULE_SYSTEM: local # The system or provider your Terraform module targets (ex. local, aws, google)
TERRAFORM_MODULE_VERSION: ${CI_COMMIT_TAG} # Tag commits with SemVer for the version of your Terraform module to be published
TERRAFORM_MODULE_DIR: ${CI_PROJECT_DIR} # The relative path to the root directory of the Terraform project.
TERRAFORM_MODULE_NAME: ${CI_PROJECT_NAME} # The name of your Terraform module, must not have any spaces or underscores (will be translated to hyphens).
TERRAFORM_MODULE_SYSTEM: local # The system or provider your Terraform module targets (ex. local, aws, google).
TERRAFORM_MODULE_VERSION: ${CI_COMMIT_TAG} # The version - it's recommended to follow SemVer for Terraform Module Versioning.
script:
- TERRAFORM_MODULE_NAME=$(echo "${TERRAFORM_MODULE_NAME}" | tr " _" -) # module-name must not have spaces or underscores, so translate them to hyphens
- tar -vczf ${TERRAFORM_MODULE_NAME}-${TERRAFORM_MODULE_SYSTEM}-${TERRAFORM_MODULE_VERSION}.tgz -C ${TERRAFORM_MODULE_DIR} --exclude=./.git .
- tar -vczf /tmp/${TERRAFORM_MODULE_NAME}-${TERRAFORM_MODULE_SYSTEM}-${TERRAFORM_MODULE_VERSION}.tgz -C ${TERRAFORM_MODULE_DIR} --exclude=./.git .
- 'curl --fail-with-body --location --header "JOB-TOKEN: ${CI_JOB_TOKEN}"
--upload-file ${TERRAFORM_MODULE_NAME}-${TERRAFORM_MODULE_SYSTEM}-${TERRAFORM_MODULE_VERSION}.tgz
--upload-file /tmp/${TERRAFORM_MODULE_NAME}-${TERRAFORM_MODULE_SYSTEM}-${TERRAFORM_MODULE_VERSION}.tgz
${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/terraform/modules/${TERRAFORM_MODULE_NAME}/${TERRAFORM_MODULE_SYSTEM}/${TERRAFORM_MODULE_VERSION}/file'
rules:
- if: $CI_COMMIT_TAG
......
# To contribute improvements to CI/CD templates, please follow the Development guide at:
# https://docs.gitlab.com/ee/development/cicd/templates.html
# This specific template is located at:
# https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/templates/Terraform-Module.gitlab-ci.yml
include:
- template: Terraform/Module-Base.gitlab-ci.yml # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Terraform/Module-Base.gitlab-ci.yml
- template: Jobs/SAST-IaC.gitlab-ci.yml # https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/ci/templates/Jobs/SAST-IaC.gitlab-ci.yml
stages:
- validate
- test
- deploy
fmt:
extends: .terraform-module:fmt
# See the included job template at `Terraform/Module-Base.gitlab-ci.yml` to learn about supported variables.
deploy:
extends: .terraform-module:deploy
rules:
- if: $CI_COMMIT_TAG
# Terraform/Module-Base
#
# The purpose of this template is to provide flexibility to the user so
# they are able to only include the jobs that they find interesting.
#
# Therefore, this template is not supposed to run any jobs. The idea is to only
# create hidden jobs. See: https://docs.gitlab.com/ee/ci/yaml/#hide-jobs
#
# There is a more opinionated template which we suggest the users to abide,
# which is the lib/gitlab/ci/templates/Terraform-Module.gitlab-ci.yml
# These variables may be overridden by the pipeline including it to control how the Terraform module is being deployed.
variables:
TERRAFORM_MODULE_DIR: ${CI_PROJECT_DIR} # The relative path to the root directory of the Terraform project.
TERRAFORM_MODULE_NAME: ${CI_PROJECT_NAME} # The name of your Terraform module, must not have any spaces or underscores (will be translated to hyphens).
TERRAFORM_MODULE_SYSTEM: local # The system or provider your Terraform module targets (ex. local, aws, google).
TERRAFORM_MODULE_VERSION: ${CI_COMMIT_TAG} # The version - it's recommended to follow SemVer for Terraform Module Versioning.
.terraform-module:fmt:
stage: validate
image:
name: hashicorp/terraform:latest
entrypoint: ['']
script: terraform -chdir="${TERRAFORM_MODULE_DIR}" fmt -check -diff -recursive
allow_failure: true
.terraform-module:deploy:
stage: deploy
image: curlimages/curl:latest
script:
- TERRAFORM_MODULE_NAME=$(echo "${TERRAFORM_MODULE_NAME}" | tr " _" -) # module-name must not have spaces or underscores, so translate them to hyphens
# Builds the Terraform module artifact: a gzipped tar archive with the contents from `$TERRAFORM_MODULE_DIR` without a `.git` directory.
- tar -vczf /tmp/${TERRAFORM_MODULE_NAME}-${TERRAFORM_MODULE_SYSTEM}-${TERRAFORM_MODULE_VERSION}.tgz -C ${TERRAFORM_MODULE_DIR} --exclude=./.git .
# Uploads the Terraform module artifact to the GitLab Terraform Module Registry, see
# docs/user/packages/terraform_module_registry/index.html#publish-a-terraform-module
- 'curl --fail-with-body --location --header "JOB-TOKEN: ${CI_JOB_TOKEN}"
--upload-file /tmp/${TERRAFORM_MODULE_NAME}-${TERRAFORM_MODULE_SYSTEM}-${TERRAFORM_MODULE_VERSION}.tgz
${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/terraform/modules/${TERRAFORM_MODULE_NAME}/${TERRAFORM_MODULE_SYSTEM}/${TERRAFORM_MODULE_VERSION}/file'
# frozen_string_literal: true
require 'spec_helper'
RSpec.describe 'Terraform/Module-Base.gitlab-ci.yml', feature_category: :continuous_integration do
subject(:template) { Gitlab::Template::GitlabCiYmlTemplate.find('Terraform/Module-Base') }
describe 'the created pipeline' do
let(:default_branch) { 'main' }
let(:pipeline_branch) { default_branch }
let_it_be(:project) { create(:project, :custom_repo, files: { 'README.md' => '' }) }
let(:user) { project.first_owner }
let(:service) { Ci::CreatePipelineService.new(project, user, ref: pipeline_branch) }
let(:pipeline) { service.execute(:push).payload }
let(:build_names) { pipeline.builds.pluck(:name) }
before do
stub_ci_pipeline_yaml_file(template.content)
allow(project).to receive(:default_branch).and_return(default_branch)
end
it 'does not create any jobs' do
expect(build_names).to be_empty
end
end
end
# frozen_string_literal: true
require 'spec_helper'
RSpec.describe 'Terraform-Module.gitlab-ci.yml', feature_category: :continuous_integration do
before do
allow(Gitlab::Template::GitlabCiYmlTemplate).to receive(:excluded_patterns).and_return([])
end
subject(:template) { Gitlab::Template::GitlabCiYmlTemplate.find('Terraform-Module') }
shared_examples 'on any branch' do
it 'creates fmt and kics job', :aggregate_failures do
expect(pipeline.errors).to be_empty
expect(build_names).to include('fmt', 'kics-iac-sast')
end
it 'does not create a deploy job', :aggregate_failures do
expect(pipeline.errors).to be_empty
expect(build_names).not_to include('deploy')
end
end
let_it_be(:project) { create(:project, :repository, create_branch: 'patch-1', create_tag: '1.0.0') }
let_it_be(:user) { project.first_owner }
describe 'the created pipeline' do
let(:default_branch) { project.default_branch_or_main }
let(:service) { Ci::CreatePipelineService.new(project, user, ref: pipeline_ref) }
let(:pipeline) { service.execute(:push).payload }
let(:build_names) { pipeline.builds.pluck(:name) }
before do
stub_ci_pipeline_yaml_file(template.content)
allow_next_instance_of(Ci::BuildScheduleWorker) do |instance|
allow(instance).to receive(:perform).and_return(true)
end
allow(project).to receive(:default_branch).and_return(default_branch)
end
context 'when on default branch' do
let(:pipeline_ref) { default_branch }
it_behaves_like 'on any branch'
end
context 'when outside the default branch' do
let(:pipeline_ref) { 'patch-1' }
it_behaves_like 'on any branch'
end
context 'when on tag' do
let(:pipeline_ref) { '1.0.0' }
it 'creates fmt and deploy job', :aggregate_failures do
expect(pipeline.errors).to be_empty
expect(build_names).to include('fmt', 'deploy')
end
end
end
end
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment