GitLab CI template for Helm

This project implements a generic GitLab CI template for Helm.

Usage

In order to include this template in your project, add the following to your gitlab-ci.yml:

include:
  - project: 'to-be-continuous/helm'
    ref: '4.0.1'
    file: '/templates/gitlab-ci-helm.yml'

Understand

This chapter introduces key notions and principle to understand how this template works.

Managed deployment environments

This template implements continuous delivery/continuous deployment based on Helm for projects hosted on Kubernetes platforms.

It provides several features, usable in different modes (by configuration).

Review environments

The template supports review environments: those are dynamic and ephemeral environments to deploy your ongoing developments (a.k.a. feature or topic branches).

When enabled, it deploys the result from upstream build stages to a dedicated and temporary environment. It is only active for non-production, non-integration branches.

It is a strict equivalent of GitLab's Review Apps feature.

It also comes with a cleanup job (accessible either from the environments page, or from the pipeline view).

Integration environment

If you're using a Git Workflow with an integration branch (such as Gitflow), the template supports an integration environment.

When enabled, it deploys the result from upstream build stages to a dedicated environment. It is only active for your integration branch (develop by default).

Production environments

Lastly, the template supports 2 environments associated to your production branch (master by default):

• a staging environment (an iso-prod environment meant for testing and validation purpose),
• the production environment.

You're free to enable whichever or both, and you can also choose your deployment-to-production policy:

• continuous deployment: automatic deployment to production (when the upstream pipeline is successful),
• continuous delivery: deployment to production can be triggered manually (when the upstream pipeline is successful).

Deployment context variables

In order to manage the various deployment environments, this template provides a couple of dynamic variables that you might use in your hook scripts and Helm charts (as values):

Generated environment name

The ${environment_name} variable is generated to designate each deployment environment with a unique and meaningful application name. By construction, it is suitable for inclusion in DNS, URLs, Kubernetes labels... It is built from:

• the application base name (defaults to $CI_PROJECT_NAME but can be overridden globally and/or per deployment environment - see configuration variables)
• GitLab predefined $CI_ENVIRONMENT_SLUG variable (sluggified name, truncated to 24 characters)

The ${environment_name} variable is then evaluated as:

• <app base name> for the production environment
• <app base name>-$CI_ENVIRONMENT_SLUG for all other deployment environments
• 💡 ${environment_name} can also be overriden per environment with the appropriate configuration variable

Examples (with an application's base name myapp):

Deployment and cleanup scripts

The Helm template requires you to provide a Helm chart (either in the project or located in an external repository) to deploy and delete the application.

The environment deployment is processed as follows:

1. optionally executes the helm-pre-deploy.sh script in your project to perform specific environment pre-initialization (for e.g. create required services),
2. helm upgrade the chart with the configured parameters, using $environment_name as release name,
3. optionally executes the helm-post-deploy.sh script in your project to perform specific environment post-initialization stuff,

The environment deletion is processed as follows:

1. optionally executes the helm-pre-delete.sh script in your project to perform specific environment pre-cleanup stuff,
2. helm uninstall, using $environment_name as release name,
3. optionally executes the helm-post-delete.sh script in your project to perform specific environment post-cleanup (for e.g. delete bound services).

⚠️ each of the above hook scripts needs to be executable, you can add flag execution with: git update-index --chmod=+x helm-pre-cleanup.sh

Environments URL management

The Helm template supports two ways of providing your environments url:

• a static way: when the environments url can be determined in advance, probably because you're exposing your routes through a DNS you manage,
• a dynamic way: when the url cannot be known before the deployment job is executed.

The static way can be implemented simply by setting the appropriate configuration variable(s) depending on the environment (see environments configuration chapters):

• $HELM_ENVIRONMENT_URL to define a default url pattern for all your envs,
• $HELM_REVIEW_ENVIRONMENT_URL, $HELM_INTEG_ENVIRONMENT_URL, $HELM_STAGING_ENVIRONMENT_URL and $HELM_PROD_ENVIRONMENT_URL to override the default.

ℹ️ Each of those variables support a late variable expansion mechanism with the %{somevar} syntax, allowing you to use any dynamically evaluated variables such as ${environment_name}.

Example:

variables:
  HELM_BASE_APP_NAME: "wonderapp"
  # global url for all environments
  HELM_ENVIRONMENT_URL: "https://%{environment_name}.nonprod.acme.domain"
  # override for prod (late expansion of $HELM_BASE_APP_NAME not needed here)
  HELM_PROD_ENVIRONMENT_URL: "https://$HELM_BASE_APP_NAME.acme.domain"
  # override for review (using separate resource paths)
  HELM_REVIEW_ENVIRONMENT_URL: "https://wonderapp-review.nonprod.acme.domain/%{environment_name}"

To implement the dynamic way, your deployment script shall simply generate a environment_url.txt file in the working directory, containing only the dynamically generated url. When detected by the template, it will use it as the newly deployed environment url.

Deployment output variables

Each deployment job produces output variables that are propagated to downstream jobs (using dotenv artifacts):

• $environment_type: set to the type of environment (review, integration, staging or production),
• $environment_name: the application name (see below),
• $environment_url: set to the environment URL (whether determined statically or dynamically).

Those variables may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).

Working with repositories & OCI-based registries

The Helm template supports indifferently the use of chart repositories and OCI-based registries (requires Helm 3 or above).

Those can be used both for pulling and/or pushing charts.

Configuring pull repositories

The pulling repositories/registries can be configured with the $HELM_REPOS. The value is expected as a (whitespace-separated) list of repo_name@repo_url (defaults to stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami).

⚠️ When using OCI-based registries, simply prefix the url with oci://.

The Helm template also supports user/password authentication for each, simply by defining HELM_REPO_<NAME>_USER and HELM_REPO_<NAME>_PASSWORD (as project or group secret variables).

⚠️ The <NAME> part is the repo_name transformed in SCREAMING_SNAKE_CASE (uppercase words separated by underscores).

Example: declare the GitLab chart repository from another GitLab project

variables:
  HELM_REPOS: "stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami other-proj@${CI_API_V4_URL}/projects/1234/packages/helm/release"
  HELM_REPO_OTHER_PROJ_USER: "gitlab-token"
  # HELM_REPO_OTHER_PROJ_PASSWORD set as a project secret variables

Configuring the push repository

All configuration parameters are extensively documented in the helm-publish job chapter.

Configuration reference

Secrets management

Here are some advices about your secrets (variables marked with a 🔒):

1. Manage them as project or group CI/CD variables:
   • masked to prevent them from being inadvertently displayed in your job logs,
   • protected if you want to secure some secrets you don't want everyone in the project to have access to (for instance production secrets).
2. In case a secret contains characters that prevent it from being masked, simply define its value as the Base64 encoded value prefixed with @b64@: it will then be possible to mask it and the template will automatically decode it prior to using it.
3. Don't forget to escape special characters (ex: $ -> $$).

Global configuration

The Helm template uses some global configuration used throughout all jobs.

Review environments configuration

Review environments are dynamic and ephemeral environments to deploy your ongoing developments (a.k.a. feature or topic branches).

They are disabled by default and can be enabled by setting the HELM_REVIEW_ENABLED variable (see below).

Here are variables supported to configure review environments:

Integration environment configuration

The integration environment is the environment associated to your integration branch (develop by default).

It is disabled by default and can be enabled by setting the HELM_INTEG_ENABLED variable (see below).

Here are variables supported to configure the integration environment:

Staging environment configuration

The staging environment is an iso-prod environment meant for testing and validation purpose associated to your production branch (master by default).

It is disabled by default and can be enabled by setting the HELM_STAGING_ENABLED variable (see below).

Here are variables supported to configure the staging environment:

Production environment configuration

The production environment is the final deployment environment associated with your production branch (master by default).

It is disabled by default and can be enabled by setting the HELM_PROD_ENABLED variable (see below).

Here are variables supported to configure the production environment:

helm-lint job

This job examines your chart for possible issues and uses the following variables:

helm-values-*-lint job

These jobs perform a Yaml Lint of your Helm values file and uses the following variables:

helm-*-score job

This job runs Kube-Score on the resources to be created by Helm and uses the following variables:

helm-package job

This job packages the Helm chart. It uses the following variables:

semantic-release integration

If you activate the semantic-release-info job from the semantic-release template, the helm-package job will automatically use the generated next version info for the chart version (--version).

If no next version info is determined by semantic-release, the package will be created, but without versioning info.

Note: You can disable the semantic-release integration described herebefore the HELM_SEMREL_RELEASE_DISABLED variable.

Chart version management

Depending on the branch and the step in the CI/CD pipeline, the chart will be packaged with a different version.

The general version format will be <x.y.z>-<label>:

• <x.y.z>:
  • if semantic-release integration is enabled: uses the version determined by semantic-release,
  • otherwise uses the version from the chart file
• <label>:
  • on the production branch (main or master by default), no trailing label is used
  • on any other branch, $CI_COMMIT_REF_SLUG is used as trailing label
    (ex: review-feature-12 on branch review/feature-12)
  • ⚠️ when HELM_PUBLISH_SNAPSHOT_ENABLED is enabled, the chart is additionally packaged (and published) with a label suffixed with snapshot
    (ex: snapshot on production branch and review-feature-12-snapshot on branch review/feature-12)

helm-publish job

This job publishes the packaged chart to a chart repository or OCI-based registry. It uses the following variables:

Supported publish methods

The Helm publish supports several methods, configurable with the $HELM_PUBLISH_URL variable:

ℹ️ The default configuration will use GitLab's container registry to publish your charts

If you wish to use GitLab's Helm package repository instead, simply override:

variables:
  # use channel 'release' (can be changed)
  HELM_PUBLISH_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/release"

and leave default $HELM_PUBLISH_USER/$HELM_PUBLISH_PASSWORD values.

Custom publish script

If supported methods don't fit your needs, you may provide a helm-publish.sh script (with execution permissions) in your $HELM_SCRIPTS_DIR directory to implement the required publish method.

This script may use the following variables:

• $helm_package: the packaged chart to publish,
• $HELM_PUBLISH_USER, $HELM_PUBLISH_PASSWORD and $HELM_PUBLISH_URL (see above).

helm-test job

This job runs Helm tests. The job definition must contain the helm test hook annotation: helm.sh/hook: test You are welcome to nest your test suite under a tests/ directory like $HELM_CHART_DIR/templates/tests/ for more isolation.

It is disabled by default and can be enabled by setting the HELM_TEST_ENABLED variable (see below).

It uses the following variables:

Variants

Vault variant

This variant allows delegating your secrets management to a Vault server.

Configuration

In order to be able to communicate with the Vault server, the variant requires the additional configuration parameters:

Usage

Then you may retrieve any of your secret(s) from Vault using the following syntax:

@url@http://vault-secrets-provider/api/secrets/{secret_path}?field={field}

With:

Example

include:
  # main template
  - project: 'to-be-continuous/helm'
    ref: '4.0.1'
    file: '/templates/gitlab-ci-helm.yml'
  # Vault variant
  - project: 'to-be-continuous/helm'
    ref: '4.0.1'
    file: '/templates/gitlab-ci-helm-vault.yml'

variables:
    # Secrets managed by Vault
    HELM_DEFAULT_KUBE_CONFIG: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/my-app/helm/noprod?field=kube_config"
    HELM_PROD_KUBE_CONFIG: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/my-app/helm/prod?field=kube_config"
    VAULT_BASE_URL: "https://vault.acme.host/v1"
    # $VAULT_ROLE_ID and $VAULT_SECRET_ID defined as a secret CI/CD variable