CI: Allow extends to accept arguments (parameters) and perform inline substitution at parsing time

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Problem to solve

Generally, the big problem to solve is code duplication in CI YAML. Example:

.deploy:
  stage: helm-deploy
  <<: *helm-production-variables
  rules:
  - if: $CI_MERGE_REQUEST_ID # https://docs.gitlab.com/ee/ci/yaml/#exclude-jobs-with-rules-from-certain-pipelines
    when: never
  - if: '$CI_COMMIT_REF_NAME == "master"'
  - when: manual
  script:
  - ./bin/gitlabci-helm-install.sh ${CI_JOB_NAME:7} # deploy-blah => blah

.delete:
  stage: helm-delete
  <<: *helm-production-variables
  when: manual
  script:
  - helm delete --purge "$CI_ENVIRONMENT_NAME" || true
  - kubectl delete jobs -l release="$CI_ENVIRONMENT_NAME" || true

# Application no 1

deploy-app-1:
  extends:
  - .deploy
  needs:
  - job: helm-app-1 # HERE 
    artifacts: true
  environment:
    name: $PRODUCTION_ENV_PREFIX-$CI_COMMIT_REF_SLUG-app-1 # HERE
    url: https://${CI_ENVIRONMENT_NAME}.$PRODUCTION_DOMAIN/
    on_stop: delete-app-1 # HERE

delete-app-1:
  extends: .delete
  needs: .deploy-app-1 # HERE
  environment:
    name: $PRODUCTION_ENV_PREFIX-$CI_COMMIT_REF_SLUG-app-1 # HERE
    action: stop

# Application no 2

# ... same as application no 1, instead of app-1, we will have app-2 at "# HERE"

(The above example is a copy-pasta from our actual CI YAML, and we have ~15 apps that use .deploy and .delete.)

Proposal

Allow extends to accept arguments (parameters) and perform inline substitution of {{variables}} at parsing time. (The moment CI YAML parser processes extends)

.deploy:
  stage: helm-deploy
  <<: *helm-production-variables
  needs:
  - job: helm-{{application}} # HERE 
    artifacts: true
  environment:
    name: $PRODUCTION_ENV_PREFIX-$CI_COMMIT_REF_SLUG-{{application}} # HERE
    url: https://${CI_ENVIRONMENT_NAME}.$PRODUCTION_DOMAIN/
    on_stop: delete-app-1 # HERE
  rules:
  - if: $CI_MERGE_REQUEST_ID # https://docs.gitlab.com/ee/ci/yaml/#exclude-jobs-with-rules-from-certain-pipelines
    when: never
  - if: '$CI_COMMIT_REF_NAME == "master"'
  - when: manual
  script:
  - ./bin/gitlabci-helm-install.sh {{application}}

.delete:
  stage: helm-delete
  <<: *helm-production-variables
  needs:
  - deploy-{{application}}
  when: manual
  script:
  - helm delete --purge "$CI_ENVIRONMENT_NAME" || true
  - kubectl delete jobs -l release="$CI_ENVIRONMENT_NAME" || true
  environment:
    name: $PRODUCTION_ENV_PREFIX-$CI_COMMIT_REF_SLUG-{{application}}
    action: stop

# Application no 1

deploy-app-1:
  extends:
  - section: .deploy
    params:
      application: app-1

delete-app-1:
  extends:
  - section: .delete
    params:
      application: app-1

Intended users

Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/

Links / references

~"devops::release" devopspackage

CC @ayufan @grzesiek @jyavorska @markglenfletcher, and this time also @markpundsack.

I understand this would be a big decision to make. And there's many different approaches we could take, and the most notable one is #22060 (closed) (not sure why it's marked as duplicate of #16094 (closed)). However, I thought keeping things enclosed in CI YAML spec, with no outside processes generating the YAML, has its advantages too.

Edited by 🤖 GitLab Bot 🤖