GitLab Next

This issue was moved (and resolved) in another project.

Problem to Solve

Some pipelines and/or jobs use unique resources or are in some way destructive to an environment. Being able to limit concurrency for them would allow users control over scenarios where there should only be one deploy at a time for an:

• Environment
• Entire Project
• Job (perhaps due to shared testing infrastructure in a testing lab)

Solution

We will have the syntax lock: semaphore-name which could be applied at a job or pipeline level, and would create a project-level semaphore preventing jobs or pipelines from running which also claim the same semaphore.

• Pipelines/jobs would wait for it to become available, and time out after some point (probably just based on pipeline time out).
• Locked jobs are guaranteed to be executed in the order created by pipeline, so the job would receive a blocked status to indicate that it waits for being unlocked by another pending/running job.

Implicit locking for environments

Because environments are much more often than not the kind of place where you'd want only one deployment to run at once, and always in the correct order, we will include implicit locking wherever environment: is used, using a semaphore with the name of the environment.

1. When environment: is used, it implies lock:, so you don't need to specify lock: and environment:,
2. When environment: is used, you can use lock: some-name to create a lock across all environment deployments,
3. When implict lock is used, you can define lock: nil to disable locking, thus run with full concurrency limit,
4. Implicit lock for the environment comes from the assumption that all deployments are by design not working very well when executed concurrently

Sample Configuration

This example will run only ever one of the project's pipeline's at once. The pipeline itself will run as normal, with all jobs running in parallel in the build stage.

lock: $CI_PROJECT_NAME
# lock: $CI_ENVIRONMENT_NAME for example would give you a way to run one entire pipeline per environment

stages:
  - build

jobA:
  stage: build
  script: 
    - echo HelloA

jobB: 
  stage: build
  script:
    - echo HelloB

This example moves the lock to a job. Multiple pipelines can run simultaneously, but jobA will only ever run one at a time, across all pipelines in the project.

stages:
  - build

jobA:
  lock: jobA
  stage: build
  script: 
    - echo HelloA

jobB: 
  stage: build
  script:
    - echo HelloB

Future Improvements

Different concurrency behaviors

At the moment, all this will do is wait for a semaphore to free up. You could imagine more possibilities:

concurrency:
   parallel: Default current value, job is launch even if an other is in progress
   cancel : Cancel job if is launch in parallel of another
   wait: Wait previous job is finish for launch current
   skip: Skips job, if lock is already acquired