Limit pipeline concurrency using named semaphores
Problem to Solve
Some pipelines and/or jobs use unique resources or are in some way destructive to an environment. Being able to limit concurrency for them would allow users control over scenarios where there should only be one deploy at a time for an:
- Entire Project
- Job (perhaps due to shared testing infrastructure in a testing lab)
We will have the syntax
lock: semaphore-name which could be applied at a job or pipeline level, and would create a project-level semaphore preventing jobs or pipelines from running which also claim the same semaphore.
- Pipelines/jobs would wait for it to become available, and time out after some point (probably just based on pipeline time out).
- Locked jobs are guaranteed to be executed in the order created by pipeline, so the job would receive a
blockedstatus to indicate that it waits for being unlocked by another
Implicit locking for environments
Because environments are much more often than not the kind of place where you'd want only one deployment to run at once, and always in the correct order, we will include implicit locking wherever
environment: is used, using a semaphore with the name of the environment.
environment:is used, it implies
lock:, so you don't need to specify
environment:is used, you can use
lock: some-nameto create a lock across all environment deployments,
- When implict lock is used, you can define
lock: nilto disable locking, thus run with full concurrency limit,
- Implicit lock for the environment comes from the assumption that all deployments are by design not working very well when executed concurrently
This example will run only ever one of the project's pipeline's at once. The pipeline itself will run as normal, with all jobs running in parallel in the build stage.
lock: $CI_PROJECT_NAME # lock: $CI_ENVIRONMENT_NAME for example would give you a way to run one entire pipeline per environment stages: - build jobA: stage: build script: - echo HelloA jobB: stage: build script: - echo HelloB
This example moves the lock to a job. Multiple pipelines can run simultaneously, but
jobA will only ever run one at a time, across all pipelines in the project.
stages: - build jobA: lock: jobA stage: build script: - echo HelloA jobB: stage: build script: - echo HelloB
Different concurrency behaviors
At the moment, all this will do is wait for a semaphore to free up. You could imagine more possibilities:
concurrency: parallel: Default current value, job is launch even if an other is in progress cancel : Cancel job if is launch in parallel of another wait: Wait previous job is finish for launch current skip: Skips job, if lock is already acquired