Run job when cache does not exist

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Proposal

To make pipelines faster, some jobs only run when certain files like composer.lock or package.lock are changed. These jobs install dependencies the and put them into a cache.

When Gitlab runner caches expire, jobs that depend on those caches fail.

To prevent this situation, it should be possible to let jobs run when caches do not exist.

Example 

+----------------------+    +-----------------+
| install-dependencies | -> | build-container |
|    (fills cache)     |    |   (uses cache)  |
+----------------------+    +-----------------+

Job install-dependencies is only run when composer.lock or package.lock changes. It runs composer install and npm install and stores the installed dependencies in a cache.

Job build-container loads this cache and builds a docker container. When the cache expired, the application container will miss the dependencies, leading to a incomplete container with a broken app.

The problem would be solved when install-dependencies would always be run when the cache does not exist.

Example .gitlab-ci.yml 

stages:
  - dependencies
  - docker

install-dependencies:
  stage: dependencies
  rules:
    - if: $CI_COMMIT_BRANCH
      changes:
        - composer.lock
        - package.lock
  cache:
    - key: composer-npm-$CI_COMMIT_REF_SLUG
      paths:
        - ./vendor/
        - ./node_modules/
      policy: push-pull
  script:
    - composer install
    - npm install

build-container:
  stage: docker
  cache:
    - key: composer-npm-$CI_COMMIT_REF_SLUG
      paths:
        - ./vendor/
        - ./node_modules/
      policy: pull
  script:
    - docker build .

Suggested "rules" extension 

install-dependencies:
  rules:
    - if: $CI_COMMIT_BRANCH
      changes:
        - composer.lock
        - package.lock
    - if: $CI_COMMIT_BRANCH
      cache-missing: composer-npm-$CI_COMMIT_REF_SLUG

This would run the "install-dependencies" job whenever the cache does not exist.

Related: #244932

Edited by 🤖 GitLab Bot 🤖