gitlab-ci.yml "changes" are ignored on "new branch" first push into gitlab CI

Problem to solve

Consider the following use case:

  • User wishes to run a specific job ONLY if a specific set of files change.
  • EXAMPLE: build a docker image ONLY if Dockerfile changes

So they leverage changes as per: https://docs.gitlab.com/ee/ci/yaml/#onlychangesexceptchanges

Example:

build docker image:
  stage: docker_build
  only:
    refs: ['branches']
    changes:
      - .gitlab-ci_docker_build.yml
      - Dockerfile
      - Pipfile
      - Pipfile.lock
  except: ['master']
  script:
    - >> do the docker build stuff <<

To explain the example:

  • .gitlab-ci_docker_build.yml is its own file (it is included by the main .gitlab-ci.yml, I contain a VAR that defines some docker build tags required, so if the user bumps the version, we'd like to rebuild the docker image
  • Dockerfile obviously if the docker build script itself changes, rebuild
  • Pipfile(.lock) for Python projects, if the application project dependencies change, rebuild the docker with all the new/updated packages

The expectation here is:

  1. If a developer pushes application code changes, DO NOT rebuild the docker image. (these are MOST of the commit/push/MR scenarios, so we don't want to needlessly build a docker image over-and-over ;o)
  2. If a developer pushes application package requirement changes (Pipfile(.lock)), DO rebuild the docker image.
  3. If a developer changes how docker is built Dockerfile or bumps the docker version in .gitlab-ci_docker_build.yml, DO rebuild the docker image.

As per https://docs.gitlab.com/ee/ci/yaml/#using-changes-with-new-branches-and-tags, it clearly states: Using changes with new branches and tags

When pushing a new branch or a new tag to GitLab, the policy always evaluates to true and GitLab will create a job. This feature is not connected with merge requests yet and, because GitLab is creating pipelines before a user can create a merge request, it is unknown what the target branch is at this point.

The annoyance here is this workflow:

  1. On a branch, developer makes application changes that SHOULD NOT result in docker rebuild.
  2. They edit/commit/etc locally, then PUSH
  3. Pipeline runs and UNDESIRABLY runs the docker build jobs (because of the above caveat) - this is a "new branch", and so all jobs are run regardless of the changes directive.
  4. Subsequent commits/pushes, and merge to master are all handled as expected. (my example above excludes master as there is a different job that controls a docker build on master)

Intended users

Developers & DevOps

Proposal

Either: A) consider if we can control/fix/prevent this "new branch" aspect B) add configuration that allows changes to compare to its own reference

Permissions and Security

TBD

Documentation

TBD

Testing

Would impact current behaviour/expectations of changes (there may be counter-example use cases where this behaviour is DESIRED).

What does success look like, and how can we measure that?

Workflow described above flows without UNDESIRED item. (that is, there is a way for "changes" to actually work on "new branch" initial push)

To repeat the workflow for EXPECTED/DESIRED:

  1. On a branch, developer makes application changes that SHOULD NOT result in docker rebuild.
  2. They edit/commit/etc locally, then PUSH
  3. Pipeline runs as usual (but docker builds DO NOT run)
  4. Subsequent commits/pushes, and merge to master are all handled as expected.

Compare with:

  1. On a branch, developer makes <CHANGE_THAT_SHOULD_TRIGGER_DOCKER_BUILD> (as per above list)
  2. They edit/commit/etc locally, then PUSH
  3. Pipeline runs as usual and docker builds DO run
  4. Subsequent commits/pushes, and merge to master are all handled as expected.

What is the type of buyer?

TBD

Links / references