Allow CI runner tag "preference"

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Problem to solve

Currently, it is only possible to designate required tags for CI jobs. If no associated runner exists with the given tag, the job will remain in a stuck state. This creates a dependency that the CI author must also know the details of the runner configuration. This is typical for single projects, but not always for CI templates or complex projects.

I commonly require similar CI jobs when working on multiple projects that use similar frameworks. In order to keep my .gitlab-ci.yml as DRY as possible, I often create my own custom CI "templates" which I reuse with the includes keyword. I also create instance-level project templates so that others in my organization or who implement my open-source projects can also utilize my CI templates. However, this creates a separation where I the CI author do not control the runner configuration of the user implementing the CI template. It is often important for my CI templates to include tags for example to separate jobs running in dev vs. prod. However, by including these tags I have now dictated to the user how their runners should be configured, which is not always best.

Intended users

Developers, change managers, release managers

Further details

Use Cases

In my organization I might use 3 separate environments for dev, staging, and prod, and I might use tags to direct certain jobs to certain runners. In a different organization, dev might be one environment and staging and prod might cohabitate in a second environment. A third organization might only use dev and prod with no staging environment at all. I want to be able to support all of these scenarios in an un-opinionated way. For example I might design that dev jobs run on dev tagged runners and staging jobs run on staging tagged runners if they exist and run on prod tagged runners otherwise.

Benefits

By allowing for flexibility between the CI and the runner configuration, more projects might be able to inherit the same code and more templates may be shareable between applications and organizations.

Proposal

Implementation could take a few different forms. I've listed my ideas. Each idea is separate and not dependent on the others:

1. A new advanced definition for tag supporting additional keys like prefer and require

   myjob:
     stage: deploy
     tags:
       prefer:
         - staging
         - production
         - prod
       require:
         - docker

   The order of the prefer block would be important and values would be searched in order. The require block would not care about order. In other words the CI engine would first determine if runner with staging and docker tags are available, then production and `docker, and so on.

2. RegEx matching for tags. For the following example we would want to match runners tagged with staging, prod, or production AND be tagged with docker:

   myjob:
     stage: deploy
     tags:
       - (staging|prod(uction)?)
       - docker

   The complex part of this approach would be setting any type of preference. Looking at the regex it would not be obvious that staging is checked first and prod or production are given a lower preference.

Sidebar: RegEx support for tags would be nice anyway. The first example could have been simplified to:

myjob:
  stage: deploy
  tags:
    prefer:
      - staging
      - /^prod-.*$/
    require:
      - docker

Permissions and Security

No permissions or security impacts.

Documentation

Testing

What does success look like, and how can we measure that?

Acceptance criteria would include the availability of any solution which would allow a user to control the evaluation logic of tags with choices including logical AND, logical OR, and both.

Links / references