Pipeline trigger documentation does not adequately explain how different ways of triggering work

Problem to solve

Developing a multiproject CI pipeline with the use of triggers is both awkward and difficult to develop without thorough insight into how GitLab CI works. This creates a frustrating experience that does not match with the documentation and forces end users to need to play around far too long to create the pipeline they desire. An end user can create a pipeline config in roughly three different ways, all of which, one would think would behave the same, but instead, all behave very differently.

Intended users

Further details

Let's first review what a multiproject pipeline CI configuration using triggers is supposed to accomplish:

graph LR
    subgraph projectA
    a1
    end
    subgraph projectB
    b1
    end

    a1[trigger] --> b1[triggered]

Very simple, projectA contains a pipeline with a job called trigger who's sole purpose is to reach out to projectB to create a pipeline containing a job called triggered. The important detail that we need to know in this experiment is that we only care about jobs that are triggered. In all examples below, we utilize the following example job:

triggered:
  stage: triggered
  script:
    - echo "$CI_PIPELINE_SOURCE"
    - echo "$CI_PIPELINE_TRIGGERED"
  only:
    refs:
      - triggers

They key item to note is the use of only.refs["triggers"].

Unfortunately, there are at least 3 ways to create this using our documentation alone. And each of the three ways operates differently. Let's dive into each one. Using our own documentation as a guide here.

Trigger Token

Documentation: https://docs.gitlab.com/ee/ci/triggers/#trigger-token

For this we must create a special Trigger Token that is project specific. The quickest and easiest way to test this is to create the following CI configuration:

trigger_token:
  stage: control
  script:
    - curl -i --request POST --form "token=$TRIGGER_TOKEN" --form ref=master https://gitlab.com/api/v4/projects/10378862/trigger/pipeline
  except:
    refs:
      - triggers

Using this method appears to be the most workable, but less featureful in terms of viewing the pipelines. The above example will trigger a pipeline:

The pluses here, are that we created a pipeline with only the job we cared about. The downsides:

  • This also requires one to setup a special Trigger Token, and configure CI to utilize it, presumably (as I did) via a CI Variable that is masked. This extra setup is automagically solved using CI_JOB_TOKEN, discussed in the second example.
  • We can't view this in projectA's pipeline. This is solved using the two below solutions.

Special item to note, the triggered job outputs this onto the console:

 $ echo "$CI_PIPELINE_SOURCE"
 trigger
 $ echo "$CI_PIPELINE_TRIGGERED"
 true

Which is what I would expect from ANY triggered pipeline.

CI_JOB_TOKEN

Documentation: https://docs.gitlab.com/ee/ci/triggers/#ci-job-token

We get a free variable, CI_JOB_TOKEN, for which we can trigger downstream pipelines in other projects. This is nifty as we don't need to stand up a dedicated token. Using the example here:

job_token:
  stage: control
  script:
    - curl -i --request POST --form "token=$CI_JOB_TOKEN" --form ref=master https://gitlab.com/api/v4/projects/10378862/trigger/pipeline
  except:
    refs:
      - triggers

While it does trigger a pipeline in the downstream project as desired, the pipeline creates all jobs regardless of the fact that the are triggered. Remember what our job definition looks like for the triggered job noted above. Now looking at our examples:

The downsides to this are that the use of except.refs["triggers"] is useless, so jobs that are not intended to be created are indeed part of the pipeline. Oddly, our triggered job does not even show up in the pipeline, and this is the specific job we care about. And as one can see in the examples I've provided, I now accidentally led GitLab CI into an infinite loop. If left unhandled, this could escalate quickly. We do know that pipelines are triggered using a different mechanism due to this single line in our docs:

Pipelines triggered that way also expose a special variable: CI_PIPELINE_SOURCE=pipeline

I disagree with this implementation. We called a trigger the same way as our first example, just swapping the authentication method. So why does this change the behavior of the trigger?

GitLab CI Trigger

Documentation: https://docs.gitlab.com/ee/ci/yaml/#trigger

And here's our example:

trigger_ci_config:
  stage: control
  trigger:
    project: skarbek/test0
  except:
    refs:
      - triggers

This doesn't even work. A trigger here does not abide by the only/except ruleset. The reason and recommended documentation are being added in issue: #212219 (closed)

Proposal

I feel like the use of trigger and pipeline have become overloaded terms. In the above examples, all of these are some sort of trigger onto a pipeline. They either work, do not work, or when they do work, they build the pipeline incorrectly.

Due to the inconsistent methods for possibilities of configuring a pipeline, this makes the first time user frustrated. All of the above is solvable in some way, which I will not go into solutions on purpose, but all of the above methods SHOULD be consolidated in a better fashion. They should operate the same way allowing a user to swap these configurations for newer capabilities with ease. These are all the same from a configuration standpoint, but deep inside of GitLab CI, they end up returning differing capabilities. None of this is well documented. Also the API endpoint /api/v4/projects/:id/trigger/pipeline is not documented either.

What does success look like, and how can we measure that?

Triggering a pipeline implements the same behavior no matter the method utilized to perform the trigger. All three examples above should be interchangeable. They, however, are not so these should each have their own documentation and guidance to provide a better outline for how an end user is to better decide their approach to building their massive CI pipeline.

What is the type of buyer?

Core/Free

Is this a cross-stage feature?

Links / references

Edited by Jason Yavorsky