CI definition using environment variables for several different types of data that comes from multiple places with lots of overlays

Problem

There is a confusion created by the CI definition using environment variables for several different types of data that comes from multiple places with lots of overlays. And then certain ones are not available at different phases, which are all obscured from the user.

Sources:

• built into gitlab (Regular)
• built into gitlab (deploy jobs)
• ci definition file in-job
• ci definition file whole pipeline
• ci definition included jobs
• parent group settings
• project settings general
• project settings scoped to environments
• project settings scoped to protected runners
• job outputs from same pipeline
• job outputs from upstream pipelines

There are probably more.

Some of those are available at various points in the flow new pipeline launches:

• pipeline creation (new commit, manual run, schedule)
• job resolution on server (uses when/rules/only/except etc)
• runner retrieval (uses tags)
• Execution of jobs (additional items for “deploy” jobs)

Proposal

I think an approach to help resolve this is to help users form a more valid mental model of what is happening. Replacing some with a different templating mechanism like jinja {{}} might help. I’m not sure, but I hear a lot of people running into this area of confusion.

Issue was originally raised by @mterhar