Document different types of predefined variables

Further details

In the Use variables with include documentation sectionwe currently don't specify all variables that can be used.

That's because in Clarify variable usage with include keyword (!114571 - merged) we ended up changing from "all but x" wording to saying "only y" instead. At the time, it seemed that this was the less complex option as we assumed the result of "all but x" would only be two specific variables. This was based on a misunderstanding.

More specifically, "all" was defined as Pipeline predefined variables – we don't define anywhere what those are. In that MR we assumed it was all CI_PIPELINE_-prefixed variables, but looking at the code we can see that it is significantly more complex:

• Some but not all CI_PIPELINE_* variables
  • comparing to the documented list, these are not included:
    • CI_PIPELINE_ID
    • CI_PIPELINE_TRIGGERED
    • CI_PIPELINE_URL
• All CI_COMMIT_* variables
• All CI_MERGE_REQUEST_* variables
  • CI_MERGE_REQUEST_APPROVED is a bit tricky for me; I don't fully grok how the EE models "extend" the base models – but I'm confident enough that this is true despite my ability to technically prove it
• Some other ones without a shared prefix:
  • CI_DEPLOY_FREEZE
  • CI_KUBERNETES_ACTIVE
  • CI_GITLAB_FIPS_MODE
  • CI_OPEN_MERGE_REQUESTS

Before the MR we explicitly mentioned that CI_PIPELINE_ID, CI_PIPELINE_URL, CI_PIPELINE_IID and CI_PIPELINE_CREATED_AT can't be used in include: – worth noting that both CI_PIPELINE_ID and CI_PIPELINE_URL are not Pipeline predefined variables per the the code-based definition, so they wouldn't even need to be called out as exceptions.

---

Similarly, Project predefined variables are also not limited to just CI_PROJECT_-prefixed variables, see relevant code. I'll wait with a more thorough analysis as above until we've had further confirmation from a dev.

Proposal

I think this complexity makes it clear that we can't define this only within the Use variables with include section itself. We need to specify what Pipeline predefined variables and Project predefined variables are in a more central place.

I'm unsure what the right place and format for doing so is. Should we break up the big list in the Predefined CI/CD variables reference to have sub-headings for this distinction? Should it become another column in the list? (Note that the existing Defined for column does not align with these code-based definitions.)

There's also a larger question if this is (partly) an implementation detail that we shouldn't even expose to customers. But as long as the place in the code is the main deciding factor for where a certain variable can and cannot be used, I'm not sure we can avoid it.

Who can address the issue

As @furkanayhan first noticed this here, I was hoping you could confirm that the way I've defined it via code analysis above is the correct way to look at it – but feel free to defer to grouppipeline security.

@marcel.amirault will be the best person to judge the bigger picture "how do we document predefined variables" question.

Other links/references