Feature request: Gitlab CI needs directive that can reference a job which extends a hidden job, by referencing the hidden job itself

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Proposal

Hello, we leverage three features pretty extensively in our pipeline manifests but they don't play well together in some circumstances.

We've created a batch of CI job template manifests in a repo which contain directives common to all of our project level jobs. Whereas in our project repos, we include these templates, the templates have several hidden jobs in them which we extend from the project pipelines with the unique directives required for each project.

This works well, except for the case when we want to use the needs: directive.

In this case, we are required to put the needs: directive into the .gitlab-ci.yml file directly instead of being able to place it in our templates because we tend to name the extend-ing job as a combination of the service name and the hidden job's name.

For example, we might have a hidden job in the pipeline templates named .scan, and in the projects, we would have job definition like this bare bones example.

In the template, we would have jobs such as:

.build:
  image: someimage:latest

.scan:
  image: someotherimage:latest

And in .gitlab-ci.yml for each project, we would have an include for the above template and use the below code to extend the template jobs:

build project1:
  extends: .build

scan project1:
  extends: .scan

We may want to add needs: to the above scan project1 job, so right now we do this by adding the additional needs: directive to .gitlab-ci.yml itself in each project:

build project1:
  extends: .build

scan project1:
  extends: .scan
  needs:
  - job: build project1

However, we would like to add this in the templates and have it apply to all projects which extend the .scan hidden job, regardless of the build job's actual name. So for example, in the templates repo, we would like to do this:

.build:
  image: someimage:latest

.scan:
  image:someotherimage:latest
  needs:
  - job: .build

And then in .gitlab-ci.yml we would not need to make any changes:

build project1:
  extends: .build

scan project1:
  extends: .scan

The template using needs:job is the issue, as the hidden job doesn't exist in the pipeline, and so the pipeline fails to be created properly with a YAML error.

So what this ticket is requesting is to add a new needs: directive, something like needs:extendedjob or needs:hiddenjob or some other name your team can come up with, which references the hidden job itself, and can detect which job actually extends that job, and apply the correct job execution order, and pull in the right artifacts.

I realize the last 2 examples above don't demonstrate the request, so I am placing the code for how this might look below.

The template might look like this:

.build:
  image: someimage:latest

.scan:
  image: someotherimage:latest
  needs:
  - extendedjob: .build

The .gitlab-ci.yml would remain unchanged:

build project1:
  extends: .build
 
scan project1:
  extends: .scan

The pipeline would see the new needs:extendedjob directive, and figure out that the build project1 job extends the .build hidden job, and still properly create the pipeline and pull in the correct artifacts, just as if we had included the needs:job directive directly within the .gitlab-ci.yml manifest itself.