Skip to content

Allow `needs:` to refer to a job in the same stage

Problem to solve

With the current implementation of the directed acyclic graph, the user has to help the scheduler a bit by defining stages for jobs, and only passing dependencies between stages. That can get complicated for large DAGs. Right now, users can deal with this by topologically sorting the DAG and greedily adding artificial “stage1”, “stage2”, etc. labels (or even one stage name per job).

Intended users

Individual contributor automators

Further details

Proposal

We will allow to depend on the jobs within the same stage. This reason we have the limitation as it is today is simply because it's simpler to do, as we can avoid the circular references, and very cleanly support dependencies: and needs:. There are also concerns around how the system would work if we start defining needs within current stage, definitely dependencies: is not prepared for that and that will also need to be looked at. Furthermore, we will need to be careful we don't introduce new complex relationship processing that results in performance issues.

Limitations

We don't have a plan to allow needs: to reference items in future stages. This is a more far out there case that could really become difficult to visualize, so we are not addressing it now.

Permissions and Security

Documentation

Testing

What does success look like, and how can we measure that?

Links / references