Allow `needs:` to refer to a job in the same stage
Problem to solve
With the current implementation of the directed acyclic graph, the user has to help the scheduler a bit by defining stages for jobs, and only passing dependencies between stages. That can get complicated for large DAGs. Right now, users can deal with this by topologically sorting the DAG and greedily adding artificial “stage1”, “stage2”, etc. labels (or even one stage name per job).
Intended users
Individual contributor automators
Further details
Proposal
We will allow to depend on the jobs within the same stage. This reason we have the limitation as it is today is simply because it's simpler to do, as we can avoid the circular references, and very cleanly support dependencies:
and needs:
. There are also concerns around how the system would work if we start defining needs within current stage, definitely dependencies:
is not prepared for that and that will also need to be looked at. Furthermore, we will need to be careful we don't introduce new complex relationship processing that results in performance issues.
Limitations
We don't have a plan to allow needs:
to reference items in future stages. This is a more far out there case that could really become difficult to visualize, so we are not addressing it now.