Allow `needs:` (DAG) to refer to a job in the same stage
Problem to solve
With the current implementation of the directed acyclic graph, the user has to help the scheduler a bit by defining stages for jobs, and only passing dependencies between stages. That can get complicated for large DAGs. Right now, users can deal with this by topologically sorting the DAG and greedily adding artificial “stage1”, “stage2”, etc. labels (or even one stage name per job).
Individual contributor automators
We will allow to depend on the jobs within the same stage instead of this being prevented by an error.
For now, we are not making stages only a "visualization hint" since they are still part of processing. In the future we are considering making all pipeline processing DAG (just, by default without needs set, it will behave just like a stage-based pipeline). At that point it may make sense to more broadly revisit what stages mean in GitLab CI.
If a job
needs another in the same stage,
dependencies should be respected and it should wait (within the stage) to run until the job it needs is done. Then, fetch its dependencies and run itself.
- Circular references will need to be detected and will still result in an error.
dependencies:will not be updated to support this (at least as part of this issue) separate from within the context of
needssince non-DAG pipelines having dependencies on something in the same stage is undefined.
- We don't yet have a plan to allow
needs:to reference items in future stages. This is a more far out there case that could really become difficult to visualize, so we are not addressing it now.