Come up with design for visualizing DAG/hybrid DAG in pipeline view
Validation track effort in preparation of the build track for issue gitlab#31877 (closed)
Problem Statement
We have implemented the DAG and are currently just displaying it over the top of stages, which doesn't really mean anything and could be confusing. We aren't taking advantage of the DAG structure in any way to render an important view. We need to determine the problem that people need to solve when they are visualizing a DAG.
Some initial discussions have taken place in gitlab#31877 (closed).
- Opportunity canvas: https://docs.google.com/document/d/17a3Jjzt0s69GRsN-JPeCf0rlmz9URyEZ2E9qOIpCbDY/edit
- Discussion guide: https://docs.google.com/document/d/1TYbi--6JyB3JtWnEdbQE3ExR_xJo_fpWLxGwX7vyGwg/edit?usp=sharing
DAG pipelines are currently rendered as follows:
. | |
---|---|
The internal representation of the data structure is something more like the following:
structure example |
---|
As part of discovering the problem here we should also investigate how people are using the pipeline view. There are interesting possibilities for an integrated view rather than having two modes, since a stage flow can be considered a more rigid subset of a DAG. For example, there culd be one pipeline style that supports both DAG and non-DAG pipelines, especially since GitLab's DAGs can be hybrids. We could ignore stages from the platform's perspective, and essentially make them a kind of annotation for a job. In this way we'd have a unified concept.
Reach
This impacts users who have advanced, complex pipelines and are some of the largest most sophisticated users of GitLab pipelines.
3.0 = Significant reach (~25% to ~50%).
Impact
We have provided these teams with a solution that works in terms of execution, but have not given them any tools in order to visualize and understand what's actually happening. The only possibility is to look at the .gitlab-ci.yml
to understand what's going on.
2.0 = High impact
Confidence
We are very confident that this is a problem. We knew when implementing the DAG that we were doing it without any UX. This was the right decision and still a valuable capability, but we knew we needed to come back and solve this problem.
100% = High confidence
Effort
This is a moderate effort requiring new frontend work to display what's actually happening in the UX, and possibly backend changes to deliver the information to the frontend. It's also a challenging problem to solve and will require some thought and careful consideration to come up with a good solution.
- PM .5
- Designer .5
- Frontend Engineer 1
- Backend Engineer .5
Total 2.5 (though this is highly speculative as no specific solution has been identified)
Current outcome
- This issue focuses on creating a feasible DAG visualization which will be shown on a separate tab on the pipeline detail view.
- We'll proceed with design for pure DAG visualization and defer design for hybrid/DAG visualization to a separate issue
- Pure DAG visualization will only include jobs with
needs:
syntax - Pure DAG visualization is not expected to show stages
- While a pure DAG visualization does not provide the full context of a pipeline because stages and non-DAG jobs are not represented, there is still value in starting with this minimal MVC
- Pure DAG visualization will only include jobs with
- We'll need a reference point for DAG visualization for FE to work off from
- The final concept needs to be stress-tested for accessibility
- To stress-test the iteration concept in terms of complexity we'll be able to derive the input based off from
.GitLab-ci.yml
files so it can be rebuilt with spoof scripts
Reference graph for iteration
Discovery required for the next steps discussed at #973 (comment 316324573)
- Handles a pure DAG pipeline only, stage-only based jobs will not be visible
- Follows rules of Scenario 1, discovery 1 (See #973 (comment 307587612))
DAG graph with connections that do not span multiple stages
https://observablehq.com/d/57f55d7a1398d9a0
Example |
---|
DAG graph with connections spanning multiple stages
https://observablehq.com/d/751c24a083851f98
Example | Manually fixed example |
---|---|