Consolidated view of entire pipeline across projects
For any given project, show all upstream and downstream projects in the same pipeline graph, clearly indicating which jobs belong to the pipeline in focus. e.g. put external jobs
Alternatively, show consolidated pipelines in the group or sub-group views.
Note, this is not about showing artifact, image, submodule or other dependencies. It's about what pipelines cause other pipelines to run.
Larger projects, especially those adopting a micro-services architecture, often have a set of interdependent components that form the complete product. When working with these projects, it is important to be able to build, test, and deploy both the individual parts as well as the whole. For example when developers are working on a low level service, it is important to ensure downstream components are not negatively impacted by changes.
With Cross Project Pipelines, GitLab will have first class support for managing and running CI/CD pipelines across projects. Users will be able to easily link a projects CI/CD stages into the full pipeline, as well as view current status.
Links / references
@markpundsack nice! I am thinking in terms of nested pipeline views in which certain builds/jobs or actionable builds/jobs are promoted to be visible in the "nested" pipeline element.
I get this idea from sidefx houdini's node system:
here it nests a subnet of nodes into 1 parent node: https://youtu.be/sPfpikUp26E?t=9m11s
here it has given that parent node controls to control its subnodes from the parent level: https://youtu.be/sPfpikUp26E?t=34m41s
it that the direction you want this to go? just first thing that comes to mind
Nesting could work, but sounds more like #22550 (closed).
In this issue I was thinking of showing one big pipeline. Maybe by expanding and collapsing "nested" pipelines, we'd knock off both issues. I honestly don't know which is more valuable in practice, but I imagine many people coming from Jenkins will expect to see a single view of their entire cross-project pipeline.
ha nice, that's both crazy and nice at the same time. I think if we'd make the pipeline view draggable, it could support such big pipeline graphs (if you look up a big pipeline graph on your mobile it gives you a sense of what I mean)
We should however think of what we want to solve exactly... meeting expectations of users from a different product with a different context... or what does the user need in that view.
say we have multiple upstream projects of which we use data... (I assume we always pick the last stable/passed pipeline data)..don't we want a visual of those pipelines if we don't use the latest HEAD?
@dimitrieh That's a good question. If we support having a pipeline depend on multiple pipelines, it would trigger when any of the upstream projects change, and pull the latest from the unchanged ones. I still wonder if we can ignore that in this simpler issue, and leave that for #22558 (closed).
As I understand it, a dependency indicates that a pipeline cannot start until another succeeds. But that doesn't necessarily mean that the dependent pipeline will automatically start when the other one succeeds, so it's a different behavior from triggers. Shouldn't we represent them differently, then?
@cperessini My assumption is that there will be only one type of dependency that you declare in pipelines, and that will trigger downstream pipelines to run. But I can see the case for wanting an easy way to declare the dependency, but not trigger on it. Again, better suited for #22558 (closed), so moving conversation there.
Thanks @dimitrieh, this looks great. I hadn't thought about displaying the current pipeline in a compressed view, but it makes sense. There's going to be some interesting graph drawing when you have multiple upstream/downstream flows. e.g. what would it look like if Prometheus also had two upstream projects, or if Gitaly had multiple downstream projects? Or even worse, what if some of those projects were the same? e.g. Prometheus depends on GitLab Shell, but not GitLab Build?
If we stick to only graphing the trigger pipeline, then there's always a 1:1 or 1:many relationship, but never a many:many relationship. That keeps the graph drawing so much simpler. And also has a nice property that all the pipelines shown were actually run together. i.e. in #22558 (comment 25260311), GitLab Build's pipeline could have been run several days earlier and you're just grabbing the latest result.
Just like we only show jobs that actually run in the current pipeline view; we hide jobs that are defined but not "run". There's an interesting parallel there.
Some more edge considerations. What would it look if Gitlab Shell had 12 stages to its pipeline? Are users going to be confused by having the same icon used to represent the overall project pipeline status and in the minigraph for that project pipeline? Can we drop the overall status and just show the mini-graph directly? How about going further and just drawing the mini graph directly, without a bounding oval around it? (I know the bounding oval presents a nice place to put the project name and other meta information.) I'd like to see an exploration of an expanded graph directly connecting the mini graphs. I wonder if users would want an even more expanded view of full pipeline graphs, not just the mini graphs. Probably gets really crazy at some point, and might not be the default view, but someone debugging or just trying to understand the full pipeline might want to see the whole thing in all its glory.
If we stick to only graphing the trigger pipeline, then there's always a 1:1 or 1:many relationship, but never a many:many relationship.
I think this actually makes the most sense.. with:
- graph drawing is simpler
- all the pipelines shown are actually run together
But also because the pipeline flow (graphically displayed with lines and nodes) is only important for showing things that are blocking other things.. this gives us a sense of what goes wrong where!
When a certain pipeline has a dependency on artifacts from another project, these are gathered from the last passed pipeline.. this will always be good code.. or at least it always will have passed the pipeline.. so not that interesting. Not to mention that it can be a dependency for multiple upstream and downstream projects at the same time.. complicating the drawing unnecessarily... However! We could show them differently, as it's still worth knowing which projects are imported to make a certain pipeline work and those will always be upstream projects.
bonus, we again don't need the trigger thingy..as the first project.. is the project pipeline that triggered everything.
Can we drop the overall status and just show the mini-graph directly?
Yes i think this makes sense here!
This would result in:
Can we drop the overall status and just show the mini-graph directly? How about going further and just drawing the mini graph directly, without a bounding oval around it?
but someone debugging or just trying to understand the full pipeline might want to see the whole thing in all its glory.
What about a list for showing cross project pipelines? showing projects as stages instead of .. stages?
@dimitrieh First mockup is great. I'm still not sure how it would look if there were more than 5 stages in the minigraph. The +1 stuff seems totally unnecessary and is still subject to all the problems of the full many:many graph. It was really not obvious what they might mean, and even after clicking on them, is still a bit confusing.
For that last mockup, I was thinking more of the "full" pipeline, ovals with job names, not just status circles. One potential advantage of showing the fully expanded pipelines is that it would obviate the need for a separate tab. The Pipelines tab could show the entire cross-project pipeline.
What about a list for showing cross project pipelines? showing projects as stages instead of .. stages?
@dimitrieh Do you mean at the group or top level? Just like I want a global list of pipelines (#18054), a global list of cross-project pipelines is probably useful. But something about having two separate tabs for project pipelines and cross-project pipelines seems wrong. It's a fine stepping stone, but I wonder what the end-game is.
I imagine #22550 (closed) would just augment the existing Pipelines tab. Is there a way to make the consolidate view an augmentation as well? Like clicking on an upstream or downstream project expands the pipeline inline to make the graph fuller, and adds any further upstream/downstream pipelines to the view. Maybe there's a button to expand them all at once. Or maybe having them always be expanded makes sense. I think the recent mockups in this issue are great for seeing relationships between pipelines, but is that the most common use-case, or do people want to see the actual jobs of the entire pipeline? Given that everything to the left of the current pipeline must necessarily be successful, is there value in showing the details? Or is the existence of optional manual jobs in the pipeline a reason to show the details? But even then, the mini-graph is still there, so you can get to individual jobs.
Now that I think of it, the mini graph is all I use most of the time now; I don't bother clicking on the full pipeline graph unless there's a problem rendering the dropdown list (which happens far too often). So I wonder if I'm thinking of this all wrong, and the focus is really on the mini graph in the MR view!? Should we think about how to show these cross-project dependencies in the MR view? It might be OK if it's a single line, but the branched trees we have will be really difficult to include. Well, unless we do the same collapsed trick we do for jobs, and show a single icon for downstream pipelines, click on it to see the list of downstream pipelines... I could imagine it even going further to then render next-level downstream pipelines dynamically as you click on them. Like see the minigraph with upstream and downstream, click on the downstream, see 2 downstream pipelines expanded to show their minigraphs, including their further downstream pipelines, click on them to see them expand to mini pipeline graphs... Crazy, I know. But as I use the mini graphs more, I really like them; both for the location (in the MR where I want them) and the compactness. I haven't really lamented the missing detail of the full pipeline graph.
At any rate, we still need the full pipeline view, so maybe we just also need a mini graph view of cross-project pipelines.
So let's think through some use-cases.
- I see a failure in a pipeline and want to track down its root cause, which may be in an upstream pipeline.
- I'm working in an MR, push a commit and want to watch the downstream pipeline to make sure it all succeeds.
- I'm new to a project and want to understand the overall pipeline.
For 1 and 2, I'd want to work from the main MR view's mini pipeline graph, or at least the Pipelines tab.
Side note: We haven't defined what a failure in a cross-project pipeline means. If a downstream pipeline fails, should it cause the upstream pipeline to fail even though individual jobs/stages in that pipeline succeeded? Unfortunately, there are cases for both.
a) Consider open source project X with an unrelated project Y that declares a dependency on X so that Y is rebuilt every time X changes. There should be no way that the result of Y's pipeline affects X's. b) Consider Omnibus which packages GitLab CE and GitLab EE. If a change in GitLab CE causes Omnibus to break, shouldn't the pipeline of GitLab CE reflect that it was the cause and thus notify the author of the change?
Perhaps that's the difference between a backward reference and a forward reference. In (a), Y would declare X as a dependency, but maybe in (b), GitLab CE would trigger Omnibus. Whoever declares the dependency claims failure status. That may not actually make sense, and even if it does, it may be too subtle to be intuitive. But one way to think about it is that if I declare the dependency in GitLab CE, then the dependency is part of CE's pipeline, and thus failure of downstream would fail CE's pipeline. Unfortunately, Omnibus would likely still need to declare what artifacts it needs from CE, so it still has to declare the dependency, and thus make things more ambiguous.
Further side note, back to (a), when looking at project X, would I really want to see anything about project Y? I think it would clutter my view if I saw all the project depended on mine, that I have no concern for. But when viewing project Y, I may need to see the pipeline for project X. Although honestly even in that case, I might not care about the details, just that the dependency exists and to know that that's why this particular pipeline was triggered. Sometimes project Y will have standalone pipeline because I pushed new code to it, sometimes it will have triggered pipelines because of an upstream push. Consumers of Ruby gems, for example, don't likely care about the pipeline for the gem. By virtue of being open source, I'd have permission to see the pipeline, but not the desire to do so (most of the time).
Can we tell the difference? Would we look at project membership and treat upstream/downstream project differently if I'm just a guess on them?
I took a spin at a view for showing expanded pipelines across projects. This view is supposed to be inside the
Pipelinetab in a Pipeline page, but I'm showing it without context because it gets pretty big. If we were to include this, we would need to add @dimitrieh's idea for an area that you can pan around and zoom in/out of.
This is not part of the design. Just showing it for reference so you can get an idea of the structure of the pipeline.
Since we are in the
Pipelinetab for a particular pipeline, we show that pipeline expanded by default. You can see its direct upstream and downstream pipelines.
Expanding the first level
When you click on a pipeline node, it expands to show you all stages and jobs. This pipeline has an upstream pipeline in the GitLab Shell project. Let's expand that one
Expanding the second level
When we expand the GitLab Shell pipeline we find out it doesn't have any upstream pipelines. We already knew it had a downstream pipeline (Gitaly), but we now see it has another one in GitLab EEP. I'm not sure if this is valuable information, since it doesn't affect the current project's pipeline, but it gives you a better sense of how all your projects communicate.
Expanding the third level
Finally, we can expand that second downstream pipeline of GitLab Shell and we see it doesn't have any more dependencies.
I forgot to include a way to collapse the pipelines back into nodes, but we should add that. We could also have a button to expand all nodes into pipelines like @markpundsack suggested, but I think it should only act on the nodes that are currently visible, and not on the ones that are about to be loaded.
@dimitrieh I think the
+2on the nodes from #22558 (comment 25293018) shouldn't be necessary. I think it'd be best to show all nodes explicitly because we'll have the space with the interactive area.
That's awesome and exactly what I'm looking for. Doesn't mean that's what we should ship, but helps me understand the option.
What do you all think?
Hi @ChadMalchow - we're really excited about this enhancement since it helps validate the use of GitLab CI for multi-stage pipelines. We are still making the internal business case for using EE over CE and keeping this feature at the EE (not EEP) level would help there.
Based on your website, EE is "Ideal for co-located teams who need additional security and workflow controls for their professional projects.". This feels like a professional project feature to me.
EEP is "ideal for distributed teams who need advanced workflow controls, High Availability, and Premium Support.". This is not an advanced workflow control, distributed or HA capability.
Having great dashboard makes a lot of sense, but I would focus now on extending only mini-pipeline graph on Merge Request Widget with additional pipelines.
This would give us a very nice way to show dependent pipelines, without spending a lot of time now on implementing full-fledged dashboard as described here. Most people will probably look for feedback on mini-graph anyway.