Cross-project pipeline triggering and visualization MVC
Note: If you are looking for multiple pipelines within a single project, please see https://gitlab.com/groups/gitlab-org/-/epics/438.
### Problem to Solve
Pipelines are great for managing individual delivery associated with repos, and there are some limited ways to connect them via API, but we want to establish a feature set that more completely addresses how to do more "legacy" (non-decoupled) releases with cross-project pipeline (trigger and artifact) dependencies.
### Description
We have [three types of pipelines](https://about.gitlab.com/direction/cicd/#pipelines):
1. CI pipeline: Multiple stages, one project, similar to naming in Jenkins, planning to build 8.8
1. Deploy pipeline: The flow of deploying code to servers through various stages
1. Project pipeline: Cross-project dependencies, particularly for micro-services, but also for complicated build dependencies: e.g. api -> front-end, ce/ee -> omnibus. Like https://ci.concourse-ci.org/
The first two types have received a lot of work lately; this issue covers the third.
We have basic triggers, but need to improve them, including adding first-class support for them in `.gitlab-ci.yml`. We also need dependencies that you define in .gitlab-ci.yml per project. All-multi project visualizations can be based on this; you need read access to the other projects. You don't have one uber pipeline defined somewhere; projects depend on each other to define the pipeline. We can show the project pipeline even in the project view without needing an entity to represent the group of pipelines.
We want to find a way to support these kinds of relationships and tracking with simple but powerful primitives (that also don’t lock us into an old school style of RM way of thinking.) Our first priority then will be to solve these problems for teams that can largely automate their pipelines, but need better tools to handle scenarios that cross project boundaries.
### Use Cases
Through this issue we want to build a solid, workable solution for having cross-project pipelines work that solve real problems. By improving the usability of connected pipelines in the UI, enhancing status passing and behavior between pipelines, and creating a more clear way for pipelines to depend on/trigger each other, we unlock a few key use cases:
- "Legacy" style non-decoupled releases where different teams are working in independent product areas, but want to be able to configure pipelines (or a master release pipeline) that can orchestrate the behavior across the otherwise (at least partially) independent projects/pipelines.
- Microservice style releases where there are complex dependency relationships embedded directly in the pipelines. For example, one microservice that requires redeployment of a related microservice (that exists in a different project) after updating.
### Future Improvements
https://gitlab.com/groups/gitlab-org/-/epics/446
epic