Explore DAG tracking and visualization
Problem to solve
As a user I want to see the pipeline graph (on an whole) for the pipelines that have been used to end up with a model. This way, a user can better understand how a model is created with what underlying steps.
Gitlab already introduced DAGs into the CI/CD pipelines, documented here: https://docs.gitlab.com/ee/ci/directed_acyclic_graph/
In our case we need to connect the different data pipeline types to get a holistic view. Furthermore, we should evaluate, what information we can integrate, such as what repos have been used, what parameters where relevant, what version of scripts / data where used, etc.
This ticket aims at doing basic research on this matter in order to define how we want to integrate DAGs into MLReef.
Proposal for Technical Solution
Permissions and Security
Documentation
https://docs.gitlab.com/ee/ci/directed_acyclic_graph/