Update 2022/08/29

All Weekly Demos: #16

Recording

https://youtu.be/NIcNr3OJw-k

Vision

Make GitLab a tool Data Scientists and Machine Learning Engineers love to use.

Mission

Explore and Collaborate with different teams to deliver features that improve the user experience for Data Scientists and Machine Learning Engineers, while increasing awareness within the company to this user groups

ML Experiment Tracking

What is Experiment Tracking

gitlab-org&8560

When Data Scientists are working on Machine Learning, it is common to run train the same model using different configuration. These configurations can take many forms, which range from a parameter to the algorithm used for learning, or the learning itself, or the data used for training, all o which being able to significantly impact the final performance. In this context, we are calling each trial a Candidate and an Experiment a collection of comparable Candidates. A candidate can eventually be promoted to a model to be released, or the whole experience can happen just for exploration.

MLFlow Experiment Tracking

The most common option at the moment is MLFlow experiment tracking. We intend to make it really simple for users to switch between MLFlow and GitLab https://www.mlflow.org/docs/latest/tracking.html

What are we not doing

We are NOT packaging MLFlow with GitLab.

What are we doing

We are implementing an Experiment Tracking Server on GitLab, and we are providing the same Rest API as MLFlow. Reasons are highlighted on gitlab-org&8560

Progress

• MR introducing the MLFlow is currently open for review
  • gitlab-org/gitlab!95689 (merged)
  • Three endpoints: Create Experiment, Get Experiment and Get Experiment By Name
• Automated documentation on the difference between MLFlow and GitLab APIs
  • https://gitlab.com/gitlab-org/incubation-engineering/mlops/mlflow_experiment/-/blob/main/results.md

Up Next

• Next two weeks: Implement remaining 8 endpoints gitlab-org/gitlab#370478 (closed)
• Then:
  • Frontend
  • Documentation