Weekly MLOps Demo - September 17th 2021
All Weekly Demos: #16
Recording
Vision
Make GitLab the perfect companion for Machine Learning Engineers and Data Scientists.
Mission
Identify opportunities in our portfolio to explore ways where GitLab can provide a better user experience for Data Science and Machine Learning across the entire Machine Learning life cycle (model creation, testing, deployment, monitoring, and iteration).
What Was Done
Improved conversion from Jupyter to Markdown
gitlab-org/gitlab#338890 (closed)
We have now a pretty decent Jupyter to Markdown converter that strips out most of the noise from the notebook into a plain text diffable version. We are now reaching out to different teams to better understand how to integrate it into the codebase.
This is what the convertion in generating: https://gitlab.com/gitlab-org/incubation-engineering/mlops/ipynb2md/-/blob/main/examples/processed_sample.md
Top ML Repos
We don't want to build every MLOps solution out there, but we don want to be able to integrate with the most popular tools used by MLE's/DSs. We tried creating a list of candidates by looking at the ML repos on GitHub and check which we should look into, and we had some surprising results: Only about 15% focus on MLOps, and they are way down the list. While this can have many explanations (mlops is way further on the ML learning path, it's new, the repos don't invest in marketing, etc), it limits the number of options we have, which is good for now.
Note as well that this only includes OSS repos, it doesn't rank private tools (including big cloud providers), which we need to find a different way to look at them.
The list can be found here: https://docs.google.com/spreadsheets/d/1N-pdrClQjVU-dQwdEzHD8h7ig51pbO4GTdEWCY0CLPw/edit#gid=1961070483
Up Next
-
ipynb2md integrationgitlab-org/gitlab#340798 (moved)