Saturation growth predictions and reporting with Jupyter notebooks, juypter books, Facebook Prophet, integrated with Prometheus saturation metrics
This would deliver FY21-Q3 Infra KR: Improve runbooks experience using Jupyter notebooks
gitlab-com/www-gitlab-com#8276 (closed)
I've been working with GitLab's Jupyter integration for the past few days, and, at present, am not convinced that this can be used in critical and time-sensitive periods, such as during an incident - the times we tend to use runbooks the most.
As a first step, I propose that we use this functionality for something not runtime critical: capacity planning.
Proposal: Capacity Planning using Facebook Prophet and Prometheus Saturation Metrics
- Integrate Juypter Notebooks, Jupyter Books, The Prometheus Pandas Library and Facebooks' Prophet library
- Provide forward forecasts for up to 90 days, based on saturation metrics over the past year
- Run this regularly in CI (on the ops instance) and notify if any saturation events are predicted
- Persist the jupyter book in GitLab Pages
- Allow operators to run the book interactively in Jupyter
As a proof-of-concept, I've already been experimenting with this, running Jupyter Notebooks locally.
Some of the graphs from this:
Edited by Steve Loyd