Memory Improvement Roadmap
The goal of this issue is to describe the roadmap we'll be following to improve memory usage of GitLab. In the past we mostly focused on response timings as this was a more severe problem. With many of those being solved it's time we start looking into memory usage more often.
This issue is not meant for reporting memory usage problems, so please refrain from posting comments such as "I'm suffering from this problem too!". Instead please create a separate issue and assign the following two labels to the issue:
- memory usage
If you can not add these labels yourself please ask another GitLab developer (e.g. you can ping me and I'll add the labels) to do so. This ensures this particular roadmap issue stays on topic and the individual issues are easier to find.
Broadly speaking the process can be broken up into 3 distinct steps:
- Add graphs to Grafana to plot memory usage of certain parts over time
- Stabilize memory usage of various parts of GitLab
- Improve (= reduce) memory usage of GitLab
Step 2 and 3 will probably be done at the same time per component, but this is not a requirement. It's vital that we first add monitoring before making any changes. Without monitoring we have no idea what impact our changes will have.
Plot Rails Controllers memory allocations over timeDone 🚢
Plot Sidekiq workers memory allocations over timeDone 🚢
8.15 & 8.16
For 8.15 and 8.16 we'll focus on two parts:
Since 8.15 will be released in roughly 2 weeks from now I expect most improvements won't land in 8.15.0, but we should try to at least get something in.
8.17 and later releases
Since these releases are still a bit off we won't be planning anything specifically for now. Since improving MR memory usage will be quite hard it's possible releases after 8.16 will also focus on MR memory usage.
I'm purposely not going to set any goals such as "N MB of memory usage by month X" as it's very hard to live up to such expectations. I'd much rather add incremental improvements until there's nothing left to improve (without rewriting large parts of GitLab).
Finally, we're also not yet aiming at reducing our RAM recommendations/requirements, instead we're going to initially focus on getting the code under control.