Resolve Memory Leaks
This is a meta issue to track what would be required to solve GitLab memory leaks, what it would allow us to do once fixed, etc.
Requirements
The most important requirement is better monitoring. Our current setup is not adequate enough to track down where memory leaks occur, when, how big their impact is, etc. Issue #2936 (closed) covers part of this by suggesting replacing New Relic & friends with something better, other issues may be added over time. Better monitoring would make it possible (and/or easier) for us to answer the following questions:
- When do memory leaks occur (e.g. what pages trigger them)
- Where do memory leaks originate from source code wise (e.g. what method, file, line, etc)
- What is the impact of this leak? Is it a small leak we can live with for the time being, or is it a big leak that has to be fixed ASAP?
- Finally, once the leak has been dealt with: did it actually have a positive impact.
The above issue already specifies some of the requirements, so I'll not replicate them here.
Outcome
Should these problems be resolved we can start looking into things such as multi-threaded application servers using Puma (#3592 (closed)). This in turn would allow for greater scalability without an increase of memory usage.
Currently Blocked Issues/MRs
The following multithreading/Ruma issues/MRs are currently blocked by this:
cc @sytses