Monitoring the memory usage of a fresh omnibus install
Ensuring the GitLab is able to run on cheap commodity hardware is a priority.
GitLab probably experiences more resource pressure in regards to physical memory (ie RAM) than other resources such as CPU or storage.
Unfortunately, I've not been able to find any metrics about how much memory a "default" GitLab Omnibus install uses, how this breaks down between the various components in the GitLab package and in particular, how this grows from release to release.
Would it be worth adding a CI job to record these metrics for each release?
I've done some (naive) basic testing of my own, and for release 11.4, came up with the following numbers: https://docs.google.com/spreadsheets/d/1kzBDMisXG8WrRlfnVwYjAxafAd_pVMJI5R8z5lKP2_Y/edit
These number are very much preliminary!
Right now on a 64-bit Ubuntu Bionic machine, without any tweaks to gitlab.rb, GitLab requires about ~3GB of memory to run.
When I did some rudimentary configuration changes to gitlab.rb, I was able to bring this down to ~2.7GB:
https://docs.google.com/spreadsheets/d/1xcPJao8prrC_GHyZ4DyT_naeq1cU7iU1cSG_fAVZnng/edit
I used smem for memory estimation, specifically using the USS and PSS values instead of RSS which is a terrible way to estimate memory usage.
The script I used to generate the data is here, written in some really ugly Perl (don't judge me
Having these metrics will help us to decide whether having an option to use Puma instead of Unicorn is worthwhile.
Related work:
- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22372
- https://gitlab.com/gitlab-org/gitlab-ce/issues/52762
- gitlab-development-kit!532 (merged)
- !2783 (merged)
cc @sytses
Update: refer to #3858 (comment 212627839).