Skip to content

Configure gitlab-exporter to run with jemalloc

Matthias Käppler requested to merge mk-gl-exporter-jemalloc into master

What does this MR do?

While investigating memory use of gitlab-exporter, we found that it does not run with the jemalloc memory allocator by default. We run Rubies with jemalloc because it is known to be more memory efficient especially in the presence of multi-threaded servers. We also found that by tweaking its config to be stricter, we were able to save ~20% of memory consumed by gitlab-exporter.

This change also updates gitlab-exporter to v9.1.0 v10.0.0, which includes GC related performance improvements going hand-in-hand with this change.

Related issues

Chart MR: gitlab-org/charts/gitlab!1777 (merged)

Test plan

We expect to see no difference in functional behavior, but a drop in memory use.

To verify that the new settings are being applied:

  • run an Omnibus container from this MR
  • make sure that gitlab-exporter is enabled (set it to enable in gitlab.rb)
  • make sure that it is running (gitlab-ctl status)
  • find its pid, e.g. pgrep -af gitlab-exporter
  • inspect cat /proc/<pid>/environ; it should list the env vars configured by default
  • inspect pmap <pid> | grep jemalloc (files mapped into process memory); it should list libjemalloc.so

To test stability, we ran Apache Bench (ab -- install via apt install apache2-utils) against the /metrics endpoint, e.g.:

ab -n 200 -c 2 -l localhost:9168/metrics

To verify memory consumption is reduced, we ran smem (install via apt install smem)

watch -n 0.5 smem -P gitlab-exporter

It should hover around 40 MB (it was closer to 60MB without this change applied).

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion

Required

  • Merge Request Title, and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com
  • Pipeline is green on dev.gitlab.org if the change is touching anything besides documentation or internal cookbooks
  • trigger-package has a green pipeline running against latest commit

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes
  • [-] Documentation created/updated
  • [-] Tests added
  • [-] Integration tests added to GitLab QA
  • Equivalent MR/issue for the GitLab Chart opened
Edited by DJ Mountney

Merge request reports