Skip to content

Log Diagnostic Reports file size

Aleksei Lipniagov requested to merge 362900-log-report-filesize into master

What does this MR do and why?

Additionally logs the size of the report (in bytes).
This would allow us to troubleshoot potential issues even easier (example using reports logs).
We also will be able to correlate report size and the time it takes to produce it.

The logging code (as well as reports dump) is behind an :ops FF report_jemalloc_stats, so could be disabled if it'll misbehave.

How to set up and validate locally

Very similar to !91283 (merged).

  1. Pull this branch
  2. Make sure you have GITLAB_DIAGNOSTIC_REPORTS_ENABLED set to true (need to restart the Puma and/or Sidekiq)
  3. Set smaller constants lib/gitlab/memory/reports_daemon.rb. e.g: DEFAULT_SLEEP_S could be 5, DEFAULT_SLEEP_MAX_DELTA_S = 1, DEFAULT_SLEEP_BETWEEN_REPORTS_S = 1. You could do it via ENV vars (in GDK/GCK) or just hacking the code.
  4. Make sure report_jemalloc_stats feature flag is enabled or run Feature.enabled(:report_jemalloc_stats)
  5. Check tail -f log/application_json.log. Could also ... | grep jemalloc.
  6. On OSX+GDK, you will notice an additional field report_size, but the value will be 0, because libjemalloc must be on LD_PRELOAD, which is not how GDK is configured by default. So, no reports would be produced on OSX, but the code shouldn't break.
  7. On GCK, this aspect is already configured similar to production, so you will be able to see real file sizes.

Screenshot_2022-07-29_at_17.31.23

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #362900 (closed)

Edited by Aleksei Lipniagov

Merge request reports