Skip to content

CI: prometheus - reduce storage amount, length. set resources

Jason Plum requested to merge ci-prometheus-reduce-20240125 into master

What does this MR do?

CI: prometheus - reduce storage amount, length. set resources

We have been seeing a lot of Prometheus "failing" due to memory pressures on the node(s) when in use.

Forcibly set the retention to 1GB over 3d.

Set the resources! They were 0, so it will "always" lose. Ask for 2G, limit to 4G. Observation shows "happy" prometheus at ~2.5G

Related issues

Author checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • Merge Request Title and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com
  • When ready for review, follow the instructions in the "Reviewer Roulette" section of the Danger Bot MR comment, as per the Distribution experimental MR workflow

For merge requests from forks, consider the following options for Danger to work properly:

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes
  • Documentation created/updated
  • Tests added/updated
  • Integration tests added to GitLab QA
  • Equivalent MR/issue for omnibus-gitlab opened
  • Equivalent MR/issue for Gitlab Operator project opened (see Operator documentation on impact of Charts changes)
  • Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.
Edited by Jason Plum

Merge request reports