Skip to content

bandwidth of the rails log stream has increased, so we need to grow the beats fleet

In: gitlab-com/gl-infra/production#6278 (closed) we saw a backlog of messages accumulating for the rails logs.

Based on:

Screenshot_from_2022-02-03_17-09-43

src: https://dashboards.gitlab.net/d/USVj3qHmk/logging?orgId=1&from=now-7d&to=now&viewPanel=23

we suspect the amount of data that pubsubbeats need to process has increased (probably due to a deployment).

The same problem is happening with fluentd-archiver, however, we are making this change only for pubsubbeats for now to confirm this is where the bottleneck is.

Signed-off-by: Michal Wasilewski mwasilewski@gitlab.com

Merge request reports