Skip to content

elastic: Increase shards_per_node on high-throughput indices from 5 to 8

Igorrequested to merge
elastic-high-throughput-8-shards into master

This is in response to gitlab-com/gl-infra/production#4653 (closed), we want to spread load more evenly across nodes.

Thus, I would like to increase number_of_shards on the busy indices (rails, gitaly, workhorse, sidekiq, and a few more).

Calculations to avoid allocation failures:

  • shards: 8
  • total shards per node: 3
  • hot nodes per zone: 9
  • warm nodes per zone: 3

8 shards / 3 shards per node = 2,6 nodes
=> this fits on our 3 warm nodes

We can rollover some of the affected indices after applying this. The expectation is that it will balance CPU utilization across the hot fleet.

Merge request reports