Skip to content

Tunes urgent-cpu-bound hpa and minReplicas

John Skarbek requested to merge jts/tune-hpa-urgent-cpu into master

What does this MR do?

  • We are scaling a little too aggressively, sometimes hitting our max pod count despite not using all available CPU capacity. Changing the target average CPU will pull down the level for which we scale.
  • We are also scaling quickly enough that we remain undersaturated. For this, let's proceed to lower our minimum replicas to only 30.
  • This leaves us with at minimum 150 available workers, which is still 10 more than what we had previously running on VM's.

Dry run results

For the gprd sidekiq urgent-cpu-bound shard:

       - name: urgent-cpu-bound
         concurrency: 5
         hpa:
-          targetAverageValue: 200m
-        minReplicas: 40
+          targetAverageValue: 300m
+        minReplicas: 30

Addresses: gitlab-com/gl-infra/delivery#972 (closed)

Edited by John Skarbek

Merge request reports