Skip to content

Zoekt: Add `cpu_to_tasks` ratio

Problem to solve

Currently, with 1:1 ratio between number of CPUs and number of concurrent tasks for a Zoekt node, we leave performance on the table.

For example, during gitlab-com/gl-infra/production#18460 (closed) we had a peak of 82 concurrent indexing operations for one Zoekt node, but in terms of CPU usage we only reached 23 out of limit 40:

image

source

Proposal

I believe that we should add a cpu_to_tasks ratio to Zoekt application settings. The default, could still be 1.0.

For .com, we can set it to 3.5 or even higher depending on the actual CPU load.

We could even skip adding the UI in the beginning and only add it as a field in application settings so that we can change it manually.

Edited by Dmitry Gruzd