Automatically set GOMAXPROCS based on CPU Quotas
What does this MR do?
Uses go.uber.org/automaxprocs/maxprocs
to automatically set GOMAXPROCS
.
GOMAXPROCS
usually matches the number of CPU cores available for the machine. Many libraries use this to inform them of the concurrency they can use.
However, GOMAXPROCS
still uses the number of CPU cores available for the machine even if cgroup quotas have been set, as is common for Kubernetes and Docker. This can lead to scenarios where the concurrent value is set way higher than is intended in these environments, leading to poor performance.
Why was this MR needed?
Specifically the fastzip
library defaults to GOMAXPROCS
, and in Kubernetes environments where the helper binary has a CPU quota, this can lead to poor utilisation of compute and memory.
However, in general, GOMAXPROCS
is used to inform the Go scheduler and it's better to not have that over commit.
What's the best way to test this MR?
What are the relevant issue numbers?
Related to gitlab#402233
Might help with #27984 (closed)