Skip to content

Automatically set GOMAXPROCS based on CPU Quotas

Arran Walker requested to merge ajwalker/autoset-maxprocs into main

What does this MR do?

Uses go.uber.org/automaxprocs/maxprocs to automatically set GOMAXPROCS.

GOMAXPROCS usually matches the number of CPU cores available for the machine. Many libraries use this to inform them of the concurrency they can use.

However, GOMAXPROCS still uses the number of CPU cores available for the machine even if cgroup quotas have been set, as is common for Kubernetes and Docker. This can lead to scenarios where the concurrent value is set way higher than is intended in these environments, leading to poor performance.

Why was this MR needed?

Specifically the fastzip library defaults to GOMAXPROCS, and in Kubernetes environments where the helper binary has a CPU quota, this can lead to poor utilisation of compute and memory.

However, in general, GOMAXPROCS is used to inform the Go scheduler and it's better to not have that over commit.

What's the best way to test this MR?

What are the relevant issue numbers?

Related to gitlab#402233

Might help with #27984 (closed)

Edited by Arran Walker

Merge request reports