SQL n+1 in /api/:version/runners - 40k+ SQL queries per request

Related to the investigations in gitlab-com/gl-infra/scalability#865


A SQL n+1 performance regression exists in API route /api/:version/runners. This can lead to thousands (>20k!) SQL queries being generated for a single request.

https://log.gprd.gitlab.net/goto/95e9e6ad3c29edc5d6bc45cd5de6766f

image

Note: at present we don't capture any context metadata for this endpoint. This makes it fairly difficult to understand exactly which account/customer/namespace is causing the problem. We should fix this too: #321831 (closed)

Edited by Andrew Newdigate