Create a new CredentialsInventoryResourceAccessTokensFinder
Related to #524159 (closed) & #511922 (closed)
Summary
During the testing of the rollout of #511922 (closed), we encountered performance issues due to the poor PAT query performance, high user count & an even higher tokens count (PATs & RATs).
Logs
- Credentials Inventory API: https://log.gprd.gitlab.net/app/r/s/LO5bZ
- Credentials Inventory UI: https://log.gprd.gitlab.net/app/r/s/2vkSk
Proposal
- Make a new
CredentialsInventoryResourceAccessTokensFinderor refactorCredentialsInventoryPersonalAccessTokensFinderto work with the resource_access_tokens API & UI for uniformity - The created
CredentialsInventoryPersonalAccessTokensFinderfixed the PAT timeouts to unblock the Credentials Inventory API rollout but, the/groups/:id/manage/resource_access_tokensendpoint's performance is still inconsistent on large groups.
Why?
- This solves the larger issue we have with our query performance
- A more permanent solution might be to the following
Database
The Afflicted Query
-
Used groups_id=4249178 (>30k users), 10575092 (>8k users)
-
Query plans:
-
Prettified: https://paste.depesz.com/s/5Xt
SELECT
personal_access_tokens.*
FROM
personal_access_tokens
WHERE
personal_access_tokens.user_id IN (
SELECT
users.id
FROM
users
JOIN user_details AS user_detail ON user_detail.user_id = users.id
WHERE
users.user_type = 6 AND
user_detail.bot_namespace_id IN (
SELECT
namespaces.traversal_ids[array_length(
namespaces.traversal_ids,
1
)] AS id
FROM
namespaces
WHERE
traversal_ids @> '{4249178}'
)
) AND
personal_access_tokens.impersonation = false
ORDER BY
personal_access_tokens.id ASC
LIMIT 20
OFFSET 0
Edited by Hakeem Abdul-Razak