Implement lazy association loader
What does this MR do and why?
This MR introduces an abstraction to preload associated records of an Active Record model lazily on GraphQL. It makes it possible to sort, limit, and filter has_many
associations without causing an N+1 query issue.
See [Draft] Use lazy association loader (!115241 - closed) for its usage.
Database review
The below queries are generated for a GraphQL query to load only the last state transition record for a list of vulnerabilities.
GraphQL query
query getVulnerabilities($project_fullpath: ID!) {
project(fullPath: $project_fullpath) {
vulnerabilities(first: 20) {
nodes {
stateTransitions(last: 1) {
nodes {
fromState
}
}
}
}
}
}
Here is an example query generated by the `TopNLoader` utility class
SELECT
"vulnerability_state_transitions".*
FROM
vulnerabilities
JOIN LATERAL (
SELECT
"vulnerability_state_transitions".*
FROM
"vulnerability_state_transitions"
WHERE (vulnerability_state_transitions.vulnerability_id = vulnerabilities.id)
ORDER BY
"vulnerability_state_transitions"."id" ASC
LIMIT 2) AS vulnerability_state_transitions ON TRUE
WHERE
"vulnerabilities"."id" IN (78324807, 78324806, 78324805, 77964096, 77896757, 77896756, 77896755, 77896754, 77625632, 77625631, 77625630, 77524835, 77524834, 77524833, 77498088, 77498087, 77498086, 77069619, 77069618, 77046383)
https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/16812/commands/57213
Old query to load all the state transitions
SELECT
"vulnerability_state_transitions".*
FROM
"vulnerability_state_transitions"
WHERE
"vulnerability_state_transitions"."vulnerability_id" IN (78324807, 78324806, 78324805, 77964096, 77896757, 77896756, 77896755, 77896754, 77625632, 77625631, 77625630, 77524835, 77524834, 77524833, 77498088, 77498087, 77498086, 77069619, 77069618, 77046383)
https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/16938/commands/57551
Note that the latter one performs better but it loads all the state transitions from the database which could introduce memory bloat and serious performance issues.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.