Backend: Add GraphQL CiCatalogResource field `latest_version` to replace `versions` when fetching multiple resources
Summary
This is a follow up issue to Backend: CI Catalog - Add additional fields to ... (#407382 - closed).
Initially we had implemented the latest_version
field for CiCatalogResource
type. Upon first maintainer review, it had been deemed an anti-pattern and an implementation of versions
was suggested, which resulted in Add GraphQL field `versions` to CiCatalogResource (!120811 - merged).
However, upon another maintainer review, it's now suggested to re-implement latest_version
for performance reasons.
Why the versions
field is not performant (ref: #413129 (comment 1419174082)):
The proposed method your previous peer reviewer has more or less similar issue that it lacks
LIMIT
clause. This type of query causes statement timeout for big tables likeci_builds
,ci_pipelines
, including a mid-sized table likereleases
that contains historical records (i.e. it just grows, never shrinks).
Proposal
Ref: #413129 (comment 1417697313)
According to the usecase, a pagination is unnecessary and should be prevented (otherwise backend has to put decent effort for making it optimized), so it's reasonable for me to use
latest_version
field. Even if we are able to batch-load eachversions
, frontend needs to specify a different cursor perciCatalogResource
, and has to re-fetch entire resources when one of pages is incremented/decremented, which sounds like unnecessary expensive.I suggest that we go for one of the following paths:
- Introduce
latest_version
field inciCatalogResources
type. For example, you can follow the same pattern with https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/types/environment_type.rb#L72-75, which show-cases a preloading logic for N entries inPreloaders::Environments::DeploymentPreloader
.- Same with (1) except using
LATERAL JOIN
instead ofUNION
.- Switch to Offset pagination from Keyset pagination. This likely needs a refactoring on frontend side.
If we go for pattern 1 or 2, frontend should fetch the resource as following:
- When a user visits the catalog resource index page, frontend shows latest version for each catalog resource.
- When a user visits the catalog resource details page, frontend show version history of the specific catalog resource. They can directly call
project.releases
query.
Additionally, the versions
field should be updated for use only when fetching a single resource.
Implementation
Description | MR |
---|---|
Add CiCatalogResource latest_version GQL field; deprecate versions . Note: The versions field will be reverted back to alpha state in the next MR (below). |
!123113 (merged) |
Limit GQL CiCatalogResource versions field to single resource request |
!123977 (merged) |