Skip to content

Backend: Add GraphQL CiCatalogResource field `latest_version` to replace `versions` when fetching multiple resources

Summary

This is a follow up issue to Backend: CI Catalog - Add additional fields to ... (#407382 - closed).

Initially we had implemented the latest_version field for CiCatalogResource type. Upon first maintainer review, it had been deemed an anti-pattern and an implementation of versions was suggested, which resulted in Add GraphQL field `versions` to CiCatalogResource (!120811 - merged).

However, upon another maintainer review, it's now suggested to re-implement latest_version for performance reasons.

Why the versions field is not performant (ref: #413129 (comment 1419174082)):

The proposed method your previous peer reviewer has more or less similar issue that it lacks LIMIT clause. This type of query causes statement timeout for big tables like ci_builds, ci_pipelines, including a mid-sized table like releases that contains historical records (i.e. it just grows, never shrinks).

Proposal

Ref: #413129 (comment 1417697313)

According to the usecase, a pagination is unnecessary and should be prevented (otherwise backend has to put decent effort for making it optimized), so it's reasonable for me to use latest_version field. Even if we are able to batch-load each versions, frontend needs to specify a different cursor per ciCatalogResource, and has to re-fetch entire resources when one of pages is incremented/decremented, which sounds like unnecessary expensive.

I suggest that we go for one of the following paths:

  1. Introduce latest_version field in ciCatalogResources type. For example, you can follow the same pattern with https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/graphql/types/environment_type.rb#L72-75, which show-cases a preloading logic for N entries in Preloaders::Environments::DeploymentPreloader.
  2. Same with (1) except using LATERAL JOIN instead of UNION.
  3. Switch to Offset pagination from Keyset pagination. This likely needs a refactoring on frontend side.

If we go for pattern 1 or 2, frontend should fetch the resource as following:

  • When a user visits the catalog resource index page, frontend shows latest version for each catalog resource.
  • When a user visits the catalog resource details page, frontend show version history of the specific catalog resource. They can directly call project.releases query.

Additionally, the versions field should be updated for use only when fetching a single resource.

Implementation

Description MR
Add CiCatalogResource latest_version GQL field; deprecate versions. Note: The versions field will be reverted back to alpha state in the next MR (below). !123113 (merged)
Limit GQL CiCatalogResource versions field to single resource request !123977 (merged)
Edited by Leaminn Ma