Skip to content

Update Ci catalog Version model to use denormalized `released_at` column

Leaminn Ma requested to merge refactor-ci-catalog-version-model-queries into master

What does this MR do and why?

In #430117 (closed), our objective is to denormalize catalog_resources_versions.released_at so that we can reduce the need to JOIN with the releases table in Ci::Catalog::Resources::Version. This allows us to make more efficient queries that improve performance and allow us to clean up the model code.

In previous MRs, we have already denormalized released_at, established a syncing process, and backfilled the data. Now we can proceed with this final MR to complete #430117 (closed).

In this MR, we refactor the code in Ci::Catalog::Resources::Version so that it utilizes the new released_at column. In doing so, we're able to remove the need to explicitly define the keyset pagination order.

The query plans below show the resulting SQL changes. There are no significant changes to the specs because there are no behavioural or visible changes.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Query plans

1. Sorting query

Ci::Catalog::Resources::Version.order_by_released_at_asc

Before:

SELECT "catalog_resource_versions".*
FROM "catalog_resource_versions"
INNER JOIN "releases" ON "releases"."id" = "catalog_resource_versions"."release_id"
ORDER BY "releases"."released_at" ASC, "releases"."id" ASC

Query plan link: https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/25861/commands/81528

After:

SELECT "catalog_resource_versions".*
FROM "catalog_resource_versions"
ORDER BY "catalog_resource_versions"."released_at" ASC

Query plan link: https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/25861/commands/81529

2. latest_for_catalog_resources query

Ci::Catalog::Resources::Version.latest_for_catalog_resources(catalog_resources)

Before:

SELECT "catalog_resource_versions".*
FROM (
  VALUES (7),(6),(5),(2),(1),(8),(9),(10)
) catalog_resources(id)
INNER JOIN LATERAL (
  SELECT "catalog_resource_versions".*
  FROM "catalog_resource_versions"
  INNER JOIN releases AS rel ON rel.id = catalog_resource_versions.release_id
  WHERE "catalog_resources"."id" = "catalog_resource_versions"."catalog_resource_id"
  ORDER BY rel.released_at DESC
  LIMIT 1
) catalog_resource_versions ON TRUE;

Query plan link: https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/25861/commands/81531

After:

SELECT "catalog_resource_versions".*
FROM (
  VALUES (7),(6),(5),(2),(1),(8),(9),(10)
) catalog_resources(id)
INNER JOIN LATERAL (
  SELECT "catalog_resource_versions".*
  FROM "catalog_resource_versions"
  WHERE "catalog_resources"."id" = "catalog_resource_versions"."catalog_resource_id"
  ORDER BY "catalog_resource_versions"."released_at" DESC
  LIMIT 1
) catalog_resource_versions ON TRUE;

Query plan link: https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/25861/commands/81532

Edited by Leaminn Ma

Merge request reports