Skip to content

Backend: Denormalize name and description in Ci::Catalog::Listing

Summary

The intention in this issue is to improve the query plan performance in Ci::Catalog::Listing.

Denormalized columns are necessary for query performance. Details can be found here: !134200 (comment 1604567481).

Proposal

Ref !134517 (comment 1609987074):

  1. Add denormalized columns name and description to the catalog_resources table. Include trigram indexes on both columns.
  2. Add a migration to backfill the data. Add logic to keep the columns in sync with the projects table when:
  • A project is marked as a catalog resource.
  • The project name/description is updated.
  1. Update the name sort filter queries in the Ci::Catalog model to use the new denormalized column.
  2. Update the search query plan in Ci::Catalog::Listing from !134517 (diffs) to use the new denormalized columns.

MR Implementation

Description MR
Step 1 & 2: Add denormalized columns name and description t... (!134708 - merged) !134708 (merged)
Step 3: Update CI catalog resources name sort to use de... (!135172 - merged) !135172 (merged)
Step 4: Update query plan in Ci::Catalog::Listing to us... (!135182 - merged) !135182 (merged)

Performance Implementation for 'All resources'

Group Issue Link
backend Backend: Denormalize name and description in Ci... (#427928 - closed) 👈 You are here
backend Backend: Denormalize visibility_level and impro... (#429056 - closed) #429056 (closed)
backend Backend: Implement background syncing to ensure... (#429376 - closed) #429376 (closed)
Edited by Leaminn Ma