Skip to content

feat: add index on layers for alternative namespace usage query (batch 1/3)

João Pereira requested to merge 779-a-index into master

Related to #779 (closed), more precisely the option A described in #779 (comment 1179923688).

The first implementation step is to create the required index on layers to speed up the upcoming alternative query described in #779 (comment 1159620397).

To avoid downtime we need to create this index concurrently. However, from the PostgreSQL docs (source):

Concurrent builds for indexes on partitioned tables are currently not supported. However, you may concurrently build the index on each partition individually and then finally create the partitioned index non-concurrently in order to reduce the time where writes to the partitioned table will be locked out. In this case, building the partitioned index is a metadata only operation.

So we'll first create the index concurrently on the 64 partitions.layers_p_N partitions and only then on public.layers.

Creating these indexes on a large database such as GitLab.com's is time-consuming (likely more than one minute per index in this case), so we can't use regular migrations. We'll therefore use a post-deployment migration for the first time. For precaution, and because this is our first post-deployment migration, we'll split the creation of the 64 + 1 indexes into a few batches. This MR is for the first, which only creates indexes on partitions 0 and 1.

This change can only be merged and released once gitlab-org/build/CNG!1208 (merged), gitlab-org/charts/gitlab!2867 (closed) and the corresponding charts bump on k8s-workloads are merged. This change will make sure that post-deployment migrations are skipped and not executed alongside regular migrations. Instead, for now, we'll need to raise a production change request to manually apply this post-deployment migration using the registry CLI from within a registry pod.

Edited by João Pereira

Merge request reports