feat: add index on layers for alternative namespace usage query (batch 1/3)
Related to #779 (closed), more precisely the option A
described in #779 (comment 1179923688).
The first implementation step is to create the required index on layers
to speed up the upcoming alternative query described in #779 (comment 1159620397).
To avoid downtime we need to create this index concurrently. However, from the PostgreSQL docs (source):
Concurrent builds for indexes on partitioned tables are currently not supported. However, you may concurrently build the index on each partition individually and then finally create the partitioned index non-concurrently in order to reduce the time where writes to the partitioned table will be locked out. In this case, building the partitioned index is a metadata only operation.
So we'll first create the index concurrently on the 64 partitions.layers_p_N
partitions and only then on public.layers
.
Creating these indexes on a large database such as GitLab.com's is time-consuming (likely more than one minute per index in this case), so we can't use regular migrations. We'll therefore use a post-deployment migration for the first time. For precaution, and because this is our first post-deployment migration, we'll split the creation of the 64 + 1 indexes into a few batches. This MR is for the first, which only creates indexes on partitions 0 and 1.
This change can only be merged and released once gitlab-org/build/CNG!1208 (merged), gitlab-org/charts/gitlab!2867 (closed) and the corresponding charts bump on k8s-workloads are merged. This change will make sure that post-deployment migrations are skipped and not executed alongside regular migrations. Instead, for now, we'll need to raise a production change request to manually apply this post-deployment migration using the registry CLI from within a registry pod.