Zoekt::PlanningService fix for setting project_namespace_id in metadata

What does this MR do and why?

Explanation

There were two issues.

• PlanningService: We were selecting the emptiest index for a project to be assigned. If there is only one index, then it is fine, but the issue with the multiple indices per namespace is that it might break the metadata of the index. Let's say we have one index with metadata project_namespace_id_from: 5. And for the next huge project_namespace(6), we don't have enough space, so we will initialize a new index with metadata project_namespace_id_from: 6. Now for the next project_namespace(7), it was selecting the empties index, it might select the first index and will overwrite the project_namespace_id_from from 5 to 6. This is not correct. It has just overwritten the already built up index. We will lose the project_namespace: 6. The fix is to just select the last index from the build collection.

Second change in PlanningService is that we will use the open range for project_namespace_id_. So if a namespace has only an index, its metadata will be {project_namespace_id_from: nil, project_namespace_id_to: nil}. If a namespace has multiple indices, the subsequent index will have project_namespace_id_from as the next integer of project_namespace_id_to of its previous index. We are doing this to cover all the project_namespaces if there is a transfer. This is the discussion regarding this topic: !185633 (comment 2419355161)

• ProvisioningService: If the condition required_storage_bytes > node.unclaimed_storage_bytes was true, we were just skipping the creation of the index. But the issue is with multiple indices. It could create partial indices for a namespace. The fix is to raise if the condition is true, so everything will get rolled back. Use insert_all to maintain the atomicity and a minute performance improvement.

AI Summary of Changes

These changes should improve performance and reliability of the Zoekt search service, particularly when provisioning large numbers of indices.

This merge request optimizes the Zoekt search service by improving how projects are assigned to indices and making the provisioning process more efficient. The key changes include:

1. Simplifying the algorithm for assigning projects to indices by preferring to use the last available index if it has enough space, rather than finding the optimal fit each time.

2. Making the index creation process more efficient by using bulk insertion (insert_all) instead of creating indices one by one, which reduces database operations.

3. Improving error handling to make the process more atomic - if one index in a batch fails to be created, the entire operation is rolled back, preventing partial provisioning.

4. Adding more comprehensive test cases that verify the system's behavior when encountering various error conditions, such as non-existent nodes or insufficient storage space.

References

Screenshots or screen recordings

Internal link: #525536 (comment 2415757317) to show that the fix works

Before	After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Query Report

INSERT INTO "zoekt_indices" ("zoekt_enabled_namespace_id", "zoekt_replica_id", "zoekt_node_id", "namespace_id", "reserved_storage_bytes", "state", "metadata", "created_at", "updated_at")
    VALUES (18710, 1039607, 1000143, 88471579, 10018244988, 0, '{"project_namespace_id_from":88843821}', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP),
    (16745, 1037358, 1000146, 7715866, 2542004353, 0, '{"project_namespace_id_from": 15158440}', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP),
    (17272, 1038201, 1000152, 86183824, 418876566, 10, '{"project_namespace_id_from": 86453415}', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP),
    (21326, 1040607, 86, 16624643, 174919723455, 2, '{"project_namespace_id_to": 102675946, "project_namespace_id_from": 66848366}', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP),
    (14346, 1036904, 1000142, 947973, 1378861433, 10, '{"project_namespace_id_from": 23963372}', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
ON CONFLICT
    DO NOTHING
RETURNING
    "id"

https://console.postgres.ai/shared/0b7a5bed-4c3a-4e92-b73c-2321641fbd42

Related to #525536 (closed)