Skip to content

Route projects by root namespace id

Madelein van Niekerk requested to merge 432840-root-projects-by-namespace into master

What does this MR do and why?

Introduces routing for advanced project search by root namespace id by:

  1. Adding es_parent for projects so that it can be picked up during indexing
  2. Reindexing projects until all documents have routing
    1. Based on MigrationReindexBasedOnSchemaVersion which indexes until all the documents have the right schema version. But because indexing with routing creates a new document instead of updating an existing document, we also need to delete the old documents.
  3. Adding routing to search query if the migration is finished
    1. Only route if a group is selected

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

before after
Screenshot_2024-01-31_at_14.41.31 Screenshot_2024-01-31_at_14.36.24

How to set up and validate locally

  1. Checkout this branch but don't run the migration worker yet
  2. Do a project search without a group selected and note that the ES query (in the performance bar) has no routing
  3. Do a project search with a group selected and note that the ES query has no routing
  4. Make sure all migrations have been executed by running Elastic::MigrationWorker.new.perform and ElasticClusterReindexingCronWorker.new.perform on repeat
    1. Verify that the migration finished: curl http://localhost:9200/gitlab-development-migrations/_doc/20240125091440
    2. You can also tail the elasticsearch logs to view the progress of the migration: tail -f log/elasticsearch.log
  5. Do a project search without a group selected and note that the ES query has no routing and the same results as before were returned
  6. Do a project search with a group selected and note that the ES query now has routing and the same results as before were returned

Related to #432840 (closed)

Edited by Madelein van Niekerk

Merge request reports