Skip to content

Zoekt: Add mark_ready task

Ravi Kumar requested to merge 442881-zoekt-add-mark_ready-task into master

What does this MR do and why?

Add a new task move_indices_to_ready in the Search::Zoekt::SchedulingService. This task will move all the initializing zoekt indices to ready if there are no zoekt_repositories in the pending state. Also, add a logging with a payload of the total count of zoekt indices which will be moved to ready in a single run.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. Open the rails console.
bin/rails c
  1. Create a new namespace for the testing.
  2. Run the following commands to create zoekt index
zoekt_node = ::Search::Zoekt::Node.find_or_create_by!(index_base_url: 'http://127.0.0.1:6080/', search_base_url: 'http://127.0.0.1:6090/', uuid: '00000000-0000-0000-0000-000000000000')
namespace = Namespace.last # Some namespace you want to enable
enabled_namespace = Search::Zoekt::EnabledNamespace.find_or_create_by(namespace: namespace)
zoekt_node.indices.create!(zoekt_enabled_namespace_id: enabled_namespace.id, namespace_id: namespace.id, state: :ready)
idx = zoekt_node.indices.last
idx.initializing!
  1. Create some projects under the selected namespace. Let's create 3 projects
  2. Create Search::Zoekt::Repository records
namespace.projects.each { |p| idx.zoekt_repositories.create!(zoekt_index: idx, project: p, state: :pending) }
  1. Run the service
Search::Zoekt::SchedulingService.new(:move_indices_to_ready).execute
  1. Check the state of the index. It must be initializing
  2. Now move just one zoekt_repository to the ready state
idx.zoekt_repositories.last.ready!
  1. Again run the service from step 6. Check the state of the index. It must be initializing 
idx.reload.state
  1. Now move all the zoekt_repositories to the ready state
idx.zoekt_repositories.map(&:ready!)
  1. Again run the service from step 6. Check the state of the index. It must be ready 
idx.reload.state

Query plan

https://console.postgres.ai/shared/8b71ba27-fdd2-4201-82db-406ae75e1fc0

SELECT
    "zoekt_indices".*
FROM
    "zoekt_indices"
WHERE
    "zoekt_indices"."state" = 1

https://console.postgres.ai/shared/6c268398-9ae7-4cea-be95-e22b567bfe5b

SELECT
    "zoekt_indices".*
FROM
    "zoekt_indices"
    INNER JOIN "zoekt_repositories" ON "zoekt_repositories"."zoekt_index_id" = "zoekt_indices"."id"
WHERE
    "zoekt_indices"."state" = 1
    AND "zoekt_indices"."id" NOT IN ( SELECT DISTINCT
            "zoekt_repositories"."zoekt_index_id"
        FROM
            "zoekt_repositories"
        WHERE
            "zoekt_repositories"."state" != 10)

Related to #442881 (closed)

Edited by Ravi Kumar

Merge request reports