Skip to content

Expose knowledge graph tasks

What does this MR do and why?

  • exposes knowledge graph tasks to be processed by zoekt nodes
  • processes callback's payload for knowledge graph tasks

Additional context: knowledge graph tasks will be processed by zoekt-indexer service running on zoekt nodes. Zoekt-indexer service periodically fetches a batch of tasks to process. We combine tasks of both types in the batch.

Related to #547436 (closed)

How to set up and validate locally

  1. set up zoekt (https://gitlab-org.gitlab.io/gitlab-development-kit/howto/zoekt/)
  2. create some zoekt and knowledge graph tasks in rails console
# disable indexing temporarily so testing tasks are not being processed
gdk stop gitlab-zoekt-indexer-development-1 gitlab-zoekt-indexer-development-2

Feature.enable(:knowledge_graph_indexing)

# create graph tasks
ns = Project.find(7).project_namespace
Ai::KnowledgeGraph::EnabledNamespace.create!(namespace_id: ns.id)
Ai::KnowledgeGraph::EnabledNamespace.first.replicas.create!(zoekt_node: Search::Zoekt::Node.first, namespace_id: ns.id, retries_left: 3)
# create one or more graph tasks:
Ai::KnowledgeGraph::Replica.last.tasks.create!(zoekt_node_id: Search::Zoekt::Node.first.id, namespace_id: Ai::KnowledgeGraph::Replica.last.namespace_id, task_type: 0)

# create zoekt tasks
zns = Group.find_by_path('gitlab-org')
ens = ::Search::Zoekt::EnabledNamespace.create!(namespace: zns)
idx = ::Search::Zoekt::Index.create!(zoekt_enabled_namespace: ens, namespace_id: zns.id, zoekt_node_id: Search::Zoekt::Node.first.id)
zrepo = ::Search::Zoekt::Repository.create!(zoekt_index: idx, project_id: zns.projects.first.id, project_identifier: zns.projects.first.id)
# create one or more zoekt tasks (for more tasks make sure to use different namespace_id):
::Search::Zoekt::Task.create(zoekt_node_id: Search::Zoekt::Node.first.id, zoekt_repository_id: zrepo.id, task_type: 0)
  1. verify list of tasks returned for the node:
::Search::Zoekt::TaskPresenterService.execute(Search::Zoekt::Node.first)

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Jan Provaznik

Merge request reports

Loading