Expose knowledge graph tasks
What does this MR do and why?
- exposes knowledge graph tasks to be processed by zoekt nodes
- processes callback's payload for knowledge graph tasks
Additional context: knowledge graph tasks will be processed by zoekt-indexer service running on zoekt nodes. Zoekt-indexer service periodically fetches a batch of tasks to process. We combine tasks of both types in the batch.
Related to #547436 (closed)
How to set up and validate locally
- set up zoekt (https://gitlab-org.gitlab.io/gitlab-development-kit/howto/zoekt/)
- create some zoekt and knowledge graph tasks in rails console
# disable indexing temporarily so testing tasks are not being processed
gdk stop gitlab-zoekt-indexer-development-1 gitlab-zoekt-indexer-development-2
Feature.enable(:knowledge_graph_indexing)
# create graph tasks
ns = Project.find(7).project_namespace
Ai::KnowledgeGraph::EnabledNamespace.create!(namespace_id: ns.id)
Ai::KnowledgeGraph::EnabledNamespace.first.replicas.create!(zoekt_node: Search::Zoekt::Node.first, namespace_id: ns.id, retries_left: 3)
# create one or more graph tasks:
Ai::KnowledgeGraph::Replica.last.tasks.create!(zoekt_node_id: Search::Zoekt::Node.first.id, namespace_id: Ai::KnowledgeGraph::Replica.last.namespace_id, task_type: 0)
# create zoekt tasks
zns = Group.find_by_path('gitlab-org')
ens = ::Search::Zoekt::EnabledNamespace.create!(namespace: zns)
idx = ::Search::Zoekt::Index.create!(zoekt_enabled_namespace: ens, namespace_id: zns.id, zoekt_node_id: Search::Zoekt::Node.first.id)
zrepo = ::Search::Zoekt::Repository.create!(zoekt_index: idx, project_id: zns.projects.first.id, project_identifier: zns.projects.first.id)
# create one or more zoekt tasks (for more tasks make sure to use different namespace_id):
::Search::Zoekt::Task.create(zoekt_node_id: Search::Zoekt::Node.first.id, zoekt_repository_id: zrepo.id, task_type: 0)
- verify list of tasks returned for the node:
::Search::Zoekt::TaskPresenterService.execute(Search::Zoekt::Node.first)
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Edited by Jan Provaznik