Part 3 of adding graph database indexing support to Zoekt Indexer

Omar Qunsulrequested to merge
79-graph-zoekt-indexer-to-create-graph-part-3 into main

What does this MR do and why?

Part 3 of adding graph database indexing support to Zoekt Indexer

Addressing: #79 (closed)

Plan

  • Part 1 !589 (merged): Adding parsing of Knowledge Graph tasks.
  • Part 2 !590 (merged) Adding Graph Indexer + Graph DB Client**
  • Part 3: (This MR) Adding clients for Knowledge Graph Bindings and Graph DB implementation. Adding support for both compiling with and without knowledge graph. Without is the default not

You can see the MR that has all the three parts: The MR that has Parts (1-3) !563 (closed)

Context

As Zoekt Indexing tasks, we have added to the Rails App Knowledge Graph indexing tasks that are also exposed on the same endpoint as Zoekt Indexing tasks. See this MR. This MR implements the first step of pulling and parsing the tasks. Not's not indexing or creating graph databases yet on the Zoekt nodes.

How to validate Locally

  • Checkout this repo and run make build-unified
  • Enable the FF knowledge_graph_indexing on some project locally in GDK
  • Enable Knowledge Graph service on all the Zoekt Nodes via Rails Console Search::Zoekt::Node.update_all(services: [0, 1])
  • In GDK stop all the zoekt-indexer instances: gitlab-zoekt-indexer-development-1 and gitlab-zoekt-indexer-development-2. Run the process manually in a shell ./gitlab-zoekt-indexer/bin/gitlab-zoekt indexer -index_dir zoekt-data/development/index -listen :6080 -secret_path /Users/omar/gdk/gitlab-shell/.gitlab_shell_secret -self_url "http://localhost:6080" -search_url "http://localhost:6090" -gitlab_url http://gdk.test:3000. Make sure to change to change the paths.
  • (ALTERNATIVE) Keep gitlab-zoekt-indexer-development-1 running after you restart it. And run gdk tail gitlab-zoekt-indexer-development-1 to see the output.
  • You should see this in the output
{"time":"2025-08-14T11:39:39.661467+02:00","level":"INFO","msg":"start IndexRepository","namespace_id":102,"force":false}
{"time":"2025-08-14T11:39:39.696951+02:00","level":"INFO","msg":"Indexing repository started"}
{"time":"2025-08-14T11:39:39.776562+02:00","level":"INFO","msg":"Indexing repository finished","result":0}
{"time":"2025-08-14T11:39:39.777303+02:00","level":"INFO","msg":"finish IndexRepository","namespace_id":102,"force":false,"indexTime":0.115830417}
  • In the Zoekt Data directory in GDK, you should see some Graph database files generated. Example:
omar ~/gdk/zoekt-data/development/index $ ls knowledge_graph_data/000/000/102.db 
database.kuzu	metadata.json
Edited by Omar Qunsul

Merge request reports