feat: add Knowledge Graph code indexing task on push to default branch

What does this MR do and why?

When code is pushed to a project's default branch, we now create a CodeIndexingTask record so the Knowledge Graph knows what to index.

  • Database table (p_knowledge_graph_code_indexing_tasks): Daily-partitioned with 1-day retention. Stores project ID, git ref, commit SHA and namespace traversal path. Short retention because these are transient work items that once consumed are not needed.
  • Model (Analytics::KnowledgeGraph::CodeIndexingTask): Creates the record after checking the project exists and has knowledge graph indexing enabled on its root namespace.
  • Worker (Analytics::KnowledgeGraph::CodeIndexingWorker): Idempotent, low-urgency Sidekiq worker. Concurrency capped at 100, defers on database health signals. Gated behind knowledge_graph_infra feature flag.
  • Service hook: EE::Git::BranchPushService enqueues the worker on default branch pushes when the feature flag and namespace-level setting are both on.
  • Project methods: #traversal_path builds the full namespace path including the project ID. #knowledge_graph_indexing_enabled? checks for an EnabledNamespace record on the root ancestor.

References

Resolves #593568 (closed)

How to set up and validate locally

  1. Run the migrations and restart Rails:

    cd gitlab-development-kit && gdk restart rails-web rails-background-jobs
  2. Enable the Knowledge Graph feature flag:

    # In rails console: bin/rails c
    Feature.enable(:knowledge_graph_infra)
  3. Enable Knowledge Graph for a namespace:

    namespace = Namespace.find_by_path('your-group')
    Analytics::KnowledgeGraph::EnabledNamespace.create!(namespace: namespace, root_namespace: namespace)
  4. Push a commit to the default branch of a project under that namespace.

  5. Check the table for the inserted task:

    Analytics::KnowledgeGraph::CodeIndexingTask.last

    image.png

Acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Jean-Gabriel Doyon

Merge request reports

Loading
Loading