Add instance setting to auto-index root namespaces for Orbit

What does this MR do and why?

Adds an instance setting to automatically enroll top-level groups into Orbit indexing on GitLab Self-Managed, similar to Zoekt. This gives administrators an "index everything" option without using the Rails console or enrolling groups one at a time. Turning the setting off keeps existing enrollments.

References

Closes #606375 (closed)

Screenshots or screen recordings

Before After
image image

After enabling the setting:

image

Database

The worker batches top-level groups in ID ranges of 1,000 rows before applying the anti-join within each bounded range. The outer batch query uses the partial index index_groups_on_parent_id_id, and the anti-join uses the unique index index_knowledge_graph_enabled_namespaces_on_root_namespace_id.

The query below shows one batch with a reduced batch size of two to demonstrate both ID boundaries generated by EachBatch:

SELECT "namespaces"."id"
FROM "namespaces"
LEFT OUTER JOIN "knowledge_graph_enabled_namespaces" "knowledge_graph_enabled_namespace"
  ON "knowledge_graph_enabled_namespace"."root_namespace_id" = "namespaces"."id"
WHERE "namespaces"."type" = 'Group'
  AND "namespaces"."parent_id" IS NULL
  AND "namespaces"."id" >= 22
  AND "namespaces"."id" < 27
  AND "namespaces"."parent_id" IS NULL
  AND "knowledge_graph_enabled_namespace"."root_namespace_id" IS NULL;
Nested Loop Anti Join  (cost=0.29..4.74 rows=1 width=8) (actual time=0.198..0.200 rows=2 loops=1)
  Buffers: shared hit=6 read=1
  ->  Index Only Scan using index_groups_on_parent_id_id on namespaces  (cost=0.13..2.36 rows=1 width=8) (actual time=0.194..0.195 rows=2 loops=1)
        Index Cond: ((parent_id IS NULL) AND (parent_id IS NULL) AND (id >= 22) AND (id < 27))
        Heap Fetches: 2
        Buffers: shared hit=2 read=1
  ->  Index Only Scan using index_knowledge_graph_enabled_namespaces_on_root_namespace_id on knowledge_graph_enabled_namespaces knowledge_graph_enabled_namespace  (cost=0.15..2.37 rows=1 width=8) (actual time=0.001..0.001 rows=0 loops=2)
        Index Cond: (root_namespace_id = namespaces.id)
        Heap Fetches: 0
        Buffers: shared hit=4
Planning:
  Buffers: shared hit=57
Planning Time: 0.343 ms
Execution Time: 0.210 ms

How to set up and validate locally

  1. Configure Orbit and enable the knowledge_graph feature flag.
  2. Sign in as an administrator and visit /admin/orbit.
  3. Select Index root namespaces automatically, then Save changes.
  4. Run Analytics::KnowledgeGraph::AutoIndexWorker.new.perform in the Rails console.
  5. Verify that top-level groups are enrolled and existing enrollments remain after clearing the setting.
  6. Run bundle exec rake gitlab:orbit:info and check the setting value.
Agent context - implementation decisions and verification

Implementation notes

  • Current master already has the Admin > Orbit page and gitlab:orbit:info backed by Analytics::KnowledgeGraph::InfoService. Unlike !246563 (closed), this MR extends those components and keeps the registry under Analytics::KnowledgeGraph instead of adding parallel Orbit classes and a bounded context.
  • Orbit accepts top-level group namespaces, so the scope excludes personal namespaces.
  • Disabling the setting never deletes enrollments.
  • The JSONB column and object constraint use separate migrations so the constraint uses the standard non-transactional helper.
  • The request spec replaces the reference MR's controller spec.
  • Review feedback from the reference MR is included: the help path omits .md, docs use current shortcodes and one sentence per line, the setting definition and OpenAPI entries are present, worker deltas are exact, and locale/gitlab.pot is regenerated.

Verification

  • Focused RSpec suite: 1,512 examples, 0 failures, 5 existing pending examples.
  • RuboCop on all changed Ruby files.
  • HAML lint, Markdown lint, Vale, OpenAPI validation, gettext extraction, queue generation, and pre-push checks.
  • Migration rollback and reapply in GDK.

The info task reports:

Deployment:                           Self-managed
License :orbit available:             yes
Index root namespaces automatically:  yes

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

  • The setting defaults to off and is gated to self-managed, licensed instances.
  • The worker is idempotent, add-only, batched, and safe against concurrent inserts.
  • The setting is documented in product docs and the application settings OpenAPI schema.
  • Generated schema, queue, and translation files are updated.
  • Run db:gitlabcom-database-testing before database review.
Edited by Dmitry Gruzd

Merge request reports

Loading
Loading