Skip to content

Indexing does not index subgroup associated data

Background

The index_namespaces task in Search::RakeTaskExecutorService uses this SQL:

      Namespace.by_parent(nil).each_batch do |batch|
        batch = batch.include_route
        batch = batch.select(&:use_elasticsearch?)

Sub-groups can be added for limiting and those would be missed by this because it only looks at top level namespaces.

Update: This bug affects all subgroup association data regardless of limiting setting

Proposal

Fixing in a slightly different way than the proposal below. The fix will walk through each root namespace and descendants in the ElasticNamespaceIndexerWorker and queue indexing for group wikis and associated database data.

Old Proposal

Use all namespaces if the limiting setting is enabled, otherwise use root level namespaces for indexing.

diff --git a/ee/app/services/search/rake_task_executor_service.rb b/ee/app/services/search/rake_task_executor_service.rb
index 7b6f785fde38..486d0acbc2e3 100644
--- a/ee/app/services/search/rake_task_executor_service.rb
+++ b/ee/app/services/search/rake_task_executor_service.rb
@@ -313,12 +313,9 @@ def index_users
     end
 
     def index_namespaces
-      Namespace.by_parent(nil).each_batch do |batch|
-        batch = batch.include_route
-        batch = batch.select(&:use_elasticsearch?)
-
+      namespaces_in_batches do |namespaces|
         ElasticNamespaceIndexerWorker.bulk_perform_async_with_contexts(
-          batch,
+          namespaces,
           arguments_proc: ->(namespace) { [namespace.id, :index] },
           context_proc: ->(namespace) { { namespace: namespace } }
         )
@@ -538,6 +535,21 @@ def projects_in_batches
       count
     end
 
+    def namespaces_in_batches
+      namespaces = if ::Gitlab::CurrentSettings.elasticsearch_limit_indexing?
+                     Namespace.all
+                   else
+                     Namespace.by_parent(nil)
+                   end
+
+      namespaces.each_batch do |batch|
+        batch = batch.include_route
+        batch = batch.select(&:use_elasticsearch?)
+
+        yield batch
+      end
+    end
+
     def helper
       @helper ||= ::Gitlab::Elastic::Helper.default
     end

The following discussion from !161475 (merged) should be addressed:

Edited by Terri Chu