NamespaceIndexerWorker should not use `all_projects`
Background
Currently, Search::Zoekt::NamespaceIndexerWorker
uses namespace.all_projects
to iterate over all projects. The problem is that this is a very expensive query for a large namespace
Proposal
I believe that we should instead iterate 1 level at a time and query more jobs to process subgroups.