Zoekt::IndexerWorker should skip projects with no or empty repository
As discovered in gitlab-com/gl-infra/production#8739 (comment 1359699471) . There are 2 scenarios where projects should be skipped for indexing:
- Projects without a repository like https://staging.gitlab.com/gitlab-org/grafana-dashboards
- Projects with empty repositories like https://staging.gitlab.com/gitlab-org/prometheus-client-mmap
We should skip indexing these projects to save wasted work by Zoekt but to also reduce noise in our error logs for Zoekt.
Technical details
It seems that for both of these cases we have project.repository.empty? == true. But this call is slow because it calls Gitaly. I hope that there is a cheaper way for (1) which is a more common case. I think GitLab should know that a project has no repository at all without needing to ask Gitaly. We should also investigate for the cases where we need to call Gitaly what is the cheapest way to check if a repository is empty. Since this stuff is implemented in the GitLab UI (if you load those projects) there should be some examples in code for how to do this.