Add feature flag to optionally index empty zoekt repos

What does this MR do and why?

Add feature flag to optionally index empty zoekt repos

AI Summary

This change introduces a new feature flag called zoekt_index_empty_repos that controls whether the Zoekt search indexing system should process empty repositories (repositories with no code content).

Previously, the system would automatically skip indexing empty repositories and mark them as "done" without processing them. Now, with this feature flag enabled (which is the default behavior going forward), the system will attempt to index even empty repositories, treating them the same as repositories with content.

When the feature flag is disabled, the system reverts to the old behavior of skipping empty repositories. This gives administrators control over whether to spend resources indexing repositories that don't contain any searchable code.

The change also updates the logic to use empty_repo? instead of repo_exists? for more accurate detection of repositories that lack content, and includes comprehensive tests to verify both the new default behavior and the fallback behavior when the feature flag is disabled.

References

Screenshots or screen recordings

Before	After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #561000 (closed)