Send v2 AST search request to zoekt webserver

What does this MR do and why?

Send v2 AST search request to zoekt

This is not functionally different than the current requests (we currently send a very large list of repo ids), but this will allow us to send more complicated queries in the next iteration.

We plan to start sending filters for traversal ids, repo access level, visibility level, etc.

AI Summary

This code introduces a new search feature for GitLab's code search functionality. The main change adds a feature flag that allows switching between two different ways of building search requests - the old method and a new "AST payload" approach.

The new approach creates a more structured way to build search queries using dedicated classes that can handle different types of search filters (like searching by text patterns, repository IDs, or combining multiple conditions). It includes a new system for creating search requests with configurable options like timeouts and result limits.

The code also adds comprehensive test coverage to ensure the new search filters and request building work correctly. The feature is currently disabled by default, allowing for safe testing and gradual rollout. This appears to be preparation for improving how GitLab searches through code repositories, making the search system more flexible and maintainable.

References

Closes #547282 (closed)

This is a continuation of gitlab-zoekt-indexer!516 (merged)

Screenshots or screen recordings

Before	After

How to set up and validate locally

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.