Convert the FFI Implementation of Zoekt Indexer / Knowledge Graph to HTTP

Context

As part of #79 (closed), we have added the Knowledge Graph Full Indexing functionality to Zoekt Indexer

As part of gitlab-org/rust/knowledge-graph#168 (closed), we have decided to adopt a multi-process (multi-container) services architecture instead of FFI.

Proposal

• move full indexing logic under GKG-webserver service (this service will be created in scope of gitlab-org/rust/knowledge-graph#191 (closed))
• when zoekt-orchestrator receives an "index repo" task, it fetches the repository through Gitaly and stores it in a directory
• it sends a HTTP request to GKG-indexer to index the repository
• for now an env variable GITLAB_ZOEKT_GKG_INDEX_URL on zoekt side will be used to decide if zoekt should run in zoekt-orchestrator mode (related discussion - gitlab-org/cloud-native/charts/gitlab-zoekt!126 (comment 2724227653))

References

• MR for updated design document gitlab-com/content-sites/handbook!15644 (merged)
• design doc section https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/knowledge_graph/#creating-or-updating-a-repository