Introduce distinct embeddings endpoints for search and indexing operations
<!-- Please read this! Before opening a new issue, make sure to search for keywords in the issues: - https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/issues and verify the issue you're about to submit isn't a duplicate. --> ## Problem to solve From https://gitlab.com/groups/gitlab-org/-/work_items/21437#note_3196131142: > It probably makes sense to split traffic between two endpoints for search and indexing See https://gitlab.com/gitlab-org/gitlab/-/work_items/594971#note_3413546252 for the summary of the discussion from https://gitlab.com/groups/gitlab-org/-/work_items/21437#note_3196131142. ## Proposal - add new endpoints specifically for indexing and search in AIGW: - current embeddings endpoint: `/v1/embeddings/code_embeddings` - split endpoints for specific functionalities: `/v1/embeddings/code_embeddings/index` & `/v1/embeddings/code_embeddings/search` - use the distinct endpoints in Rails depending on the purpose of the invocation ## Further details <!-- Include examples, use cases, benefits, goals, or any other details that help us understand the problem better. --> ## Links / references <!-- Select a type --> <!-- /label gitlab-org/modelops/applied-ml/code-suggestions/ai-assist~2278648 --> <!-- /label gitlab-org/modelops/applied-ml/code-suggestions/ai-assist~10230929 --> <!-- /label gitlab-org/modelops/applied-ml/code-suggestions/ai-assist~15119514 -->
issue