Add LiteLLM client to handle requests to self-managed OpenAI compatible models
What does this merge request do and why?
Mistral and Mixtral models are expected to be deployed behind OpenAI compatible API server (like vLLM). Let's add a handler for this type of requests.
Related issues:
Edited by Igor Drozdov