This epic is to capture work to self-hosted instance administrator of **GitLab Duo Self-Hosted** on Self-Managed to bring any model or platform for use with GitLab Duo features. This iteration draws a line between supported models (to include GitLab technical support) and compatible models (which do not include Gitlab support). This iteration focuses on enabling open model choice using compatible platforms and specs, but stipulates that GitLab will not provide technical assistance for those models and configurations that are not explicitly supported. For more information on supported vs compatible models strategy, see https://gitlab.com/gitlab-org/gitlab/-/issues/539028+ ### Background Self-Hosted customers have a variety of requirements for data residency, feature quality, and cost that drive the need for model selection. Custom Models is not able to validate every platform and feature for LLMs, and risks falling behind competitor offering for supported models and platforms. We already have seen at least [one customer ](https://gitlab.com/gitlab-org/product-feedback/-/issues/142)who has experimented with using supported model family prompts with unsupported models (i.e. the Mistral model family prompt to support Llama 3.1 before it was GA). With the rapid fire release of new models, it is impossible for Custom Models to support every model on every platform. This epic proposes create two tiers of models for use with Self-Hosted models, compatible vs supported. Supported models are those that are available on supported platforms (such as AWS Bedrock or Azure) that are reasonably expected to work with Gitlab Duo features but have not been tested and validated for feature use by Custom Models. Give customers the flexibility to choose compatible models, but GitLab will not provide technical support for compatible but not supported model selection. ### Key Decisions * **Compatible vs supported**: Give customers the flexibility to choose compatible models, but GitLab will not provide technical support for compatible but not supported model selection. * **Flexibility over quality:** Expect (but attempt to minimize) that non-supported models may not reach the quality standards of supported models. * **Optimize for general model use**: Provide optimizations in terms of feature quality for non-supported models. * **Customer validation vs Gitlab validation**: Shift the onus for validation of quality, cost, and data residency to customers in order to provide maximum flexibility. * **Supported platform over BYO platform**: Model platforms have different connection requirements that may not be currently supported with Self-Hosted configuration UI. This iteration focuses on BYOM over BYOP to enable greater model flexibility on supported platforms. ### Definition of Done * Self-Hosted admins can access centralized documentation indicating what models and platforms are confirmed compatible * Self-Hosted admins can leverage any inference platform that is OpenAI API compatible for model inference * Self-hosted admins can configure supported, compatible, or other model endpoints for use with GitLab Duo Self-Hosted features * When using a non-supported model, admins will have the option to choose an "other" prompt during configuration that offers general feature functionality and is not-specific to any model. * Once admins have configured a compatible models, they can configure it with any Duo feature in the Self-Hosted UI as normal; all GitLab Duo features available on Self-Hosted will be available for model selection with the configured compatible model * Compatible models will be tagged or otherwise visibly marked in the UI to distinguish them from supported models. * Customers will be responsible for validating the quality, cost, and data residency of compatible models; compatible models will not receive technical support from GitLab. * Customers have a mechanism to contribute to community documentation to add new compatible models and platforms, 'vote' for existing compatible models or platforms, or otherwise provide feedback on model compatability