## **Summary** This epic outlines GitLab's approach to enabling model selection capabilities for GitLab Duo AI features on GitLab.com, as well as GitLab Duo Self-Hosted on Self-Managed. * Currently, GitLab Duo on GitLab deployments using the .com AI Gateway rely on predefined default models for Duo features. This creates challenges for customers who need specific models for compliance reasons or who want to optimize for particular characteristics. * The only route for customers to choose specific models for use is via Gitlab Duo Self-Hosted, which is currently only available to self-managed customers. This epic establishes an iterative framework to address the needs of customers across all Gitlab surfaces. This epic proposes a bifurcated UX for customers using the GitLab.com AI Gateway and those using a Self-Hosted AI Gateway. This epic will be implemented in a phased approach. For more information on Custom Model's model switching strategy, see https://gitlab.com/gitlab-org/gitlab/-/issues/539029+. ## **Definitions** For the purpose of this document, the following definitions apply. * **Duo Feature**: this refers to a top-level Duo feature that may or may not include sub-features. Top-level features may or may not themselves be unit primitives. The top-level features include Code Suggestions, Duo Chat, as well as other stand-alone features that are not contained within Code Suggestions and Chat – for example, Vulnerability Resolution. * **Duo Sub-Feature**: this refers to LLM-powered functionalities captured within the top-level Duo Feature.  Sub-features can be understood as [unit primitives.](https://gitlab.com/gitlab-org/cloud-connector/gitlab-cloud-connector/-/tree/254c06bd241d7004c33b412588bbe9e205bc8f08/config/unit_primitives) * **Cascading settings**: this refers to the behavior in which controls established at higher levels of the hierarchy impose boundaries on available choices at lower levels. Higher-level configurations cannot be overridden by lower-level administrators, unless explicitly allowed. For example, a top level hierarchy that sets Duo to “Off by default” enables lower level hierarchies to enable Duo, but “Always off” cannot be overridden at lower levels. * [**Hybrid model switching**](https://gitlab.com/groups/gitlab-org/-/epics/15829): this refers to the ability of Self-Hosted customers to configure either the GitLab.com AI Gateway with its default AI Vendors OR the Self-Hosted AI Gateway with its customer-configured models interchangeably.  ## **Iteration Plan** ### [Iteration 1](https://gitlab.com/groups/gitlab-org/-/epics/17570): Namespace-level Configuration on GitLab.com via the .com AIG (:white_check_mark: delivered by 19 June 2025) This iteration enables namespace owners on GitLab.com who are using the **GitLab.com AI Gateway** to select one model from among supported GitLab AI Vendor models at the namespace level for each GitLab Duo feature/sub-feature. Supported models are defined as the existing default model(s) for that feature on GitLab.com, as well as Claude 3.5 Sonnet. Namespace owners will also be able to disable each feature/sub-feature. * **What**: Namespace-level model selection on GitLab.com * **Why**: This iteration will meet the requirements of a customer escalation, and provide general functionality that can be leveraged by GitLab.com customers who are unable for various compliance or regulatory reasons to use default models selected by GitLab. ### [Iteration](https://gitlab.com/groups/gitlab-org/-/epics/17192): Hybrid Models Selection on Self-Hosted (:white_check_mark: delivered to beta in 18.3) This iteration enables instance administrators of **GitLab Duo Self-Hosted** on Self-Managed to configure both fully self-hosted models, as well as GitLab AI Vendor models for use with Duo features –[ hybrid configurations](https://gitlab.com/groups/gitlab-org/-/epics/17192). * **What**: Hybrid configuration, enabling customers to route between both the Self-Hosted AI Gateway and the GitLab.com AI Gateway * **Why**: Self-Hosted customers seek to minimize feature calls to self-hosted models where possible, and leverage GitLab AI Vendor models for specific groups and projects that do not have stringent security or data residency requirements. ### [Iteration](https://gitlab.com/groups/gitlab-org/-/epics/19144): Instance-level Configuration on Self-Managed and Dedicated (:white_check_mark: delivered in 18.4) This iteration enables instance administrators on Self-Managed and Dedicated who are using the **GitLab.com AI Gateway** to choose a default model at the instance level. GitLab.com instance admins will be able to select a different default model for each feature/sub-feature from among supported GitLab AI Vendor models. Supported models are defined as the default model(s) for that feature on GitLab.com, as well as Claude 3.5 Sonnet. Administrators will also be able to disable each feature/sub-feature. * **What**: Instance-level model selection on Self-Managed and Dedicated * **Why**: This iteration will expand the functionality of GitLab.com to Self-Managed and Dedicated customers using the GitLab.com AIG and provide general functionality that can be leveraged by GitLab.com customers who are unable for various compliance or regulatory reasons to use default models selected by GitLab. ### [Iteration](https://gitlab.com/groups/gitlab-org/-/epics/18374): User-Level Model Selection via the .com AIG (in progress) This iteration will focus on bringing Model Selection to end-users on GitLab.com, Dedicated, and Self-Managed using the GitLab.com multi-tenant AI Gateway. This iteration will enable end-users to select from allowable models in the Chat window and IDE if their namespace administrator HAS NOT pinned a preferred model for the namespace. * **What**: End-user model selection configuration * **Why**: This iteration extends the flexibility of Model Selection to enable end-users to choose from among their preferred, allowable models while still preserving key namespace administrator governance requirements. * **Iterations**: * https://gitlab.com/groups/gitlab-org/-/epics/19251+s (:white_check_mark: delivered in 18.4) * https://gitlab.com/groups/gitlab-org/-/epics/19344+s * https://gitlab.com/groups/gitlab-org/-/epics/19345+s * https://gitlab.com/groups/gitlab-org/-/epics/19346+s ### [Iteration](https://gitlab.com/groups/gitlab-org/-/epics/17743): BYO Model and Platforms on Self-Hosted (in progress) This iteration will facilitate instance administrator of **GitLab Duo Self-Hosted** on Self-Managed to bring any compatible model or model for use with GitLab Duo features. This iteration draws a line between supported models (to include GitLab technical support) and compatible models (which do not include Gitlab support). This iteration focuses on enabling open model choice using compatible platforms and specs, but stipulates that GitLab will not provide technical assistance for those models and configurations that are not explicitly supported. For more information on supported vs compatible models strategy, see https://gitlab.com/gitlab-org/gitlab/-/issues/539028+ * **What**: Facilitation for compatible models and * platforms * **Why**: GitLab customers are heterogeneous and have a broad mix of requirements that make it difficult to impossible for GitLab to support all models and platforms for all features. Customers are already trying out and using non-supported configurations, and this iteration seeks to enable that without adding additional GitLab overhead to develop, validate, and maintain all those potential customer choices.This shifts the onus of validation to the customer for compatible models, but provides greater flexibility in model selection for use with Gitlab Duo features. ### [Iteration](https://gitlab.com/groups/gitlab-org/-/epics/17741): Cascading Control to Group-level Configuration This iteration enables namespace admins using the GitLab.com AIG and instance administrators of **GitLab Duo Self-Hosted** on Self-Managed to use cascading controls for model governance. * On Duo Self-Hosted, instance admins would declare their self-hosted models as they currently do, select multiple allowable models for each feature/sub-feature, and choose a default model. * GitLab.com, Dedicated, and Self-Managed customers using the GitLab.com AIG would select multiple allowable models for each feature/sub-feature and choose a default model. Those selections will then cascade to top-level group owners, who will be able to select a different default model from among those enabled by the instance administrator. Supported models are defined as those configured for use with Duo by the instance administrator. * **What**: Instance and top level group model selection * **Why**: This iteration enables GitLab administrators to maintain model governance and top level controls, while also allowing flexibility of choice to their user base. ### Iteration: User-Level Model Selection on Self-Hosted This iteration will extend user-level configuration to Self-Hosted. Building off work in iteration 6, this iteration will allow GitLab instance admins to provide a list of instance-administer approved and available models for use with Duo features. In this iteration, end-users will be able to choose from among the list of available models (to include both Self-Hosted and GitLab AI Vendor models) when the instance or group administrator has NOT pinned a specific model for use. * **What**: User-level model selection on Self-Hosted * **Why**: This iteration will bring user-level configuration parity to Self-Hosted users ### [Iteration](https://gitlab.com/groups/gitlab-org/-/epics/17744): Cascading Controls to Sub Group-level Configuration This iteration will further cascade configuration options for customers using the GitLab.com AIG and **GitLab Duo Self-Hosted** to Group level owners. As with iteration 3, group-level owners will be able to choose from among supported models enabled by administrators higher in the hierarchy. Supported models are defined as GL AI Vendor model option made available by the Gitlab.com AIG or configured for use with Duo by the Self-Hosted instance administrator. * **What**: model selection at the sub-group level of administrator enabled AI models * **Why**: Customers seek to specify models for specific groups and projects, particularly paired with the capacity to use hybrid configurations and BYO models on Gitlab Duo Self-Hosted. This iteration will further extend the ability of customers to configure their specific models for specific contexts and Duo feature use-cases. ### Iteration: Cascading Controls to Project-level Configuration on Self-Hosted This iteration will further cascade configuration options on **GitLab Duo Self-Hosted** to Project level owners. As with earlier cascading iterations, project-level owners will be able to choose from among supported models enabled by administrators higher in the hierarchy. Supported models here are defined as those configured for use with GitLab Duo Self-Hosted by the instance administrator, and enabled by the namespace or top-level group administrator. * **What**: model selection at the project level of administrator enabled AI models * **Why**: Customers seek to specify models for specific projects, particularly paired with the capacity to use hybrid configurations and BYO models. This iteration will further extend the ability of customers to configure their specific models for specific contexts and Duo feature use-cases. ### [Iteration](https://gitlab.com/gitlab-org/gitlab/-/issues/566609): Organization-Level Model Selection with Cascading Controls to Namespace via the .com AIG This iteration will provide higher level control to organization-level admins on Gitlab.com, Dedicated, and Self-Managed who are using DAP via GitLab.com AIG/WFS. Controls will cascade to child Organization and Namespace levels. * **What**: Organization-level model selection configuration * **Why**: This iteration will provide top-level controls for model selection on Gitlab.com and for Self-Managed and Dedicated customers using the Gitlab.com AIG/WFS ### Resources * [Model Switching: Before and After](https://docs.google.com/document/d/1Z-pXx4mvhsA9P5xDsSRwTnYDNVjUmLcGVIEbVg_uuZE/edit?tab=t.0)