## Context and Rationale Our AI product offerings are expanding, **not** just in features, but also in **which AI providers/models** a customer can use. Some add-ons grant access to **features** (often seat-based), while other add-ons grant **entitlement** to certain **providers** (providers like `amazonq` or `self-hosted-models`) - often without seat assignments. - **Example:** (https://gitlab.com/gitlab-org/fulfillment/meta/-/issues/2178+): > 1. _Keep existing Duo SKUs as-is, sold to any customer who wants Duo:_ > 1. _`GitLab Duo Pro` sold at $14 (new price)_ > 2. _`GitLab Duo Enterprise` sold at $39_ > 2. _Create two new add-on(s) SKU that provides a GitLab Duo customer with the ability to self-host:_ > 1. _`GitLab Duo Self-hosting - Duo Enterprise` sold at $10_ > 3. _Customers buy Duo SKU always, and add on self-hosting SKU if they want to self-host._ > 4. _Customers buy the self-hosting SKU must purchase seat qty equal to their Duo Enterprise (or Duo Pro) seat qty._ It is fair to say that: - **Gitlab Duo Pro** (seat-based) → unlocks the `duo_chat` feature. - **Gitlab Duo Self-Hosting** (entitlement-based) → unlocks **Self-Hosted Models** (SHM) as a provider. A user who has **both** “Duo Pro” **and** “Duo Self-Hosting” can run the `duo_chat` feature **on** SHM. We need to let users combine one or more seat-based features with one or more providers/models. For instance, a user might have a seat for duo_chat (from “Duo Pro”) and the right to use “SHM” (from “ DUO Self-Hosting”), so they can run `duo_chat` on “SHM.” ## Why We Need This Separation Traditionally, we tied a **single add-on** to a feature like `duo_chat`, covering **both** the user’s permission to use the feature (seat assigment) **and** provider/model (SHM or Duo). However, with multiple providers: 1. **Feature vs. Provider** - **Features** such as `duo_chat` or `code_generation` may still be seat-based (e.g., “Duo Pro,” “AmazonQ Pro”). - **Providers/Models** like “SHM” or “AmazonQ” can require a separate add-on, but **often without** seat licensing. (_This can be probably expanded to support model shifting as well._) 2. **Single Feature, Multiple Providers** - Users may run the **same** feature (e.g., `duo_chat`) against **different** providers/models (SHM, AmazonQ, Anthropic, etc.) if they have the relevant **provider** add-ons. 3. **Add-On Separation** - One add-on is responsible for granting seat-based access to the **feature**. - Another add-on (if needed) grants **entitlement** to a specific **provider** or model. - This doesn’t mean the feature is tied to multiple seat-based add-ons; rather, the feature’s seat-based add-on can be **combined** with a provider add-on to enable “Duo Chat” over “SHM,” for example. 4. **Avoiding Duplication** in our configuration and scopes - We don’t want a unique add-on (or seat-based scope) for every feature–provider combo (e.g., shm_duo_chat, amazonq_duo_chat). - We want one scope per feature and one provider entitlement scope/claim per model/provider. ## Core Requirement **We need to manage and enforce these two distinct dimensions**—feature access (seat-based) vs. provider/models entitlement — **without** duplicating logic for each possible combination. ### Product Use Cases and Examples #### 1. “Duo Chat” With Self-Hosted Models 1. **Duo Pro** Add-On - Seat-based. - Gives users the `duo_chat` feature (scope), assuming they have a seat. 2. **Self-Hosting Pro** Add-On - Entitlement-based (no seats). - Grants access to the “SHM” provider. A user assigned a **Duo Pro** seat has the `duo_chat` **feature**. If the organization also has **Duoi Self-Hosting** add-on, they can configure `duo_chat` to use “SHM” as the backend. **Key Point:** - The user must pass **both** a feature scope check (`duo_chat`) and a provider entitlement check (`shm`) to succeed. #### 2. “Duo Chat” With AmazonQ 1. **AmazonQ Pro** Add-On - Seat-based. - Unlocks the `duo_chat` feature for assigned seats **and** grants the “amazonq” provider entitlement. In this scenario, a single add-on covers **both** the seat-based feature scope and the provider itself. However, the **provider** portion doesn’t require a separate seat check—only the feature does. **Key Point:** - The seat check is relevant to the `duo_chat` feature. - The “amazonq” provider check is simply a yes/no entitlement based on the add-on. ## How would permission framework work We separate **feature-level access** (seat-based) from **provider-level** entitlements (often seat-free): 1. **Feature Access**: “Do I have a seat for `duo_chat`?” 2. **Provider Access**: “Is `shm` or `amazonq` or any other provider enabled by an add-on or the license plan this organisation has purchased?” ### End-User UI / Application Checks - **Seat Check** (Feature Scope): - If the user wants `duo_chat`, the system verifies if they hold a seat in an add-on that unlocks `duo_chat` (e.g., Duo Pro, AmazonQ Pro). - **Provider Check** (Entitlement Only): - If the user selects a provider (e.g., `shm`), the system verifies that the org owns an add-on that grants that provider. - No seat assignment is needed here, even if the add-on is seat-based. The seat concept is only for **features**. ### AI Gateway (Final Enforcement) To prevent tampering, the AI Gateway enforces the same logic via a **JWT** with: - **`"scopes"`** → e.g., `["duo_chat", "code_generation", ...]` - **`"providers"`** → e.g., `["duo","shm"]` **At runtime**: If a user requests `duo_chat` with `provider=shm`, the AI Gateway requires `"duo_chat"` in `scopes` and `"shm"` in `providers`. Otherwise, it returns `403 Forbidden`. ## **Analytics & Tracking** GitLab aims to provide AI-powered features across multiple models and deployment configurations. To keep our tracking extensible and avoid duplication, we recommend: 1. **Model and provider agnostic feature identifiers**: - Don’t embed provider/model names in the feature name (e.g., avoid `amazon_q_duo_chat`). - Instead, capture **feature** (e.g., `duo_chat`) and **provider** (e.g., `amazonq`) as **separate attributes** in tracking events. 2. **Future-proof design**: - Tracking new models or integrations should not require new unit primitives; We will simply pass a new **provider/model** attribute. - This approach **extends** seamlessly to Duo Pro/Enterprise or any upcoming product SKUs. **Example:** ```python internal_event_client.track_event( "ai_feature_used", category="ai_usage", provider="self-hosted-models" ) ``` By **separating** features from providers in our event data, we gain: - **Clear, scalable analytics**: Track feature usage and provider adoption independently. - **Simplified maintenance**: No need to create multiple primitives for every feature–provider combination. - **Greater flexibility** for future partnerships, additional AI models, or changing product offerings. ## Why This Separation Works 1. **Avoids Multiple Seat Checks**: - Only features check seats. Providers are a simple “yes/no” entitlement. 2. **No Duplication of “Unit Primitives”**: - We don’t create a separate feature scope for every provider combination (`shm_duo_chat`, `amazonq_duo_chat`, etc.). Instead, we keep one scope per feature and one per provider. 3. **Scalable for Future Providers**: - Adding a new model vendor or AI provider is as simple as adding a new `provider`/`unit_prmitive` to the system. - The seat-based feature logic remains unchanged. 4. **Consistent UX and Security**: - Front end can hide or disable providers the customer lacks. - The AI Gateway confirms entitlements in the JWT, preventing unauthorized usage. ## Conclusion Our requirement is **not** that “multiple add-ons exist for the same feature.” Rather, we’re **separating** the seat-based feature from the provider entitlement. A user may **need** both an add-on that provides seats for the feature (e.g., Duo Chat) **and** an add-on that grants a particular provider (e.g., Self-Hosting). By splitting these concerns—**seat-based scopes** vs. **provider entitlements**—we achieve a fain-grained permission control and better internal event tracking. This would also allow us to avoid having unit primitive duplication duo to the possible combinations of current and future AI offerings. ## Relevant links and discussions: - https://gitlab.com/gitlab-com/ops-sub-department/aws-gitlab-ai-integration/integration-motion-planning/-/issues/407# - https://gitlab.com/gitlab-org/fulfillment/meta/-/issues/2178 - https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/merge_requests/2042#note_2372236569 - Model switching blueprint: https://gitlab.com/gitlab-com/content-sites/handbook/-/merge_requests/11970/diffs