[Discussion] Support different providers (SHM, AmazonQ) with unit primitives and cloud connector permission framework

Context and Rationale

Our AI product offerings are expanding, not just in features, but also in which AI providers/models a customer can use. Some add-ons grant access to features (often seat-based), while other add-ons grant entitlement to certain providers (providers like amazonq or self-hosted-models) - often without seat assignments.

  • Example: (https://gitlab.com/gitlab-org/fulfillment/meta/-/issues/2178+):

    1. Keep existing Duo SKUs as-is, sold to any customer who wants Duo:
      1. GitLab Duo Pro sold at $14 (new price)
      2. GitLab Duo Enterprise sold at $39
    2. Create two new add-on(s) SKU that provides a GitLab Duo customer with the ability to self-host:
      1. GitLab Duo Self-hosting - Duo Enterprise sold at $10
    3. Customers buy Duo SKU always, and add on self-hosting SKU if they want to self-host.
    4. Customers buy the self-hosting SKU must purchase seat qty equal to their Duo Enterprise (or Duo Pro) seat qty.

    It is fair to say that:

    • Gitlab Duo Pro (seat-based) → unlocks the duo_chat feature.
    • Gitlab Duo Self-Hosting (entitlement-based) → unlocks Self-Hosted Models (SHM) as a provider.

A user who has both “Duo Pro” and “Duo Self-Hosting” can run the duo_chat feature on SHM.

We need to let users combine one or more seat-based features with one or more providers/models. For instance, a user might have a seat for duo_chat (from “Duo Pro”) and the right to use “SHM” (from “ DUO Self-Hosting”), so they can run duo_chat on “SHM.”

Why We Need This Separation

Traditionally, we tied a single add-on to a feature like duo_chat, covering both the user’s permission to use the feature (seat assigment) and provider/model (SHM or Duo). However, with multiple providers:

  1. Feature vs. Provider

    • Features such as duo_chat or code_generation may still be seat-based (e.g., “Duo Pro,” “AmazonQ Pro”).
    • Providers/Models like “SHM” or “AmazonQ” can require a separate add-on, but often without seat licensing. (This can be probably expanded to support model shifting as well.)

  2. Single Feature, Multiple Providers

    • Users may run the same feature (e.g., duo_chat) against different providers/models (SHM, AmazonQ, Anthropic, etc.) if they have the relevant provider add-ons.

  3. Add-On Separation

    • One add-on is responsible for granting seat-based access to the feature.
    • Another add-on (if needed) grants entitlement to a specific provider or model.
    • This doesn’t mean the feature is tied to multiple seat-based add-ons; rather, the feature’s seat-based add-on can be combined with a provider add-on to enable “Duo Chat” over “SHM,” for example.

  4. Avoiding Duplication in our configuration and scopes

    • We don’t want a unique add-on (or seat-based scope) for every feature–provider combo (e.g., shm_duo_chat, amazonq_duo_chat).
    • We want one scope per feature and one provider entitlement scope/claim per model/provider.

Core Requirement

We need to manage and enforce these two distinct dimensions—feature access (seat-based) vs. provider/models entitlement — without duplicating logic for each possible combination.

Product Use Cases and Examples

1. “Duo Chat” With Self-Hosted Models

  1. Duo Pro Add-On
    • Seat-based.
    • Gives users the duo_chat feature (scope), assuming they have a seat.
  2. Self-Hosting Pro Add-On
    • Entitlement-based (no seats).
    • Grants access to the “SHM” provider.

A user assigned a Duo Pro seat has the duo_chat feature. If the organization also has Duoi Self-Hosting add-on, they can configure duo_chat to use “SHM” as the backend.

Key Point:

  • The user must pass both a feature scope check (duo_chat) and a provider entitlement check (shm) to succeed.

2. “Duo Chat” With AmazonQ

  1. AmazonQ Pro Add-On
    • Seat-based.
    • Unlocks the duo_chat feature for assigned seats and grants the “amazonq” provider entitlement.

In this scenario, a single add-on covers both the seat-based feature scope and the provider itself. However, the provider portion doesn’t require a separate seat check—only the feature does.

Key Point:

  • The seat check is relevant to the duo_chat feature.
  • The “amazonq” provider check is simply a yes/no entitlement based on the add-on.

How would permission framework work

We separate feature-level access (seat-based) from provider-level entitlements (often seat-free):

  1. Feature Access: “Do I have a seat for duo_chat?”
  2. Provider Access: “Is shm or amazonq or any other provider enabled by an add-on or the license plan this organisation has purchased?”

End-User UI / Application Checks

  • Seat Check (Feature Scope):
    • If the user wants duo_chat, the system verifies if they hold a seat in an add-on that unlocks duo_chat (e.g., Duo Pro, AmazonQ Pro).
  • Provider Check (Entitlement Only):
    • If the user selects a provider (e.g., shm), the system verifies that the org owns an add-on that grants that provider.
    • No seat assignment is needed here, even if the add-on is seat-based. The seat concept is only for features.

AI Gateway (Final Enforcement)

To prevent tampering, the AI Gateway enforces the same logic via a JWT with:

  • "scopes" → e.g., ["duo_chat", "code_generation", ...]
  • "providers" → e.g., ["duo","shm"]

At runtime:
If a user requests duo_chat with provider=shm, the AI Gateway requires "duo_chat" in scopes and "shm" in providers. Otherwise, it returns 403 Forbidden.

Analytics & Tracking

GitLab aims to provide AI-powered features across multiple models and deployment configurations. To keep our tracking extensible and avoid duplication, we recommend:

  1. Model and provider agnostic feature identifiers:
    • Don’t embed provider/model names in the feature name (e.g., avoid amazon_q_duo_chat).
    • Instead, capture feature (e.g., duo_chat) and provider (e.g., amazonq) as separate attributes in tracking events.
  2. Future-proof design:
    • Tracking new models or integrations should not require new unit primitives; We will simply pass a new provider/model attribute.
    • This approach extends seamlessly to Duo Pro/Enterprise or any upcoming product SKUs.

Example:

internal_event_client.track_event(
    "ai_feature_used",
    category="ai_usage",
    provider="self-hosted-models"
)

By separating features from providers in our event data, we gain:

  • Clear, scalable analytics: Track feature usage and provider adoption independently.
  • Simplified maintenance: No need to create multiple primitives for every feature–provider combination.
  • Greater flexibility for future partnerships, additional AI models, or changing product offerings.

Why This Separation Works

  1. Avoids Multiple Seat Checks:
    • Only features check seats. Providers are a simple “yes/no” entitlement.
  2. No Duplication of “Unit Primitives”:
    • We don’t create a separate feature scope for every provider combination (shm_duo_chat, amazonq_duo_chat, etc.). Instead, we keep one scope per feature and one per provider.
  3. Scalable for Future Providers:
    • Adding a new model vendor or AI provider is as simple as adding a new provider/unit_prmitive to the system.
    • The seat-based feature logic remains unchanged.
  4. Consistent UX and Security:
    • Front end can hide or disable providers the customer lacks.
    • The AI Gateway confirms entitlements in the JWT, preventing unauthorized usage.

Conclusion

Our requirement is not that “multiple add-ons exist for the same feature.” Rather, we’re separating the seat-based feature from the provider entitlement. A user may need both an add-on that provides seats for the feature (e.g., Duo Chat) and an add-on that grants a particular provider (e.g., Self-Hosting).

By splitting these concerns—seat-based scopes vs. provider entitlements—we achieve a fain-grained permission control and better internal event tracking.

This would also allow us to avoid having unit primitive duplication duo to the possible combinations of current and future AI offerings.

Relevant links and discussions:

Edited by Nikola Milojevic