Skip to content

Model Switching - Top namespace level model selection

For the Model Swithcing feature, the namespace admins will need to select which model they allow for use per feature. He will be define the model it wants for each feature.

The top namespaces admin will only be able to choose from the models made available by the instance admin. This work is covered here.

The child namespaces admin will only be able to choose from the models made available by their parent namespace.

  • Put the code behind the ai_model_switching feature flag.
  • Make sure this system is compactible with Model Swithcing (= model metadate stored in the AIGW) as well as with GitLab Duo Self-Hosted models (= model metadata stored in GitLab rails).
  • Make sure it is required to select a default model for each modified instance ai feature.
  • Make CRUD operation create and upsert with GraphQL
    • Index
    • Upsert
      • Make sure the model is compatible with the feature by fetching the offered model metadata in the AIGW
{
  "models": [
    {
      "name": "Claude Sonnet 3.7 - Anthropic",
      "identifier": "claude_sonnet_3_7_20250219"
    },
    {
      "name": "Claude Sonnet 3.7 - Vertex",
      "identifier": "claude_sonnet_3_7_20250219_vertex"
    },
    {
      "name": "Claude Sonnet 3.5 - Anthropic",
      "identifier": "claude_3_5_sonnet_20240620"
    }
  ],
  "unit_primitives": [
    {
      "feature_setting": "duo_chat",
      "default_model": "claude_sonnet_3_7_20250219",
      "selectable_models": [
        "claude-3-7-sonnet-20250219",
        "claude_3_5_sonnet_20240620"
      ],
      "beta_models": [],
      "unit_primitives": [
        "ask_build",
        "ask_commit"
      ]
    },
    {
      "feature_setting": "code_generations",
      "default_model": "claude_sonnet_3_7_20250219",
      "selectable_models": [
        "claude-3-7-sonnet-20250219",
        "claude_3_5_sonnet_20240620"
      ],
      "beta_models": [],
      "unit_primitives": [
        "generate_code"
      ]
    },
    {
      "feature_setting": "code_completions",
      "default_model": "claude_sonnet_3_7_20250219",
      "selectable_models": [
        "claude-3-7-sonnet-20250219",
        "claude_3_5_sonnet_20240620"
      ],
      "beta_models": [],
      "unit_primitives": [
        "complete_code"
      ]
    }
  ]
}

Proposal

Data model

The work to fetch these metada has been started in this draft for organization. But there are some reusable abstraction like:

erDiagram

    NAMESPACES["NAMESPACES (existing)"] {
        bigint id PK
        bigint parent_id FK
        bigint owner_id FK
        varchar name
        varchar path
        bigint organization_id FK
    }

    NAMESPACE_AI_FEATURES {
        bigint id PK
        bigint namespace_id FK
        varchar offered_model_ref
        varchar offered_model_name
    }
    NAMESPACES ||--o{ NAMESPACE_AI_FEATURES : "has many"

GraphQL request to get the feature settings should look like this:

query getAiFeatureSetting {
  aiNamespaceFeatureSettings(namespaceId: $ID) {
    nodes {
      feature
      title
      mainFeature
      provider
      releaseState
      offeredModelRef
      offeredModelName
      offeredModels {
        nodes {
          ref
          name
        }
      }
    }
  }
}

The mutation for update should look like this:

mutation updateAiFeatureSetting($input: AiFeatureSettingUpdateInput!) {
  aiNamespaceFeatureSettingUpdate(input: $input) {
    aiFeatureSettings {
      feature
      title
      mainFeature
      provider
      releaseState
      offeredModelRef
      offeredModelName
      offeredModels {
        nodes {
          ref
          name
        }
      }
    }
    errors
  }
}

The input for this mutation would be:

{
  "namespeceId": 1,
  "feature": "CODE_SUGGESTION", # Enum
  "offeredModelRef": "claude-3-7-sonnet-20250219"
}
Edited by Patrick Cyiza