Multiple Model Configuration Per Feature - Instance level model selection

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Work on this issue
• Close this issue

(The current issue is subject to change)

For the Model Swithcing feature, the instance admin will need to select which model they allow to for use per feature. Model switching was enabled at the namespace level on GItLab.com in #526009 (closed)).

Instance admins define default AI models for each feature to ensures a structured decision-making process and gives users autonomy without breaking organizational policies.

Proposal

Consider this flow diagram

sequenceDiagram
    Self Managed->>Cloud Connector: Scheduled Sidekiq job calls API
    Cloud Connector->>AI Gateway: Fetch Model List
    AI Gateway->>AI Gateway: Fetch Model List
    AI Gateway-->>Cloud Connector: Return list of models
    Cloud Connector-->>Self Managed: Return list of models
    Self Managed->>Self Managed: Insert model records and set defaults

• Create the ai_model_switching feature flag and it's roll out issues. Put all the model switching code behind it.

• Ping the endpoint of the AIGW to get the offered models (covered here). The response should look something like:

   [
     {
       'name': 'GitLab: Claude 3.7 Sonnet',
       id: 'claude_3_7_sonnet',
       release_state: 'GA' 
       'features':  ['duo_chat', 'completion']
      },
      {
       'name': 'Gemini Pro', 
       'id': 'gemini_pro',
       release_state: 'BETA'  
       'features': ['summarize_mr_review']
      }
    ]

• For each feature multiple model can be selected, these models will be available for selection in namespaces.

• Make sure it is required to select a default model for each modified instance ai feature.

• Make CRUD operation create and upsert with GraphQL

  • Index
  • Upsert (make sure the model is compatible with the feature)
  • Delete (roles routes to default the default model)

Data model

To insert model records and set defaults consider this data model.

erDiagram


    INSTANCE_AI_FEATURES["INSTANCE_AI_FEATURES "] {
        bigint id PK
        bigint namespace_id FK
        timestamp created_at
        timestamp updated_at
        varchar feature_category
        varchar feature
        varchar default_model
    }

    INSTANCE_AI_FEATURE_MODELS["INSTANCE_AI_FEATURE_MODELS (Future Iteration)"] {
        bigint id PK
        bigint instance_ai_feature_id FK
        varchar model
    }

    INSTANCE_AI_FEATURES ||--o{ INSTANCE_AI_FEATURE_MODELS : "has many"