Instance-Level Self-Hosted Model serving configuration

When a customer self-hosts a model, the existence of this model along with it's connectivity information needs to be declared to GitLab Rails. GitLab Rails is responsible for exposing this information to the AI Gateway.

Once a model has been declared, it can be assigned to specific GitLab Duo functions as described in Configuration option to select model (#454323 - closed)

This configuration is added in the instance-level configuration settings.

See also Self-Hosted Model Deployment Blueprint (!148599 - merged)

Screenshots

Taken from the MR Draft: Instance-Level Self-Hosted Model serving... (!151505 - closed)

Screenshot_2024-05-02_at_17.35.42

Edited by Sean Carroll