AI Feature Proposal - Self-hosted/Air-gapped AI Model Solution

Proposal

For our customers running GitLab on-prem, the AI features running in the SaaS is often a problem. They fear Intellectual Property infringement.

The 2 customers I talked to yesterday have precisely this same concern. They are using GitLab on-prem due to privacy constraints and so cannot use our AI.

Since we are using GitLab on-prem as a proxy to GitLab.com for most calls to AI features, this could be highly helpful to allow customers to select the target of the API calls to AI endpoints.

One of our customers told me yesterday that they are experiencing Llama from Meta yesterday. Allowing to define the target of the API calls would allow them to send data to their own model, and get answers out of it. This way the source code stays in their own infrastructure, respecting the privacy constraints. And the AI infrastructure deployment does not belong to us but solely to our customers.

From a technical perspective, we need to be able to select the target of the API call (maybe for each call?) and to expose our API specifications. Then, as long as the customers implement these specifications, they are free to use the model they want and even train the model on their own data.