Self-hosted Mistral/Mixtral models for Code generation

What does this MR do and why?

This MR enables the setting/use of self-hosted models (Mistral/Mixtral) for code suggestions.

This MR is based on !152380 (merged) and the FE is being handled in !152107 (merged)

Screenshots or screen recordings

Screen_Recording_2024-05-17_at_13.53.40

Prompt Evaluation Control (~0.85) and Test similarity score (~0.87)

• Control : 0.85
• Test with one setup: 0.87 : Variance: +2%
• Test with second simpler set up: 0.86 : Variance : +1%
• Sample Size:425
• Success: Yes , for both set ups , but to optimise latency choosing simple instructions , the simpler set up as the main aim was to be better than the control?

On GCP

SELECT count(*), avg(similarity_score) FROM `dev-ai-research-0e2f8974.code_suggestion_experiments.mhamda_mistral_20240517_161540__similarity_score

How to set up and validate locally

• Ensure you have purchased Duo Pro and that the instance is running as an SM instance.

• Verify the value of GITLAB_SIMULATE_SAAS by executing echo $GITLAB_SIMULATE_SAAS; it should be 0.

If you don't have a Duo Pro record, you can create one from rails c using:

add_on = GitlabSubscriptions::AddOn.find_or_create_by!(name: "code_suggestions") {|e| e.description = "Test"}
add_on_purchase = GitlabSubscriptions::AddOnPurchase.create!(
  add_on: add_on, expires_on: 1.month.from_now, quantity: 5, purchase_xid: 'A-S0001'
)

• In a separate terminal, if gdk is running, execute the following:

gdk stop rails-web && GITLAB_RAILS_RACK_TIMEOUT_ENABLE_LOGGING=false PUMA_SINGLE_MODE=true GITLAB_SIMULATE_SAAS=0 gdk rails s

1. In the Rails console, enable the experiment fully:

   > Feature.enable(:self_managed_code_suggestions)
   > Feature.enable(:ai_custom_model)
   > ApplicationSetting.first.update(duo_features_enabled: true)

   This MVP requires the Ai::SelfHostedModel record to be created in the database.

   The table/model exists on the target branch, but the UI doesn't yet exist there: Front-end iteration for self-deployed model con... (!152107 - merged).

Please create a record in the Rails console: ::Ai::SelfHostedModel.create!(model: 0, endpoint: 'http://127.0.0.1:11434/v1', name: 'mistral-7B', api_token: 'token')

2. Log in as an admin.

3. Visit the index page Admin Area > AI-Powered Features > Features.

4. Use Mistral for Code Generations.

   

Make sure that you have the AI gateway up and running in the GDK.

Set up a locally hosted Mistral model using OLLAMA. In a separate terminal, run ollama serve.

Follow the above recording to verify the changes.