Skip to content

Make code completion provider-specific

Jan Provaznik requested to merge jp-prompt-ant into master

What does this MR do and why?

  • adds support for Anthropic provider to code suggestion prompt
  • when model_family parameter is set to anthropic, it builds anthropic-specific prompt used for code completion and builds addition prompt parameter (similar to code_generation) to the request forwarded to AI gateway
  • prompts are extracted to their own classes

Related to #425875 (closed)

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. Because there is no Anthropic support yet on AI gateway, this code can not be verified using API yet and can be verified only from console with checking TaskSelector result with something like ::CodeSuggestions::TaskSelector.task(params: {model_family: :anthropic}).body

Alternatively it can be verified e2e from VScode by adding following snippet into REST API endpoint:

diff --git a/ee/lib/api/code_suggestions.rb b/ee/lib/api/code_suggestions.rb
index ba38b39de205..25e299340640 100644
--- a/ee/lib/api/code_suggestions.rb
+++ b/ee/lib/api/code_suggestions.rb
@@ -140,16 +140,6 @@ def gitlab_realm
           body = task.body
           file_too_large! if body.size > MAX_BODY_SIZE
 
-          # Because Anthropic endpoint is not available on AI gateways yet,
-          # for development purposes only we just use Rails' Anthropic client
-          # directly, before the flag is rolled-out, this workaround should be removed:
-          # https://gitlab.com/gitlab-org/gitlab/-/issues/426111#specific-rollout-on-production
-          if safe_params[:model_family] == :anthropic
-            text = Gitlab::Llm::Anthropic::Client.new(current_user).complete(prompt: body)
-            text = text&.dig("completion").to_s.strip
-            break { choices: [{ text: text }] }
-          end

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Jan Provaznik

Merge request reports