Skip to content

Reorganize the `deps.py` module

Alexander Chueshev requested to merge ac/reorganize-deps into main

What does this merge request do and why?

Our existing deps.py module contains all project dependencies. It's getting hard to maintain and read the module while adding new features or extending new ones with updated business requirements. This MR reorganizes deps.py following https://python-dependency-injector.ets-labs.org/examples/application-multiple-containers.html.

Changes:

  • deps.py has been renamed to container.py that contains the main ContainerApplication class with all required dependencies.
  • ContainerApplication includes 5 sub-containers code_suggestions, x_ray, chat, snowplow, and fastapi.
  • Every sub-container has been placed in a separate package outside the main ai_gateway/container.py module.
  • Every sub-container is able to extend/override model dependencies according to the business requirements.

Addresses https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/issues/393.

How to set up and validate locally

  1. Check out to this merge request's branch.

  2. Ensure a local Docker image built successfully.

    docker buildx build --platform linux/amd64 \
      -t ai-gateway:dev .
  3. Run a local service on Docker.

    docker run --platform linux/amd64 --rm \
      -p 5052:5052 \
      -e AIGW_AUTH__BYPASS_EXTERNAL=true \
      -v $PWD:/app -it ai-gateway:dev
  4. Run the following cURL commands to test code generations, completions, x-ray and chat

  5. Re-run with fake models environment enabled

    docker run --platform linux/amd64 --rm \
      -p 5052:5052 \
      -e AIGW_AUTH__BYPASS_EXTERNAL=true \
      -e AIGW_USE_FAKE_MODELS=true \
      -v $PWD:/app -it ai-gateway:dev
Code Generations - Default
curl --request POST \
  --url http://127.0.0.1:5052/v2/code/generations \
  --header 'Content-Type: application/json' \
  --header 'X-Gitlab-Authentication-Type: oidc' \
  --data '{
  "current_file": {
    "file_name": "main.go",
    "content_above_cursor": "# Hello world",
    "content_below_cursor": "\n"
  },
  "prompt_version": 2,
  "prompt": "Human: You are a coding autocomplete agent. We want to generate new Go code inside the\nfile '\''hello.go'\'' based on instructions from the user.\nThe existing code is provided in <existing_code></existing_code> tags.\nThe new code you will generate will start at the position of the cursor, which is currently indicated by the <cursor> XML tag.\nIn your process, first, review the existing code to understand its logic and format. Then, try to determine the most\nlikely new code to generate at the cursor position to fulfill the instructions.\n\nWhen generating the new code, please ensure the following:\n1. It is valid Go code.\n2. It matches the existing code'\''s variable, parameter and function names.\n3. It does not repeat any existing code. Do not repeat code that comes before or after the cursor tags. This includes cases where the cursor is in the middle of a word.\n4. If the cursor is in the middle of a word, it finishes the word instead of repeating code before the cursor tag.\n\nReturn new code enclosed in <new_code></new_code> tags. We will then insert this at the <cursor> position.\nIf you are not able to write code based on the given instructions return an empty result like <new_code></new_code>.\n\nHere are a few examples of successfully generated code by other autocomplete agents:\n\n<examples>\n\n  <example>\n    H: <existing_code>\n         package main\nimport (\n  \"net/http\"\n)\nfunc main() {\n  err := http.ListenAndServe(httpPort, nil)\n  if err != nil { panic(err) }\n}\n// create an HTTP handler that fetches the current user preferences and returns them as JSON\n<cursor>\n\n// function to print the current user'\''s name\n       </existing_code>\n\n    A: <new_code>func userPrefsHandler(w http.ResponseWriter, r *http.Request) {\n\n  // Get current user\n  user := getCurrentUser(r)\n\n  // Fetch user prefs from database\n  prefs, err := fetchUserPrefs(user.ID)\n  if err != nil { http.Error(w, \"Error fetching prefs\", 500); return }\n\n  // Encode prefs to JSON\n  jsonPrefs := json.Marshal(prefs)\n  if err != nil { http.Error(w, \"Error encoding prefs\", 500); return }\n\n  w.Header().Set(\"Content-Type\", \"application/json\")\n  w.Write(jsonPrefs)\n}</new_code>\n  </example>\n\n  <example>\n    H: <existing_code>\n         // filter out any non-prime numbers from the list\nfunc filterPrimes(list []int) []int {\n<cursor>\n  return primes\n}\n\n// calculate the square root of a number\n       </existing_code>\n\n    A: <new_code>var primes []int\n  for _, num := range list {\n    isPrime := true\n    for i := 2; i <= num/2; i++ {\n      if num%i == 0 {\n        isPrime = false\n        break\n      }\n    }\n    if isPrime { primes = append(primes, num) }\n  }</new_code>\n  </example>\n\n</examples>\n\n\n<existing_code>\n// Generate a function to print hello world\nfunc print<cursor>\n</existing_code>\n\n\nHere are instructions provided in <instruction></instruction> tags.\n\n<instruction>\nCreate more new code for this file. If the cursor is inside an empty function,\ngenerate its most likely contents based on the function name and signature.\n\n</instruction>\n\n\nAssistant: <new_code>"
}
'
Code Generations - Anthropic - Default
curl --request POST \
  --url http://127.0.0.1:5052/v2/code/generations \
  --header 'Content-Type: application/json' \
  --header 'X-Gitlab-Authentication-Type: oidc' \
  --data '{
  "current_file": {
    "file_name": "main.go",
    "content_above_cursor": "# Hello world",
    "content_below_cursor": "\n"
  },
  "prompt_version": 2,
  "prompt": "Human: You are a coding autocomplete agent. We want to generate new Go code inside the\nfile '\''hello.go'\'' based on instructions from the user.\nThe existing code is provided in <existing_code></existing_code> tags.\nThe new code you will generate will start at the position of the cursor, which is currently indicated by the <cursor> XML tag.\nIn your process, first, review the existing code to understand its logic and format. Then, try to determine the most\nlikely new code to generate at the cursor position to fulfill the instructions.\n\nWhen generating the new code, please ensure the following:\n1. It is valid Go code.\n2. It matches the existing code'\''s variable, parameter and function names.\n3. It does not repeat any existing code. Do not repeat code that comes before or after the cursor tags. This includes cases where the cursor is in the middle of a word.\n4. If the cursor is in the middle of a word, it finishes the word instead of repeating code before the cursor tag.\n\nReturn new code enclosed in <new_code></new_code> tags. We will then insert this at the <cursor> position.\nIf you are not able to write code based on the given instructions return an empty result like <new_code></new_code>.\n\nHere are a few examples of successfully generated code by other autocomplete agents:\n\n<examples>\n\n  <example>\n    H: <existing_code>\n         package main\nimport (\n  \"net/http\"\n)\nfunc main() {\n  err := http.ListenAndServe(httpPort, nil)\n  if err != nil { panic(err) }\n}\n// create an HTTP handler that fetches the current user preferences and returns them as JSON\n<cursor>\n\n// function to print the current user'\''s name\n       </existing_code>\n\n    A: <new_code>func userPrefsHandler(w http.ResponseWriter, r *http.Request) {\n\n  // Get current user\n  user := getCurrentUser(r)\n\n  // Fetch user prefs from database\n  prefs, err := fetchUserPrefs(user.ID)\n  if err != nil { http.Error(w, \"Error fetching prefs\", 500); return }\n\n  // Encode prefs to JSON\n  jsonPrefs := json.Marshal(prefs)\n  if err != nil { http.Error(w, \"Error encoding prefs\", 500); return }\n\n  w.Header().Set(\"Content-Type\", \"application/json\")\n  w.Write(jsonPrefs)\n}</new_code>\n  </example>\n\n  <example>\n    H: <existing_code>\n         // filter out any non-prime numbers from the list\nfunc filterPrimes(list []int) []int {\n<cursor>\n  return primes\n}\n\n// calculate the square root of a number\n       </existing_code>\n\n    A: <new_code>var primes []int\n  for _, num := range list {\n    isPrime := true\n    for i := 2; i <= num/2; i++ {\n      if num%i == 0 {\n        isPrime = false\n        break\n      }\n    }\n    if isPrime { primes = append(primes, num) }\n  }</new_code>\n  </example>\n\n</examples>\n\n\n<existing_code>\n// Generate a function to print hello world\nfunc print<cursor>\n</existing_code>\n\n\nHere are instructions provided in <instruction></instruction> tags.\n\n<instruction>\nCreate more new code for this file. If the cursor is inside an empty function,\ngenerate its most likely contents based on the function name and signature.\n\n</instruction>\n\n\nAssistant: <new_code>",
  "model_provider": "anthropic"
}
Code Generations - Anthropic - Claude 2.1
curl --request POST \
  --url http://127.0.0.1:5052/v2/code/generations \
  --header 'Content-Type: application/json' \
  --header 'X-Gitlab-Authentication-Type: oidc' \
  --data '{
  "current_file": {
    "file_name": "main.go",
    "content_above_cursor": "# Hello world",
    "content_below_cursor": "\n"
  },
  "prompt_version": 2,
  "prompt": "Human: You are a coding autocomplete agent. We want to generate new Go code inside the\nfile '\''hello.go'\'' based on instructions from the user.\nThe existing code is provided in <existing_code></existing_code> tags.\nThe new code you will generate will start at the position of the cursor, which is currently indicated by the <cursor> XML tag.\nIn your process, first, review the existing code to understand its logic and format. Then, try to determine the most\nlikely new code to generate at the cursor position to fulfill the instructions.\n\nWhen generating the new code, please ensure the following:\n1. It is valid Go code.\n2. It matches the existing code'\''s variable, parameter and function names.\n3. It does not repeat any existing code. Do not repeat code that comes before or after the cursor tags. This includes cases where the cursor is in the middle of a word.\n4. If the cursor is in the middle of a word, it finishes the word instead of repeating code before the cursor tag.\n\nReturn new code enclosed in <new_code></new_code> tags. We will then insert this at the <cursor> position.\nIf you are not able to write code based on the given instructions return an empty result like <new_code></new_code>.\n\nHere are a few examples of successfully generated code by other autocomplete agents:\n\n<examples>\n\n  <example>\n    H: <existing_code>\n         package main\nimport (\n  \"net/http\"\n)\nfunc main() {\n  err := http.ListenAndServe(httpPort, nil)\n  if err != nil { panic(err) }\n}\n// create an HTTP handler that fetches the current user preferences and returns them as JSON\n<cursor>\n\n// function to print the current user'\''s name\n       </existing_code>\n\n    A: <new_code>func userPrefsHandler(w http.ResponseWriter, r *http.Request) {\n\n  // Get current user\n  user := getCurrentUser(r)\n\n  // Fetch user prefs from database\n  prefs, err := fetchUserPrefs(user.ID)\n  if err != nil { http.Error(w, \"Error fetching prefs\", 500); return }\n\n  // Encode prefs to JSON\n  jsonPrefs := json.Marshal(prefs)\n  if err != nil { http.Error(w, \"Error encoding prefs\", 500); return }\n\n  w.Header().Set(\"Content-Type\", \"application/json\")\n  w.Write(jsonPrefs)\n}</new_code>\n  </example>\n\n  <example>\n    H: <existing_code>\n         // filter out any non-prime numbers from the list\nfunc filterPrimes(list []int) []int {\n<cursor>\n  return primes\n}\n\n// calculate the square root of a number\n       </existing_code>\n\n    A: <new_code>var primes []int\n  for _, num := range list {\n    isPrime := true\n    for i := 2; i <= num/2; i++ {\n      if num%i == 0 {\n        isPrime = false\n        break\n      }\n    }\n    if isPrime { primes = append(primes, num) }\n  }</new_code>\n  </example>\n\n</examples>\n\n\n<existing_code>\n// Generate a function to print hello world\nfunc print<cursor>\n</existing_code>\n\n\nHere are instructions provided in <instruction></instruction> tags.\n\n<instruction>\nCreate more new code for this file. If the cursor is inside an empty function,\ngenerate its most likely contents based on the function name and signature.\n\n</instruction>\n\n\nAssistant: <new_code>",
  "model_provider": "anthropic",
  "model_name": "claude-2.1"
}

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Tan Le

Merge request reports