Skip to content

Disable suffix truncation experiment

Tan Le requested to merge tl-disable-suffix-truncation into main

What does this merge request do and why?

Disable suffix truncation experiment

We suspect that this experiment causes high CPU load on certain input so disabling this for now until we confirm the root cause (see https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/issues/385). Furthermore, we also can confirm that the acceptance rate of suffix truncation is not much different from that of the baseline (see #387).

How to set up and validate locally

  1. Check out to this merge request's branch.
  2. Ensure a local Docker image built successfully.
    docker buildx build --platform linux/amd64 \
      -t ai-gateway:dev .
  3. Run a local service on Docker.
    docker run --platform linux/amd64 --rm \
      -p 5052:5052 \
      -e AUTH_BYPASS_EXTERNAL=true \
      -v $PWD:/app -it ai-gateway:dev
  4. Fire a few cURL requests sequentially and confirm they are all using variant 0 of exp_truncate_suffix.
    for _ in $(seq 1 5); do
      curl -sSL --request POST \
        --url http://codesuggestions.gdk.test:5052/v2/completions \
        --header 'Content-Type: application/json' \
        --header 'X-Gitlab-Authentication-Type: oidc' \
        --data '{
          "prompt_version": 1,
          "project_path": "awesome_project",
          "project_id": 23,
          "current_file": {
            "file_name": "main.py",
            "content_above_cursor": "// create a function to add two number\n",
            "content_below_cursor": ""
          }
        }' | jq '.experiments'
    done
    
    [
      {
        "name": "exp_truncate_suffix",
        "variant": 0
      }
    ]
    [
      {
        "name": "exp_truncate_suffix",
        "variant": 0
      }
    ]
    [
      {
        "name": "exp_truncate_suffix",
        "variant": 0
      }
    ]
    [
      {
        "name": "exp_truncate_suffix",
        "variant": 0
      }
    ]
    [
      {
        "name": "exp_truncate_suffix",
        "variant": 0
      }
    ]

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Tan Le

Merge request reports