Vertex AI Search endpoint for Documentation tool of Duo Chat (AI Gateway)

What does this merge request do and why?

Related to Draft: Retrieval Augmented Generation (RAG) for... (gitlab-org/gitlab!142787 - closed)

How to set up and validate locally

curl -X 'POST' \
  'http://0.0.0.0:5052/v1/search/docs' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
    "type": "string",
    "metadata": {
      "source": "string",
      "version": "17.0.0-pre"
    },
    "payload": {
      "query": "how to create an issue?",
    }
}'

Evaluation results - Independent LLM Judge - Correctness

SHA 9b27f21885731cc7c201382f25e648c556e81a58

• Before: ai_gateway_docs_search disabled
• After: ai_gateway_docs_search enabled

grade	before_percentage	after_percentage
4	76.7	86
3	16.3	9.3
2	2.3	2.3
1	2.3	2.3

query

WITH grades as (
  SELECT 4 as grade union all
  SELECT 3 as grade union all
  SELECT 2 as grade union all
  SELECT 1 as grade
), before_base_table AS (
  SELECT *
  FROM `dev-ai-research-0e2f8974.duo_chat_experiments.lulalala-mr778-disabled_20240508_154921__independent_llm_judge`
  WHERE answering_model = 'duo-chat'
), after_base_table AS (
  SELECT *
  FROM `dev-ai-research-0e2f8974.duo_chat_experiments.lulalala-mr778-enabled_20240508_160229__independent_llm_judge`
  WHERE answering_model = 'duo-chat'
), before_correctness_grade AS (
  SELECT correctness as grade, COUNT(*) as count
  FROM before_base_table
  GROUP BY correctness
), after_correctness_grade AS (
  SELECT correctness as grade, COUNT(*) as count
  FROM after_base_table
  GROUP BY correctness
)

SELECT grades.grade AS grade,
       ROUND((COALESCE(before_correctness_grade.count, 0) / (SELECT COUNT(*) FROM before_base_table)) * 100.0, 1) AS before_percentage,
       ROUND((COALESCE(after_correctness_grade.count, 0) / (SELECT COUNT(*) FROM after_base_table)) * 100.0, 1) AS after_percentage,
FROM grades
LEFT OUTER JOIN before_correctness_grade ON before_correctness_grade.grade = grades.grade
LEFT OUTER JOIN after_correctness_grade ON after_correctness_grade.grade = grades.grade;

Evaluation results - Similarity score

similarity_score_range	before_percentage	after_percentage
1	2.3	2.3
0.9	62.8	60.5
0.8	27.9	30.2
0.7	4.7	4.7
0.6	2.3	2.3
0.5	0	0
0.4	0	0
0.3	0	0
0.2	0	0
0.1	0	0

query

WITH buckets as (
  SELECT 1.0 as bucket union all
  SELECT 0.9 as bucket union all
  SELECT 0.8 as bucket union all
  SELECT 0.7 as bucket union all
  SELECT 0.6 as bucket union all
  SELECT 0.5 as bucket union all
  SELECT 0.4 as bucket union all
  SELECT 0.3 as bucket union all
  SELECT 0.2 as bucket union all
  SELECT 0.1 as bucket
), before_similarity_score AS (
  SELECT *
  FROM `dev-ai-research-0e2f8974.duo_chat_experiments.lulalala-mr778-disabled_20240508_154921__similarity_score`
  WHERE answering_model = 'duo-chat'
), after_similarity_score AS (
  SELECT *
  FROM `dev-ai-research-0e2f8974.duo_chat_experiments.lulalala-mr778-enabled_20240508_160229__similarity_score`
  WHERE answering_model = 'duo-chat'
)

SELECT buckets.bucket AS similarity_score_range,
    (
        SELECT ROUND((COUNT(*) / (SELECT COUNT(*) FROM before_similarity_score)) * 100.0, 1)
        FROM before_similarity_score
        WHERE buckets.bucket = ROUND(before_similarity_score.comparison_similarity, 1)
    ) AS before_percentage,
    (
        SELECT ROUND((COUNT(*) / (SELECT COUNT(*) FROM after_similarity_score)) * 100.0, 1)
        FROM after_similarity_score
        WHERE buckets.bucket = ROUND(after_similarity_score.comparison_similarity, 1)
    ) AS after_percentage,
FROM buckets

Merge request checklist

• Tests added for new functionality. If not, please raise an issue to follow up.
• Documentation added/updated, if needed.