Skip to content

feat: fallback to stable datastore in search API

Shinya Maeda requested to merge fallback-to-stable-datastore-vertex-ai-search into main

What does this merge request do and why?

This MR adds a retry mechanism to search API that if a datastore based on GL version is not found, it falls back to a previous stable datastore so that users can keep using the doc tool.

FYI, this MR implements the following spec in the doc https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/blob/main/docs/search.md:

  1. If the request encountered an error, it falls back to a stable data store which is hard-coded in AI Gateway. This process would be improved in the future to be more resilient. Example scenarios:

    1. The data store ID doesn't exist yet (for example, GitLab-Rails bump the VERSION and its deployed to gitlab.com, but the ingestion pipeline has not run yet).

    2. The data is still being processed/indexed by Vertex AI Search, which usually takes 5-10 minutes.

Closes Duo Chat documentation search may not work duri... (gitlab-org/gitlab#461729 - closed)

How to set up and validate locally

  1. Bump the VERSION file in GitLab-Rails. e.g. 18.0.0-pre
  2. Checkout this feature branch in AI Gateway.
  3. gdk start
  4. Login to the Rails console via gdk rails console:
  5. Try a query Gitlab::Llm::AiGateway::DocsClient.new(User.first).search(query: 'how to create an issue?').
  6. Confirm that a result is returned.

Example:

[1] pry(main)> Gitlab::Llm::AiGateway::DocsClient.new(User.first).search(query: 'how to create an issue?')
  User Load (1.3ms)  SELECT "users".* FROM "users" ORDER BY "users"."id" ASC LIMIT 1
=> {"response"=>
  {"results"=>
    [{"id"=>"7e85d5fbdaf1414e0469f41bec5c7b9e",
      "content"=>
       "# Create an issue\n\nDETAILS:\n**Tier:** Free, Premium, Ultimate\n**Offering:** GitLab.com, Self-managed, GitLab Dedicated\n\nWhen you create an issue, you are prompted to enter the fields of the issue.\nIf you know the values you want to assign to an issue, you can use\n[quick actions](../quick_actions.md) to enter them.\n\nYou can create an issue in many ways in GitLab:\n\n- [From a project](#from-a-project)\n- [From a group](#from-a-group)\n- [From another issue or incident](#from-another-issue-or-incident)\n- [From an issue board](#from-an-issue-board)\n- [By sending an email](#by-sending-an-email)\n- [Using a URL with prefilled values](#using-a-url-with-prefilled-values)\n- [Using Service Desk](#using-service-desk)\n\n## From a project\n\nPrerequisites:\n\n- You must have at least the Guest role for the project.\n\nTo create an issue:\n\n1. On the left sidebar, select **Search or go to** and find your project.\n1. Either:\n\n   - On the left sidebar, select **Plan > Issues**, and then, in the upper-right corner, select **New issue**.\n   - On the left sidebar, at the top, select the plus sign (**{plus}**) and then, under **In this project**,\n     select **New issue**.\n\n1. Complete the [fields](#fields-in-the-new-issue-form).\n1. Select **Create issue**.\n\nThe newly created issue opens.\n\n## From a group\n\nIssues belong to projects, but when you're in a group, you can access and create issues that belong\nto the projects in the group.\n\nPrerequisites:\n",
      "metadata"=>
       {"source_type"=>"doc",
        "title"=>"Create an issue",
        "md5sum"=>"7984b425ebb2afbbf53eabfb1cdfc712adcbd672c982df02c9a7675bd9a82993",
        "source_url"=>"https://gitlab.com/help/user/project/issues/create_issues",
        "source"=>"user/project/issues/create_issues.md"}},
...

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Shinya Maeda

Merge request reports