Skip to content

Fix empty response scenarion in documentation tool

Alexandru Croitor requested to merge fix_documentation_tool_exception_on_empty_responses into master

What does this MR do and why?

Fix empty response scenarion in documentation tool

DRYed some of of the TanukiBot service code by making it return a response modifier, which is being used both in completions and chat agent. Adjusted specs accordingly.

Updated llm tools matcher to lookup used tools in the context rather than parse the prompt

Extracted real requests examples from zero shot spec into a separate file.

Also added chat resetting for every example on real requests. Previously every new question used to contain all previous questions history.

re #424302 (closed)

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. To reproduce this locally make sure you have documentation embeddings setup locally. If you do not check out the documentation https://docs.gitlab.com/ee/development/ai_features/#populating-embeddings-and-using-embeddings-fixture
  2. Once you have embeddings populated ask a documentation question in chat. You should get a response
  3. open rails console and change the current version of the embeddings:
[153] pry(main)> ::Embedding::TanukiBotMvc.get_current_version
=> 2
[154] pry(main)> ::Embedding::TanukiBotMvc.set_current_version!(3)
=> 0
  1. ask the documentation question in the chat again
Observation:
  • Without this fix the chat looks like it is being stuck answering the question.
  • With the fix you should expect to get a default answer "I'm sorry, I was not able to find any documentation to answer your question."

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Alexandru Croitor

Merge request reports