Skip to content

Add skip_cache option

Jan Provaznik requested to merge jp-skip-cache into master

What does this MR do and why?

Add skip_cache option

It's possible that some AI chain tools use completion service (e.g. SummaryComments tool), in this case we want to avoid storing request/response in cache because it's only intermediate step.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. Ask a question which uses SummarizeComments tool (e.g. "what is summary of comments on flightjs/Flight#1 issue?")
  2. Or alternatively can be verified from console with something like: res = Gitlab::Llm::Chain::Agents::ZeroShot::Executor.new(user_input: 'What is summary of comments on flightjs/Flight#1 issue?', tools: [Gitlab::Llm::Chain::Tools::IssueIdentifier,Gitlab::Llm::Chain::Tools::SummarizeComments], context: ::Gitlab::Llm::Chain::GitlabContext.new(current_user: user,container:nil,resource: <issue>, ai_request: ::Gitlab::Llm::Chain::Requests::Anthropic.new(user))).execute
  3. Verify in rails console that no "intermediate" messages are stored in cache using Gitlab::Llm::Cache.new(user).find_all

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Jan Provaznik

Merge request reports