Skip to content

chore: change cache ttl for cheaper one

Tetiana Chuprynarequested to merge
brytannia-main-patch-96d8 into main

What does this merge request do and why?

We're using expensive 1h cache when we can use cheaper 5m.

this is what Anthropic doc says about using 1h cache

if you have prompts that are used at a regular cadence (i.e., system prompts that are used more frequently than every 5 minutes), continue to use the 5-minute cache, since this will continue to be refreshed at no additional charge.

And about cache lifetime:

The cache’s default minimum lifetime (TTL) is 5 minutes. This lifetime is refreshed each time the cached content is used.

I believe we have enough traffic to justify this change. That will save us some costs, however I don't have any numbers to back up this MR.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
  • If this change requires executor implementation: verified that issues/MRs exist for both Go executor and Node executor or confirmed that changes are backward-compatible and don't break existing executor functionality.

Merge request reports