Fix categorize_question timeout
What does this MR do and why?
Enforce JSON format further.
Limit output token for categorize_question to save cost in edge cases when prompt becomes chatty.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
How to set up and validate locally
- Comment out https://gitlab.com/gitlab-org/gitlab/-/blob/6d0b85d8140d296116aa41b8c352639407ad0d41/ee/lib/gitlab/llm/completions/chat.rb#L74
- ask chat a question
- observe
llm.log
file to see that request to AI is sent - observe snowplow-micro for newly added attributes
Related to #439665 (closed)
Edited by Mark Chao