Skip to content

Draft: Summarize notes draft

Alexandru Croitor requested to merge summarize_notes into master

What does this MR do and why?

Related to https://gitlab.com/gitlab-org/gitlab/-/issues/404510

high level plan

This is how I see it, open to debate any point you may think can be changed.

This list may continue to be extended as I keep thinking about it. I will strike out items that are done or changed.

  • Required
    • ensure this works for issue, work item, MR, epic!
    • make sure this works on large enough resources
      • This may require AI Token limit/transformer layer as we may need to split the input into multiple chunks or take other strategies to fit into model's tokens limits
  • Can be a follow-up but not GA in any form, not even alpha
    • opt-in setting at namespace level
    • store api keys at namespace level
    • enable ai experimentation FFs per namespace - Done
      • as a faster alternative to namespace settings, while we implement those
    • ability to turn off AI BG jobs in case of total AI shut down - Done. The worker is behind openai_experimentation FF that is used as a global on/off FF.
    • make sure this works on large enough resources
      • This may require AI Token limit/transformer layer as we may need to split the input into multiple chunks or take other strategies to fit into model's tokens limits - MR in works: !117176 (closed)
  • Can be a follow-up iteration preferably before alpha so that we can effectively iterate
    • retry & backoff mechanism, i.e. circuit breaker mechanism - MR in review !117210 (merged)
    • a way to switch/test LLM configurations without having to deploy. E.g. store LLM configurations and be able to switch between them though chatops for instance?
    • track data on user satisfaction(ai performance) in relation to specific LLM configuration
  • Iterative improvements
    • opt-in setting at namespace level - MR in works !116984 (closed)
    • store api keys at namespace level
    • caching layer

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Alexandru Croitor

Merge request reports