Skip to content

Add GPT5 for Duo Chat/Software Development Flow behind a FF

GPT5 was officially released on August, 7th.

Let's enable it behind a default disabled feature flag for Duo Chat and Software Development Flow. Once model switching is released, we should be able to remove the feature flag related code.

Proposal

  • Let's open a MR which adds GPT5 support for our agentic features/workflows - we could potentially re-use the MR that @Joey_Khabie already prepared in https://gitlab.com/gitlab-org/gitlab/-/issues/555574#note_2633168725 ?
  • We should put this functionality behind a feature flag, i.e., by default (flag disabled) our existing workflows will use Sonnet 4. With the flag enabled, we can switch to GPT.
  • We can use this MR to test GPT5 locally as described in my previous comment
  • Prompt caching can go in the same MR or in a separate MR.
  • Provided that there are no significant performance issues with GPT5, we can start rolling out the feature flag gradually on August, 20th.
  • Later, when model switching for Duo Workflow Service is available, we can get rid of the feature flag.

Prompt caching

Prompt caching cannot be disabled. To ensure that no user-specifc/dynamic data gets cached, the static content (instructions, examples, tool schemas) should be placed first, with the dynamic user-specific content afterwards. See https://gitlab.com/gitlab-org/gitlab/-/issues/555574#note_2679410916

Edited by Alexander Chueshev