Consolidate experimentation feature flags
Continuing conversation here: !130598 (comment 1536622928)
We currently have a handful of experimentation API endpoints for the following LLMs: OpenAI, Anthropic, and Vertex. These are documented here.
Each of these endpoints is guarded behind 2 feature flags:
- the
ai_experimentation_api
feature flag, which is enabled for members of thegitlab_team_members
group on GPRD. - an LLM-specific feature flag.
-
tofa_experimentation_main_flag
(for Vertex) -> disabled on GPRD -
anthropic_experimentation
-> disabled on GPRD -
openai_experimentation
-> enabled globally on GPRD
-
Both the general feature flag and the LLM-specific feature flag must be enabled for a user to be able to use these experimental endpoints.
Suggestion: Remove the 3 LLM-specific feature flags.
Reason: I don't see a technical reason why we need to rely on additional flags in order to enable these endpoints. The general ai_experimentation_ai
feature flag is enabled per-user and the provider-specific ones are global. By relying on fewer feature flags, we can make our local testing steps simpler and close out the feature flag rollout issues for each of these feature flags.
One potential problem with this change is that the removal will enable the Anthropic and Vertex experimentation endpoints for gitlab-team-members
group on GPRD. Many of those team members also have the ability to enable feature flags on GPRD so it doesn't seem like a huge risk but maybe there is a legal reason we don't want these endpoints enabled on GPRD for anyone?