## Background Blueprint: https://gitlab.com/gitlab-com/content-sites/handbook/-/merge_requests/7311+ The LLM prompts were developed in the Rails codebase to leverage existing Ruby expertise and as the AI Gateway was evolving. Now that the AI Gateway is a stable and key part of the GitLab infrastructure the prompts can be migrated from Rails into the AIGW. The Rails monolith will remain the persistence and control layer, with AI features becoming a thin entrypoint which is a referer to the prompt and wrapper code in the AI Gateway. This effort is seen as a "lift and shift" where the prompt code will moved to yaml files on the AI Gateway with Python wrappers. GitLab AI functionality is expected to be unchanged during evaluations. Moving prompt to the AI Gateway offers the following advantages: 1. Native access to data science libraries written in Python 2. Ability to iterate on AI features without needed a GitLab Rails upgrade 3. IDE code can access the prompts independently of Rails ## Risks This effort is seen to be low-risk from a feature perspective, but any rewriting of code includes risks. The original prompts will be initially left unchanged in Rails and routing to the prompts in the AI Gateway controlled by Rails and feature flags. After the appropriate number of milestones and when all parties are comfortable, the original Rails-based prompts can be deleted and the feature flag removed. ### Description Migration of all LLM prompts into the AI Gateway and out of GitLab Rails. GitLab Rails retains a thin entry point for each prompt, and the prompt processing is managed by the AI Gateway. ### Feature Flag Rollout #### .com Prompts The process will be protected by a specific feature flag for each feature, so that existing code and functionality remains unchanged when the feature flag is not enabled. #### Self-Hosted Models Prompts These are in `Experimental` status and already behind the `:ai_custom_model` feature flag. These will be updated directly. ## Project Plan ### Pre-work - [x] https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/merge_requests/922+ - [x] https://gitlab.com/gitlab-org/gitlab/-/merge_requests/152429+ ## Communication Plan - [x] Communicate to AI Powered EMs in 2024-06-20 sync call - [x] Prework planning on 2024-07-08 sync call [meeting notes](https://docs.google.com/document/d/14Z5A-kqqnIVMgxzUdU-BEJu5fd_dCM-tDJCoSS2NRn0/edit) - [x] Communicate AI Powered Stage-wide of project launch via Slack - [x] Planning call 2024-07-10 - [ ] %"17.3" Release notes - [ ] %"17.4" Release notes - [ ] Close this epic and Communicate Project completion via Slack ## Delivery ### Milestone 17.3 - [ ] https://gitlab.com/groups/gitlab-org/-/epics/14430+ ### Milestone 17.4 - [ ] https://gitlab.com/groups/gitlab-org/-/epics/14431+ ### Prompts | Feature | Prompt File Link | Issue | | -------------------------- | ------------------------------------------------------------------------------------------------------------ | --------------------------------------------------- | | CI Editor | https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/gitlab/llm/chain/tools/ci_editor_assistant/prompts | | | GitLab Documentation | https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/gitlab/llm/chain/tools/gitlab_documentation | | | Epic Reader | https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/gitlab/llm/chain/tools/epic_reader/prompts | | | Issue Reader | https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/gitlab/llm/chain/tools/issue_reader/prompts | | | Refactor | https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/gitlab/llm/chain/tools/refactor_code/prompts | | | Write Tests | https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/gitlab/llm/chain/tools/write_tests/prompts | | | Explain Code | https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/gitlab/llm/chain/tools/explain_code/prompts | | | Explain Vulnerability | https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/gitlab/llm/chain/tools/explain_vulnerability | | | Summarize Comments Chat | https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/gitlab/llm/chain/tools/summarize_comments/prompts | | | Fill MR Description | Being created: https://gitlab.com/gitlab-org/gitlab/-/issues/456968#note_1918771471 | | | RCA | Being added via https://gitlab.com/gitlab-org/gitlab/-/merge_requests/154040+s | | | Code Suggestions | https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/code_suggestions/prompts | | | Chat Zero Shot | https://gitlab.com/gitlab-org/gitlab/-/tree/master/ee/lib/gitlab/llm/chain/agents/zero_shot/prompts | | | ResolveVulnerability | `Gitlab::Llm::Templates::Vulnerabilities::ResolveVulnerability` | https://gitlab.com/gitlab-org/gitlab/-/issues/482070 | | SummarizeReview | `Gitlab::Llm::Templates::SummarizeReview` | https://gitlab.com/gitlab-org/gitlab/-/issues/495131 | | ExplainCode | `Gitlab::Llm::VertexAi::Templates::ExplainCode` | https://gitlab.com/gitlab-org/gitlab/-/issues/500094 | | GenerateDescription | `Gitlab::Llm::Templates::GenerateDescription` | https://gitlab.com/gitlab-org/gitlab/-/issues/500542 | | GenerateCommitMessage | `Gitlab::Llm::Templates::GenerateCommitMessage` | https://gitlab.com/gitlab-org/gitlab/-/issues/500549 | | SummarizeNewMergeRequest | `Gitlab::Llm::Templates::SummarizeNewMergeRequest` | https://gitlab.com/gitlab-org/gitlab/-/issues/501320 | | GenerateCubeQuery | `Gitlab::Llm::VertexAi::Templates::GenerateCubeQuery` | | | CategorizeQuestion | `Gitlab::Llm::Templates::CategorizeQuestion` | https://gitlab.com/gitlab-org/gitlab/-/issues/500741 | | ReviewMergeRequest | `Gitlab::Llm::Templates::ReviewMergeRequest` | https://gitlab.com/gitlab-org/gitlab/-/issues/501321 | | AiGitCommand | `::Llm::GitCommandService` | https://gitlab.com/gitlab-org/gitlab/-/issues/500895 |