As a next step we are going to extend code suggestions to distinguish clearly between 2 different use cases: * Code Generation based on a comment, new function name, etc. * Code Completion mid code This will allow us to specifically focus on the specifics of the use cases and especially build a central point for code generation so it can be used in code suggestions and GitLab Chat (as its own tool).  ## Code Generation We can do much better code generation suggestions with adding additional information from the project. As a general problem of code tools is that the user will only write a small sentence what they are expecting to be created, but to accomplish certain tasks there are multiple ways to write code for it. The target is to create code that matches the project as good as possible. This means we create specific templates for code generation and maybe use also different models then for code completion. Read more here - https://gitlab.com/gitlab-org/gitlab/-/issues/415210 ### Code Generation Use Cases There will be multiple different use cases for generating code within the product, which should all use the same "engine" underneath. * As part of a code file by writing a comment, new function, new class * By using the chat * single functions or code snippets (e.g. regex for XYZ) * Complete files (e.g. spec file) * Setup a new project * Scaffold a new part of the project * Rewrite from one language/setup to another ## Code completion The target for mid code work is to have fast and short code completions that make sense. For this we might use a different model and/or prompts. ## Steps forward * [ ] Implement an REST endpoint with the 1:1 Input/Output of current [Model Gateway API endpoint](https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist#completions) to call code-gecko * [x] Investigate incoming request if code generation or code completion * [x] Send incoming code generation requests to code-bison * [x] 2 different paths of completing requests in our setup, iterate towards different prompts for the different 2 things * [ ] Start enrichment pipeling for code generation which is then used by Code Suggestions and Chat as a Tool ### Notes * use the same multi provider setup we use in Chat to actual build prompts differently for each provider * Self Manage implementation is going to connect to a new system (based on the current model gateway) which is called AI gateway to forward requests to 3rd party providers, which will also enable us to bring all AI features with this to SM * We can move from REST to GRPC in connection from SM to AI Gateway * Investigate if .com is going through AI gateway or directly to AI providers (first step for sure directly to be fast)