This epics is targetted around collecting all issues + discussion that are targetted especially to improve performance for users of Code Suggestions. Performance contains both Latency improvements and Perceived performance. ## Possible Improvement topics ### Latency improvements * Look for Improvements in * Rails Layer * Model gateway * Infrastructure setup * Prompt improvements for code generation * Fix long running prompts * Websockets might be a possibility to cut out DNS + Rails time * Grpc from GL Instance to Model Gateway * Local Code LLM (next to SM instance - self hosted) * Local Machine Code Completion LLM ### Perceived performance improvements * [Better IDE UX Indicator when we request next to line](https://gitlab.com/gitlab-org/gitlab/-/issues/418999) * [larger better status bar indicator is in latest version](https://gitlab.com/gitlab-org/gitlab-vscode-extension/-/issues/765#note_1672638789) * Streaming for code generation at least (later maybe another code completion model supports streaming) if we move forward * Better local caching, type-in-cache