Additional borrow and redirection for Code Suggestions
To help with the additional 3rd party investigations into AI and long term support of WebIDE/AI Enablement integration we are adding additional members to the team. The goal is to provide effectively 8 people to this effort. Here's is the proposed strategy for resolving who and when these people are assigned.
Proposed focus:
- 3rd party models - Google/Anthropic evaluations, integrations, and vendor relations. More than likely additional vendors to consider.
- AI Enablement - Long term integration point for all requests of AI modeled features. Provides controls and abstractions to allow easier development from domain experts. Also needs to be the control point to switch between vendors dynamically.
3rd party models
To support 3rd party model evaluation we will redirect people involved in AI scaling from here https://gitlab.com/product-private/private/code-suggestions/ide-borrow/-/issues/2. This means that short term we will be limited in scaling for our internal solution, but we believe this is the right decision given the options provided by 3rd party vendors.
AI Enablement
To support the AI Enablement effort we need to evaluate and execute on the following:
- Supporting the GitLab Workflow Extension and other IDEs as an integration for AI features
- Supporting Code Suggestions within the abstraction layer
Strategy
Objectives 1 and 2 can be done in parallel, and will require 2 + 2 ICs to pursue the work separately. Objective 1 is decoupled from the Code Suggestions feature, because we must support an IDE + Abstraction Layer integration in order to offer support for other AI features. Objective 2 will accomplish moving the Code Suggestions functionality itself out of the Model Gateway and into the Abstraction Layer, without impacting the Code Suggestions functionality that exists in production today. Finally, we will be able to connect these 2 pieces and offer support in the new GitLab Workflow Extension for Code Suggestions within the Abstraction Layer. This completes the exit criteria for the borrow.
Team Members
Team and members identified for this effort
Member | Title | Role |
---|---|---|
Mikołaj Wawrzyniak | Staff Backend Engineer, Analytics:Analytics Instrumentation | IDE + Abstraction layer support |
Pavel Shutsin | Senior Backend Engineer, Plan:Optimize | IDE + Abstraction layer support |
Max Woolf | Staff Backend Engineer, Analyze:Product Analytics | Code Suggestions + Abstraction layer support |
Tan Le | Senior Fullstack Engineer, AI Assist | Code Suggestions + Abstraction layer support |
Andras Herczeg | Backend Engineer, AI Assist | Code Suggestions + Abstraction layer support |
Alexander Chueshev | Senior Backend Engineer, Machine Learning | Code Suggestions + Abstraction layer support |
Sebastian Rehm | Engineering Manager, Analytics Instrumentation | DRI for this borrow and exit criteria |
Timeline for transition:
- Objective 1- IDE + Abstraction Layer has some research required and unknowns. Engineers should wrap up any in-flight product group work beginning 2023-06-19 with the expectation to complete anything in-flight by 2023-06-23. During this time, they should expect to be available to begin research and investigation discussions on 2023-06-23.
- Objective 2 - Code Suggestions + Abstraction Layer can begin work immediately. Engineers should expect to transition on 2023-06-19.
Onboarding
-
Join the #ai_enablement_team Slack channel -
Use this template to open an access request for the group's GCP account and use the snippet to configure it -
Use this template to open an access request to Anthropic -
Configure your GDK to test AI features locally -
This script can be used to bootstrap
-
-
Keep track of all work in the form of child-epics and issues -
Know who to reach out to help: - AI Enablement PM: Torsten Linz or Sarah Waldner
- AI Enablement EM: Michelle Gill or Tim Zallmann
- AI Enablement Team Members: In the above Slack channel or labeled here as AI Enablement
- Eligible Reviewers or Maintainers: Prioritizing AI Enablement first, and others familiar with AI features in general next
Objective 1
-
Collaborate with the groupide on a direction for integrating features into the GitLab Workflow Extension. Solidify a path forward regarding integrating feature-by-feature, one endpoint to the Abstraction Layer, or some other thing -
Align this plan with all other IDEs that are currently supporting Code Suggestions -
Collaborate with Torsten on integrating the GitLab Chat to the IDE extensions -
Create an architectural diagram and implementation plan for supporting Code Suggestions and the GitLab Chat within the IDE extensions. This epic is not yet defined but can be used for this purpose. -
Implement usage tracking/telemetry that will enable us to understand quality of Code Suggestions (via acceptance rate at this time), and quality of the GitLab Chat responses (undefined at this time)
Objective 2
-
Understand how to implement a new AI action in order to add support for a new Code Suggestions endpoint -
Keep track of any feature flags you create -
Understand how Code Suggestions is currently implemented in the IDE today - what prompts does it pass, how often does it work (on keystroke for example)? This should be backwards compatible. -
Understand how to support multiple providers -
Collaborate with Quality in order to make sure performance is optimal
Information: