Learning GitLab Duo - Shem Gyll

module-name: "GitLab Duo"
area: "Product Knowledge"
gitlab-group: '"AI Framework" "Cloud Connector" "Duo Chat" "Editor Extensions" "Code Creation"'
maintainers:
  - Austin_Pierce

Introduction

This training module is intended to provide Support Engineers with a better understanding of GitLab Duo and the underlying AI technology.

Goals of this training module

At the end of this module, you should be able to:

• answer Duo-related tickets (or have a good idea of how to start troubleshooting Duo-related tickets/problems)
• demonstrate an understanding of the role Duo plays in GitLab
• support peers with questions about Duo

General Timeline and Expectations

• This issue should take you 2 weeks to complete.

Reminders:

• This is a Public issue; don't include anything confidential.
• You should do stage 0 first and the final stage last. Outside of that, you can perform tasks from the stages in any order you like.

Stage 0: Create Your module

1. Create an issue using this template by making the Issue Title: "Learning GitLab Duo - your name"
2. Add yourself as the assignee
3. Consider setting a milestone and/or a due date to help motivate yourself!
4. Update your Support Team yaml file to indicate that you've started learning this knowledge area:

knowledge_areas:
- name: GitLab Duo
  level: 1

Consider using the Time Tracking functionality so that the estimated length for the module can be refined.

Stage 0.5: Foundational Concepts

• Done with Stage 0.5

Goal Gain a basic understanding of the underlying AI technologies and methods that power GitLab Duo.

1. Watch What is Machine Learning? (approx. 3 minutes)
2. Watch How Neural Networks Work (approx. 5 minutes)
3. Watch How Chatbots and Large Language Models Work (approx. 8 minutes)

For further exploration, consider the following resources:

• AWS Cloud Computing Concepts Hub: Deep Learning
• AWS Cloud Computing Concepts Hub: Large Language Models (LLMs)
• Khan Academy/Code.org Computing Unit 2: How AI works

Stage 1. Overview of GitLab Duo

• Done with Stage 1

Goal Understand the basics of what GitLab Duo does, what is supported, how to purchase, and how data collected is used.

1. Read the GitLab Duo overview to familiarize yourself with the generally available (GA), beta, and experimental Duo features.
2. Read GitLab's Support policy for experiment, beta, and GA features to understand the level of support provided for each.
3. Read Subscription Add-ons to understand how customers purchase Duo and assign Duo seats to users on GitLab.com and self-managed instances.
4. Learn about Duo Enterprise, a higher-tier Duo add-on. Note that only Ultimate customers are eligible to purchase Duo Enterprise.
   1. GitLab Duo Enterprise Blog Post
   2. View a walkthrough of GitLab Duo Enterprise features
5. Read about GitLab Duo data usage to learn how customer data is used (or not).

Keeping up with changes

GitLab is moving fast on AI. Below are some ways you can keep up with recent and upcoming developments.

1. Consider subscribing to the AI Powered Weekly Updates label
2. Consider turning on notifications for new #spt_pod_ai posts
3. Review release notes for new major and minor versions of GitLab. As of writing this in July 2024, there's a very good chance that Duo improvements and additions will be in each release.
4. Consider adding some of these Slack channels
   • #ai_strategy
   • #s_ai-powered
   • The Slack channel for any AI product group you're particularly interested in. For example:
     • #g_duo_chat
     • #g_editor-extensions
     • #g_code_creation
     • #g_ai_framework
     • #g_cloud_connector
5. Consider attending All Hands meetings, where AI announcements and progress are discussed.

Stage 2. GitLab Duo in practice

• Done with Stage 2

Goal Learn how to configure GitLab Duo, and practice using core features like Chat and Code Suggestions.

Set up a self-managed instance and enable GitLab Duo

If you already have a working Sandbox Cloud instance on GitLab 17.3 or higher, skip steps 1-2.

1. If you haven't done so already, take the time to get acquainted with the GitLab Sandbox Cloud. Ensure you're capable of deploying a basic Omnibus instance.
2. Using GitLab Sandbox Cloud, deploy a single node Omnibus instance on version 17.3.0 or higher. The template support-resources-template-v2-*** can be used for that. Confirm that it has at least a Premium license associated with it.
3. Follow the instructions to obtain a GitLab Duo license by "purchasing" a subscription in customersdot staging (Option 2).
   • A Team Member license is a Legacy license type which cannot sync to customersdot; ordering a subscription is the only way to get a cloud license currently.
   • The linked documentation is for GDK instances, but it also works for standard Omnibus instances.
4. Activate the subscription license on the instance.
   • If Duo features don't work after adding the license, try manually synchronizing your subscription. The initial automatic synchronization can take up to 24 hours.
5. Read Configure GitLab Duo on a self-managed instance. By default, a basic Omnibus instance deployed with GitLab Sandbox Cloud meets the connectivity requirements for a self-managed instance, so you shouldn't have to change anything.

Control GitLab Duo access

1. For some GitLab Duo features, like Code Suggestions, you must assign seats to users for them to be able to use the features. Create a user on your instance, and assign a Duo Pro seat to both your root user and newly created user.
2. Read the introductory section of Turn off GitLab Duo features.
3. Practice turning Duo off and on at the:
   1. Group level
   2. Project level
   3. Instance level
4. Turn on beta and experimental features on your instance.

Try Duo Chat

1. Read about The context Duo Chat is aware of.
   • Go to this issue, open Duo Chat on that page, and ask a question about the issue.
   • Go to this file, open Duo Chat on that page, and ask a question about the file.
2. Review Ask Duo Chat. Try out three different uses of Duo, and record which ones you tried:
   • Use:
   • Use:
   • Use:
3. Review the error codes in Duo Chat troubleshooting.

Try Duo Code Suggestions and Explanation in the IDE

1. Read the docs overview of Code Suggestions
2. Read about supported extensions and languages
3. Configure GitLab Duo in the VSCode IDE
   1. Click on the VSCODE IDE editor extension link under Supported Editor Extensions. Follow the instructions for configuring the GitLab Duo extension, and connect it to your Duo-enabled self-managed instance. If you have questions or trouble setting up the extension, don't hesitate to ask for help in #spt_pod_ai.
4. Once your IDE is configured to use GitLab Duo:
   1. Do one or more of our documented Code generation prompts in the language of your choice.
   2. Optional. if you want to dive deeper, do one of the documented development challenges or code along with a GitLab Duo Coffee Chat recording.

Try out other GitLab Duo features

1. Optional. There are several more Duo features. Try out one or more of them to further familiarize yourself with the capabilities of GitLab Duo.

Stage 3. Understanding the GitLab Duo Infrastructure

• Done with Stage 3

Goal Gain an understanding of the technical architecture of GitLab Duo.

1. Read AI Architecture to learn about the AI Gateway, the model-agnostic abraction layer built for developing Duo features.
2. Optional: Read the AI Gateway blueprint to learn about the historical motivations and goals of this architecture.
3. Read Cloud Connector Architecture to understand how the AI Gateway handles traffic from GitLab.com, self-managed instances, and Dedicated instances.
   1. Optional: Bookmark and watch Life of a Cloud Connector Request on GitLab Unfiltered (recorded September 2024)
4. Read the introduction to the GitLab Language Server, a common interface for all GitLab IDE extensions.
5. Read about GitLab Duo data usage.
6. Consider reading about Action Cable
   • Action Cable is used to integrate websockets with the rails application. Subscriptions are the real-time websocket channels that communicate with rails. This is how communication is done with the Duo Chat feature.
   • Note that ActionCable connection and subscription events are logged in production_json.log.
7. Optional: Watch a walkthrough of setting up a Mistral vLLM instance and configuring a Duo Self-Hosted Model - video, slides.

Stage 4. Handling GitLab Duo tickets

• Done with Stage 4

Goal Learn how to troubleshoot Duo, the scope of support we offer for Duo features, the product groups involved in Duo, and where to get help with difficult Duo-related tickets

Troubleshooting GitLab Duo

1. Review and bookmark the GitLab Duo Triage runbook. This is an excellent resource for troubleshooting a wide range of Duo-related issues.
2. Review running a health check for GitLab Duo
   1. Run a health check on the self-managed instance you set up in Stage 2
3. The editor extension.
   1. Many Duo-related customer tickets are about questions or issues with the editor extensions. It's important to understand how to use the core troubleshooting options available for these types of problems.
   2. Review Troubleshooting Code Suggestions
   3. Review this ticket, noting the troubleshooting steps suggested by the Support Engineer.
   4. Try enabling and reviewing the GitLab:Debug logs and Language Server debugs logs.
4. Duo Chat
   1. Review this ticket, and take special note of the role licenses can play in troubleshooting Duo.
   2. Review this ticket, which highlights how important it is to pay attention to the GitLab version when troubleshooting a Duo feature. This is true for any part of the product, but is especially true for Duo at the moment, where major feature additions and changes are rolling out quickly.
5. Review the description section of gitlab-com/gl-infra/production#18329 (closed), and this comment from the AI Framework Engineering Manager, David O'Regan. Take note of the common trends and pain points identified from Duo incidents.
6. Know where to look in Kibana to identify issues
   1. Consider checking out the Duo Chat dashboard
   2. Check out the ApplicationCable::Connection controller in the rails logs (production_json.log)
7. Optional but recommended. Search Zendesk for closed Duo tickets, and review 3-4 of them. This will give you a sense of the Duo-related problems customers encounter, and how to troubleshoot them. (Use search keywords like "Duo", "Duo Pro", "Duo Chat", "Code suggestions", etc.)
8. Optional. Update the docs and/or one of the Duo runbooks to clarify how a Duo feature works, or add/update a troubleshooting tip for a Duo feature.
   1. Docs MR:
   2. Docs MR:

Scope of support for Duo features

1. Read the GitLab Duo section of the Statement of Support. Note that customer problems with "the accuracy of, [or] the responses generated by GitLab Duo Pro" are out of scope.

Where to go for help

1. You can ask for help in the #spt_pod_ai channel in Slack
2. Duo is developed across multiple product groups, and sometimes it's hard to figure out which group to ask for help. This is a known challenge that the AI teams are trying to address. In the meantime:
   1. Bookmark the Features by Group page. It can help you determine which group(s) own any given generally available Duo feature.
   2. Don't worry too much about asking the wrong group. Do your best and you'll be redirected if needed.
3. Knowing about the key AI groups, how they're connected to each other, and how to contact them is invaluable when navigating a problem with Duo. Note that this is not an exhaustive list of groups that own AI features.
   1. Review the Core Platform Section > Systems > Cloud Connector group, which ensures that Duo features are available across all deployment types (GitLab.com, self-managed, and Dedicated).
   2. Review the organizational charts of the Data Science section > AI-powered stage, and the Dev section > Create stage.
   3. The AI-powered stage includes two groups involved in work relevant to Support:
      1. Review the Duo Chat group group, which owns (no surprise) Duo Chat.
      2. Review the AI Framework group, which owns the AI Gateway (a model-agnostic abstraction layer used by other product groups to develop Duo features) and Duo settings.
      3. Custom Models is a future feature that will allow users to customize Duo features, including training Duo on their own data. You can read more in Group direction - Custom Models.
   4. The Create stage includes three groups involved in Duo features:
      1. Review the Code Creation group, and note the features they own.
      2. Review the Code Review group, which owns Suggested Reviewers.
      3. Review the Editor Extensions group, which implements Duo features in various IDEs.
4. Note that you can create a Request for Help issue for each of these groups:
   • Cloud Connector
   • Duo Chat
   • AI Framework
   • Editor Extensions
   • Code Creation
   • Code Review

Stage 5. Final Stage

• Done with Stage 5

1. Have your manager review this issue.

2. Manager: schedule a call (or integrate into 1:1) to review how the module went once you have reviewed this issue.

3. Update your Support Team yaml file to indicate that you're ready to work on tickets in this knowledge area:

   modules
   - GitLab Duo
   
   knowledge_areas:
   - name: GitLab Duo
     level: 2

You will now be listed as "Ready to work tickets" in GitLab Duo on Skills by Person page.

🎉 Congratulations! You made it, and are well-prepared to assist customers and colleagues with Duo-related tickets. 🎉

If you think of any improvements to this module, please submit an MR! The file is located in an issue template in the 'support-training` repository.