Using Gitlab DUO Agent to create a Foundational Agent workflow

STELLA

(Support Through Engaged LLM Assistant)

AI Support Chatbot to assist customers with self-Service. My goal is to provide Guided Flows navigating options to desired outcome. This allows our customers and internal users the flexibility to choose what they want (sometimes they don't know and have to be prompted), giving them the best outcome. We plan to launch internally first to allow us to test fully before customer visibility.

  • Who would use Stella: Customers, GitLab Employees, Partners
  • Where would Stella be launched from: Support Portal

Objective

Launch STELLA (Support Through Engaged LLM Assistant), an AI-powered chatbot that guides customers through self-service resolution paths using intelligent conversational flows, reducing dependency on support while improving resolution speed and customer satisfaction.

In Scope

  • Guided flow experiences that navigate users through decision trees to resolve common issues
  • Integration with GitLab Duo LLM for natural language understanding and response generation
  • Authenticated user experience with personalized context and recommendations
  • Initial use cases available (example: 2FA (Two-Factor Authentication) issue resolution)
  • Internal launch to GitLab team members as pilot users
  • Connection to existing knowledge base content for supplementary information
  • Analytics and telemetry to track chatbot performance, user satisfaction, and resolution rates
  • Feedback mechanisms for continuous improvement
  • Handoff workflow to human support when chatbot cannot resolve issue

Stella features & requirements:

  • Stella must be accessible from Support.GitLab.com (Support Portal)
  • Paths to resolution: Customer has an issue, needs an answer. Customers can ask, or choose an option that aligns with their needs. Selections are made to drive them towards an end goal (Answer) based on choices they take along a path to get to the exact information they require.
  • Download accessible (At end of path) - Once an answer is provided (Details) the customer should be able to download the information to retain history of interaction
  • Contact Support with prepopulated info from journey - If support is still required, the information based on the choices already made by the customer should be retained and sent to support, so that the customer doesn't need to start over with all the same information.
  • Return to start - The customer should be able to navigate back to the start (home) to restart a path or search as needed.
  • Breadcrumb trail (navigatable) - The path should be navigatable, tracked at each selection, so that if the user should need to see where they are (the path choices) it's apparent to the customer
  • Back Button - a back button should allow the customer to navigate back a step as needed should they need to make a different choice
  • Keyword Search to assist guidance
  • Text Fields to allow customers to add info manually
  • Add attachments
  • Radio button (?) Option for choices options to choose from > make them clickable to continue forward

USE CASES and Slides:

KPIs and Outcomes:

  • 75% self-service resolution rate for guided flow interactions
  • 40% reduction in L1 support tickets for targeted use cases
  • 4.0+ average user satisfaction rating (5-point scale)
  • 80%+ completion rate for initiated guided flows
  • < 2 minute average time to resolution for successful self-service

Metrics to Track:

  • Self Service Resolution Rate
  • First Contact Resolution Rate (FCR)
  • Completion Rate (through paths)
  • Average Time to Resolution
  • Escalation Rate (Did they have to contact support)
  • Resolution Accuracy
  • number of Users per week/Month/Year
  • Return user rates
  • Abandon Rate
  • Feedback rates
  • Ticket Deflection
  • Cost Savings
  • Support Engineer Capacity (freed up for other things)
  • Consistency (Paths)

Deflection KPIs:

  • Stella resolution rate: 60% of conversations resolved without human handoff
  • Ticket deflection rate: 45% reduction in tickets for covered use cases provided (more will re-evaluate)
  • Self-service completion rate: 70% of Stella interactions ending in "problem solved" confirmation
  • Containment rate: 55% of all support inquiries handled end-to-end by Stella

Abandonment KPIs:

  • Conversation abandonment rate: <25(?)% (users leaving mid-conversation)
  • First-response abandonment: <10(?)% (users leaving after Stella's first response)
  • Handoff request rate: <30(?)% (users explicitly Submitting ticket for human intervention)
  • Fallback frequency: <10% (Stella unable to provide relevant response and ticket has to be opened)

Expected Outcomes:

  • Increased article Consumption
  • Increased Knowledge Base
  • Decreased support tickets
  • Increase in Self-Service Rate
Edited by Kirsty Allen