## Background Root cause analysis has been well received among the early adopters of the feature as it drastically reduces the time required for scanning through long job logs and understanding the core reason for job failure. This Epic lays down the vision for the feature pre and post GA to move it forward in maturity and integrate with Duo workflow. ## Informing research/studies 1. [Heuristic evaluation of RCA](https://gitlab.com/gitlab-org/gitlab-design/-/issues/2561#note_1896771577 "Heuristic Evaluation of each AI capability") 2. [RCA (Duo integrated workflow) usability study](https://gitlab.com/gitlab-org/ux-research/-/issues/3077#note_1995784873 "RCA + Duo Usability study") 3. [Deep dive study - Workflow: Fix failing pipeline](https://gitlab.com/gitlab-org/ux-research/-/issues/3105#note_1993407540 "[Draft] Duo Workflow - Deep Dive Workflow Interviews - Fix a failing Test / Fix Pipeline") 4. [UX Themes exercise](https://www.figma.com/board/CtUfA874FYaaL0qfndE6X6/Pipeline-execution---AI%2FML-UX-theme?node-id=1-574&t=WWeSy3p7R7jPBDBO-0) ## Video walkthrough TBD <details> <summary> #### :white_check_mark: Changes to be made before GA </summary> <table> <tr> <th>Change</th> <th>Status</th> <th>Dependency outside PE</th> </tr> <tr> <td>Integrate RCA flow with Duo</td> <td>In progress</td> <td> Problems with slash command not appearing, response not streaming. 1. [Missing loading animation](https://gitlab.com/gitlab-org/gitlab/-/issues/466191 "Missing loading state when quick (slash) command is triggered") 2. [Missing streaming responses when using slash commands](https://gitlab.com/gitlab-org/gitlab/-/issues/461179 "DuoChat: Missing streaming on Slash commands") 3. [Slash commands not displaying](https://gitlab.com/gitlab-org/gitlab/-/issues/469530 "Slash command not displaying in Duo Chat after switching pages") </td> </tr> <tr> <td> [Implement the slash command](https://gitlab.com/gitlab-org/gitlab/-/issues/471244 "Implement /troubleshoot on failing job page [GitLab]") `/troubleshoot` </td> <td> :white_check_mark: </td> <td> In partnership with AI framework team [Issue](https://gitlab.com/groups/gitlab-org/-/epics/14404#note_1987424709 "/troubleshoot slash command on GitLab Duo Chat") </td> </tr> <tr> <td> [Make layout changes to the response generated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/159018 "Change the prompt of the RCA feature") </td> <td> :white_check_mark: </td> <td>None</td> </tr> <tr> <td> [Improve discoverability of `Troubleshoot` button](https://gitlab.com/gitlab-org/gitlab/-/issues/473091 "Make troubleshoot button more discoverable on job logs page") </td> <td> :white_check_mark: </td> <td>None</td> </tr> </table> ## Vision themes: NOW </details> #### :books: Theme: Boost user trust in RCA's response By including as much available context as possible and allowing them to make a judgement on the effectiveness of a suggested fix. <table> <tr> <th>Issue</th> <th>Purpose</th> <th>Scheduled ?</th> </tr> <tr> <td> [Provide .gitlab-ci.yml as additional context](https://gitlab.com/gitlab-org/gitlab/-/issues/461511 "Root Cause Analysis: Provide .gitlab.ci.yaml as additional context") </td> <td>Improve response quality</td> <td> * [ ]   </td> </tr> <tr> <td> [Add runner context to RCA](https://gitlab.com/gitlab-org/gitlab/-/issues/473760 "Add runner context to root-cause analysis response") </td> <td>Improve response quality</td> <td> * [ ]   </td> </tr> <tr> <td> [Provide job variable context](https://gitlab.com/gitlab-org/gitlab/-/issues/476603 "Provide Job Variable context") </td> <td>Improve response quality</td> <td> * [ ]   </td> </tr> <tr> <td> [Provide RCA with code or commit context](https://gitlab.com/gitlab-org/gitlab/-/issues/473068 "Provide Root cause analysis with MR or commit context") </td> <td>Improve response quality</td> <td> * [ ]   </td> </tr> <tr> <td> [Analyze how to improve correctness](https://gitlab.com/gitlab-org/gitlab/-/issues/479164 "Analyze the example fix responses scored < 2") </td> <td>Improve response quality</td> <td> * [x]   </td> </tr> <tr> <td> [Allow users to verify the context that RCA's response is based on](https://gitlab.com/gitlab-org/gitlab/-/issues/477904 "Allow users to see the context that RCA response is based on") </td> <td>Provide transparency into the working of the model</td> <td> * [ ]   </td> </tr> <tr> <td> [Allow users to provide feedback on response generated](https://gitlab.com/gitlab-org/gitlab/-/issues/461667 "Allow users to provide feedback on the response generated from the RCA feature") </td> <td>Enable users to provide feedback and use the feedback to improve response quality</td> <td> * [x]  17.4 </td> </tr> <tr> <td> [Improve feedback mechanism in Duo chat](https://gitlab.com/gitlab-org/gitlab/-/issues/426503/ "Improve the feedback mechanism in the Duo chat") </td> <td>Enable users to provide feedback and use the feedback to improve response quality</td> <td> * [x]   </td> </tr> <tr> <td> [Discussion: How to use the feedback received to improve responses?](https://gitlab.com/gitlab-org/gitlab/-/issues/478537 "Discussion: How to use the feedback received to improve responses?") </td> <td>Enable users to provide feedback and use the feedback to improve response quality</td> <td> * [x]   </td> </tr> <tr> <td> [Improve response when errors and failures are sometimes outside the truncation limit](https://gitlab.com/gitlab-org/gitlab/-/issues/474303 "Improve response when errors and failures are sometimes outside the log truncation window") </td> <td>Ensure the logs outside of truncation limit is considered</td> <td> * [ ]   </td> </tr> </table> #### :books: Theme: Make it easier to understand and act on the failure root cause Make enhancements to the drawer component and response layout for better usability. <table> <tr> <th>Issue</th> <th>Purpose</th> <th>Scheduled for</th> </tr> <tr> <td> [Improve example fix presentation in RCA to help users act without confusion](https://gitlab.com/gitlab-org/gitlab/-/issues/478120 "Improve example fix presentation in RCA to help users act without confusion") </td> <td>Improve usability</td> <td> * [ ]   </td> </tr> <tr> <td> [Allow drawer to be resized](https://gitlab.com/gitlab-org/gitlab-services/design.gitlab.com/-/issues/1588 "Allow for a drawer to be resized") </td> <td>Improve usability</td> <td> * [ ]   </td> </tr> <tr> <td> [Investigate when chat should remain open when switching between pages](https://gitlab.com/gitlab-org/gitlab/-/issues/460659 "Investigate when chat should remain open when switching between pages") </td> <td>Improve usability</td> <td> * [ ]   </td> </tr> <tr> <td> [Reduce response verbosity](https://gitlab.com/gitlab-org/gitlab/-/issues/473333 "RCA prompt iteration to improve 'very unhelpful' responses and reduce verbosity") </td> <td>Improve usability</td> <td> * [x]  17.5 </td> </tr> </table> ## #### :books: Theme: Streamline the general workflow for fixing failed pipeline Boost SCM to CI funnel by making core CI offerings more attractive. <table> <tr> <th>Issue</th> <th>Purpose</th> <th>Scheduled?</th> </tr> <tr> <td> [Improve the job log search experience](https://gitlab.com/gitlab-org/gitlab/-/issues/367917 "Improve the job log search experience") </td> <td>Shorten time to a successful job</td> <td> * [ ]   </td> </tr> <tr> <td> [Present failure overview for a job in the drawer](https://gitlab.com/gitlab-org/gitlab/-/issues/478548 "Present failure overview for a job in the drawer") </td> <td>Shorten time to a successful job</td> <td> * [ ]   </td> </tr> </table> ## Vision themes: NEXT #### :books: Theme: Help users to locate a pipeline failure faster and more efficiently Harness GitLab ecosystem to differentiate from competitor products + Improve developer productivity <table> <tr> <th>Issue</th> <th>Purpose</th> <th>Scheduled for</th> </tr> <tr> <td> [Allow users to access root cause analysis at pipeline level](https://gitlab.com/gitlab-org/gitlab/-/issues/478139 "Allow users to access root cause analysis at pipeline level") </td> <td>Shorten time to a successful pipeline</td> <td> * [ ]   </td> </tr> <tr> <td></td> <td></td> <td></td> </tr> </table> #### :books: Theme: Provide end to end automated support to user for fixing 
failed pipeline Harness GitLab ecosystem to differentiate from competitor products + Improve developer productivity <table> <tr> <th>Issue</th> <th>Purpose</th> <th>Scheduled for</th> </tr> <tr> <td> [Duo workflow: Fix failed pipeline](https://gitlab.com/gitlab-org/gitlab/-/issues/478552 "Duo workflow: Fix failed pipeline") </td> <td>Shorten time to a successful pipeline</td> <td> * [ ]   </td> </tr> </table> ## Future vision themes: LATER #### :books: Theme: Automate the complete workflow of identifying and
fixing a pipeline failure Make enhancements to the drawer component and response layout for better usability. | Issue | Purpose | Scheduled for | |-------|---------|---------------| | | | |