RFC: On Implementing a Retrospective Bot Across all RFH sections and groups

Need

Currently, retrospectives for Request for Help issues rely on manual processes. Team members must remember to conduct retrospectives and document learnings, which can lead to:

  • Missed retrospective opportunities when issues are closed quickly
  • Inconsistent retrospective practices across the team
  • Lost learning opportunities from resolved issues
  • No standardized prompts to guide retrospective thinking

The Security team has successfully implemented an automated retrospective bot (see example) that triggers when issues are closed, ensuring consistent retrospective practices.

image

Approach

Implement a triage-policies bot similar to the Security team's retrospective bot, configured for all Request for Help issues

Proposed Workflow

Issue Closed → Bot checks conditions → Matches criteria + no retro labels
    ↓
Bot posts comment + adds ~"retrospective::started"
    ↓
Support Engineer responds with retrospective thoughts
    ↓
Support Engineer changes label to ~"retrospective::completed" or ~"retrospective::not needed"
    ↓
Process complete (bot won't trigger again due to forbidden labels)

Technical Implementation

The Bot would be based on the Security team's existing approach (triage-policies.yml) and would work as follows:

  1. Trigger Conditions: Bot activates when:

    • Issue is closed
    • Issue matches specific labels (e.g., Help group::xxx")
    • Issue does NOT have ~"retrospective::started", ~"retrospective::completed", or ~"retrospective::not needed"

  2. Bot Actions:

    • Posts a standardized comment prompting for retrospective
    • Adds ~"retrospective::started" label
    • Optionally mentions relevant team members or DRI

  3. Required Labels (to be created):

    • ~"retrospective::started" - Bot has triggered, awaiting response
    • ~"retrospective::completed" - Retrospective documented
    • ~"retrospective::not needed" - Team determined retrospective not necessary

  4. Bot Comment Template (example):

    Retrospective Requested

    This issue has been closed. Please take a moment to reflect on:

    • What went well? What worked effectively?
    • What could be improved? What challenges did we face?
    • Action items: What specific changes should we make?
    • Documentation: Should any handbook pages be updated?

    Once you've added your retrospective thoughts, please update the label to:

    • ~"retrospective::completed" if retrospective is documented
    • ~"retrospective::not needed" if retrospective is not applicable

Benefit

  1. Consistency: Ensures all qualifying issues receive retrospective consideration
  2. Learning Culture: Reinforces continuous improvement practices
  3. Documentation: Creates a standardized record of learnings
  4. Reduced Cognitive Load: Team members don't need to remember to do retrospectives
  5. Metrics: Easier to track retrospective completion rates
  6. Knowledge Sharing: Retrospectives become discoverable for future reference

The Ask for Engineering Managers

  1. Please review this issue and Vote 1 - if you would like this implemented for your Engineering Group
  2. Please review this issue and Vote 2 - if you would not like this implemented for your Engineering Group
  3. Please let us know if the proposed workflow (bot comment + labels) work for your team, or would you prefer a different approach?
  4. Do you have any concerns or feedback you would like to share?

Competition / Alternatives

Alternative 1: Manual Retrospective Reminders

  • Pros: No technical implementation needed, flexible
  • Cons: Relies on memory, inconsistent, easy to skip

Alternative 2: Different Bot Approach

Engineering managers may prefer:

  • Different trigger conditions
  • Alternative workflow (e.g., bot creates separate retrospective issue)
  • Integration with existing Support Ops tooling
  • Custom prompts for different issue types

Alternative 3: No Automation

  • Pros: Maximum flexibility, no maintenance
  • Cons: Misses the benefits of consistency and automation

Next Steps

Based on feedback:

  1. Implement the bot across RFH Issues
  2. As iteration 2, we could consider using the approach as a gateway to further innovations such as implementing dashboards similar to: Sec Section - Escalation Reduction Dashboard

References

Edited by John Lyttle