RFC: Implement Offline-Crush via tag in Zendesk

Problem Statement

Support Engineers sometimes escalate tickets to engineering prematurely without exhausting internal Support resources first.

Contributing factors might include:

  • Difficulty syncing with Crush sessions due to timezone/availability constraints
  • Slack messages requesting support can sometimes be overlooked, particularly if posted to busy channels outside their home region/working hours
  • Engineers holding onto tickets too long before seeking assistance

Proposal: "Offline Crush" Macro

Implement an "Offline Crush" Zendesk macro that enables asynchronous peer collaboration through a dedicated tag and view.

Core Concept

Any ticket where an SE is seeking assistance from the broader Support team can use the Offline Crush macro, which will:

  1. Add the offline-crush tag to make the ticket visible in a dedicated Zendesk view
  2. Add an internal note with instructions to open a fieldnote

This allows any Support team member to review and comment/provide assistance on their own schedule.

Unlike a live Crush Session that is generally limited to those on the call, support could be offered by anyone with Zendesk access (not just support engineers, but developers and potentially the sales account team).

This is an extension to the first step in the How to Get Help workflow, by providing an async option to bring the ticket before a group of peers.

Required Components

  1. Zendesk macro: "Offline Crush" that:
    • Applies the offline-crush tag
    • Adds an internal note with instructions to open a fieldnote
    • Note: Due to current system constraints, the fieldnote must be opened manually by the SE rather than automatically by the macro
  2. Dedicated Zendesk view: Displays all tickets tagged for offline crush assistance
  3. Fieldnote template: <fieldnote_offline_crush> that prompts SE to provide:
    • Clear description of the issue/blocker
    • What has been tried so far
    • Specific help needed
    • Next steps being considered

Sample Internal Note

When the "Offline Crush" macro is applied, the following internal note will be added to the ticket:

---
OFFLINE CRUSH REQUEST

This ticket has been tagged for async peer collaboration.

Please open a fieldnote to request assistance from the Support team:
1. Use the <fieldnote_offline_crush> template
2. Provide details about what you've tried and where you're blocked
3. Submit the fieldnote as an internal note on this ticket

Your request will be visible in the Offline Crush view for the team to review and assist.
---

Process Flow

When to use Offline Crush:

  • SE is stuck and needs a second opinion
  • Unclear about next troubleshooting steps
  • Needs expertise outside their comfort zone
  • Before escalating to engineering

How to request team help:

  1. Apply the "Offline Crush" macro to the ticket
  2. Follow the instructions in the internal note to open a fieldnote using the <fieldnote_offline_crush> template
  3. Complete the fieldnote template:
    • Problem: Brief description of the customer issue
    • What I've tried: Steps already taken
    • Blocker: What's preventing progress (e.g., "Can't find the error in the SOS file")
    • Help needed: Specific question or guidance requested
    • Next steps: What you're considering doing next
  4. Submit the fieldnote as an internal note on the ticket

Example fieldnote:

OFFLINE CRUSH REQUEST

Problem: Customer experiencing 500 errors on merge requests
What I've tried: Reviewed logs, checked Redis, verified Sidekiq queues
Blocker: Can't identify the root cause in the SOS file - errors seem inconsistent
Help needed: Need help analyzing the SOS file or suggestions for next troubleshooting steps
Next steps: Considering asking for additional logs, but want to confirm this is the right approach

How to provide help:

  1. Review tickets in the Offline Crush Zendesk view
  2. Add internal notes with suggestions, questions, or offers to pair
  3. Pairing sessions/live crush session can evolve from the discussion
  4. Remove tag once issue is resolved or a clear path forward is established

Benefits

  • Asynchronous collaboration - No need to wait for scheduled Crush sessions
  • Timezone-friendly - Global team can participate regardless of location
  • Increases cross-region collaboration - Enables support engineers from different regions to assist each other beyond their typical working hours
  • Lower barrier to asking for help - One-click macro makes it easy
  • Structured requests - Fieldnote template ensures consistent, helpful information
  • Keeps tickets clean - Fieldnotes keep async collaboration separate from customer-facing ticket timeline
  • Leverages collective expertise - Both senior and peer engineers can contribute
  • Reduces premature escalations - Ensures internal Support knowledge is tapped first
  • Knowledge sharing - Team learns from each other's troubleshooting approaches
  • Visibility - Leadership can identify common blockers and training needs

Implementation Considerations

Questions for discussion:

  1. Tag naming: Should it be offline-crush, async-crush, peer-assist, or something else?
  2. Macro design:
    • Is the proposed internal note format clear and actionable?
    • Should the macro set any other ticket properties (priority, status, etc.)?
  3. Fieldnote template:
    • What fields should be in the <fieldnote_offline_crush> template?
    • Should fields be mandatory or guidance text?
  4. Response expectations:
    • Should there be a target response time? (e.g., within 24 hours)
    • Who monitors the view to ensure tickets get attention?
    • What happens if no one responds?
  5. Relationship to existing processes:
    • How does this complement live Crush sessions?
    • When should SEs still use Support Pod escalations?
    • When should SSCs be contacted directly vs. using Offline Crush?
  6. Zendesk view configuration:
    • What filters/sorting should the view have?
    • Should it show ticket age, priority, or other metadata?
    • Should there be separate views by region or category?
  7. Success metrics:
    • Reduction in engineering escalations
    • Time from tag application to first response
    • Percentage of offline crush tickets resolved without escalation
    • SE satisfaction with the process
  8. Pilot approach:
    • Should we test with a specific team or region first?
    • What's the timeline for evaluation?
  9. Future automation:
    • Can the fieldnote creation be automated in the future to reduce manual steps?

Proposed Next Steps

  1. Gather feedback from Support team on this RFC (2 weeks)
  2. Define process guidelines and finalize the <fieldnote_offline_crush> template based on feedback
  3. Work with Support Ops to create:
    • Zendesk macro with the internal note
    • <fieldnote_offline_crush> template
    • Dedicated Zendesk view with appropriate filters
  4. Document the process in Support handbook
  5. Launch pilot program with volunteer team
  6. Collect feedback and iterate (4-6 weeks)
  7. Roll out globally with training and communication

Open Questions

  • Should we have different macros for different types of help requests?
  • How do we encourage participation from senior engineers without creating obligation?
  • Should there be a companion macro to remove the tag and close the request?
  • What's the best way to handle tickets that receive no response within a reasonable timeframe?

Please share your thoughts, concerns, and suggestions below. All feedback is welcome!

Edited by Chantal Lawrence