Next iteration of Slack bot breach notifications for all SGGs
Update: Changed focus
Through the issue lifetime, this changed from the original proposal. Instead of aligning on a single type of notification that Ops maintains for us, we agreed to maintain notifications ourselves and only relying on Ops to notify us via Zendesk.
In https://gitlab.com/gitlab-com/support/toolbox/sgg-notifications we can define custom notifications per SGG. @manuelgrabowski is acting as the DRI for that part / maintains that project.
Problem Statement
What is the problem?
We have a current KPI goal of hitting 95% FRT SLA on High priority tickets. Based on the feedback provided in the ideas docs (available in this epic under the SGG column), all SGGs agreed that having a Slack channel notification on High's would be helpful. This proposal seeks to identify a single Slack bot alert that will help support our goal in a unified way (across all SGGs).
Context: There was mention that the Slack bot may become fully deprecated, thus this proposal was created to identify a streamlined alert that could be applied to all SGG's as a compromise (this will help reduce the overhead management efforts that is required to implement customized Slack bot alerts on a per-SGG basis).
Why is this a problem?
To date, we have achieved 95%+ FRT SLA on High's one out of the last 8 weeks. The current Slack notification is too generic and does not highlight when a High
priority ticket is close to breach.
Additionally, the Support team Slack bot is a useful tool for SGG collaboration, thus losing that entirely to help our groups stay on top of FRT / High priority tickets would be considered a detriment to group collaboration and visibility/consistency. The ZD notification App does not provide the group with an easy way to collaborate in the same way as Slack, and depends on each individual to subscribe to the notifications.
Proposal
To help minimize the overhead in management so we can retain the bot, this issue proposes that we agree to a single streamlined alert and messaging that would be applied to all SGG's.
Since our current goal is to achieve 95% FRT SLA on new High priority tickets, a couple of ideas for alerting include:
- Keep the current Slack bot notification (alert on any ticket set to breach in 2 hrs), but enhance the messaging (as was proposed under #4736 (closed)), with the addition to include the priority of the ticket, along with an emoji (siren or ... ?) to highlight when a new High priority ticket is about to breach.
Example of Option 1 is in this comment.
or
- Only alert in the channel when a High priority ticket is close to breach (2 hrs or less), with the same improved messaging (ticket ID, ticket subject, org, and form).
Example of Option 2 could look like:
‼ This HIGH priority SaaS Ticket #123456 is set to breach within 2 hours‼
If there's another idea someone wants to add, please comment below.
Otherwise, perhaps as a quick way to see where we are at:
- Add a
1️⃣ for the first idea (keep existing bot, improve messaging) - Add a
2️⃣ for the second idea (alert on High's only) - Add a
🎱 for something else (and then comment with your idea)
We'll set the timeline to something a little shorter (end of this week) to gather all feedback and then make the final decision from there. Thank you, everyone.
DRI
@izzyfee to act as the DRI for this issue with @jcolyer to act as DRI for Ops implementation.
Required Resources
This will require collaboration with Ops to agree to a single (shrinked scope) for the bot alert that will be applied to all SGG's. The bot will be used globally in nature (all SGGs), and not customized per-SGG.
All SGG's need to agree on the alert.
Desired Outcome
What does success look like?
Single streamlined bot alert applied to all SGG's to help maintain / improve our FRT achievement for High's + retain SGG collaboration to work together to achieve those goals as a team.
How do we measure success?
We will continue to monitor our week-over-week FRT SLA goals.
Where would future feedback go?
Feedback can go here in this issue.