Commit bc632c8f authored by Jason Colyer's avatar Jason Colyer Committed by Lyle Kozloff
Browse files

Updating main support-ops page

* Removing unneeded links
* Adding issue board links
* Linking to new workflows page
* Adding workflows folder
* Adding test page in workflows
parent e77812bd
......@@ -265,6 +265,6 @@ We understand you may have plans outside of your normal workspace while you're o
 
## US Federal on-call
 
[US Federal](/handbook/support/support-ops/processes/us-federal-zendesk.html) on-call support is provided 7 days a week between the hours of 0600 and 1800 Pacific Time.
US Federal on-call support is provided 7 days a week between the hours of 0600 and 1800 Pacific Time.
 
The current on-call schedule can be viewed in [PagerDuty](https://gitlab.pagerduty.com/schedules#P89ZYHZ)(Internal Link), or in the [Support Team on-call page](https://gitlab-com.gitlab.io/support/team/oncall.html)(Public Link). The schedule is currently split into two, 6 hour shifts, an AM and a PM shift. The AM shift starts at 0600 Pacific Time and runs until 1200 Pacific Time. The PM shift starts at 1200 Pacific Time and runs until 1800 Pacific Time.
---
layout: handbook-page-toc
title: Support Operations Team
description: Introduction to the responsiblities of the Support Operations Team.
description: Support Operations Team main page
---
 
# Introduction to Support Operations Team (Support-Ops):
# Introduction to Support Operations Team (Support-Ops)
Support Operations Team is responsible for supporting the day-to-day operations
Support Operations team is responsible for supporting the day-to-day operations
and software systems used by GitLab’s global Technical Support team, primarily
our Zendesk instance, and integrations with internal business processes and
tools. The Support Operations Team is exclusively responsible for handling
Zendesk Support and Explore related queries.
 
Further details and responsibilities are explained [here](#responsibilities).
You can locate us in GitLab issues or via the `#support_operations` channel
in slack.
You can locate us in GitLab issues or via the
[`#support_operations`](https://gitlab.slack.com/archives/C018ZGZAMPD)
channel in slack.
 
## On this page
{:.no_toc .hidden-md .hidden-lg}
......@@ -23,13 +22,13 @@ in slack.
- TOC
{:toc .hidden-md .hidden-lg}
 
## Mission:
## Mission
To help the Technical Support Team spontaneously and make their day to day
workflow simple and easier so the efficiency and result values of Technical
Support Team increases.
 
## Vision:
## Vision
To provide the “Best in Class” support to our support team specifically and
GitLab to increase the overall value of GitLab as a team, organization and
......@@ -40,20 +39,7 @@ product.
- To solve Technical Support Team Zendesk related issues within 3 months.
- Increase efficiency of Zendesk to enable Support to achieve KPIs
 
## Responsibilities
The core responsibilities of the Support Ops team are:
* Zendesk (Global and US Federal instance)
* [Shared Organizations](responsibilities.html#shared-organizations-in-zendesk)
* [Salesforce - Zendesk sync](responsibilities.html#salesforce-zendesk-sync)
* [SFDC<>US Federal Zendesk sync](responsibilities.html#sfdcus-federal-zendesk-sync)
* [Customer Satisfaction Survey (CSAT)](responsibilities.html#customer-satisfaction-survey-csat)
* [Calendly](responsibilities.html#calendly)
* [PagerDuty](responsibilities.html#pagerduty)
* [Slack Notifications](responsibilities.html#slack-notifications)
## The Support Operations Team:
## The Support Operations Team
 
| Name | Role | Slack Handle |
|--|--|--|
......@@ -62,33 +48,36 @@ The core responsibilities of the Support Ops team are:
| [Dan Nolan](https://gitlab.com/dnolan1) | Support Operations Specialist (US Federal)} | @dnolan1
| [Alyssa Villa](https://gitlab.com/avilla4) | Support Operations Specialist (Main Instance) | @avilla4 |
 
## Support Operations Important Links:
## Support Operations Important Links
 
* [Main Project](https://gitlab.com/gitlab-com/support/support-ops/support-ops-project)
* [Progress Board](https://gitlab.com/gitlab-com/support/support-ops/support-ops-project/-/boards/1171115)
* [Priority Vice Issues Board](https://gitlab.com/gitlab-com/support/support-ops/support-ops-project/-/boards/1738899)
* [Project Labels](https://gitlab.com/gitlab-com/support/support-ops/support-ops-project/-/blob/master/README.md)
* [Our workflows](workflows/)
* [Project Labels](https://gitlab.com/gitlab-com/support/support-ops/support-ops-project#what-do-all-the-labels-mean)
* Various issue boards:
* [By category issue board](https://gitlab.com/groups/gitlab-com/support/support-ops/-/boards/2325911)
* [By priority issue board](https://gitlab.com/groups/gitlab-com/support/support-ops/-/boards/564195)
* [By progress issue board](https://gitlab.com/groups/gitlab-com/support/support-ops/-/boards/2325921)
* [Zendesk issue board](https://gitlab.com/groups/gitlab-com/support/support-ops/-/boards/2325976)
 
## Support Operations Workflow:
## Support Operations Workflow
 
![](https://lh6.googleusercontent.com/gLFocegPFVnk9wx4YbHDZV78N1rLlymzeekgu3c-YgtWN22kKiXnE7HTtzhn-mnb7ZafZZRTAr9Igw2zK748T-eun36I3ecLJs1OzC1HqbsDgpBwzal2D-LRafKUZQr7h2RgFRUM)
 
This diagram explains how the general fixes are handled by the Support Operations Team.
This diagram explains how the general fixes are handled by the Support
Operations team.
 
## Support Operations Changelogs:
## Support Operations Changelogs
 
The Support Operations Team usually make the critical changes usually in weekend
and communicate them via Slack channel #support_team-chat
([https://gitlab.slack.com/archives/CCBJYEWAW](https://gitlab.slack.com/archives/CCBJYEWAW))
and SWIR
([/handbook/support/#support-week-in-review](/handbook/support/#support-week-in-review)).
The Support Operations Team usually make the critical changes over weekends
and communicate them via Slack channel [`#support_team-chat`](https://gitlab.slack.com/archives/CCBJYEWAW)
and SWIR ([/handbook/support/#support-week-in-review](/handbook/support/#support-week-in-review)).
 
For an overview of what issues/MRs involve Support Ops, please see our
[changelog page](https://gitlab-com.gitlab.io/support/support-ops/changelog/)!
 
## Division of Responsibilities
 
To help ensure the team doesn't get overwhelmed and has thr ability to focus on
To help ensure the team doesn't get overwhelmed and has the ability to focus on
specialization and learning, we divide out the responsibilties amongst our
team. The current division of responsibilities is:
 
......@@ -103,7 +92,7 @@ team. The current division of responsibilities is:
| | Ticket Forms and Fields | @jcolyer | @nabeel.bilgrami |
| | Theme/Guide | @jcolyer | @nabeel.bilgrami |
| | Triggers | @nabeel.bilgrami | @avilla4 |
| | Users and Orgs | @avilla4 | @nabeel.bilgrami |
| | Users and Orgs | Support Ops team | Support Ops team |
| | Views | @avilla4 | @nabeel.bilgrami |
| | ZD<>SFDC Sync Main | @jcolyer | @nabeel.bilgrami |
| | ZD<>SFDC Sync Partners | @jcolyer | @nabeel.bilgrami |
......@@ -128,23 +117,23 @@ team. The current division of responsibilities is:
| | Audits | @jcolyer | @nabeel.bilgrami |
| | ADWR | @jcolyer | @avilla4 |
 
## Frequently Asked Questions (F.A.Q.):
## Frequently Asked Questions (F.A.Q.)
 
* If we receive any probelm in using Zendesk, can we contact Zendesk directly?
** If we receive any probelm in using Zendesk, can we contact Zendesk directly? **
 
Please contact Support-Ops team first. Discuss the problem by asking a question in channel and tagging @support-ops. It is a high probability that we can help you resolve the problem at hand. In cases where we cannot and we do need to contact Zendesk support directly, it is best to have Support-Ops handle that.
 
* What will happen if Zendesk is down globally?
** What will happen if Zendesk is down globally? **
 
Zendesk will only go down when the internet is globally effected because they use Pods for services. This ensures that if a region is facing is downtime, Zendesk can quickly mitigate that while making sure services run smoothly. However, if you are still facing any problem accessing Zendesk, please contact the support-ops team. In the case that Zendesk is down globally, we have email support option available.
Zendesk will only go down when the internet is globally effected because they use Pods for services. This ensures that if a region is facing downtime, Zendesk can quickly mitigate that while making sure services run smoothly. However, if you are still facing any problem accessing Zendesk, please contact the support-ops team. In the case that Zendesk is down globally, we have email support option available.
 
* Is there any disaster recovery plan available?
** Is there any disaster recovery plan available? **
 
Zendesk keeps the data in backup servers will all due diligence. This ensures that we can recover data when it is needed. These backups are utilized to restore Zendesk in the case it fails due to a problem on Zendesk's end.
 
Also, the support-ops team ensures all triggers, automations, views, macros, forms, fields, conditions, etc are documented to save the hassle of writing up everything from scratch.
 
* Why do we allow users to open support tickets without being required to login to Zendesk via some authentication?
** Why do we allow users to open support tickets without being required to login to Zendesk via some authentication? **
 
According to Lee Matos:
 
......
---
layout: handbook-page-toc
title: Support Operations Audits Processes
description: Audits Processes for Support Operations.
---
# Audits Processes
## On this page
{:.no_toc .hidden-md .hidden-lg}
- TOC
{:toc .hidden-md .hidden-lg}
Once a quarter, Support Operations is to conduct audits on all the tech stacks
we manage. These currently include:
* Zendesk
* Calendly
* Pagerduty
All of these are conducted via issues in the
[support-ops/audits](https://gitlab.com/gitlab-com/support/support-ops/audits)
project.
## Main Zendesk
These are done and tracked via the
[Zendesk Issue Template](https://gitlab.com/gitlab-com/support/support-ops/audits/-/issues/new?issuable_template=Zendesk).
This is a more complex audit, requiring a lot of checking and following up. To
start, run
[this script](https://gitlab.com/gitlab-com/support/support-ops/audits/-/blob/master/bin/zendesk_audit).
It will take a considerable amount of time, but reduce a good portion of the
work required on your part. Use the output to fill out the notes section of the
issue you generated.
From there, you need to go through the items reported and ping the person
in the issue to ask for the issue to be fixed (or clarify if this is
intentional). This can take time, so wait about 72 hours after pinging someone
before following back up. If the issue is not addressed within a week, ping
that person's manager.
After that, you need to review the
[API tokens](https://gitlab.zendesk.com/agent/admin/api/settings) currently in
use. The script above will output the basic details of what to put into the
issue. You will need to fill it out and seek out the maintainer/requester of the
API token to enter the justification/use case. This can take time, so wait about
72 hours after pinging someone before following back up. If the issue is not
addressed within a week, ping that person's manager.
### Based on script output
```mermaid
graph TD;
A-->B;
B-->C;
C-->D;
D-->E;
E--No-->F
E--Yes-->K;
F-->G;
G--No-->H;
G--Yes-->E;
H-->I;
I--No-->J;
I--Yes-->K;
K-->L;
L-->M;
A(Run ./bin/zendesk_audit);
B(Put the output into the notes of the issue);
C(Ping each person to address the problems)
D(Wait 72 hours)
E(Have all issues been addressed?)
F(Ping them again on the issue, wait 4 days)
G(Has the person addressed the issue?)
H(Ping their manager, wait 72 hours)
I(Has the issue still not been addressed?)
J(Ping Support Operations Manager for assistance)
K(Ping Support Operations Manager for review)
L(Support Operations Manager reviews)
M(Support Operations Manager marks complete and closes issue)
```
### API token review
```mermaid
graph TD;
B-->C;
C-->D;
D-->E;
E--No-->F
E--Yes-->K;
F-->G;
G--No-->H;
G--Yes-->E;
H-->I;
I--No-->J;
I--Yes-->K;
K-->L;
L-->M;
B(Go through Zendesk API tokens, listing out token name and owner)
C(Ping the owners asking for justifcation)
D(Wait 72 hours)
E(Have all issues been addressed?)
F(Ping them again on the issue, wait 4 days)
G(Has the person addressed the issue?)
H(Ping their manager, wait 72 hours)
I(Has the issue still not been addressed?)
J(Ping Support Operations Manager for assistance)
K(Ping Support Operations Manager for review)
L(Support Operations Manager reviews)
M(Support Operations Manager marks complete and closes issue)
```
## US Federal Zendesk
These are done and tracked via the
[Zendesk Issue Template](https://gitlab.com/gitlab-com/support/support-ops/audits/-/issues/new?issuable_template=US%20Federal%20Zendesk).
This is a more complex audit, requiring a lot of checking and following up. To
start, run
[this script](https://gitlab.com/gitlab-com/support/support-ops/audits/-/blob/master/bin/us_federal_audit).
It will take a considerable amount of time, but reduce a good portion of the
work required on your part. Use the output to fill out the notes section of the
issue you generated.
From there, you need to go through the items reported and ping the person
in the issue to ask for the issue to be fixed (or clarify if this is
intentional). This can take time, so wait about 72 hours after pinging someone
before following back up. If the issue is not addressed within a week, ping
that person's manager.
After that, you need to review the
[API tokens](https://gitlab-federal-support.zendesk.com/agent/admin/api/settings)
currently in use. The script above will output the basic details of what to put
into the issue. You will need to fill it out and seek out the
maintainer/requester of the API token to enter the justification/use case. This
can take time, so wait about 72 hours after pinging someone before following
back up. If the issue is not addressed within a week, ping that person's
manager.
### Based on script output
```mermaid
graph TD;
A-->B;
B-->C;
C-->D;
D-->E;
E--No-->F
E--Yes-->K;
F-->G;
G--No-->H;
G--Yes-->E;
H-->I;
I--No-->J;
I--Yes-->K;
K-->L;
L-->M;
A(Run ./bin/us_federal_zendesk_audit);
B(Put the output into the notes of the issue);
C(Ping each person to address the problems)
D(Wait 72 hours)
E(Have all issues been addressed?)
F(Ping them again on the issue, wait 4 days)
G(Has the person addressed the issue?)
H(Ping their manager, wait 72 hours)
I(Has the issue still not been addressed?)
J(Ping Support Operations Manager for assistance)
K(Ping Support Operations Manager for review)
L(Support Operations Manager reviews)
M(Support Operations Manager marks complete and closes issue)
```
### API token review
```mermaid
graph TD;
B-->C;
C-->D;
D-->E;
E--No-->F
E--Yes-->K;
F-->G;
G--No-->H;
G--Yes-->E;
H-->I;
I--No-->J;
I--Yes-->K;
K-->L;
L-->M;
B(Go through Zendesk API tokens, listing out token name and owner)
C(Ping the owners asking for justifcation)
D(Wait 72 hours)
E(Have all issues been addressed?)
F(Ping them again on the issue, wait 4 days)
G(Has the person addressed the issue?)
H(Ping their manager, wait 72 hours)
I(Has the issue still not been addressed?)
J(Ping Support Operations Manager for assistance)
K(Ping Support Operations Manager for review)
L(Support Operations Manager reviews)
M(Support Operations Manager marks complete and closes issue)
```
## Calendly
These are done and tracked via the
[Calendly Issue Template](https://gitlab.com/gitlab-com/support/support-ops/audits/-/issues/new?issuable_template=Calendly).
As the API is not yet able to handle this, this process will be a bit more of a
manual process. To starter, you will need will first make an audit issue via the
[Calendly Issue Template](https://gitlab.com/gitlab-com/support/support-ops/audits/-/issues/new?issuable_template=Calendly).
After doing so, you will want to run
[this script](https://gitlab.com/gitlab-com/support/support-ops/audits/-/blob/master/bin/calendly_audit).
Use the output to fill out the notes section of the issue you generated.
From there, you need to go through the items reported and ping the person
in the issue to ask for the issue to be fixed (or clarify if this is
intentional). This can take time, so wait about 72 hours after pinging someone
before following back up. If the issue is not addressed within a week, ping
that person's manager.
After that, you need to go into calendly and confirm who is and isn't there. It
helps to make a list of who all is in the
[Calendly group](https://calendly.com/app/organization/users) and then correlate
it to what the script outputed. Any users in Calendly but not outputted from the
script need to be noted and we will need confirmation of why they are in there.
If no justifcation can be made, they will need to be removed.
### Based on Script output
```mermaid
graph TD;
A-->B;
B-->C;
C-->D;
D-->E;
E--No-->F
E--Yes-->K;
F-->G;
G--No-->H;
G--Yes-->E;
H-->I;
I--No-->J;
I--Yes-->K;
K-->L;
L-->M;
A(Run ./bin/calendly_audit);
B(Put the output into the notes of the issue);
C(Ping each person to address the problems)
D(Wait 72 hours)
E(Have all issues been addressed?)
F(Ping them again on the issue, wait 4 days)
G(Has the person addressed the issue?)
H(Ping their manager, wait 72 hours)
I(Has the issue still not been addressed?)
J(Ping Support Operations Manager for assistance)
K(Ping Support Operations Manager for review)
L(Support Operations Manager reviews)
M(Support Operations Manager marks complete and closes issue)
```
### Based on Calendly review
```mermaid
graph TD;
B-->C;
C-->D;
D-->E;
E--No-->F
E--Yes-->K;
F-->G;
G--No-->H;
G--Yes-->E;
H-->I;
I--No-->J;
I--Yes-->K;
K-->L;
L-->M;
B(Go through calendly users and find who is in the group but not part of the script's output)
C(Ping those who should not be in calendly asking for justifcation)
D(Wait 72 hours)
E(Have all issues been addressed?)
F(Ping them again on the issue, wait 4 days)
G(Has the person addressed the issue?)
H(Ping their manager, wait 72 hours)
I(Has the issue still not been addressed?)
J(Ping Support Operations Manager for assistance)
K(Ping Support Operations Manager for review)
L(Support Operations Manager reviews)
M(Support Operations Manager marks complete and closes issue)
```
## Pagerduty
These are done and tracked via the
[Pagerduty Issue Template](https://gitlab.com/gitlab-com/support/support-ops/audits/-/issues/new?issuable_template=Pagerduty).
This is a simpler audit, since it is mostly just checking for valid IDs and
schedules. As this doesn't require much human interaction, it is largely
scripted.
To do these, first make an audit issue via the
[Pagerduty Issue Template](https://gitlab.com/gitlab-com/support/support-ops/audits/-/issues/new?issuable_template=Pagerduty).
Then run
[this script](https://gitlab.com/gitlab-com/support/support-ops/audits/-/blob/master/bin/pagerduty_audit).
Use the output to fill out the notes section of the issue you generated.
From there, you need to go through the items reported and ping the person
in the issue to ask for the issue to be fixed (or clarify if this is
intentional). This can take time, so wait about 72 hours after pinging someone
before following back up. If the issue is not addressed within a week, ping
that person's manager.
Once all that has been done, ping the Support Operations Manager for review.
```mermaid
graph TD;
A-->B;
B-->C;
C-->D;
D-->E;
E--No-->F
E--Yes-->K;
F-->G;
G--No-->H;
G--Yes-->E;
H-->I;
I--No-->J;
I--Yes-->K;
K-->L;
L-->M;
A(Run ./bin/pagerduty_audit);
B(Put the output into the notes of the issue);
C(Ping each person to address the problems)
D(Wait 72 hours)
E(Have all issues been addressed?)
F(Ping them again on the issue, wait 4 days)
G(Has the person addressed the issue?)
H(Ping their manager, wait 72 hours)
I(Has the issue still not been addressed?)
J(Ping Support Operations Manager for assistance)
K(Ping Support Operations Manager for review)
L(Support Operations Manager reviews)
M(Support Operations Manager marks complete and closes issue)
```
---
layout: handbook-page-toc
title: Support Operations GitLab Processes
description: List of Support Operations GitLab Processes which includes Time Tracking, Triage, Priority, Stage and Category.
---
# GitLab Processes
## On this page
{:.no_toc .hidden-md .hidden-lg}
- TOC
{:toc .hidden-md .hidden-lg}
## Time Tracking
Support-Ops utilizes
[Time Tracking](https://docs.gitlab.com/ee/user/project/time_tracking.html)
in the issues/MRs we work. We utilize this time tracking to help define our
hiring model, measure workload, and etc. As you work issues and merge requests,
make sure you utilize this feature.
## Triage
At least once a day, Support-Ops should triage the
[issues](https://gitlab.com/groups/gitlab-com/support/support-ops/-/issues) and
[MRs](https://gitlab.com/groups/gitlab-com/support/support-ops/-/merge_requests)
that appear under the
[support-ops group](https://gitlab.com/gitlab-com/support/support-ops).
Support-Ops also makes use of the
[GitLab Triage Gem](https://gitlab.com/gitlab-org/gitlab-triage) to assist in
ensuring all issues/merge requests are triaged.
## Labels
For a list of the labels we use and what they mean, please see the [Support Ops Project README](https://gitlab.com/gitlab-com/support/support-ops/support-ops-project#what-do-all-the-labels-mean)
## Changelog
Our [changelog](https://gitlab-com.gitlab.io/support/support-ops/changelog/) is
auto-generated once a day. It locates issues and MRs using the quarter's epic
milestone. The format of these is FYxxQy.
When working issues/MRs, make sure to apply the correct epic/milestone. You can
do this during creation, using the right-hand side panel, or via a comment on
the issue/MR. Examples of doing this via a comment can be seen below:
| What to do | Comment box text |
|---|---|
| Add issue to epic FY22Q1 | `/epic &3` |
| Add MR to milestone FY22Q1 | `/milestone %FY22Q1` |
---
layout: handbook-page-toc
title: Support Operations Processes