Nathan Black: Self-managed Support Engineer (AMER) onboarding checklist

We need to keep iterating on this checklist so please submit MR's for any improvements that you can think of. The file is located in an issue template in the 'support-training` repository If an item does not start with a role or someone else's name, it's yours to do.

Goal of this entire checklist: Set a clear path for Support Engineer training

Your trainer: tpresa

⚠ This issue is Not confidential, so everybody can see it. Do not mention sensitive or confidential personal, customer, or GitLab related information in this issue. That includes e.g. mentioning customer names for tickets that you worked on.

Stage 0: Complete general onboarding to have all necessary accounts and permissions

  • Done with Stage 0

Typically completed within the first week

  1. General expectations: it is expected that you will start to tackle Stages 0, 1, 2, and 3 as early as your first week, but you are not expected to complete all items for a number of weeks. We believe that you often learn best and fastest by diving into (relatively easy) tickets, and learning by doing. However, this has to be balanced with making time to learn some of the tougher topics. The expectation is therefore that you are sufficiently advanced in your onboarding by the end of your first week that you can answer 2-5 "simple" tickets. Over the following weeks, your manager will set higher targets for the number and difficulty of tickets to be tackled, but you should always save about 2-3 hrs per day to spend on working through this bootcamp. Some days it will be less, some days it will be more, depending on ticket load and how deep "in the zone" you are in either the bootcamp or the tickets you are responding to; but an average of 2-3 hrs per day should allow you to complete everything through to the end of Stage 6 within about four weeks.
  2. General Onboarding Checklist: this should have been created for you on the People Ops Employment issue tracker when you were hired.
    1. Finish every item on that list starting with New team member: until you are waiting for someone to answer a question or do something before you can continue that list.
    2. Ensure that as part of general onboarding, you and your manager check in that you have been added to all relevant Engineering communications channels. Access can be requested via a Single Person Access Request. Relevant channels include:
      • Email
      • Slack
        • Join #support-team-chat, #support_gitlab-com, #support_self-managed, #support-managers
        • Added to @support-dotcom and/or @support-selfmanaged group
        • Added to @support-team-emea or @support-team-apac group
      • GitLab groups and aliases:
        • Manager: add team member to gitlab-com/support as Maintainer
        • Manager: add team member to regional group gitlab-com/support/amer, gitlab-com/support/apac or gitlab-com/support/emea as Owner
    3. Start Stage 1 here, as well as the first steps of Stage 2 and Stage 3.
    4. Check back daily on what was blocking you on your General Onboarding Checklist until that list is completely done, then focus on this one.
  3. Team Calls: To keep up with the changes and happenings within the Support team, we have team calls every week. Every member of the support team is encouraged to join so they can also state their opinions during the call.
    1. Read up on the calls the Support Team uses to sync up and make sure you have the ones that pertain to you on your calendar.
    2. Identify agendas for those meetings, and read through a few past meetings in the document.
    3. Verify that you have a 1:1 scheduled with your manager and you have access to the agenda for that meeting.
  4. Introduce yourself to your team!
    1. Write an entry with your name, location, the date you started, a quick blurb about your experience, personal interests and what drew you to GitLab support in the Support Week in Review doc before Friday.
      1. Manager: Make sure to introduce the team member in the Engineering Week-in-Review document and copy the introduction over!
      2. Manager: Talk to the new team member about doing an introduction at the next regional team meeting, and add it to the agenda.
    2. Now, also format and send the introduction post you just created to the #support-team-chat Slack channel. Welcome to multi-modal communication, key to effective communication at GitLab!
  5. Manager: add team member to the Support Shared Drive in Google Drive with Contributor permissions.

Stage 1: Become familiar with git and GitLab basics

  • Done with Stage 1

Typically started in the first week, completed by the end of the second week

If you are already comfortable with using Git and GitLab and you are able to retain a good amount of information by just watching or reading through, that is okay. But if you see a topic that is completely new to you, stop the video and try it out for yourself before continuing.

  1. Let your manager know you're ready to be assigned a trainer.
  2. Just quickly check on your Zendesk account to make sure that is ready for you when you need it.
  3. Add a profile picture to your Zendesk account
  4. Let your manager know if you were not able to create an account in Zendesk for some reason.
  5. Under your profile in Zendesk, it should read Support Staff under role. If it reads Light Agent, inform your manager.
  6. Get familiar with our support workflows
  7. Request access to Chatops
  8. Manager: Add team member to the Support Team Drive.

Go over these topics in GitLab University:

  1. Under the topic of Version Control and Git
    1. About Version Control
    2. Try Git
    3. Explore Git internals and go back to it from time to time to learn more about how Git works
  2. Under the topic of GitLab Basics
    1. All the GitLab Basics that you don't feel comfortable with. If you get stuck, see the linked videos under GitLab Basics in GitLab University
    2. GitLab Flow
    3. Take a look at how the different GitLab versions compare
  3. Any of these topics that you don't feel comfortable with in the user training we use for our customers: env_setup.md, feature_branching.md, stash.md, git_log.md. For the rest of the topics in user training, just do a quick read over the file names so you start remembering where to find them.
  4. Get familiar with the services GitLab offers
    1. The differences between CE and EE (Core, Starter, Premium, Ultimate)
  5. Get familiar with the different teams in-charge of every stage in the DevOps cycle and what they are responsible for. This will assist you in adding the right labels when creating issues.

Stage 2. Installation and Administration basics.

  • Done with Stage 2

Typically started in the first week, completed by the end of the third week

Goals

  • Get your development machine set up.
  • Familiarize yourself with the codebase.
  • Be prepared to reproduce issues that our users encounter.
  • Be comfortable with the different installation options of GitLab and configure LDAP.
  • Have an installation available for reproducing customer bug reports.
  1. Check back on your Zendesk account to see if you are an Support Staff yet.
  2. Manager: Add team member to the GitLab Support Google Calendar.
  3. Add team member to the GitLab Support Google Cloud Project with Support Permissions.

Set up your development machine

  1. Install the GDK

Installations

You will keep one installation continually updated on Digital Ocean (managed through terraform), just like many of our clients, but you need to choose where you would like to test other installations. (TODO: We need to list some benefits of each choice here.)

  1. Manager: Add new team member to the dev-resources project.
  2. Use terraform to create a new test instance.
    1. Clone the dev-resources project.
    2. Read up on how to create a new instance.
    3. Create a new file and name it <your-full-name>.tf in a new branch. You can copy another team member's file and modify it or you can create your own from scratch. If you have any issues or questions ask in the #support channel.
    4. After you've created it, open up a merge request and if the tests pass then merge it.
  3. Set up your test environments.
  4. Choose between Local VM's or Digital Ocean for your preferred test environment, and note it in a comment below.
  5. Perform each of the following Installation Methods on the test environment you chose above:
    1. Install via Omnibus
    2. Populate with some test data: User account, Project, Issue
    3. Backup using our Backup rake task
    4. Install via Docker
    5. Restore a backup to your Docker VM using our Restore rake task
    6. You can check this box but this one never stops as long as you are a Support Engineer for GitLab: Keep this installation up-to-date as patch and version releases become available, just like our customers would.
  6. Ask as many questions as you can think of in the #support chat channel

Stage 3. Customer Interaction through Zendesk

  • Done with Stage 3

Typically started in the first week, and completed by the end of the fourth week

Goals

  • Have a good understanding of ticket flow in Zendesk and how to interact with our various channels.
  • See some common issues that our customers face and how to resolve them.

Initial Zendesk training

Zendesk is our Support Center and our main communication line with our customers. We communicate with customers through several other channels too, see the support handbook for the full list. Set aside around 40 minutes for completion of this section.

  1. Complete Zendesk Agent training
    1. Navigate to Zendesk university and register. You'll receive an email with information on accessing the Zendesk course
    2. Complete the "Zendesk Suite Overview: Support" course (approx. 10 min)
  2. Review additional Zendesk resources
    1. UI Overview
    2. Updating Tickets
    3. Working w/ Tickets (Read Avoiding Agent Collisions carefully).

Learn about the Support process

  1. Perform 15 one-hour pairing sessions with Support Engineers. Contact each in Slack to find a time that works, then create a Calendar event, inviting that person. When it's time for the pairing session, create a new Support project Issue and use the Ticket Pairing template for the call. Have at least 3 calls with a Support GitLab.com member.
    1. call with tpresa; issue link: #984 (closed)
    2. call with tpresa; issue link: #989 (closed)
    3. call with Harish; issue link: #1016 (closed)
    4. call with TPresa; issue link: #1034 (closed)
    5. call with Greg ; issue link: #1012 (closed)
    6. call with artychan; issue link: #1033 (closed)
    7. call with Phil ; issue link: #1065 (closed)
    8. call with Thiago; issue link: #1068 (closed)
    9. call with Gabriel; issue link: https://gitlab.com/gitlab-com/support/support-pairing/issues/84
    10. call with James; issue link: https://gitlab.com/gitlab-com/support/support-pairing/issues/16
    11. call with Julianne/Greg; issue link: https://gitlab.com/gitlab-com/support/support-pairing/issues/60
    12. call with Eldridge; issue link: https://gitlab.com/gitlab-com/support/support-pairing/issues/176
    13. call with Nourdin; issue link: https://gitlab.com/gitlab-com/support/support-pairing/issues/128
    14. call with John; issue link: https://gitlab.com/gitlab-com/support/support-pairing/issues/300
    15. call with VladS; issue link: https://gitlab.com/gitlab-com/support/support-pairing/issues/325
  2. Dive into our ZenDesk support process by reading how to handle tickets.
  3. Learn about the hot queue.
  4. Learn about the Zendesk SLA settings and Breach Alerts.
  5. Start getting real world experience by handling real tickets, all the while gaining further experience with GitLab.
    1. First, learn about our Support Channels.
    2. Proceed on to Regular email support tickets.
      • Here you will find tickets from our GitLab EE Customers and GitLab CE Users
      • Tickets here are extremely varied and often very technical
      • You should be prepared for these tickets, given the knowledge gained from the previous steps of your training
      • Tickets in Zendesk contain a wealth of knowledge from past interactions. Often times, the issue has occurred at least once and a solution has been provided, or an issue created on the customer's behalf. Reading through past tickets can aide in the learning process, getting you up to speed quicker.
        • Look for errors, keywords, etc. to search Zendesk like so: "keyword or error".
  6. Check out your colleagues' responses.
    1. Hop on to the #zd-self-managed-feed channel in Slack and see the tickets as they come in and are updated.
    2. Read through about 20 old tickets that your colleagues have worked on and their responses.
  7. Watch the GitLab Debugging Techniques: A Support Engineering Perspective video on GitLab Unfiltered (YouTube). This dives into some common issues that customers might run into.

Learn about the Escalation process for Issues

Some tickets need specific knowledge or a deep understanding of a particular component and will need to be escalated to a Senior Support Engineer or a Developer.

  1. Read about creating issues and issue prioritization.
  2. Take a look at the GitLab.com Team page to find the resident experts in their fields.

Learn about raising issues and fielding feature proposals

  1. Understand what's in the pipeline at GitLab as well as proposed features: Direction Page.
  2. Practice searching issues and filtering using labels to find existing feature proposals and bugs.
  3. If raising a new issue always provide a relevant label and a link to the relevant ticket in Zendesk.
  4. Add customer labels for those issues relevant to our subscribers.
  5. Take a look at the existing issue templates to see what is expected
  6. Raise issues for bugs in a manner that would make the issue easily reproducible. A Developer or a contributor may work on your issue.
  7. Schedule a 45 minute call with your trainer where you share your screen with them while you answer tickets on Zendesk, and they give you feedback and answer your questions. The goal of this call is for your trainer to pass on tactical knowledge about the ticket handling process. Repeat this step a few times if it would be beneficial to you.

Congratulations. You now have your Zendesk Wings!

From now on you can spend most of your work time answering tickets in Zendesk. Try to set aside 2 hours per day to make it through Stage 4-6 of this list. Never hesitate to ask as many questions as you can think of in the #support_self-managed chat channel.

Stage 4. Customer Calls

  • Done with Stage 4

Typically started in week 2 or 3, and completed by the end of week 4.

Goal Gain experience in scheduling and participating in calls with customers.

Look at the GitLab Support Google Calendar to find customer calls you can listen in on. Contact the person leading the call to check if it is okay for you to jump in on the call, and if they could stay on with you for a few minutes after the call, so you can ask them a few questions about the things you didn't understand. Also, ask them to ask you a few questions to make sure you understand the points they want to highlight from the call.

  1. Manager: make sure that the new team member has a "Pro" account within the GitLab team account on Zoom.us.

  2. Start arranging to pair on calls with other Support Engineers. Aim to cover two of each type of call. Comment on this issue with the type of call you were in, who it was with, and the link to the relevant ticket (if one exists).

    1. Learn about Cisco WebEx.
    2. call with tpresa & Hough 24 sept 2019
    3. call with Blair
    4. call with khugues 25 Oct 2019 136616
    5. Reverse Shadowing (You lead calls with a GitLab Support Engineer giving you feedback and support)
    6. call with https://gitlab.zendesk.com/agent/tickets/157363
    7. call with ___
    8. call with ___
    9. call with ___

  3. Manager: invite the new team member to the group Calendly account (Account->Users->New User)

  4. Set up Zoom integration on your Calendly account by going to calendly integrations and following the instructions for Zoom (make sure to update the calendly event's location to use "Zoom")

  5. Calculate your desired work hours in UTC time and ping @dstanley to add you to the calendly Team Support Call event with that info.

  6. Read Customer Call process documentation

Stage 5. Gathering Diagnostics

  • Done with Stage 5

Typically done around the third week.

Goal Understand the gathering of diagnostics for GitLab instances

  1. Learn about the GitLab checks that are available
    1. Environment Information and maintenance checks
    2. GitLab check
    3. Omnibus commands
      1. Status
      2. Starting and stopping services
      3. Starting a rails console

Stage 6. On Call Duty

  • Done with Stage 6

Typically to be completed within 8 weeks of joining.

Goal Aim to become a fully-fledged Support Engineer!

  1. Read about on-call duty:

    1. GitLab's on-call guide. On that page, the Customer Emergency On-Call Rotation Section is what applies to Support Engineers, NOT the Production Team section.
    2. GitLab Support's On-Call Guide contains even more information specific to handling emergencies in Support.

  2. Schedule time with your trainer to have them show you how to respond to Customer Emergencies.

  3. Ping your manager to add you to PagerDuty.

    1. Manager: add the Engineer to the rotation with at least 6 weeks of lead time before their first on-call shift.

  4. Sign up on PagerDuty with the link that will be emailed to you, and install the app on your phone.

    1. Familiarize yourself with the interface and the functionality.

  5. Configure your personal notification rules in PagerDuty under "My Profile" > "Notification Rules"

    1. Currently, customer emergency escalation policy is set to 10 minutes. That means if you do not respond to the notification within this period, the emergency will escalate to the rest of the team. Make sure your personal notification rules take this into account.
    2. Remember to update this accordingly when your details changed

  6. Use this PagerDuty guide to subscribe to your on-call schedule.

  7. In the weeks leading up to your first on-call shift, shadow the on-call engineers who are handling customer emergencies.

    1. ZD Ticket: ___
    2. Repeat this as many times as you are able.

  8. After your first on-call shift, add the "on-call hero" moniker to your entry on the team page and assign the MR to your manager.

Stage 7. Make the path better for those who come after.

You probably noticed that there was something broken or non-ideal in the onboarding process. You might have brought it up to your manager. They likely told you to look at Stage 7 of your onboarding issue.

  1. Make an improvement to the Support Engineer Bootcamp issue.

Stage 8. Optional Advanced GitLab Topics

Discuss with your training manager if you should stop here and close your issue or continue. Also, discuss which of the advanced topics should be followed. Do not do all of them as they might not be relevant to what customers need right now and can be a significant time investment.

These are some of GitLab's more advanced features. You can make use of GitLab.com to understand the features from an end-user perspective and then use your own instance to understand setup and configuration of the feature from an Administrative perspective

Edited by Nathan Black