The goal of this issue is to develop the goals and requirements for a recurring survey to evaluate customers' experiences with GitLab's professional services. This is one of a series of surveys we are developing to measure customer satisfaction at different points in the customer success lifecycle.
Who is the target user of the feature?
GitLab - Would be sent by @dsakamoto or @dtacheny
Customer - Would be sent to our primary point of contact for the PS engagement
What questions are you trying to answer?
Were you satisfied with our services? (either NPS or extremely, very, moderately dissatisfied / neutral / moderately very extremely satisfied)
Were the deliverables high quality and met the expectations set defined ?
Were communications and status during the project clearly communicated?
Were expectations clearly communicated prior to the services starting?
@dsakamoto I've opened this issue to start gathering goals and requirements for the professional services survey we discussed in our Customer Surveys meeting earlier today.
To kick things off, please edit the issue description above with responses to the questions listed. Don't hesitate to reach out if you need any help filling these out.
@eileenux I updated the description last week but realized you may not be notified by this. Think we have a good example that we can leverage to start measuring NPS. @dtacheny please queue up SOWs that we're closing including who we would want to send the survey to... Can we get a first iteration survey going? cc @brendan
@dsakamoto Ah you're right, I don't get notifications about edits to the issue description. Sorry about that! In the future please leave a comment about any changes you want me to review and I will definitely take a look!
Thanks for the additional context in the issue description. Based on the questions and examples you've shared I think this gives me enough to start on building a first draft of the survey in Qualtrics. I'll share the link here for your review once I have a first iteration together.
Thanks @eileenux! No worries. I should have realized it would not set an alert to you. Also not @ mention in email doesn't translate always in GitLab as it tagged another Eileen. Good to know.. LMK as we're ready to put it to use! cc @dtacheny@brendan
All,
In the interest of speed and lack of scale, we're doing these direct so I think they are except from those regulations (i.e., direct human to human emails vs. automated). Please LMK if this is incorrect. @eileenux@KimLock
@dsakamoto Here's a preview link for the professional services survey draft. Please take a look and let me know if you have any feedback on the format, phrasing of the questions, etc.
@KimLock I tried to include a couple of the questions you had written in the related issue you mentioned (https://gitlab.com/gitlab-com/marketing/product-marketing/issues/961). Since the goals of the two surveys are a little different (provide constructive feedback on professional services vs. provide a positive case study), my suggestion would be to include a question at the end of this satisfaction survey to people who provided high ratings about their experience to see if they would be interested in providing additional feedback for a possible case study/testimonial. Would that work for you? (cc @brendan)
My suggestions:
Think we may need to clarify expectations.
Suggest:
Expectations (i.e., schedule, deliverables, needs from you) were clearly communicated prior to the start of service
Progress and status were clearly communicated throughout the process.
Project deliverables were of high quality and met the expectations set at the beginning of the project.
The consultant(s) were skilled and/or knowledgeable in the area of expertise required for the project.
The consultant(s) were personable and worked well with you and your staff.
For the free form, thoughts on this? I would like to leave it open for whatever feedback they have, but also request at least one area even if they were happy. I know it's long. If too long, maybe strike "As we strive to continuously improve,"
What could we have done to make it a better service for you? As we strive to continuously improve, we would greatly appreciate at least one suggestion.
@dtacheny can you prepare this for any SOWs closed/closing this quarter? cc @brendan
@eileenux how do we attribute the survey to the specific person/company? Do we need to add that to the survey or is it captured when I sent it (through Qualtics)?
@dsakamoto I believe the only way to attribute the survey to the specific person/company is to add that as an explicit question directly in the survey. Let me know specifically what information you'd like to capture and I can go ahead and add it.
@eileenux looks great! We can evolve the questions over time, but I think this is a great start. Shifting gears, how do I send these out? Do we send them out through Qualtics to track the feedback? Do we send the link direct, meaning we would need to capture the name / company info / etc. I'm hoping we don't need to require they fill out that information. I would like to start surveying when we close SOWs.
@dsakamoto If we distribute the survey via an email from Qualtrics, it looks like it's possible to dynamically capture the respondent's email address (and with that, we can extrapolate some basic info about them like their name + company).
I think we'd need to manually upload a list of contacts and manually send the email with the survey link.
@eileenux yeah. Let's go with the path that gets us sending out surveys faster as the scale of surveys is less than 10. We can manually enter / attach the contact and customer information manually.
@dsakamoto Could you clarify which option you'd like to move forward with?
Option 1: Static link
Pros:
A reusable link that can be pasted anywhere you or members of your team are communicating with customers.
Requires no intervention from the UX Research team to send.
Cons:
Unable to track identifying information of respondents. In other words, customers will have to manually fill out this info as part of the survey.
Option 2: Distribute over email via Qualtrics
Pros:
Can automatically track identifying information of respondents.
Cons:
If someone forwards the email to a colleague, the info we auto-capture may be inaccurate. Sometimes recipients share with coworkers, so the initial name/contact we send it to may be different than the person who ultimately fills it out.
Requires intervention from a UXR team member to send in the Qualtrics UI each time you want to send to a customer, adding a lot more back and forth to the process.
It sounds like Option 1 would be the most efficient overall, so that would be my recommendation. Let me know your thoughts!
Thanks @eileenux! Let's go with option #1 (closed). can you add name and company to survey?
@dtacheny can you have someone provide me information to send including name, email and service (simple description 1-3 lines) for SOWs completed this quarter? also we need a basic process to have me send the survey around the signature time?
is it recommended that we have a "back" button on the survey?
It looks like this option is disabled by default in Qualtrics, so I've gone ahead and enabled it. You should be able to see it now in both the preview and published version of the survey.
Question: is there a reason we don't integrate Qualtrics and Salesforce? I'm new here at GitLab, so I may be missing some important details on why we keep them separate. Survey data would be great to attach to contact records — not to mention reduce administrative work of manually copying results into Salesforce.
@jdbeaumont This is a great idea! We only recently adopted Qualtrics and I wasn't part of that setup process so I don't have a good answer to this question. I'll do some investigation within the UX Research team (who own Qualtrics) and let you know what I find out.
Thanks, David! I took a look at the survey preview and it's a great start. I think it is best to keep it short and sweet as you have done.
For purposes of gathering feedback on PS-delivered training, it seems that a dedicated survey sent to customers whom we know have received PS training services would be especially helpful. I believe there is a separate issue where the team is already working on that survey and I'll provide input there.
We've received 2 survey responses so far. I've emailed those to you as both a CSV (raw results) + PDF (report format).
I set up a trigger in Qualtrics to email both of us as soon as each response gets submitted, but it appears those weren't coming through. I believe I managed to get to the bottom of that problem, so you should be receiving those responses as they're submitted moving forward!
@KimLock Hello! Here is the real-time report of all the results we've received for this survey so far: https://ql.tc/ZiohEk
It doesn't look like there is a question asking if participants would like to be interviewed further, and I'm not seeing any offers for that in the open response fields.
Closing this as the survey has been created and the Customer Success team has a reliable way to monitor the survey results and be notified of new submissions as they're received.