Surveys for JTBD - Build Artifacts

Overview

We want to validate the Build Artifacts JTBD with a survey. Has this been done before and is there a survey we can model off of?

changed due date to March 31, 2022

assigned to @gdoyle and @jreporter

@rayana @enf - Is there is a current survey example we can use to validate JTBDs?

@jreporter I don't think so, but we can use the JTBDs research questions to populate a survey if you need to increase the confidence of a JTBD.

Here is an example survey used by the Package stage to help prioritize the JTBD.

Got it thanks @rayana. Do you have access to add me as a collaborator to that survey so I can make a copy and edit?

mentioned in merge request www-gitlab-com!100287 (merged)

@gdoyle - Erika has created a template survey for us to fill in: https://gitlab.fra1.qualtrics.com/jfe/preview/SV_3lnF88fZnPjHLjo?Q_CHL=preview&Q_SurveyVersionID=current

How do you want to collaborate on this?

@jreporter I'm already at capacity for 15.0 Verify::Pipeline Insights & Runner Design 15.0 ... (gitlab-org/gitlab#352037 - closed). An option is that we could push back https://gitlab.com/gitlab-org/ux-research/-/issues/1859+, as this research will help us understand some of the JTBDs better, and therefore create a better design for the artifacts page. I could then collaborate with you to put together the survey, and analyze the data myself in 15.0.

If that isn't possible, then I wouldn't be able to collaborate on this until 15.1.

Thanks @gdoyle - I will take this forward in 14.10-15.0 while you manage the solution validation - I do think I will need your support in coming up the JTBD tasks, so maybe we can use our 1:1 next week to do that

Bumping the date to next milestone

mentioned in issue gitlab-org/gitlab#356793 (closed)

changed due date to April 29, 2022

@gdoyle @rayana - I have crafted this survey https://gitlab.fra1.qualtrics.com/jfe/preview/SV_7Nycndf5sHHLWJ0?Q_CHL=preview&Q_SurveyVersionID=current

I could use some help iterating on this! (Gina I know you said 15.1, which is okay! Maybe @rayana can take a look now)

Bumping to %15.1

@jocelynjane I am also going to assign this to you to bring you along on this exercise

@jreporter Can you add me as a collaborator in that Qualtrics survey please?

Done @gdoyle

@gdoyle - would a sync be helpful to line out next steps on this? I think we should review the questions, distribute on social media, and open a recruiting request. After we run the survey for 2 weeks look at results and then see if any trends related to artifacts emerge.

WDYT?

@jreporter @jocelynjane I added some questions to the survey on Friday, so it would be great to sync to review those! I'll look to set something up for this week with us 3.

@jreporter I'm not sure if you've done this already, but you'll have to add Jocelyn as a collaborator as well to the project! I tried but the option was disabled for me

yes I have added her @gdoyle

added Jobs to be Done (JTBD) grouppipeline security workflowproblem validation labels

changed milestone to %15.1

changed due date to May 31, 2022

assigned to @jocelynjane

mentioned in merge request www-gitlab-com!104894 (merged)

@enf Hello! We were looking to get your feedback on this survey by Tuesday May 31st so we can send it out by EOD Tuesday. I have 2 specific questions:

1. Should I open an incentives request for this before we send it out? Another question around incentives - Does everyone who takes the survey receive something?
2. We'd like to use this as an opportunity to follow-up with users, so we added this as the last question right now If you'd like to be entered to win one of 5 $60 gift cards, please enter your email address in the field below to participate in a GitLab Research Interview. I've never seen the one of 5 $60 gift cards - should we reword this?

cc: @jocelynjane @jreporter I suggested we don't send the survey out until Tuesday next week, since it is known that we get pretty low traffic on responses during the weekends (especially given it is a long weekend in the US).

That is a good suggestion thanks @gdoyle

I'd suggest Wednesday or Thursday. @gdoyle @jreporter

Generally, Tuesdays and Thursday get the best response rate. On Mondays, respondents are busy catching up with work and on Friday they're closing things out. So, mid-week pings are the most successful.

Since it's holiday, Wednesday (6/1) is the new Tuesday for next week.

Thanks @enf! Will do!

@gdoyle ~ tldr: We want to set up a promotional game for this survey. And ,we shouldn't talk about the potential gratuity (promotional game) when we ask users to opt-in for future contact because we don't want them to feel like they have to opt-in to receive a gratuity.

Should I open an incentives request for this before we send it out?

Yes, we should create an incentive request. In that request, you can tell Caitlin that I'll support you by setting up a promotional game. She'll need to process the gratuity after we select the winner(s).

If you need Caitlin to help you field your survey, you should fill out a recruitment request as well. I'd suggest leveraging her expertise here, but it's your call. If you send it yourself, let me know if you need help coordinating an email merge / campaign in gmail.

Let's start another thread below to talk about the promotional game.

Another question around incentives - Does everyone who takes the survey receive something?

We usually run promotional games because it's too costly to pay every person who responds to a survey.

should we reword this?

We need to have separate questions, or sets of questions. we don't want users to feel coerced into participating in future research in order to get a gratuity. So, we need (1) one question where they opt-in for future research, and another (2) where they opt into the promotional game.

Here are 2 examples of questions that you can use to invite respondents to opt-in to be contacted about future research:

I like this format better because it's easier to extract the data when they're not entered as form fields. It does add questions though.

Promotional Game Thread

@gdoyle Here are some instructions on how to set up the block of questions that you need for our promotional game. I created a question block in the library that you can import.

For the promotional game, we need to start with the number of responses that you need and work backwards from there. @gdoyle

I would ping Caitlin in your recruitment issue and ask her (1) how many responses we typically need for these surveys, and (2) in her experience how long that will take us.

Based on what she says, we'll fill in the following details so that I can set up the

• Start of promotional game / start of survey:
• End of promotional game / end of survey:
• Prize draw will be on
• We will only offer the promotional game in Australia, Canada, Ireland, Japan, Switzerland, UK and USA
• Prize: one(1) $120 Amazon Gift card for one randomly selected respondent

Another approach would be to time-box how long we field the survey and then proceed with the number of responses we get within a set time period.

It will be a lot of work to change course, so we want to commit to one of those up front.

closed

mentioned in issue gitlab-org/gitlab#366348