KR: Identify and address UXR recruitment challenges => 100%
-
Establish and document (in the handbook) procedures to address the following recruitment situations: -
GDPR and CAN-SPAM requirements -
Over capacity -
Time constraints -
UXR Coordinators unavailable -
Quarterly planning challenges
-
-
Document all sources, processes, and instructions to self-serve in the handbook (aka: training material for UXR)
GDPR and CAN-SPAM requirements
At GitLab, when we communicate with our research study participants, we take GDPR and CAN-SPAM requirements seriously.
- Anyone can submit a GDPR request to see which lists they are subscribed to, and be removed from any and all of them. The UX research coordinator routinely processes these requests by searching for the contact in the UX research Qualtrics directory. If the contact is not present, the coordinator comments in the issue to document. If the contact is present in the directory, the coordinator deletes the contact, then confirms in the issue that the contact has been removed.
- Every email sent to the research panel members contains an unsubscribe link. When a recipient clicks the link, Qualtrics automatically confirms that they have been unsubscribed from the mailing list.
Over capacity & time constraints
Presently, our Research Coordinators each have the capacity to simultaneously handle up to 10 total research requests. When the number of requests exceeds 10, the Research Coordinator is ‘over capacity’. When this occurs, there are too many projects to successfully manage at once, and the following side effects are experienced internally and externally to GitLab:
- Timelines slip
- Outbound communications from the Research Coordinator are reduced, resulting in the feeling of going dark
- Inbound communications to the Research Coordinator are increased, from people wanting to know statuses and help with next steps
- Errors occur in the process
To deal with situations of over capacity, we have two plans of action:
1) Quarterly planning
Given the nature of how we conduct research at GitLab, it’s difficult to have an accurate view into the pipeline at any given time to understand what research projects will be coming up when. However, we can rely on research project volume data from the past, predict the cadence of some projects quarterly, then plan accordingly. An example of identified projects that require extra coordination are:
- SUS measuring
- Category Maturity Scorecarding
- Large N surveys (n=200+)
The change we’ll make:
Going forward, a quarterly planning session will be conducted at the start of each quarter. The planning exercise will map out the upcoming research requests with known research projects, predicted research projects, upcoming events requiring research participants, anticipated changes to our processes (ex: new hires, unmoderated testing, etc), and Research Coordinator unavailability. Monthly check-ins at the beginning of each month will also take place between Research Coordination and the UX Research Director.
The goal is to have a fairly accurate view into the quarter at any given time, be able to identify any potential delivery risks, and to work early on addressing those risks with mitigation plans.
The Research Coordinator is responsible for driving and maintaining the research coordination quarterly planning exercise, the monthly check-ins, and keeping the Google Sheet up to date and accurate.
2) A back-up plan
Presently, there isn’t a back-up plan in place when a Research Coordinator becomes unavailable, most likely due to the unique and specialized nature of the Research Coordinator role. This means when a Research Coordinator is unavailable, progress slows. This model is not sustainable and has a negative impact on all parties involved.
The change we’ll make:
To keep research projects on track, each Research Coordinator must be able to fill-in for each other, should one become unavailable. Additionally, at least 2 other UX Researchers (Andrej and Katherine) will be trained to work as Research Coordinators when Research Coordinators are unavailable or if they are at over capacity and need additional assistance.
This means proper training needs to take place:
- To have Research Coordinators fill-in for each other
- To get UX Researchers up to speed on the Research Coordinator role
Ideally, a back-up plan is rarely used if proper planning was conducted.
The existing Research Coordinator, Emily, will be responsible for developing training materials for training and to conduct the training. Example coverage areas for training:
- Recruiting intake process (ex: how to respond to all parties, Service Level Agreement (SLA), expectations of a Research Coordinator, etc)
- Communication guidelines when reaching out to panelists or recruitment sources (provide examples for each, special logins/accounts to use, etc)
- Handling gratuities for different countries
- Understanding the different types of issues created for a recruiting request vs. gratuity request - and wrapping those up.
When unplanned over-capacity happens
When unplanned over-capacity occurs, Research Coordinators need to be transparent in communicating this is happening. They have several actions they can take:
- Inform the team - Be transparent. Let the #ux_research channel know, along with managers, that research coordination is over capacity at the moment along with an estimated timeframe on when they expect things to get back to normal. This automatically sets expectations for existing and new requests. Lastly, follow-up with a post when capacity returns to normal to let the channel know.
- Reset expectations on any deliverable dates - Communicate to the key people on the projects impacted that their deliverable dates may change.
- Inform participants - Be transparent. If participants are negatively impacted, let them know that we’re busy, express a sincere apology, and reset timing expectations with them.
- Ask for help - Research Coordinators need to ask their trained UX Research Team back-ups for help. Research Coordinators will need to delegate the right duties to the UX Researchers to result in the biggest impact.
- New incoming requests will take longer - We don’t stop taking on new research when we’re at over capacity. Instead, we set expectations early with new requests. New requests will be buffered accordingly to accommodate being at over capacity and project owners are educated as to why.
- Encourage self-service - Research Coordinators can point people to self-serve resources with a link to instructions in the handbook (ex: using Respondent and User Interviews to find participants, etc).
When UXR Coordinators are unavailable
Since Research Coordinators have unique duties that are critical to the success of research at GitLab, when they are unavailable, recruitment efforts slow down. This results in a series of problems for all parties.
The change we’ll make:
To address this, at least two UX Researchers will be available to act as back-up when Research Coordinators are unavailable. These back-up Research Coordinators will fill in for the unavailable Research Coordinator by completing any in-progress tasks, accepting new requests, and keeping existing requests on track.
When possible, prior to the Research Coordinator becoming unavailable, they will create an issue to provide their assigned back-up(s) with a clearly detailed list of:
- Research projects in process
- Actions needed for each project - and when to do them
- Key contacts for each project
- Links to all applicable issues
- Flag exactly what needs to be done by the back-up Research Coordinators
It’s the responsibility of the Research Coordinator to make sure their back-up understands their duties and responsibilities.
Increasing the participant pool
As research scales across GitLab, the demand for qualified participants increases. Presently, Research Coordinators recruit most of their participants from the First Look panel, which contains approximately 2900 participants. Given the number of our segments, cadence of studies being conducted, and limited panel, we run several risks:
- Noise: bombarding the panel with too many research requests, which may result in fewer responses to requests.
- Quantity: we may eventually be asking too much of our panel with so many studies. This may also result in fewer responses to our research requests.
- Familiarization: we may start to see the same small group of participants within a segment over and over again and they may be shaping our product, which may not be representative of our users.
- Composition: the First Look panel consists of participants who are advocates of GitLab and may not represent the user base accurately
The change we’ll make:
To scale properly, we will be expanding our participant reach by including a consistent call-to-action to join the First Look panel when we recruit participants from the following existing sources:
- Warehouse
- Community
- Support ticket link
- TAMs
- +new sources
As new participant sources are identified, we will have a formal and consistent way to: 1) educate them on the First Look panel, and 2) instructions on how to join. This will be done via email communication by the Research Coordinator.
Additionally, to make sure we don’t have too many repeat participants taking part in our research studies, the Research Coordinator will explore options to efficiently track the frequency in which participants participate in our studies. If the team starts to see certain participants taking part in X amount of studies within Y amount of time, we can avoid sending them recruitment invitations until Z amount time has passed.