GitLab Runner Terminology Survey
Results
We obtained responses from 177 participants who use CI/CD pipelines between July 21st and August 16th.
Concept 1 (The tool that allocates jobs and adjusts runner resources based on what's needed to support the application at that time) = Runner manager (n=40, 48%)
Users use runners manager to... | Related quotes |
---|---|
Automatically manage runner resources in order to optimize job performance (Majority - n=28, 43%) | "Distribute tasks across available CI runners in an optimal way to minimize job execution time. This is especially important for tasks that rely on giant docker images (8+ GB) that or that need access to scarce resources (e.g. GPUs)." "Dynamic job allocation and resource management tool for optimizing application performance and resource utilization." View all verbatim |
Run CI (n= 15, 23%) | "execute CI pipelines" "running a CI/CD pipeline" View all verbatim |
Ranked names:
- Runner manager
- CI coordinator
- Resource allocator
Concept 2 (Resources that execute a job on a platform, from the machine that does the execution itself to the environment where the job is executed on) = Runner (n=38, 53%)
Users use runner to... | Related quotes |
---|---|
Execute or run jobs (Majority - n=26, 51%) | "Taking tasks from a queue and performing them one by one on a dedicated machine/in a dedicated environment" "Running a job" "execute a job" View all verbatim |
Build, test, or deploy (n= 16, 31%) | "Building and deploying" "Deploying, (re)starting services, e2e tests" View all verbatim |
Ranked names:
- Executor
- Runner
- Compute
Concept 3 (where you can configure runners to perform in the ways you desire): Runner configuration (n=50, 72%)
Users use runner configuration to... | Related quotes |
---|---|
Configure the runner (tags, resources, parallel jobs, grouping them, behavior) (Majority - n=35, 73%) | "I use the runner configuration section to: (a) enable / disable runners; (b) create runner groups; (c) define and assign runner & group labels; (d) assign runner & group priorities (i.e. use this runner group first, until fully loaded. When all runners are of this group are busy, continue with the next group.)" "Set tags" "I feel like this is for configuring which tags are used for which runners, managing concurrency and names" View all verbatim |
Ranked names:
- Runner configuration
- Runner settings
- Execution configuration
Concept 4 (Where you can control access to who can use and manage certain runners): Runner access control (n=50, 72%)
Users use runner access control to... | Related quotes |
---|---|
Configure which users can use, manage, and edit runners (Majority - n=32, 60%) | "Controlling access to who can use and manage certain runners." "I use the Runner administration section to control / delegate admin-level access to Runner configuration. I would prefer if the configuration and administration sections where near each other and easy to find." "Defining who can use, control runner (executors)" View all verbatim |
Restrict which projects/groups can use certain runners (n= 10, 19%) | "Controlling what user can run what jobs, on which runners; If there are any tasks, runners, or variables that require special permissions." "Setting project runner targets" "Assign runners to projects/groups" View all verbatim |
Ranked names:
- Runner access control
- Access controller
- Runner profile
What’s this issue all about?
From https://gitlab.com/gitlab-org/ux-research/-/issues/2444+, we learned that there are 4 buckets of concepts that users conceptualize runners in. We want to know what users name those buckets and learn when they use them in their workflows.
Qualtrics survey: https://gitlab.eu.qualtrics.com/survey-builder/SV_2fvTIGEDUeZPGjc/edit?resourceTypeForNavContext=SingleSurveySolution
Who is the target user of the feature?
- Software devs
- Platform engineers
What questions are you trying to answer?
- Which names will they select for each concept?
- When are they using them in their workflows?
What hypotheses and/or assumptions do you have?
What decisions will you make based on the research findings?
If we see clear patterns, we'll choose the names selected by users.
Timeline/to-dos
-
@gdoyle Create recruitment issue and tag @enf , @cfaughnan
and add that it'll be promotional game. -
@gdoyle Meet with the team to come up with 5 max terms for each bucket. -
@gdoyle Add those to Qualtrics ^ and publish the survey. -
@enf Set up the promotional game and send out survey. -
@gdoyle At 100 responses or September 1st (whichever comes first) - review the data and identify if there are clear winners. If not, re-evaluate the plan.
Edited by Gina Doyle