Planning in GitLab as it relates to velocity or capacity | Being visible about limits

What’s this issue all about?

Katherine summed up the scope of this research very well with:

Learning more about how external users of GitLab do capacity or velocity planning and backlog grooming (tools, processes, constraints, etc), how they prioritize issues, what they like about GitLab's current planning features and how they feel they can be improved.

This can be part of ongoing research to enrich our planning tools in ways users expect. We also want to investigate a few solutions for https://gitlab.com/gitlab-org/gitlab-ee/issues/11403

When thinking about velocity vs capacity, this helps me:

Teams that use velocity for planning typically base velocity on the empirical observation of previously completed sprints. Capacity is based on the team's expected or projected future availability. Velocity is based on actual points completed, which is typically an average of all previous sprints. Velocity is used to plan how many product backlog items the team should bring into the next sprint. Capacity is how much availability the team has for the sprint. This may vary based on team members being on vacation, ill, etc. The team should consider capacity in determining how many product backlog items to plan for a sprint. The team may want to consider taking on fewer product backlog items if capacity is expected to be less for the sprint. Likewise, if more team members are recently added, the team may want to take on more product backlog items. Velocity should only be used for the team for planning purposes. The success of the team should always be based upon the delivery of value--i.e. a working increment of the product delivered to the customer.

What questions are you trying to answer?

Milestone planning

In general, how do our users utilize GitLab to plan and prioritize work/milestones/sprints?
- What do they like about this experience?
- What don't they like?
- What is missing that they would expect to have and be able to do?
How does a user utilize the backlog/issue list in general AND for capacity or velocity planning/backlog grooming? What are they doing at this touchpoint?
- What do they like about this experience?
- What don't they like?
- What is missing that they would expect to have and be able to do?
How does a user utilize boards in general AND for capacity or velocity planning/backlog grooming? What are they doing at this touchpoint?
- What do they like about this experience?
- What don't they like?
- What is missing that they would expect to have and be able to do?
How does a user determine their (team) capacity or velocity?
- What is that process like and who is involved?
- When are they doing this in GitLab? Is this what they would expect?
  - The objective here is understand what users would do with issues under that "cut line"
Are users bringing more work into their milestones than they have agreed upon completing or feel they can complete? If so-why?
- Do users have "stretch" goals like we at GitLab do? What are they doing with these?
Are users adding more work into their milestone or sprint while the milestone or sprint after it has started? Why?
Are users setting or wanting to set limits on what work can be brought into each column list?
- When in the process would they expect to be able to do this?
- What would they expect this to look like?
- How would they expect to be warned that they are going over their limit?
- Do they want to set maximum AND minimum?
- Do they want to set limits by weight or issue count?
How would users imagine GitLab notifying others that either them or their team is over capacity or velocity or at their limit during a milestone or sprint?
Is going over velocity or capacity a "bad" thing? What does this mean to them?
Once it is decided that some issues are above the (team?) velocity or capacity, what happens to those issues?
Are issue cards pretty stable once they are prioritized in list, or are they getting moved around a lot?
What different ways are users utilizing GitLab boards (ie: tracking personal work, setting agendas, tracking team work)?
- I'd love to see here how internal and external users differ.

Issue related questions and preference testing

Which of these solutions best fits a user's needs and why? Options 1 and 2 are actually pretty similar except 2 allows a user to set limits before adding and cards into the column. It also calls out a clear min and max but option 1 could theoretically do that by users adding another line. Both issues make assumptions for the user by calling out what is above and below the line (for better or worse). This is where option 3 differs the most, it doesn't assume which issues are "above" and "below" and instead just alerts the user to the fact that they are over or under a limit and have broken that capacity or velocity agreement which encourages them to move some issues out. Option 3 is most similar to what I have seen in boards of other products.

Option one

General question: Are users comfortable not being able to set limits until they have added some issue cards? Within these limits, do users want us to call out the issues "below the line or limit"? Do they prefer that we explicitly call out what is "under the line"? Are they comfortable having more issues in their column than they have agreed upon completing?

Probably focusing in on: C, D, E, F

What does that line indicate to users?
What do issues below the line mean to users?
What would users expect to see on the line?
Would users want more than one line? Why?
What would users expect to happen if they dragged and issue "below" the line "above" the line?
Usability questions if possible (ie: do they understand how to drag and drop, do they understand the weight and issue count changing as the drag the line up and down, where would they go to add a line, how would they delete the line) but we could maybe do this in another round after narrowing?

Option two

General question: Would users prefer to set limits before adding any issue cards? Within these limits, do users want us to call out the issues "below the line or limit"? Do they prefer that we explicitly call out what is "under the line"? Are they comfortable having more issues in their column than they have agreed upon completing?

Probably focusing in on: D, E

Why are some issues in the list red?
What would users expect to happen if they dragged a "red" issue to the top of the list (ie if they had a list with 4 issues and 2 are red, if they dragged one red issue up, would it still be 2 red or now only 1)?
Do they want to set a min AND max?
Usability questions if possible (ie: which limit are they exceeding, what is their max, what is their min, how many issues are over) but we could maybe do this in another round after narrowing?

Option three

General question: Within these limits, do users want us to leave it up to them to determine what "below the line or limit" and prioritize their list accordingly? Do they prefer that we let them decide which issues to move out and encourage them not to put more issues in their column than they have agreed upon completing?

Probably focusing in on: D, E

What does it mean that the issue list turned red?
What does it mean that the issue list turned yellow?
What would users expect to happen if they dragged all but 3 issues out of their list?
Do they want to set a min AND max?
Usability questions if possible (ie: which limit are they exceeding, what is their max, what is their min, how many issues are over) but we could maybe do this in another round after narrowing?

What assumptions do you have?

I am assuming that external users expect to plan and use their boards differently than internal users do.
I am assuming that migrating from other products like Jira (which from limited research I've done seems where many of our users NOT using GitLab are doing their planning) is confusing to users or they find feature missing.
I am assuming external users may want our issue list to behave more like a backlog.
I am assuming that perhaps external users expect to be able to do capacity or velocity planning BEFORE they move to a board.
That users would expect something like Option 1 in a backlog but not necessarily boards(that is it useful in the context of planning a long prioritized list in the backlog). An example of this in Jira:

What decisions will you make based on the research findings?

Long term: This will help the plan team prioritize requests and help the UX team with OKRs such as: #183 (closed)
Short term: This will help make informed decisions in https://gitlab.com/gitlab-org/gitlab-ee/issues/11403

What's the latest milestone that the research will still be useful to you?

Ideally a week or two before 12.2 to be useful for me to apply meaningful research to https://gitlab.com/gitlab-org/gitlab-ee/issues/11403

@uhlexsis @katokpara @ebrinkman @sbouchard1

Progress Checklist

Key Findings

&1578 (closed)

Summary 4/7 users chose option 2 as best overall, 3/7 chose option 1. Option 1 and Option 2 were both favorable in terms of concept and design, however. They just may have different use cases, depending on how rigid the team's planning processes are and how they perceive the functionalities.

For example, User 1 stated that option 1 provides capabilities to organize what he sees while option 2 is an aid for a management process (there would need to be an established reason for why he should set limits).
User 4 stated that Option 2 was for hard limits while Option 1 was for informational processes.

Overall, users were receptive to both option 1 and 2, even if they didn't fully understand all aspects of the designs on the first impression. Some users do not need to set specific limits (option 2/3) but still understood the concept well. Other users may not have stretch goals in their own workflow but see use cases for using the line to group issues on the board or prioritize issues in the backlog (option 1).

Edited Jul 10, 2019 by Katherine Okpara