Prioritization Framework Engine
Problem
There are a number of inputs that can be taking into account when determining how to prioritize an issue. We currently maintain a rather arbitrary stack ranked prioritization table based on type of issue that PMs are supposed to reference when making decisions, but it does not take into account all of the possible inputs to understand ROI.
Exploring Prioritization Frameworks
There have been a number of requests for GitLab the product to support a variety of prioritization frameworks. Out of the 9 most popular, we've had requests to add support for:
- RICE
- Value vs. Effort (gitlab-org/gitlab#202041)
- Story Mapping ( Lots of internal dogfooding use cases...case 1, case 2, case 3
- Cost of Delay (also known as Weighted Shortest Job First) Feature Request, Benefit-weight ratio issue
Most of these prioritization frameworks struggle from ambiguity around determining what the value is for any given piece of work. Solving for that is going to be a long term challenge but it is one of the largest opportunities I picked up on when speaking with with the several analysts from Gartner regarding the biggest challenges enterprises are currently facing (including ourselves) and how they are thinking about value stream management.
After lightly researching each model, I believe the best framework to start with is Cost of Delay because:
- When we get to building it into the product, it would allow us to support WSJF, which is the primary prioritization mechanism used in SAFe. SAFe is also the defacto scaled agile framework with adoption rates speeding up, not slowing down. Note: I am in no way advocating that GitLab adopt SAFe. Please no.
- It allows you to quantify backlog in terms of monetary value, helps PMs make better decisions based on the value that matters most to the company, and changes the team's mindset around features from cost/efficiency to speed/value.
One of the better learning resources I've found for understanding CoD is here. The thing that works well about their approach is separating value from urgency. By additionally layering on Duration (weight), you can generate a more total value in a given time for a scarce or constrained capacity. There are a broad spectrum for Cost of Delay approaches. Ideally, we would work towards Quantified Cost of Delay.
Better Understanding Cost of Delay (CoD) Divided By Duration
CoD is based on what appears a relatively simple surface level formula -- (Value (V) x Urgency (U)) / Duration (D).
Value Profiles
Value in this formula is considering monetary units -- or in GitLab's case, U.S. Dollars. Value can be further classified into its own framework based on what type of business outcomes we are hoping to achieve. These are referred to as Benefit Types:
- Increase Revenue: Increasing sales to new or existing customers. Delighting or disrupting to increase market share and size.
- Protect Revenue: Improvements and incremental innovation to sustain current market share and revenue figures.
- Reduce Costs: Cost that we are currently incurring that can be reduced. More efficient, improved margin or contribution.
- Avoid Costs: Improvements to sustain current cost base. Costs we are not currently incurring but may in the future.
It's critical to keep top of mind that what matters is not the $ figures. What matters is that you have the conversations about what is valuable. Don’t get distracted, or worked up, by the emphasis on numbers – you’re simply trying to find a way to simply and quickly communicate the value of what it is that you’re trying to achieve.
Read more about this approach to a Value framework
Estimating Value
How value is estimated will is incredibly variable and depends on specific business models and products, but there are a few general good practices to follow:
- Estimate the beneficial effects of the change
- Make the value equal to the cost of alternatives
- Make assumptions visible
- The goal is accuracy, not precision.
- The Lean “5-Whys” technique can help in identifying the value of a feature. Keep asking “why” until you are able to identify one or more of these benefit types.
In GitLab's case, we have a prioritization table, which states the organization's relative priorities. This is where things get nuanced because a performance issue that only impacts 1% of customers that do not have any sense of urgency for it to be addressed could be a lower priority than a UX Debt issue that impacts 50% of customers.
Urgency Profiles
Short life-cycle; Peak affected by delay
Long life-cycle; Peak affected by delay
Long life-cycle, peak unaffected by delay
Impact of External deadline
Where requirements have a specific deadline the Urgency Profile is slightly different. The benefits profile itself can look like any of the above three, but the benefits only ramp up around a certain date. As a result, the Cost of Delay is zero until the point where you need to start development – to avoid incurring any delay cost.
Read more about Urgency Profiles
Duration
Duration is most often calculated in week units.
Example Scenario
| Cost of Delay | Duration | CD3 Score | |
|---|---|---|---|
| Feature A | $1,000/week | 5 weeks | 200 |
| Feature B | $4,000/week | 1 week | 4,000 |
| Feature C | $5,000/week | 2 weeks | 2,500 |
Using these three features we can look at the impact to two alternatives to how we might schedule them. We could choose to work on and deliver these features one at a time in the order they arrived. A, then B, then C. (This is called First In, First Out. It is a common scheduling approach in manufacturing). After all, the person asking for Feature A will have been waiting for the longest time so we really should serve them first. Then B, and then C.
For the 5 weeks we are working on Feature A we incur the Cost of Delay of all three features: $5,000/wk + $4,000/wk + $1,000/wk. This adds up to $10,000/week times 5 weeks giving us a total Delay Cost incurred so far of $50,000.
We then move on to developing Feature B. For the 1 week this takes us to deliver we incur the Cost of Delay of Features B and C: $4,000/week + $5,000/week = $9,000/week. So the Delay Cost is an additional $9,000, bringing us to a total of $59 worth of Delay Cost incurred so far.
At last, we can start working on Feature C. incurring the Cost of Delay of C during its development of $5,000/week for the two weeks it takes to build Feature C. This is another $10,000 of Delay Cost to add to our previous of $59,000 for a total of $69,000 Delay Cost incurred.
Let’s consider another way of processing these Features. If we work on the features based on whichever has the highest CD3 score we would do Feature B first, followed by Feature C, and finally Feature A.
- For the 1 week we are working on Feature B we incur Cost of Delay of $(4k+5k+1k)/week. Delay Cost = $10,000
- For the 2 weeks we are working on Feature C we incur Cost of Delay of $(5k+1k)/week. Delay Cost = $12,000
- For the 5 weeks we are working on Feature A we incur Cost of Delay of $1k/week. Delay Cost = $5,000
Total Delay Cost using CD3 is $27,000 a 61% decrease in the Delay Cost.
As you can see, using CD3 to order your backlog and prioritize features or projects can have a big difference.
Other Prioritization Framework To Explore
...