Pipeline Execution FY23 Async RICE scoring
Summary
As we look ahead to FY23 we have a big audacious goal ahead of us, increase the number of users triggering pipelines by 40%. We also have a backlog of almost 2500 issues and 58 epics currently open. Prioritizing within this so we can stay focused on our long term vision is a team effort and I want to ensure we are all involved.
Over the next 1-2 weeks, we would like to conduct a prioritization exercise with the team using the RICE framework to help shape Pipeline authoring direction for FY23. This will be an async-first exercise using a scoring template that team members can complete when time allows.
Participation is optional, though we highly encourage everyone to get involved as this is an opportunity to provide input on what the team will work on over the next 6-12 months.
Instructions
- Read about the RICE framework in the handbook
- Watch this Video from Dov.
- Go to the RICE scoring spreadsheet. Follow the instructions to enter your scores.
- If you would like to suggest any additional/existing epics or issues for consideration, please comment in this issue and add it to the spreadsheet along with your scoring.
- For each epic in the scoring spreadsheet, choose a score for each RICE factor. The final RICE score should be automatically calculated in column F.
- We understand that you may not feel comfortable scoring each factor for an epic. That's ok. There are no right or wrong answers. Just provide your best guess.
- If you think an epic or issue is not worth our consideration please comment here and leave the scoring blank. This input is equally if not more valuable!
Epics and issues for Scoring
- Add a pipeline performance overview at group level
- Add configurable pipeline priority
- API Support for Merge Trains
- Bugs: MR Mergeability
- CI/CD Observability
- Configure CI minute limits per project and namespace
- Ensure after_script is called for cancelled and timed out pipelines
- Fast-forward merge support
- Improve 'Merge' button experience for different states
- Improve the experience around debugging jobs and analyzing pipelines
- Improve time to script execution
- Job Filter and Search
- Make GitLab CI/CD work better with external repositories
- Make pipeline permissions more controllable and flexible
- Merge Trains Improvements
- MVC & Follow-Ups: Persistent, shared work areas between pipeline jobs
- Passing Variables Downstream
- Pipeline create is slow
- Present information around running pipeline activities in GitLab CI
- Provide pipeline analytics to GitLab CI Users
- Secure CI_JOB_TOKEN Permissions
- Secure CI_JOB_TOKEN workflows
@jheimbuck_gl on 2022-02-04)
Results (From

Walk through of this content on GitLab Unfiltered

My initial thoughts
You all are amazing. I really appreciate everyone taking time to look at all of this, add thoughts and critically think about what we can do to impact our users.
What was surprising?
For me it was surprising not see as much customer interaction in the Merge Train epics/issues and the improve time to script execution epic. My hypothesis is that we don’t have as many users utilizing merge trains as maybe we think and for the second more and more users utilize private runners and not SaaS runners so start times aren't as important. As we think about SaaS runner MAU as a Performance Indicator for us we may lean into understanding this more.
What wasn’t surprising?
We have a bunch of epics that overlap each other, are dated, belong in other groups, etc. With one of the oldest and most mature areas of the product I really expected this and glad to see we identified some key areas we can cleanup. Next steps for me will be to get all those analytics/observability/etc. epics into a single epic and then break them down into JTBD. The other area that needs organization was the Secure CI_JOB_TOKEN / pipeline permissions work.
What were the results?
In the aggregate the following epics were our top 10 and my notes on the epics
Epic/Issue | Notes |
---|---|
Pipeline create is slow | This is work in progress. Next steps would be to identify clear success criteria, scope down the work to be done and/or set any other guardrails on how long should keep this going. I think we could deliver this by the end of June maybe. |
Improve 'Merge' button experience for different states | This is a small epic that is closed. |
Ensure after_script is called for cancelled and timed out pipelines | This is the most popular item in the backlog. There is recent interaction with Field Teams. How we do this is unknown and likely involves runner team. I’ve tagged it for FY23 so we can start looking into it. |
Job Filter and Search | The MVC is close but further search/filters should be validated. I suggest we close out that MVC and then let users chime in with what other problems they are trying to solve on the page. |
Bugs: MR Mergeability | Includes work with Create::Source Code Team but we should rip this apart and confirm and then try to move forward with it. I've tentatively put the epic into Q3 (Aug - Oct) for now. |
Improve time to script execution | There's no/little customer interaction within the epic/issues most of which are Runner related. I don't think we'll prioritize this in FY23 |
Secure CI_JOB_TOKEN Permissions / Secure CI_JOB_TOKEN workflows | Popular items but a relatively unknown amount of work to be done / what are the right use cases we should support and what should we actively NOT support? |
Provide pipeline analytics to GitLab CI Users / Improve the experience around debugging jobs and analyzing pipelines | These are called out as key things that move maturity on the direction page. |
What's next?
- I've started applying the ~fy23 labels to these epics and issues
- Post video recap here / create direction updates.
- Direction updates based on the FY labels applied
- Start discussion about the group Performance indicator - #95 (closed)
- Combine/better differentiate the CI_JOB_TOKEN/Pipeline permissions epics and the "pipeline analytics" epics targeted later in the year. - @jheimbuck_gl
- I'll start looking at epics in Q2 and beyond to set parameters for the opportunities, possible iterations and any solution validation we need to do with the team later in February. - @jheimbuck_gl @v_mishra