Value Stream Analytics: Customizable Stages
#35823 (closed) for final steps we're taking to make this feature Generally Available.
Please seeProblem to solve
Many customers like the idea of cycle analytics, but would like to see the stages defined as per their own workflows. While we prefer convention over configuration, it seems necessary to allow for the representation of different workflows before we can fully automate the calculation of the stages in cycle analytics. We do see the 2 approaches converging in the future.
Intended users
PM, EM, Executives
Further details
This issue represents the MVC for Value Stream Analytics as it would allow an end user to map and monitor the flow of work from idea to production as it happens in their organization.
Solution
Default state
When you first enter Cycle Analytics, you will still see the default stages we have today.
Since customizable stages need to be reordered after they are added, the elements in the list need to have that ability. The Design System pattern for reorderable lists is to use cards, so each stage will be represented by a draggable card.
The selected card will display a blue background and border, as well as the vertical bar we currently display for the selected stage in Cycle Analytics.
At the bottom of the list, there will be a 'ghost' card with the label Add a stage
. This card will be styled with a dashed border. When you click that card, a form will show up in the right-hand side panel of the table so you can specify its parameters.
Custom stage form
The custom stage form will contain the following fields:
- Stage name: The name that will be displayed in the list.
- Object type: The type of object that will be monitored.
- This design accounts for issues and merge requests, but backend will investigate whether it's possible to include other objects such as consider pipelines, commits, deployments, etc.
- This field is single-select.
- Start event: The event that will trigger the stage time to start counting.
- The options available will depend on the object type that's being monitored (e.g. we can monitor the last build associated to MRs but this option will not be present for issues).
- If the user selects a type of event that allows a parameter (e.g.
Label added
-> Which label?), a second dropdown will appear to allow the user to make this selection. We will use the default dropdowns we use in the issuable sidebar or dropdown filters, except these can only be single-select.
- Stop event: The event that will trigger the stage time to stop counting.
- This field will be disabled until a Start event is selected. This way, we can control which Stop events to offer depending on their compatibility with the selected Start event. Incompatibilities are listed below.
Once all the fields have been completed, the Add stage
button will become clickable.
Object type | Start event | Start parameter, Stop enabled | Filled out |
---|---|---|---|
Events
The events available for each object type are the following. Some events admit a parameter, which is marked in code styling (we will trim down on the rules in future iterations):
IssueCreated => IssueClosed, IssueFirstAddedToBoard, IssueFirstAssociatedWithMilestone, IssueFirstMentionedInCommit, IssueLabelAdded, IssueLabelRemoved, IssueLastEdited IssueFirstAddedToBoard => IssueClosed, IssueFirstAssociatedWithMilestone, IssueFirstMentionedInCommit, IssueLabelAdded, IssueLabelRemoved, IssueLastEdited IssueFirstAssociatedWithMilestone => IssueClosed, IssueFirstAddedToBoard, IssueFirstMentionedInCommit, IssueLabelAdded, IssueLabelRemoved, IssueLastEdited IssueFirstMentionedInCommit => IssueClosed, IssueFirstAssociatedWithMilestone, IssueFirstAddedToBoard, IssueLabelAdded, IssueLabelRemoved, IssueLastEdited IssueLabelAdded => IssueClosed, IssueFirstAddedToBoard, IssueFirstAssociatedWithMilestone, IssueFirstMentionedInCommit, IssueLabelAdded, IssueLabelRemoved, IssueLastEdited IssueLabelRemoved => IssueClosed, IssueFirstAddedToBoard, IssueFirstAssociatedWithMilestone, IssueFirstMentionedInCommit, IssueLabelAdded, IssueLabelRemoved, IssueLastEdited IssueClosed => IssueLabelAdded, IssueLabelRemoved, IssueLastEdited MergeRequestCreated => MergeRequestClosed, MergeRequestFirstDeployedToProduction, MergeRequestLabelAdded, MergeRequestLabelRemoved, MergeRequestLastBuildStarted, MergeRequestLastBuildFinished, MergeRequestLastEdited, MergeRequestMerged MergeRequestClosed => MergeRequestFirstDeployedToProduction, MergeRequestLabelAdded, MergeRequestLabelRemoved, MergeRequestLastEdited MergeRequestFirstDeployedToProduction => MergeRequestLabelAdded, MergeRequestLabelRemoved, MergeRequestLastEdited MergeRequestLabelAdded => MergeRequestClosed, MergeRequestFirstDeployedToProduction, MergeRequestLabelAdded, MergeRequestLabelRemoved, MergeRequestLastBuildStarted, MergeRequestLastBuildFinished, MergeRequestLastEdited, MergeRequestMerged MergeRequestLabelRemoved => MergeRequestClosed, MergeRequestFirstDeployedToProduction, MergeRequestLabelAdded, MergeRequestLabelRemoved, MergeRequestLastBuildStarted, MergeRequestLastBuildFinished, MergeRequestLastEdited, MergeRequestMerged MergeRequestLastBuildStarted => MergeRequestClosed, MergeRequestFirstDeployedToProduction, MergeRequestLabelAdded, MergeRequestLabelRemoved, MergeRequestLastBuildFinished, MergeRequestLastEdited, MergeRequestMerged MergeRequestLastBuildFinished => MergeRequestClosed, MergeRequestFirstDeployedToProduction, MergeRequestLabelAdded, MergeRequestLabelRemoved, MergeRequestLastEdited, MergeRequestMerged MergeRequestMerged => MergeRequestFirstDeployedToProduction, MergeRequestLabelAdded, MergeRequestLabelRemoved, MergeRequestLastEdited
Incompatible events
Only compatible options will be displayed in the Stop dropdown (e.g. Merge request created
can never be a stop event). The incompatible pairings are:
Start event | Stop event | Reason |
---|---|---|
Merge request created | Merge request created | Same event |
Merge request merged | Merge request created | Creation cannot happen before other events |
Merge request closed | Merge request created | Creation cannot happen before other events |
Merge request edited | Merge request created | Creation cannot happen before other events |
Label added | Merge request created | Creation cannot happen before other events |
Label removed | Merge request created | Creation cannot happen before other events |
Last build started | Merge request created | Creation cannot happen before other events |
Last build finished | Merge request created | Creation cannot happen before other events |
... | ... | ... |
If a user selects a start and stop event, and they then change the start event to something that creates an incompatible combination, the stop event field will be cleared and an error message will be displayed inline:
The stop event you had selected is incompatible with the new start event
In the future, it would be good to display why a start and stop event are incompatible.
Stage added successfully
Once the stage is created, it appears at the bottom of the list and can be dragged to a different position.
Editing
Custom stage | Default stage |
---|---|
Hovering over a card will reveal a vertical ellipsis icon, that when clicked will reveal a dropdown.
For custom stages, the options will be:
- Edit stage
- Hide stage
For default stages, the only option will be:
- Hide stage
Choosing the edit option for a custom stage will open the form again and the user will be able to change all the fields. The confirmation button will change from Add stage
to Save changes
.
Recovering hidden stages - NOT TO BE DONE FOR THIS ITERATION
In order to recover a previously hidden stage, there will be a dropdown in the top-right corner of the form. When opened, it will display default and custom stages that had been previously hidden:
Default stages available | Default stages not available |
---|---|
If there are no stages to be recovered in either category, an 'empty state' message will be displayed:
- Default stages:
There are no hidden default stages
- Custom stages:
There are no hidden custom stages
When the user clicks on a custom stage, the form will be filled out with the information and the user will be able to click the Add stage
button. If the user edits any parameters before adding it, the stage will be considered a new one.
If the user selects a default stage, the fields will be disabled because the events we use for default stages are more complex than what we allow in custom stages.
Note: The copy in this last mockup is not accurate. The correct copy will be detailed in the following table:
Stage | Object type | Start event | Start event parameter | Stop event | Stop event parameter |
---|---|---|---|---|---|
Issue | |||||
Plan | |||||
Code | |||||
Test | |||||
Review | |||||
Staging | |||||
Production |
Persistence
For the MVC, customizations will be persisted per namespace. For example, the custom stages I may define for the gitlab-org group will be seen by everyone who visits analytics for that group. Those customizations will not affect other groups or projects. For the first iteration, they will have to be defined for each namespace individually.
Permissions and Security
Similarly to the other analytical pages, it should be visible to everyone, but a user can only see the data if:
- self-managed clients Premium +
- projects and namespace on Silver + (for gitlab.com).
We only allow for users Reporter
to view (basically similar to Productivity Analytics, so @pshutsin can probably explain what he did).
Old Proposal 2
We will upload a new mock up, but the premise of the customizable cycle analytics will be based on event start and end date, similar to https://gitlab.com/gitlab-org/gitlab-ee/issues/12196#note_191361423/ & https://gitlab.com/gitlab-org/gitlab-ee/issues/12196#note_188808164.
- A user will be able to add/remove new stages
- Every stage can be defined based on the type of event and their start and end date, for example on the default stages:
- we have issue defined as: start: issue creation, end: assigned to a milestone / assigned to a board list
The addition / removal of a label will also be considered an event, so something like in review can be defined as: start: workflow::in test, end: workflow::in staging.
As an MVC, we will only look include issues/MRs/jobs, etc with an end event during a stage. WIP will be worked on in a different issue.
- Let's have the same graphs as per the default cycle analytics per stage, which we are building here https://gitlab.com/gitlab-org/gitlab-ee/issues/12078? This probably means that we will have the drop-downs in the graphs based on the stages that the user defines, @cperessini?
Old Proposal
In 12.1, we are introducing Group Cycle Analytics. As a next step, we would like to introduce custom stages in Cycle Analytics.
- Let's have a on-off
customize
button on the top right corner of the page that if slid would flip to a customizable version of the page? We can allow a user to quickly search for a label on the specific stage similarly to how we do on assigning labels for an issue.
- We will recommend users to make use of scoped labels to define the stages (below can be used as default/example):
- workflow::In defintion
- workflow::Ready for Development
- workflow::In Development
- workflow::Ready for Review
- workflow::In Review
- workflow::Ready for UAT
- workflow::In UAT
- workflow::Ready for Deployment
- workflow::In Production
In our current review process, when a reviewer gives a comment that requires information / fixes, the issue should be changed back from In review
to Ready for Development
. The sum of all the time spend in a stage even if not linear, would be attached to a stage. In order to calculate the stages, we will use the difference between the timestamps for the labels.
With time we should try to allocate the waiting
vs active
programmatically, so that on a high level management can aggregate as New
, Waiting
, Active
, Done
.
The drawback of this approach is that it's fully manual, i.e. an engineer needs to push a label such as workflow: In Dev
together with their commit/on the MR. Furthermore, more than 1 MR can be used to solve 1 issue and is often times recommended. How we can combine with information from multiple MRs will be followed up in another issue.
- Let's have the same graphs as per the default cycle analytics per stage, which we are building here https://gitlab.com/gitlab-org/gitlab-ee/issues/12078? This probably means that we will have the drop-downs in the graphs based on the labels that the user defines, @cperessini?
Documentation
Testing
What does success look like, and how can we measure that?
What is the type of buyer?
Links / references
Questions:
-
Do we already save somewhere the timestamps of adding and removing labels in order to calculate the time?
-
Can we push labels to JIRA as an update in our commits
-
Do we get the time an issue is created or scheduled for a milestone from JIRA?
Answer: https://gitlab.com/gitlab-org/gitlab-ee/issues/12196#note_187215251