Show DORA API-based Deployment Frequency metric for Premium customers
Overview
In 13.12, the Deployment Frequency metric in group-level VSA switched over to using the DORA4 API so that users will see the exact same deployment frequency metric regardless of whether they are getting that data from the API, CI/CD Analytics, or VSA. This change caused confusion for a number of reasons.
- The DORA4 API is restricted to Ultimate customers. Premium customers had access to the deployment frequency metric in group-level API prior to this change but now they do not. To further confuse things, they are still able to see a deployment frequency metric in project-level VSA.
- In project-level VSA, the deployment metric does not use the DORA4 API. It is showing the deployment frequency for all environments, even if they are not production environments, and the metric is measured based on the time a deployment is created. In contrast, the group-level VSA metric only shows deployments to production environments, and measures from the time a deployment finishes. This is confusing to users who don't have a production environment properly configured because they see a deployment frequency metric at the project level but can't see one at the group level and don't understand why.
Four separate customers have contacted GitLab over the past few weeks indicating that the Deployment Frequency metric in VSA isn't working the way they would expect. As an example, see #335053 (closed).
Proposal
- As a first iteration to reduce the inconsistencies for Premium and Ultimate customers, do a license check for the deployment frequency metric in both project and group level VSA and display a DORA4 API-based deployment frequency metric tile for all Premium and Ultimate customers.
- Update the documentation to clearly explain the difference in how the deployment frequency is calculated for the Free tier and the paid tiers.
Out of scope
For now, Free users will still see the deployment metric in project-level VSA, calculated the old way (all environments and from the time a deployment is created). They will not see a deployment metric at the group level
Edited by Larissa Lane