Create DORA 4 Widget Tiles in Project CICD Analytics

changed milestone to %13.7

added Category:Release Orchestration UX devopsrelease [DEPRECATED] needs weight workflowready for design + 1 deleted label

added to epic &4404 (closed)

added cicdactive label

mentioned in issue #241756 (closed)

changed title from Create DORA 4 Widget Tiles in CICD Analytics to Create DORA 4 Widget Tiles in Project CICD Analytics

added GitLab Premium label

added [deprecated] Accepting merge requests label

added sectionops label

changed milestone to %13.8

Moving to %13.7

changed milestone to %13.7

@rayana - I think this is one we can pull into your design queue - I have weighting for this in %13.6 I am imagining the tiles to like this image below, WDYT?

@jreporter coolio! For some reason I thought we already had this in the ux pipeline? please feel free to add it to my epic so it can be stack-ranked accordingly

Okay perfect, it has been moved @rayana and stack ranked within your epic

changed epic to &2439 (closed)

Frontend weight estimation:

I'm going to make a few big assumptions:

The backend will already have a way to capture/compute all 4 of these metrics and deliver them to the frontend
The UI is a very basic: just four tiles, each showing one data point.

If these assumptions are correct, I think it's safe to say this is a frontend-weight1. The vast majority of the work for this feature will be in figuring out how to accurately capture these metrics on the backend and deliver them through an API.

Some other ideas for iteration:

If we will have some metrics ready before others (which I think is likely), we could split each tile into its own issue/MR and deliver them separately. If we went this route I think it would make sense to call this a frontend-weight4.
Or, instead of delivering them separately, we could show some kind of "coming soon" text for tiles that aren't yet implemented (so users are aware the feature is in development).

@nfriend I think that if we don't have the metrics implemented, we shouldn't show them in the card at all.

Based on the current scope and your estimation, I created these Figma prototypes

Please leave any feedback using Figma's comment tool (press the key C to leave a comment anywhere in the prototype).

I added:

Tooltip with explanation for DORA4 metrics
Not enough data state
Skeleton loader

These are Stretch, so feel free to suggest moving to different issues.

@rayana Those prototypes look great! I'm assuming for this issue we'd only be implementing the "DORA 4 metrics" box?

I think those small additions (empty states, tooltips, skeleton loaders) can all be included in the MVC.

I think that if we don't have the metrics implemented, we shouldn't show them in the card at all.

So just to be clear, if we don't implement the "change failure rate" metric right away, for example, we shouldn't include that metric in the card at all? Or we should show the "Not enough data" message as is shown in the mockups?

@nfriend cool! Thanks for reviewing it

So just to be clear, if we don't implement the "change failure rate" metric right away, for example, we shouldn't include that metric in the card at all? Or we should show the "Not enough data" message as is shown in the mockups?

That's right. I think we can make a differentiation between:

the metric/feature not being implemented yet
the application does not have enough data to display

In case the 1. point is valid, we simply don't show anything.

But when we do implement Change failure rate in the API, but there is not enough that to display on this view, we can then display the Not enough data message. This way, we reinstate that customers/users can make use of the DORA 4 key metrics to measure DevOps performance. cc @jreporter

But when we do implement Change failure rate in the API, but there is not enough that to display on this view, we can then display the Not enough data message. This way, we reinstate that customers/users can make use of the DORA 4 key metrics to measure DevOps performance.

Is this something we should have docs for?

Agree with the current weight

Question: Do we have an issue like #208947 (closed) to create an API to fetch the relevant data?

Deployment frequency: total / day, e.g. 30 / day
Lead time for changes: hh:mm:ss, e.g. 20h 30m 12s
Time to restore service: hh:mm:ss, e.g. 20h. 30m 12s
Change failure rate: %, e.g. 15%

@jakeburden - that is right we have #241730 for the group and #250657 (closed) for the project

added frontend-weight1 label

@jreporter Just a side note/question - in all of our issues and discussions about this feature, we've been referring to these metrics as "DORA 4". But so far all of the UI mockups I've seen haven't mentioned the name "DORA 4". Should we consider putting this text in the UI somewhere? Maybe a heading called "DORA 4 Metrics?" (Or even an entire "DORA 4 dashboard"?)

Or maybe this term is copyrighted?

/cc @rayana

Solid question, at first glance, it is not copyrighted and does make sense to call it that @nfriend @rayana

@jreporter @nfriend My gut feeling says that the CI/CD term is:

Easier to digest but also reconcile with GitLab as a whole.
We don't have to explain CI/CD but we have to explain DORA 4 metrics.
The DORA 4 are a subset of metrics we can add to this dashboard. @jreporter as per our discussion of the CI dashboard.
Dora the Explorer, anyone?

If we feel strong about naming it, let's ask our users

I am satisfied with CICD, maybe the widget name can be DORA 4 since that is already an industry term @rayana

Lolz on Dora

For what it's worth, I talked to a large enterprise customer recently who had never heard of DORA 4. When we implement this in the Analytics pages I'm thinking we'll explain DORA 4 in the docs but probably not use that term in the UI

@jreporter Some questions/thoughts on how these metrics will be displayed:

Timeframe

What time frame are we going to use in order to display the metrics?

All time
per week
per day
per hour, min -- how granular?

I'm looking at &4358 (closed) for reference

Lead time

Do we display it in hours? Days?

Filtering

In the future, would we want to add some sort of navigation so users can select how they want to see the metrics over time? See example https://gitlab.com/groups/gitlab-org/-/security/dashboard

Timeframe:

Per day

Lead Time

Hours until it reaches 24 then convert to Days and Hours?

Filtering

Yes we would want this to be an enhancement down the road

@rayana FYI ^

Thanksss <3

@nfriend thoughts on making hours turn into days?

@jreporter I understood that we show the numbers per day (last 24h) for all metrics.

On a second thought regarding converting hours into days: are days more straightforward than hours? What I worry here is that people still would need to find a way to see the actual time spent on time to restore service, for example.

Right now, when we turn hours into days on the UI, users need to hover over the item to see actual timestamp... that creates extra friction when trying to uncover when something happened/how long it took in the system.

If we use per day, then having 24h turn into days would be confusing... wdyt? Should we display the last week data, for example?

Okay, per day makes perfect sense if that is a paradigm we already have @rayana

@jreporter ok! I updated the prototypes and proposal here #250658 (closed) let me know! I also added a couple of extra things on the frontend for Nathan to evaluate, but I'm expecting them to be delivered in different issues #250658 (comment 428361192)

@rayana - can you advise on what you'd like @nfriend to evaluate? I did not see specific @ mentions for him. Additionally I added the permission expectations

thoughts on making hours turn into days?

@rayana I think this should be doable! I haven't looked at what's available to us, but most date/time libraries include some kind of "humanize" feature that converts the unit into the nearest round unit.

changed the description

added typefeature label

assigned to @rayana

added frontend label and removed typefeature label

changed the description

added Technical Writing label

added UI text label

@aqualls this issue will require Technical Writing. Pinging you here for awareness. See issue description for latest SSOT.

Here's a draft of the docs &4358 (closed)

I just heard that @ngaskill is now TW for Release Management.

Nick, can you have a look at this issue and provide the necessary UI text and Technical Writing feedback? Thanks!

@rayana I looked at the Figma mockups linked in the issue description above, and they all look good to me. However, I don't see any docs drafts at &4358 (closed). I see a Documentation section in that epic with information about DORA in the Details dropdown, but no drafts in MRs.

@ngaskill in the documentation section you'll find the draft of the docs hidden in an expandable area &4358 (closed) There's no MR so far. I pinged you ahead of time in case you want to provide early feedback and gather more context around the topic. Hope that helps!

@rayana Got it. Thanks. When we create the docs, I don't think we need to go into that much detail about the background of DORA itself. We can just mention it briefly and link to the various sources mentioned in that epic. What we'll want to focus on is the GitLab-specific functionality mentioned in the section, "How to measure DORA metrics in GitLab?"

Cool! Thanks, @ngaskill

@ngaskill I just found this handbook page that mentions the DORA 4 https://about.gitlab.com/handbook/marketing/product-marketing/devops-metrics/#step-1-dora-metrics

removed [deprecated] Accepting merge requests label

mentioned in issue #259691

marked this issue as related to #259691

removed the relation with #259691

changed the description

added workflowplanning breakdown label and removed workflowready for design label

removed needs weight label

set weight to 1

changed epic to &4358 (closed)

Moving over to workflowplanning breakdown

marked this issue as related to #241730

added workflowready for design label and removed workflowplanning breakdown label

added 1 deleted label

added workflowscheduling label and removed workflowready for design label

@nicolewilliams - I have added the ~"Release::P2" and moved in workflowscheduling

marked this issue as related to #250657 (closed)

added typefeature label

mentioned in issue #273172

changed milestone to %Backlog

mentioned in epic &4404 (closed)

@rayana @dimitrieh @icamacho - There are two issue that talk about project level Dora4 metrics - this is one and the other is &4783 (closed).

I am pulling in @dimitrieh 's proposal of &4783 (closed) and would like to close this one - please let me know if there are any concerns before closing this

@ogolowinski this sounds good to me. Looking at the concept of this issue I do think the ideas could ultimately be combined. This issue focuses more on highlighting a few key metrics in a widget, rather than extending with more graphs.

I agree with the direction taken, where we extend first with &4783 (closed) after which we can review this proposal again.

cc: @rayana

added grouprelease [DEPRECATED] label and removed 1 deleted label

unassigned @rayana

added [deprecated] Accepting merge requests label

mentioned in issue #275996 (closed)

@ogolowinski We have implemented the tiles for two of the DORA metrics in Group Value Stream Analytics. Are you able to reuse any of that code for this issue? The key difference I see is you are planning to show data for the last 24 hours where as the VSA tiles support selecting a date range going back up to 180 days.

@ljlane we are planning to use the VSA titles for GitLab Core . For paid tiers - the tiles will be present and serve as links to the DORA4 charts

added severity4 label

added 1 deleted label

Closing since this will be delivered under the VSA page - Split VSA metrics into 2 rows (#359060 - closed)

closed

Create DORA 4 Widget Tiles in Project CICD Analytics

Release notes

Problem to solve

Intended users

User experience goal

Proposal

Further details

Permissions and Security

Documentation

Availability & Testing

What does success look like, and how can we measure that?

What is the type of buyer?

Is this a cross-stage feature?

Links / references

Designs

Child items ...

Activity

Timeframe

Lead time

Filtering

Create DORA 4 Widget Tiles in Project CICD Analytics

Release notes

Problem to solve

Intended users

User experience goal

Proposal

Further details

Permissions and Security

Documentation

Availability & Testing

What does success look like, and how can we measure that?

What is the type of buyer?

Is this a cross-stage feature?

Links / references

Relates to

Activity

Timeframe

Lead time

Filtering