MR Percentage Powered by Duo

When having a conversation with @kkyrala, he presented a pretty novel idea that I'd like to bring to the wider group.

Background

We have a directive to be dogfooding our AI offerings. This is to provide feedback to the product groups and to get the benefit of AI assistance in our work. One clear usage of AI across our group is for assistance with coding tasks.

It's very difficult to show if we are using AI or not. There aren't a lot of per-user analytics built into the system, and it's hard to quantify how helpful Duo or Claude are on our commits.

Proposal

Measure the percentage of Production Engineering MRs that are powered by Duo or Claude.

How?

In the MR, add a label: Powered by Duo or Powered by Claude if you used these tools in creating the MR. It does not matter if you used the results of AI or not, but the fact that you engaged with AI to complete this task.

Start with a simple script that scrapes user activity looking at MRs authored by team members in our group. Count the MRs and calculate a percentage of MRs that have this label. (On both the ops and main instance).

Please note percentage. I don't want to get into the relm of MR count.

To make this effective, we would need everyone in Production Engineering to do this.

Feedback

I'd like to hear your feedback on this. Any thoughts, comments or concerns?