MAU based metric for Agent usage

Problem to solve

At GitLab we are looking at Monthly Active User (MAU) based metrics to evaluate the success of various product categories. We want to define an AMAU (Actionable MAU) for the core GitLab Managed Agent features. This is a candidate for the group level MAU metric.

This metric should be collected by usage ping, and we should be able to break it down by namespace on gitlab.com.

Proposal

For pull-based deployments

• Distinct user count of users who made commits to agent configuration or manifest files (the files watched under the glob configuration in the agent config)

For the CI/CD tunnel

• Distinct user count of users who started a pipeline with a job that interacts with the CI/CD tunnel
• Distinct user count of users who were active in a project where the CI tunnel was used

Questions

• If we are to use the CI/CD tunnel for other features, like frontend access to Kube API, shall we differentiate between these use cases?
  • No, as we are interested in overall user-based usage of the agent features in this case. There should be metrics, not necessarily MAU based ones to account for specific feature usage
• Should the overall agent MAU be the sum of the pull-based and CI/CD tunnel MAUs?
  • This would require us to de-duplicate the users. Overall, we want this number to have a comparable metric to other GitLab features, but if it takes too much effort, then I am happy with having separate metrics. We would need the separate metrics as well as feature level metrics.

Iterations

1. Let's start with the CI/CD tunnel MAU.

Definition of done

• Usage ping metric installed for For pull-based deployments
• Usage ping metric installed for For the CI/CD tunnel