Collect consumption measures for devops:package

Problem to solve

As a product manager, I need accurate, timely data in order to better prioritize and invest engineering resources. As a company, we need dashboards at both the stage and company level that measure usage and engagement with the product. https://gitlab.com/gitlab-org/gitlab-ce/issues/61583 aims to collect and measure general adoption of Package features. However, it does not capture consumption metrics, which would allow us to better understand Stage Monthly Active Users (SMAU).

We will identify and collect these metrics, so that they can be added to a Package specific dashboard as well as the GitLab Monthly SMAU dashboard

Intended users

  • The Package team at GitLab, which will use the data to make decisions.
  • The Executive team at GitLab, which will use the data to gauge usage and investment.
  • The Data team, which will use the data to help build dashboards and other reports.

Proposal

Begin to track the following items: For 12.2 we will only look at the P1 items

No Priority Category Metric Aggregated by SMAU Eligible? Data available? Being worked on by:
1 P1 Container Registry # of page views of /container registry
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
 Yes ? Complete
2 P3 Container Registry # of unique users that visit /container_registry
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
 Yes ?
3 P2 Container Registry # of docker push events for the Container Registry
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
  • Tool: Command Line/CI/CD
 Yes ?
4 P2 Container Registry # images deleted from the UI
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
 Yes ? Complete
5 P3 Container Registry # API calls to the container registry
  • Delete Repository tag
  • Delete Repository tags in bulk
  • Get details of repository tag
  • List repository tags
  • Delete registry repository
  • List registry repository
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
 Yes ? Complete
6 P2 Container Registry # garbage collection commands executed sudo gitlab-ctl registry-garbage-collect or sudo gitlab-ctl registry-garbage-collect -m
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
 Yes ?
7 P2 Container Registry # Instances with the container registry turned on/off
  • Registry[enabled]: true in /home/git/gitlab/config/gitlab.yml
  • Registry[enabled]: false in /home/git/gitlab/config/gitlab.yml
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
 This is useful, but probably not useful for SMAU yes Complete
8 P1 Package Registry # of page views of /packages
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
 Yes ? Complete
9 P3 Package Registry # of unique users that visit /packages
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
 Yes ?
10 P2 Package Registry # of mvn deploy to the Maven Repository
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
  • Tool: Command Line/CI/CD
 Yes ?
11 P2 Package Registry # of npm push events to the Maven Repository
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
 Yes ?
12 P3 Package Registry # of packages deleted from the UI
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
 Yes ? Complete
13 P1 Dependency Proxy # of visits to /dependency_proxy
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
 Yes ? Complete
14 P3 Dependency Proxy # of docker pull gitlab.com/groupname/dependency_proxy/containers commands executed
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
 Yes ?
15 P2 Dependency Proxy # Instances with the dependency proxy turned on/off
  • dependency_proxy:enabled: true in config/gitlab.yml
  • dependency_proxy:enabled: false in config/gitlab.yml
  • Time: day/week/month/quarter/year
  • Instance: Self-managed CE/Self-managed EE
  • Account type: Core/Starter/Premium/Ultimate
 Useful but not for SMAU ? Complete

Permissions and Security

  • Access to data and any related dashboards should be limited to internal GitLab employees only.

Documentation

  • We will need to document data definitions and exceptions. The data model is being tracked here
  • As the PM for the package team, I will want to create and circulate documentation on data definitions and make them easily accessible for others.

Testing

  • Test to ensure that data is being tracked accurately
  • No degradation in performance

What does success look like, and how can we measure that?

Success looks like we are tracking usage data, reporting on it and that data is being used to make important decisions.

Links / references

Edited by Giorgenes Gelatti