Define SMAU for Package
As we move to stage monthly active users being a better representation of usage, we should re-evaluate the set of criteria for which a user is considered a SMAU in Package.
This issue should either confirm the existing definition, or include the proposal and implementation of a new definition.
Problem to solve
As a product manager, I need accurate, timely data in order to better prioritize and invest engineering resources. As a company, we need dashboards at both the stage and company level that measure usage and engagement with the product. https://gitlab.com/gitlab-org/gitlab-ce/issues/61583 aims to collect and measure general adoption of Package features. However, it does not capture consumption metrics, which would allow us to better understand SMAU.
We will identify and collect these metrics, so that they can be added to a Package specific dashboard as well as the GitLab Monthly SMAU dashboard
Intended users
- The Package team at GitLab, which will use the data to make decisions.
- The Executive team at GitLab, which will use the data to gauge usage and investment.
- The Data team, which will use the data to help build dashboards and other reports.
Proposal
Begin to track the following items:
| Priority | Category | Metric | Aggregated by | SMAU Eligible? | Data available? |
|---|---|---|---|---|---|
| P1 | Container Registry | # of page views of /container registry
|
|
Yes | gitlab.com only |
| P3 | Container Registry | # of unique users that visit /container_registry
|
|
Yes | ? |
| P2 | Container Registry | # of docker build events for the Container Registry |
|
Yes | ? |
| P2 | Container Registry | # of docker push events for the Container Registry |
|
Yes | ? |
| P2 | Container Registry | # images deleted from the UI |
|
Yes | ? |
| P3 | Container Registry | # API calls to the container registry
|
|
Yes | ? |
| P2 | Container Registry | # garbage collection commands executed sudo gitlab-ctl registry-garbage-collect or sudo gitlab-ctl registry-garbage-collect -m
|
|
Yes | ? |
| P2 | Container Registry | # Instances with the container registry turned on/off
|
|
This is useful, but probably not useful for SMAU | ? |
| P1 | Package Registry | # of page views of /packages
|
|
Yes | gitlab.com only |
| P3 | Package Registry | # of unique users that visit /packages
|
|
Yes | ? |
| P2 | Package Registry | # of mvn deploy to the Maven Repository |
|
Yes | ? |
| P2 | Package Registry | # of npm push events to the Maven Repository |
|
Yes | ? |
| P3 | Package Registry | # of packages deleted from the UI |
|
Yes | ? |
| P1 | Dependency Proxy | # of visits to /dependency_proxy |
|
Yes | gitlab.com only |
| P3 | Dependency Proxy | # of docker pull gitlab.com/groupname/dependency_proxy/containers commands executed |
|
Yes | ? |
| P2 | Dependency Proxy | # Instances with the dependency proxy turned on/off
|
|
Useful but not for SMAU | ? |
Permissions and Security
- Access to data and any related dashboards should be limited to internal GitLab employees only.
Documentation
- We will need to document data definitions and exceptions. The data model is being tracked here
- As the PM for the package team, I will want to create and circulate documentation on data definitions and make them easily accessible for others.
Testing
- Test to ensure that data is being tracked accurately
- No degradation in performance
What does success look like, and how can we measure that?
Success looks like we are tracking usage data, reporting on it and that data is being used to make important decisions.
Links / references
Edited by Tim Rizzi