Define SMAU for Release
As we move to stage monthly active users being a better representation of usage, we should re-evaluate the set of criteria for which a user is considered a SMAU in Release.
This issue should either confirm the existing definition, or include the proposal and implementation of a new definition.
Progressive Delivery:
Proposal
Begin to track the following items:
| Priority | Category | Metric | Aggregated by | SMAU Eligible? | Data available? |
|---|---|---|---|---|---|
| P1 | Feature Flags | Feature flag page access https://gitlab.com/ogolowinski/testing-project/-/feature_flags |
|
Yes | ? |
| P2 | Feature Flags | Toggle On/Off |
|
Yes | ? |
| P3 | Feature Flags | Percentage of Rollout |
|
Yes | ? |
| P4 | Feature Flags | Age of flags (created date) |
|
Yes | ? |
| P5 | Feature Flags | Stale flags (rollout=100% and feature flag on) |
|
Yes | ? |
| P6 | Feature Flags | # of unleash clients request to gitlab.com for fetching toggles |
|
Yes | ? |
| P1 | Continuous Delivery | Actively turning merge train setting off |
|
Yes | ? |
| P2 | Continuous Delivery | # of merge requests re-queued |
|
Yes | ? |
| P3 | Continuous Delivery | # of pipelines for merge train |
|
Yes | ? |
| P4 | Continuous Delivery | # of MRs on merge trains |
|
Yes | ? |
| P5 | Continuous Delivery | # of projects that enabled "Merge pipelines will try to validate the post-merge result prior to merging". |
|
Yes | ? |
| P1 | Review Apps | # of environments with deployment |
|
Yes | ? |
| P2 | Review Apps | # of stale environments (without deployment) |
|
Yes | ? |
| P1 | Incremental Rollout | Percentage of Rollout |
|
Yes | ? |
| P2 | Incremental Rollout | # of Rollbacks |
|
Yes | ? |
| P3 | Incremental Rollout | # of users that accessed the deploy board |
|
Yes | ? |
| P4 | Incremental Rollout | # of projects that have chosen "Continuous deployment to production using timed incremental rollout". |
|
Yes | ? |
| P1 | Release Orchestration | Release page access https://gitlab.com/gitlab-org/gitlab-ce/-/releases |
|
Yes | ? |
| P2 | Release Orchestration | Environments page access https://gitlab.com/gitlab-org/gitlab-ce/environments |
|
Yes | ? |
Edited by Orit Golowinski