Product Analytics Usage Quota MVC Implementation Issue

Problem

  • As the person writing the check for Product Analytics, I want to see the projects contributing to the bill, so I can verify/feel good about what we're paying for.

Proposal

User Personas

  • Group Admins who have received a bill or will receive a bill for Product Analytics at the end of the month and want to see which projects are using the managed service.

Use Cases to support in the MVC

Release/Iteration 1

  • View each project's usage for the current billing period to date on the group usage quota page
  • View each project's usage for the last billing period - on the group usage quota page
  • Do not show the tab if they do not have Product Analytics enabled for the project
    • we will need to retrieve a list of projects with product analytics settings or projects with product analytics usage under the namespace/group before we decide if the tab should be shown or not. - backend task

Release/Iteration 2

  • View total group usage for the current billing period to date
  • View the total group usage for the last billing period

Future use cases/validated in Solution Validation

  • Usage data on the project's usage quota page
  • Compare this billing period's usage to date to last billing period's to date for the group
    • For Example, by day 14 of the billing period are we at, ahead or below last billing period's usage?
  • Compare this billing period's usage to date to last billing period's to date for a specific project

Assumptions made

  1. The user who gets a bill has access to the Usage Quota page / knows where to find the page to see group/project usage.
  2. The SSOT for Data will be Gitlab.com
  3. It is easier/faster to get individual project data than sum up for the group
  4. Historic data will be slow to query / may require additional database fields in the GitLab database

Open Questions

  1. Product - Where should this live?
    • Group -> Usage quota. This is where SaaS Admins look for things like Storage, CI Minutes, etc. which are generally allocated to Groups.
  2. Technical - Can the data from each project be queried in a single group view?
    • If the usage is stored against the project then we should be able to do a relatively simple model call Namespace > Projects > Product Analytics Usage within the GraphQL resolver. - from this comment
  3. Technical - Can historic project usage be queried in a single group view?
    • As long as we restrict the history usage to the previous billing period or another fixed time period such as per year this should make this query very possible and quick to do 👍 - From this comment
  4. Technical - How often does the event total need to be updated in GitLab.com from the Product Analytics Stack?

How will we measure success

Feature Tracking

  1. Add instrumentation for total unique users who view the usage quota page -> Product Analytics tab

Measures of success

This will be successful if

  1. The graphs on the page can be rendered within 2 seconds of data selection (eg; which month to get data for)
  2. 4 out of 5 customers can complete the following micro jobs in Usability testing
    • Figure out how many events have been processed so far this billing period
    • Figure out if that number is more or less than this time last billing period.
    • Determine how many more projects are using Product Analytics this month than last month.
Edited by James Heimbuck