Improve CI usage quotas performance
What does this MR do and why?
This MR addresses performance concerns from Improve CI usage quotas performance (#386442 - closed).
Previously we had per month per project data fetched all at once — for all months in namespace history we fetched month overview data and all projects data for each month at once.
This change introduces "on demand" per month per project fetching: when user changes the selected month — that per month per project data is fetched from the server.
On page load — we're fetching all months overview and current month projects data ("current month" is defined by ciMinutesLastResetDate BE provided value). This aligns with page behaviour before the change.
Biggest change UX-wise is that now month selector switches data not only for per month per project charts, but also for the projects table below it.
Biggest change code-wise I described in Code changes section.
P.S. since we're operating with Dates, it might be wise to test this MR with different system timezones.
Screenshots or screen recordings
| Before | After |
|---|---|
 |
 |
Code changes
Most of the changes is just component and methods movement.
For reviewers' convenience, here's an overview of the changes
BEFORE AFTER
┌──────────────────────────────────────────────────────────────┐ ┌──────────────────────────────────────────────────────────────┐
│ app.vue │ │ app.vue │
│ │ │ │
│ Fetches all at once: summary, all years per month, │ │ Fetches separately: summary + all years per month, │
│ and all monthly per project data │ │ and selected [MONTH] per project data │
│ │ │ │
│ ┌──────────────────────────────────────────────────────────┐ │ │ ┌──────────────────────────────────────────────────────────┐ │
│ │ usage_overview.vue │ │ │ │ usage_overview.vue │ │
│ │ │ │ │ │ │ │
│ │ Displays summary and usage since ciMinutesLastResetDate. │ │ │ │ Displays summary and usage since ciMinutesLastResetDate. │ │
│ └──────────────────────────────────────────────────────────┘ │ │ └──────────────────────────────────────────────────────────┘ │
│ │ │ │
│ ┌──────────────────────────────────────────────────────────┐ │ │ [YEAR selector] - no refetch │
│ │ minutes_usage_charts.vue │ │ │ ┌──────────────────────────────────────────────────────────┐ │
│ │ │ │ │ │ minutes_usage_per_month.vue + _chart.vue │ │
│ │ [YEAR selector] - no refetch │ │ │ │ │ │
│ │ │ │ │ │ Displays per year per month data. │ │
│ │ Displays two charts: per month and per project data. │ │ │ └──────────────────────────────────────────────────────────┘ │
│ │ ┌──────────────────────────────────────────────────────┐ │ │ │ │
│ │ │ minutes_usage_month_chart.vue │ │ │ │ [MONTH selector] - refetches selected month data │
│ │ │ │ │ │ │ ┌──────────────────────────────────────────────────────────┐ │
│ │ │ Displays per year per month data. │ │ │ │ │ minutes_usage_per_project.vue + _chart.vue │ │
│ │ └──────────────────────────────────────────────────────┘ │ │ │ │ │ │
│ │ │ │ │ │ Displays per month per project data. │ │
│ │ ┌──────────────────────────────────────────────────────┐ │ │ │ └──────────────────────────────────────────────────────────┘ │
│ │ │ minutes_usage_project_chart.vue │ │ │ │ │
│ │ │ │ │ │ │ ┌──────────────────────────────────────────────────────────┐ │
│ │ │ [MONTH selector] - no refetch │ │ │ │ │ project_list.vue │ │
│ │ │ │ │ │ │ │ │ │
│ │ │ Displays per month per project data. │ │ │ │ │ Displays usage since ciMinutesLastResetDate │ │
│ │ └──────────────────────────────────────────────────────┘ │ │ │ │ │ │
│ │ │ │ │ │ [pagination] - will paginate selected month project │ │
│ └──────────────────────────────────────────────────────────┘ │ │ └──────────────────────────────────────────────────────────┘ │
│ │ │ │
│ ┌──────────────────────────────────────────────────────────┐ │ │ │
│ │ project_list.vue │ │ │ │
│ │ │ │ │ │
│ │ Displays usage since ciMinutesLastResetDate │ │ │ │
│ │ │ │ │ │
│ │ [pagination] - will paginate current sonth project │ │ │ │
│ └──────────────────────────────────────────────────────────┘ │ │ │
│ │ │ │
└──────────────────────────────────────────────────────────────┘ └──────────────────────────────────────────────────────────────┘How to set up and validate locally
Open a Usage Quotas > Pipelines tab for a namespace (e.g. http://localhost:3000/groups/gitlab-org/-/usage_quotas#pipelines-quota-tab) and for a profile (e.g. http://localhost:3000/-/profile/usage_quotas#pipelines-quota-tab).
Try switching Year and Month selector, see if charts and the table display the new information.
this section uses awesome instructions shared by @sheldonled on creating CI minutes usage (aka compute units, aka compputinnies)
Start a rails console via rails c in your GitLab project.
To add Namespace ci minutes:
# Create data
Ci::Minutes::NamespaceMonthlyUsage.new(namespace_id: 77, date: Date.new(2023, 8, 01), amount_used: 20, shared_runners_duration: 30).save
# Update data:
Ci::Minutes::NamespaceMonthlyUsage.where(namespace_id: 77, date: Date.new(2023, 8, 01)).update(amount_used: 40, shared_runners_duration: 60)To add Project ci minutes:
# Create data:
Ci::Minutes::ProjectMonthlyUsage.new(project_id: 22, date: Date.new(2023, 8, 01), amount_used: 70, shared_runners_duration: 50).save
# Update data:
Ci::Minutes::ProjectMonthlyUsage.where(project_id: 35, date: Date.new(2023, 8, 01)).update(amount_used: 140, shared_runners_duration: 100)Some details to pay attention:
- The date has to be the first day of the month.
- Use create instructions if you don't have the minutes for the given month
- Use update instructions if there's a record already and you want to change the amount (create will throw an error)
To find namespace and project ids:
Group.all # list all groups with ids
Group.fin(<id>).projects # list all projectsTo test user's profile U/Q page — use the same commands as above. To find your user's namespace and projects ids, do:
admin = User.find(1)
admin.namespace # will output user's namespace
admin.namespace.projects # will output that namespace projectsMR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.