Cache DORA aggregate metrics and performance scores

What does this MR do and why?

DORA dashboards on large groups run into the 15 s statement timeout, and every refresh pays the full cost again because nothing is cached. This MR is the first step (MR 1, "cache per group and range") of the plan in https://gitlab.com/gitlab-org/gitlab/-/work_items/636172.

Behind the new dora_aggregate_results_cache flag (gitlab_com_derisk, off by default, checked against the root namespace):

  • Dora::AggregateMetricsService caches the dora_daily_metrics aggregate for 10 minutes. The key is the container, metrics, interval, start and end date, environment tiers and group_project_ids. It contains no user, because the result depends only on the container's projects.
  • Dora::AggregateScoresService caches its payload for 10 minutes per user, project filters and month. The user is in the key because GroupProjectsFinder filters by what the user can see.
  • The scores payload now also includes authorized_projects_count. PerformanceScoresCountResolver reads it from there instead of running its own count over every project in the group, which a cache hit wouldn't have saved.

Authorization and validation run on every call, outside the cache. One known trade-off: if a user loses access to a project inside a group they can still read, the cached score counts can include that project for up to 10 minutes. These are aggregate counts only.

Why a 10-minute TTL

dora_daily_metrics is already not live: DeploymentWatcher schedules the refresh 5 minutes after each deployment, and incident refreshes go through Sidekiq. A 10-minute TTL at most doubles the staleness users already see for deployment metrics. It still catches repeat dashboard loads (reloads, tab switches, several people on the same group and range). There's no explicit invalidation: clearing entries from RefreshWorker would need a key bump for every ancestor of the project.

No changelog entry: the change is behind a flag that is off by default.

References

Screenshots or screen recordings

Not applicable: backend-only change, no UI difference.

How to set up and validate locally

  1. In the Rails console, enable the flag for that group's root namespace:
    Feature.enable(:dora_aggregate_results_cache, Group.find_by_full_path('your-group'))
  2. Open Analyze > Analytics dashboards for the group.
  3. Open any dashboard. Verify rails cache is written:
    Gitlab::Redis::Cache.with { |r| r.scan_each(match: '*aggregate_metrics*').to_a }

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Serhii Yarynovskyi

Merge request reports

Loading
Loading