Skip to content

Cached sidebar issues count in group pages

What does this MR do?

Related to #243753 (closed)

This MR implements caching the issues count in the sidebar for group pages with the following conditions:

  • There are two type of values cached, public count and total count. Total count is only visible for signed-in users with at least reporter role and includes confidential issues.
  • Only values >= 1000 are cached, under this threshold the COUNT query will continue to be executed with every page load.
  • Cached values expire after 24 hours.
  • As count values over 1000 are "imprecise" they will be truncated to two or three digits. For example, 2350 -> 2.3k, 14300 -> 14.3k, 112560 -> 112.6k

The main reason for this change is performance improvement as this query is quite expensive for groups with a large number of issues (for GitLab.org, for example, it currently takes ~300ms) For this reason, it's only a basic implementation that would benefit from refactoring to use exiting code if it proves beneficial. The changes will be behind a feature flag so we can monitor the performance changes with a limited impact.

Follow-up improvements

  • Refactor Groups::OpenIssuesCountService and Projects::OpenIssuesCountService to DRY duplicated code.
  • Extend Groups::OpenIssuesCountService to include a #refresh_cache method that can be used to refresh this value after significant changes like issues import or bulk update. Similarly to existing #update_project_counter_caches for project issues.
  • Implement caching for MRs and epics count.

Feature flag

This changes are behind the FF cached_sidebar_open_issues_count that is disabled by default, for testing use Feature.enable(:cached_sidebar_open_issues_count, group).

Screenshots (strongly suggested)

Current query times for GitLab.org

Screen_Shot_2021-01-12_at_15.03.41

Examples

900 9,900 9,999 999,000 9,900,000 999,900,000
900 9900 9999 999000 9900000 999900000

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Eugenia Grieff

Merge request reports