[FF] cache_total_count_on_group_merge_requests_api -- cache the X-Total count across pages on the group merge requests API

Summary

Roll out caching the X-Total count across pages on GET /groups/:id/merge_requests currently behind the cache_total_count_on_group_merge_requests_api feature flag.

  • DRI: @marc_shaw
  • Team Slack channel: #g_code_review

Note

Process and guidance live in the docs - this issue is just the commands and a place to track the rollout. "Rolling out" means incrementally enabling the flag on GitLab.com to validate stability - it is not the same as releasing the feature, which happens when the flag is removed. Feature flag controls · Feature flag lifecycle

What could go wrong?

With the flag on, GET /groups/:id/merge_requests caches the X-Total count in Redis for 15 minutes per user and filter set. Page 1 always runs the count. Pages after the first and before the last reuse the cached count instead of running it again, so X-Total can be up to 15 minutes stale on those pages. The page the cached count calls the last one always recounts, so X-Next-Page and the Link header stay correct when merge requests are created during a walk. Nothing is written to the database, and cached counts expire within 15 minutes of turning the flag off. To measure the effect, compare json.db_duration_s for this endpoint on requests with page greater than 1 before and after enabling it for a group.

Events

Feature Flag events are only logged by default for feature flags marked for the current or future milestones. To enable while the feature flag is active, see https://docs.gitlab.com/development/feature_flags/#logging

Rollout

Run all production /chatops in #production and cross-post the results to #g_code_review. Background: incremental rollout process, feature actors.

Non-production

/chatops gitlab run feature set cache_total_count_on_group_merge_requests_api 50 --actors --dev --pre --staging --staging-ref
/chatops gitlab run feature set cache_total_count_on_group_merge_requests_api true --dev --pre --staging --staging-ref

Production - percentage rollout (wait ≥15 min between steps, watch dashboards):

/chatops gitlab run feature set cache_total_count_on_group_merge_requests_api <percentage> --actors

Or target specific actors instead:

/chatops gitlab run feature set --project=gitlab-org/gitlab,gitlab-org/gitlab-foss cache_total_count_on_group_merge_requests_api true
/chatops gitlab run feature set --group=gitlab-org,gitlab-com cache_total_count_on_group_merge_requests_api true
/chatops gitlab run feature set --user=marc_shaw cache_total_count_on_group_merge_requests_api true

Before global rollout

Confirm the relevant gotchas before going to 100% - see enabling a feature for GitLab.com:

Cleanup

Remove the flag once deemed stable - see cleaning up. Track it here, or open a follow-up Feature Flag Cleanup issue. Remove the flag and its YAML definition from the codebase, then:

/chatops gitlab run release check <merge-request-url> <milestone>
/chatops gitlab run feature delete cache_total_count_on_group_merge_requests_api --dev --pre --staging --staging-ref --production

Rollback

/chatops gitlab run feature set cache_total_count_on_group_merge_requests_api false                                         # production
/chatops gitlab run feature set cache_total_count_on_group_merge_requests_api false --dev --pre --staging --staging-ref     # non-production
/chatops gitlab run feature delete cache_total_count_on_group_merge_requests_api --dev --pre --staging --staging-ref --production  # remove entirely
Edited by Marc Shaw