Draft: Shard namespace usages

What does this MR do and why?

Shard namespace usages to mitigate write lock contention on ci_namespace_monthly_usages.

Currently there is 1 row per namespace per month, meaning every CI build in a namespace contends for the same row lock when updating compute minutes. This MR introduces sharding to distribute writes across up to 10 rows per namespace per month, using a CRC32 hash of the build_id.

Changelog: other EE: true

Rollout plan

This change will be split into 3 individually feature-flagged MRs:

MR 1: Read-path aggregation (lowest risk)

Switch all read consumers to use aggregated results across shards, while writes still go to a single row (shard 1):

• Migration: Add shard_number column with default: 1 and new composite unique index (namespace_id, date, shard_number)
• New query methods: all_current_usages, total_current_minutes_used, total_shared_runners_duration, previous_usages
• Usage#total_minutes_used: Switch to SUM aggregation
• Usage#current_usage: Use all_current_usages(...).first
• Quota#previous_amount_used: Use previous_usages with SUM

Fully backward-compatible since with only shard 1 existing, SUM over one row returns the same value. Can be validated in production with zero behavioral change.

MR 2: Email notification sharding

Update the notification path to be shard-aware:

• any_usage_notified? / any_total_usage_notified?: Cross-shard notification checks
• Usages are already aggregated from usage.rb and quota.rb

Depends on MR 1. Notification writes are isolated and can be validated independently.

MR 3: Write-path sharding (the actual contention fix)

The high-impact piece that changes write distribution:

• find_or_create_current_shard with CRC32-based shard selection
• generate_shard_number
• UpdateProjectAndNamespaceUsageService#namespace_usage: Feature-flagged switch to sharded writes
• First-shard detection logic for quota recalculation

Note: The exact boundary between MR 2 and MR 3 will be refined during implementation, as some pieces (e.g., find_or_create_current_shard, is_first_shard logic) are intertwined between the notification and write paths.

References

Related to #490968

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.