Geo: Secondary site returns 500 errors due to CI minutes tracking write attempts on read-only database

Summary

Geo secondary sites return 500 errors when browsing certain projects because Ci::Minutes::NamespaceMonthlyUsage.find_or_create_current attempts to INSERT into the read-only database.

Root Cause

The CI minutes tracking code path is not Geo-aware. When rendering the shared runners minutes limit alert, it attempts to write to ci_namespace_monthly_usages which fails on secondary sites.

Steps to reproduce

  1. Set up Geo primary and secondary
  2. Browse a project on secondary where CI minutes alert renders and the usage record doesn't exist
  3. 500 error occurs

Workaround: Browse the project on primary first to create the record.

Technical Details

Exception: PG::ReadOnlySqlTransaction: ERROR: cannot execute INSERT in a read-only transaction

Failed SQL:

INSERT INTO "ci_namespace_monthly_usages" ("namespace_id", "date", "created_at")
VALUES ($1, $2, $3) RETURNING "id"

Stack Trace

ee/app/models/ci/minutes/namespace_monthly_usage.rb:47 in `block in find_or_create_current`
ee/app/models/ci/minutes/namespace_monthly_usage.rb:36 in `find_or_create_current`
ee/app/models/ci/minutes/usage.rb:71 in `current_usage`
ee/app/models/ci/minutes/usage.rb:38 in `total_minutes_used`
ee/app/models/ci/minutes/usage.rb:34 in `current_balance`
ee/app/models/ci/minutes/usage.rb:79 in `total_minutes_remaining`
ee/app/models/ci/minutes/usage.rb:30 in `percent_total_minutes_remaining`
ee/app/models/ci/minutes/context.rb:8 in `percent_total_minutes_remaining`
ee/app/models/ci/minutes/notification.rb:44 in `percentage`
ee/app/models/ci/minutes/notification.rb:75 in `calculate_notification_stage`
ee/app/models/ci/minutes/notification.rb:15 in `initialize`
ee/app/components/namespaces/pipelines/alerts/shared_runners_minutes_limit_alert_component.rb:18 in `initialize`
ee/app/views/shared/_shared_runners_minutes_limit.html.haml:5
app/views/projects/_flash_messages.html.haml:14
app/views/projects/empty.html.haml:6
app/controllers/projects_controller.rb:429 in `render_landing_page`
app/controllers/projects_controller.rb:181 in `show`

Environment

  • GitLab version: 18.5.5
  • Deployment: Omnibus multi-node with Praefect
  • Site: Secondary

Implementation Plan

Skip the code that would perform a DB write, if Gitlab::Geo.secondary? . There are other examples in the codebase.

Edited by 🤖 GitLab Bot 🤖