All three findings below came out of the first full end-to-end verification of https://gitlab.com/gitlab-org/gitlab/-/merge_requests/250118 on a local two-Caproni logical replication environment (https://gitlab.com/dbalexandre/geo-caproni-lr), running GitLab v19.2.4 images with the MR branch code loaded. The verification itself passed. These are hardening gaps found around it. ## Why 1. Stale feature flag on the secondary can make promotion silently skip the sequence sync. The `features` table row for `geo_postgresql_replication_agnostic` replicates to the subscriber instantly over logical replication, but the secondary evaluates flags through its own Redis L2 Flipper cache (about 1 hour TTL) plus per-process memoization. In this run, the flag was enabled on the primary and the row was present on the subscriber, yet `Gitlab::Geo.postgresql_replication_agnostic_enabled?` still returned false on the secondary. As a result, `Gitlab::Geo::GeoTasks.set_secondary_as_primary` printed "Streaming replication mode: skipping sequence synchronization" and the site promoted without syncing sequences. That reproduces the exact incident this MR is meant to fix. 2. The sequence discovery query fails hard on unreadable sequences. `Gitlab::Database::SyncSequencesWithTableData` runs `pg_sequence_last_value()` across every sequence in the database. If any sequence is not readable by the application role, the whole run aborts. This run hit it with a superuser-owned utility table, and extensions can create the same situation on customer databases. The error observed was `PG::InsufficientPrivilege: permission denied for sequence geocap_replication_test_id_seq`. 3. ~~Small verification gap around a shared sequence. On the GitLab v19.2.4 schema, the run reported `public.shared_audit_event_id_seq` as having no consumer tables and skipped it. If consumer resolution also misses it on current master, that shared sequence would never advance at promotion, and the first audit event insert on a promoted site would collide with an existing id.~~ **Resolved 2026-08-24: the skip is correct behavior, not a gap. See the status comment below.** ## How 1. Workaround used during this run: on the secondary, `Rails.cache.delete('flipper/v1/feature/geo_postgresql_replication_agnostic')` plus `Feature.reset`. A real fix should not rely on trusting a cached flag read during promotion. Options include bypassing the Flipper cache for this predicate in `Gitlab::Geo::LogicalReplication.configured?`, making promotion fail loudly when a subscription exists (or existed) while the predicate says streaming, or documenting a mandatory cache-reset step in the promotion runbook. 2. ~~The discovery query should not let one foreign object block a promotion. **Update 2026-08-24: partially addressed by the merged MR (per-sequence rescue, full failure report, fail-safe abort at the end). The skip-instead-of-abort fix is deprioritized to Phase 3 since Dedicated subscribers are freshly provisioned databases: https://gitlab.com/gitlab-org/gitlab/-/work_items/621927.~~ ** Filter it with `has_sequence_privilege()`, or rescue per sequence and log a warning instead of aborting the whole run. 3. ~~Check on master whether the discovery SQL (nextval defaults plus OWNED BY resolution) correctly attributes the audit event tables to `shared_audit_event_id_seq`. Fix the resolution if it is broken, or confirm this was a 19.2-only artifact and close the finding.~~ **Done: confirmed correct on master, see the status comment below.** ## What Finding 1: stale feature flag - [x] Decide the fix direction for `Gitlab::Geo::LogicalReplication.configured?`: bypass cache, fail loudly on mismatch, or runbook step - [x] Implement the chosen fix - [x] Add a test that reproduces stale-flag promotion and confirms the new behavior - [x] Update the promotion runbook if a manual step remains necessary Finding 2: sequence discovery permission errors (deprioritized 2026-08-24 to Phase 3, see https://gitlab.com/gitlab-org/gitlab/-/work_items/621927) - [x] ~~Add a privilege check (or per-sequence rescue) to `Gitlab::Database::SyncSequencesWithTableData`~~ - [x] ~~Add a test with an unreadable sequence present to confirm the run completes and logs a warning~~ - [x] ~~Confirm the fix does not skip sequences the application role can actually read~~ Finding 3: shared audit event sequence (resolved 2026-08-24) - [x] ~~Reproduce the consumer resolution for `shared_audit_event_id_seq` on current master~~ - [x] ~~Determine whether the audit event tables are correctly attributed to it~~ - [x] ~~Fix the resolution logic, or close this finding as a 19.2-only artifact with evidence~~ ## Exit criteria - Finding 1: promotion cannot silently skip sequence sync due to a stale flag cache, either by design or by a documented and enforced runbook step. - ~~Finding 2~~: an unreadable sequence anywhere in the database no longer aborts the whole sequence sync run.~~ **Deprioritized on 2026-08-24 to Phase 3, see https://gitlab.com/gitlab-org/gitlab/-/work_items/621927** - ~~Finding 3: `shared_audit_event_id_seq` is confirmed correctly resolved on master, or the master behavior is confirmed different from the 19.2 finding.~~ **Met: confirmed correctly resolved (the sequence has no consumers by design since 19.1).** Related: - https://gitlab.com/gitlab-org/gitlab/-/merge_requests/250118 - https://gitlab.com/gitlab-org/gitlab/-/issues/593337 - https://gitlab.com/gitlab-org/gitlab/-/issues/614167 - https://gitlab.com/dbalexandre/geo-caproni-lr