Skip to content

Add test coverage for postgres upgrade on macOS

🎪 Theme

gdk-reliability

📜 Description

During upgrade effort of Postgres 16, a macOS issue was identified which our current testing setup was not able to detect due to the fact that tests are executed using Linux runners.

Since we're a mac-centric company and PG upgrade is frequently occurring, this represents a high risk gap in our GDK test coverage that directly impacts developer productivity and reliability.

🎯 Success Metrics & KPIs

Primary KPIs:

  • Test Coverage: Achieve 100% macOS test coverage for Postgres operations (currently 0%)
  • Issue Prevention: Reduce macOS-specific Postgres issues by 90% (baseline: 3 issues in Q2 2025)

Secondary KPIs:

  • Developer Experience: Reduce macOS setup failures from Postgres issues by 80%
  • Release Confidence: Enable same-day Postgres upgrades for macOS users (currently requires hotfixes)

🤝 Acceptance Criteria

  • Postgres Fresh Install Testing
    • macOS CI job validates fresh Postgres installation
    • Test covers all supported macOS versions (latest 2 major versions)
    • Validates installation success rate >95%
  • Postgres Upgrade Testing
    • macOS CI job tests upgrade path from N-1 to N version
    • Migration validation with real GDK data
    • Rollback scenario testing
  • CI Integration
    • Tests run on every MR affecting Postgres configuration
    • Results block merge if macOS compatibility fails
    • Test execution time <45 minutes

💡 Proposal

Implement dedicated macOS runners for Postgres testing with:

  1. Matrix testing across supported macOS versions
  2. Automated upgrade/downgrade scenarios
  3. Failure notification system

🚫 Blockers

  • macOS runner availability/cost approval
  • Test data migration strategy definition

Out of scope

  • Performance regression testing will be addressed separately
Edited by Mohga Gamea