Special Coverage Days - Post GPRD Main Database Cluster PG16 Upgrade support
Summary
We seek special coverage During the Postgres 16 upgrade rollback window of our production MAIN database cluster. Only specific functional groups are necessary and they are elaborated below.
The coverage window is from the Switchover until the end of the PCL, from 2024-11-03 06:00 UTC to ~ 2024-11-05 09:00 UTC.
Communication
During the rollout window
Out of rollout window: wg_saas_pg16_upgrade Slack Channel
Replacement Days
Team Members covering these shifts will be encouraged to take 1.25 days (or 2 if called) in exchange for weekend/holiday coverage, or 1 full day PTO in exchange for weekday coverage, regardless of whether there are any active incidents during these shifts.
DBRE
| Shift | Team Member |
|---|---|
| 0000 UTC 2024-11-03 ~ 0600 UTC 2024-11-03 - Pre-Switchover Window | @rhenchen.gitlab @daveyleach |
| 0600 UTC 2024-11-03 ~ 1200 UTC 2024-11-03 - Switchover Window | @rhenchen.gitlab @daveyleach @bshah11 |
| 1200 UTC 2024-11-03 ~ 1800 UTC 2024-11-03 | @alexander-sosna |
| 1800 UTC 2024-11-03 ~ 0000 UTC 2024-11-04 | @jjsisson @bshah11 (I'll be around for a few hours, not the whole shift.) |
For weekdays (11-04 and 11-05) the Coverage is made through engineers regular work hours:
- APAC: @rhenchen.gitlab @daveyleach
- EMEA: @alexander-sosna
- AMERICAS: @bshah11 @jjsisson
SRE
We will rely on the standard EOC schedule
Database Maintainers
| Shift | Team Member |
|---|---|
| 0600 UTC 2024-11-03 ~ 1200 UTC 2024-11-03 - Switchover Window | |
| 1200 UTC 2024-11-03 ~ 1800 UTC 2024-11-03 | @stomlinson starting at 1400 UTC |
| 1800 UTC 2024-11-03 ~ 0000 UTC 2024-11-04 |
Database maintainers who sign up are expected to be able to respond within 30 minutes. If you have a review that will require a database review, please be sure to loop in the currently on-call database maintainer as early as possible to ensure there are no delays.
QE
One POC to help if we have questions with QA runs after the Postgres upgrades are done.
Quality Engineering On-call Rotation
- https://about.gitlab.com/handbook/engineering/quality/quality-engineering/oncall-rotation/#schedule
- https://gitlab.com/gitlab-org/quality/pipeline-triage#dri-weekly-rotation-schedule