Initial Siphon test plan on staging
## Overview This issue tracks the initial test plan for running Siphon on the GitLab staging environment. The goal is to validate snapshot creation and logical replication message processing. ## Configuration Steps 1. Configure Siphon producer with PostgreSQL credentials and hostname(s). 2. Set `initial_data_snapshot_threads_per_table` to `3`. 3. Configure table mapping: - `bulk_import_entities` (small table) 4. Ensure Siphon producer logs are preserved for post-test analysis. ## Execution Steps **Note: these steps shouldn't affect deployments/DB migrations** 1. Start the Siphon producer. 2. Verify that snapshots are being taken. 3. Verify logical replication messages are being processed. 4. Keep it running until snapshots complete, plus a few additional minutes. 5. Re-configure Siphon producer and add a new table: - `projects` (small table) 6. Restart Siphon producer and do steps 2-4 again. ### Monitor DB health Check replication lag: ```sql SELECT slot_name, ROUND( pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn) / 1024.0 / 1024.0, 2 ) AS lag_mb FROM pg_replication_slots WHERE slot_name = 'stg_main_siphon_slot_1'; ``` ## Teardown Steps 1. Drop the logical replication slot (publication remains), so WAL files are not accumulating. ```sql SELECT pg_drop_replication_slot('stg_main_siphon_slot_1'); ```
issue