Initial Siphon test plan on staging

Overview

This issue tracks the initial test plan for running Siphon on the GitLab staging environment. The goal is to validate snapshot creation and logical replication message processing.

Configuration Steps

1. Configure Siphon producer with PostgreSQL credentials and hostname(s).
2. Set initial_data_snapshot_threads_per_table to 3.
3. Configure table mapping:
   • bulk_import_entities (small table)
4. Ensure Siphon producer logs are preserved for post-test analysis.

Execution Steps

Note: these steps shouldn't affect deployments/DB migrations

1. Start the Siphon producer.
2. Verify that snapshots are being taken.
3. Verify logical replication messages are being processed.
4. Keep it running until snapshots complete, plus a few additional minutes.
5. Re-configure Siphon producer and add a new table:
   • projects (small table)
6. Restart Siphon producer and do steps 2-4 again.

Monitor DB health

Check replication lag:

SELECT
  slot_name,
  ROUND(
    pg_wal_lsn_diff(pg_current_wal_lsn(), confirmed_flush_lsn) / 1024.0 / 1024.0,
    2
  ) AS lag_mb
FROM pg_replication_slots
WHERE slot_name = 'stg_main_siphon_slot_1';

Teardown Steps

1. Drop the logical replication slot (publication remains), so WAL files are not accumulating.

   SELECT pg_drop_replication_slot('stg_main_siphon_slot_1');