Skip to content

Sec Decomp testing in db-benchmarking

With gitlab-com/gl-infra/db-migration!566 (merged) merged, we can begin testing the database-side automation for Security Decomp (breaking physical replication, setting up forward/reverse logical replication, pausing/unpausing pgbouncer, etc...).

Ref: https://ops.gitlab.net/gitlab-com/gl-infra/config-mgmt/-/merge_requests/10334

Current status:

# Primary Cluster (01 is broken and needed to be reinitialized.  This is going to take a bit to get back in sync).
root@patroni-sec-decomp-main-v16-01-db-db-benchmarking.c.gitlab-db-benchmarking.internal:~# gitlab-patronictl list
+ Cluster: db-benchmarking-patroni-sec-decomp-main-v16 (7432562132117319567) ---------+----------------+---------+------------------+----+-----------+---------------------+
| Member                                                                              | Host           | Role    | State            | TL | Lag in MB | Tags                |
+-------------------------------------------------------------------------------------+----------------+---------+------------------+----+-----------+---------------------+
| patroni-sec-decomp-main-v16-01-db-db-benchmarking.c.gitlab-db-benchmarking.internal | 10.255.103.101 | Replica | creating replica |    |   unknown |                     |
+-------------------------------------------------------------------------------------+----------------+---------+------------------+----+-----------+---------------------+
| patroni-sec-decomp-main-v16-02-db-db-benchmarking.c.gitlab-db-benchmarking.internal | 10.255.103.102 | Replica | streaming        |  4 |        14 | nofailover: true    |
|                                                                                     |                |         |                  |    |           | noloadbalance: true |
+-------------------------------------------------------------------------------------+----------------+---------+------------------+----+-----------+---------------------+
| patroni-sec-decomp-main-v16-03-db-db-benchmarking.c.gitlab-db-benchmarking.internal | 10.255.103.103 | Leader  | running          |  4 |           |                     |
+-------------------------------------------------------------------------------------+----------------+---------+------------------+----+-----------+---------------------+

# Standby Cluster
root@patroni-sec-decomp-sec-v16-01-db-db-benchmarking.c.gitlab-db-benchmarking.internal:~# gitlab-patronictl list
+ Cluster: db-benchmarking-patroni-sec-decomp-sec-v16 (7432562132117319567) ---------+----------------+----------------+---------------------+----+-----------+-----------------+----------------------------+---------------------+
| Member                                                                             | Host           | Role           | State               | TL | Lag in MB | Pending restart | Pending restart reason     | Tags                |
+------------------------------------------------------------------------------------+----------------+----------------+---------------------+----+-----------+-----------------+----------------------------+---------------------+
| patroni-sec-decomp-sec-v16-01-db-db-benchmarking.c.gitlab-db-benchmarking.internal | 10.255.104.101 | Replica        | in archive recovery |  4 |         0 | *               | max_connections: 1200->400 |                     |
+------------------------------------------------------------------------------------+----------------+----------------+---------------------+----+-----------+-----------------+----------------------------+---------------------+
| patroni-sec-decomp-sec-v16-02-db-db-benchmarking.c.gitlab-db-benchmarking.internal | 10.255.104.102 | Replica        | in archive recovery |  4 |         0 |                 |                            | nofailover: true    |
|                                                                                    |                |                |                     |    |           |                 |                            | noloadbalance: true |
+------------------------------------------------------------------------------------+----------------+----------------+---------------------+----+-----------+-----------------+----------------------------+---------------------+
| patroni-sec-decomp-sec-v16-03-db-db-benchmarking.c.gitlab-db-benchmarking.internal | 10.255.104.103 | Standby Leader | in archive recovery |  4 |           | *               | max_connections: 1200->400 |                     |
+------------------------------------------------------------------------------------+----------------+----------------+---------------------+----+-----------+-----------------+----------------------------+---------------------+
Edited by Jonathon Sisson