Add monitoring for Pgbouncer and PostgreSQL ports

Background

After the incident on January 18th, 2022, where Staging couldn't connect to the Patroni CI cluster, we realized the monitoring of the database-related services could be improved.

Proposal

We could verify if Pgbouncer and PostgreSQL services are up and running in the nodes. The idea would be akin to Kubernetes liveness probe.

Elaborating better, we could add monitoring to verify if the ports for the Pgbouncer and PostgreSQL are open and accepting connections.

Ideally, the monitoring should also use the ports configured via Chef (here, here, and here), and via Helm value files (here, here, here, here, and here).

For further context, here's the Database Architectural Diagrams.

Acceptance criteria

  • PostgreSQL port is monitored
  • Pgbouncer port(s) is/are monitored
  • The monitored ports can be dynamically configured using Chef attributes or Helm values
Edited by Natan Hoppe