Skip to content

Consul agent does not cleanup unused watch configuration

consul::watchers recipe does not remove the watcher configuration files for services that are removed from consul['watchers'] list. This will cause triggering the handler script for services that no longer exist.

An example of this arises when during near zero-downtime upgrade of a Patroni cluster the PostgreSQL service is renamed from postgresql to something else, for example postgresql_new. In its current state the Consul agent end up with two separate watchers, one for postgresql and and another for postgresql_new, both competing to update Pgbouncer databases.ini which breaks the process. To workaround this the user has to manually delete the unused postgresql watch config from /var/opt/gitlab/consul/config.d/watcher_postgresql.json.

It should be also noted that the in the same recipe postgresql service name is hardcoded and should be replaced with node['consul']['internal']['postgresql_service_name'].