Consul agent does not cleanup unused watch configuration
consul::watchers
recipe does not remove the watcher configuration files for services that are removed from consul['watchers']
list. This will cause triggering the handler script for services that no longer exist.
An example of this arises when during near zero-downtime upgrade of a Patroni cluster the PostgreSQL service is renamed from postgresql
to something else, for example postgresql_new
. In its current state the Consul agent end up with two separate watchers, one for postgresql
and and another for postgresql_new
, both competing to update Pgbouncer databases.ini
which breaks the process. To workaround this the user has to manually delete the unused postgresql
watch config from /var/opt/gitlab/consul/config.d/watcher_postgresql.json
.
It should be also noted that the in the same recipe postgresql
service name is hardcoded and should be replaced with node['consul']['internal']['postgresql_service_name']
.