PGbouncer role reconfigure failure when using docker image
Summary
Reconfigure of pgbouncer fails on omnibus docker starting in GitLab 16.0, and the error from the logs is a permission issue.
Steps to reproduce
You can either setup a full HA setup with pgbouncer on a docker container, and see the exact error in the pgbouncer logs.
Or if you just want to see the error but not in a proper setup, you can boot an omnibus docker image, and add this to the gitlab.rb
consul['enable'] = true
pgbouncer['enable'] = true
pgbouncer['databases_ini'] = '/var/opt/gitlab/consul/databases.ini'
And run reconfigure. Then check the pgbouncer logs. This turns on consul and pgbouncer, and puts the database.ini inside the consul directory. Which is what happens in a setup where you've setup the postgresl consul watchers like https://docs.gitlab.com/ee/administration/reference_architectures/50k_users.html#configure-pgbouncer
What is the current bug behavior?
2023-07-14_00:10:40.51819 2023-07-14 00:10:40.518 UTC [2459] FATAL cannot load config file
2023-07-14_00:10:41.52046 2023-07-14 00:10:41.520 UTC [2468] ERROR could not load file "/var/opt/gitlab/consul/databases.ini": Permission denied
2023-07-14_00:10:41.52047 2023-07-14 00:10:41.520 UTC [2468] ERROR error processing include file in configuration (/var/opt/gitlab/pgbouncer/pgbouncer.ini:324), stopping loading
What is the expected correct behavior?
No errors in pgbouncer log