Skip to content

Database migrations are attempted incorrectly when Puma is disabled

When reviewing some settings for setting up some node types that don't have Omnibus Roles, on our monitor node it started to fail as it attempted to run migrations even though Puma is disabled:

Recipe: gitlab::database_migrations
  * ruby_block[check remote PG version] action nothing (skipped due to action :nothing)
  * rails_migration[gitlab-rails] action run
    * bash[migrate gitlab-rails database] action run
      [execute] rake aborted!
                ActiveRecord::ConnectionNotEstablished: could not connect to server: No such file or directory
                	Is the server running locally and accepting
                	connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
                /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:60:in `block (3 levels) in <top (required)>'
                /opt/gitlab/embedded/bin/bundle:23:in `load'
                /opt/gitlab/embedded/bin/bundle:23:in `<main>'

                Caused by:
                PG::ConnectionBad: could not connect to server: No such file or directory
                	Is the server running locally and accepting
                	connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
                /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:60:in `block (3 levels) in <top (required)>'
                /opt/gitlab/embedded/bin/bundle:23:in `load'
                /opt/gitlab/embedded/bin/bundle:23:in `<main>'
                Tasks: TOP => gitlab:db:configure
                (See full trace by running task with --trace)

      ================================================================================
      Error executing action `run` on resource 'bash[migrate gitlab-rails database]'
      ================================================================================

      Mixlib::ShellOut::ShellCommandFailed
      ------------------------------------
      Expected process to exit with [0], but received '1'
      ---- Begin output of "bash"  "/tmp/chef-script20210611-4525-tdszqx" ----
      STDOUT: rake aborted!
      ActiveRecord::ConnectionNotEstablished: could not connect to server: No such file or directory
      	Is the server running locally and accepting
      	connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
      /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:60:in `block (3 levels) in <top (required)>'
      /opt/gitlab/embedded/bin/bundle:23:in `load'
      /opt/gitlab/embedded/bin/bundle:23:in `<main>'

      Caused by:
      PG::ConnectionBad: could not connect to server: No such file or directory
      	Is the server running locally and accepting
      	connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
      /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:60:in `block (3 levels) in <top (required)>'
      /opt/gitlab/embedded/bin/bundle:23:in `load'
      /opt/gitlab/embedded/bin/bundle:23:in `<main>'
      Tasks: TOP => gitlab:db:configure
      (See full trace by running task with --trace)
      STDERR:
      ---- End output of "bash"  "/tmp/chef-script20210611-4525-tdszqx" ----
      Ran "bash"  "/tmp/chef-script20210611-4525-tdszqx" returned 1

The settings for the node are intended to only enable Prometheus and Grafana and disable all the rest of the services that are enabled by default in Omnibus:

# Avoid running unnecessary services on the Prometheus server
gitaly['enable'] = false
postgresql['enable'] = false
redis['enable'] = false
puma['enable'] = false
sidekiq['enable'] = false
gitlab_workhorse['enable'] = false
alertmanager['enable'] = false
gitlab_exporter['enable'] = false

Looking at the code it appears it's specifically following if the node['gitlab']['gitlab-rails']['enable'] setting is true. This looks to be an internal setting however that's also set to true by default and I can't see guidance elsewhere that we need to specifically need to disable gitlab-rails along with Puma.

With a quick look I wonder if this setting should only be enabled if Puma is but the setting is used elsewhere so unsure of the impact.