gitlab 13.1.4 to 13.2.1 and gitlab-ctl reconfigure is broken with the psql check (external pg 12)

When we upgrade from 13.1.4 to 13.2.1 gitlab-ctl reconfigure is broken with the psql check. We are using psql 12 external DB. And we need to overwrite the psql path to /bin/psql.

Case 1: We can run backup, but gitlab-ctl recofigure failed

/opt/gitlab/embedded/bin/psql -> /bin/psql
/opt/gitlab/embedded/bin/pg_dump -> /bin/pg_dump

Case 2: We can't run backup, but gitlab-ctl reconfigure success.

/opt/gitlab/embedded/bin/psql -> /opt/gitlab/embedded/postgresql/11/bin/psql
/opt/gitlab/embedded/bin/pg_dump -> /opt/gitlab/embedded/postgresql/11/bin/pg_dump

This work around used to be just set the path /bin, but it is broken at gitlab 13.

export PATH=/bin:$PATH ### This works till gitlab 12 but not 13

# pwd
/opt/gitlab/embedded/bin
# ls -la pg_dump psql
lrwxrwxrwx. 1 root root 12 Jul 27 23:57 pg_dump -> /bin/pg_dump
lrwxrwxrwx. 1 root root  9 Jul 27 23:57 psql -> /bin/psql

Broken Recipe posgresql:bin

Recipe: postgresql::bin
  * ruby_block[check_postgresql_version] action run (skipped due to not_if)
  * ruby_block[check_postgresql_version_is_deprecated] action run (skipped due to not_if)
  * ruby_block[Link postgresql bin files to the correct version] action run

    ================================================================================
    Error executing action `run` on resource 'ruby_block[Link postgresql bin files to the correct version]'
    ================================================================================

    RuntimeError
    ------------
    Could not find PostgreSQL binaries

    Cookbook Trace:
    ---------------
    /opt/gitlab/embedded/cookbooks/cache/cookbooks/postgresql/recipes/bin.rb:66:in `block (2 levels) in from_file'

    Resource Declaration:
    ---------------------
    # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/postgresql/recipes/bin.rb

     61: ruby_block "Link postgresql bin files to the correct version" do
     62:   block do
     63:     # Fallback to the psql version if needed
     64:     pg_path = db_path || Dir.glob("#{postgresql_install_dir}/#{pg_helper.version.major}*").min
     65:
     66:     raise "Could not find PostgreSQL binaries" unless pg_path
     67:
     68:     Dir.glob("#{pg_path}/bin/*").each do |pg_bin|
     69:       FileUtils.ln_sf(pg_bin, "#{node['package']['install-dir']}/embedded/bin/#{File.basename(pg_bin)}")
     70:     end
     71:   end
     72:   only_if do
     73:     !File.exist?(File.join(node['postgresql']['data_dir'], "PG_VERSION")) || \
     74:       pg_helper.version.major !~ /^#{pg_helper.database_version}/ || \
     75:       (Services.enabled?('geo_postgresql') && geo_pg_helper.version.major !~ /^#{geo_pg_helper.database_version}/) || \
     76:       !node['postgresql']['version'].nil?
     77:   end
     78:   notifies :restart, 'runit_service[postgresql]', :immediately if omnibus_helper.should_notify?("postgresql") && resource_exists['runit_service[postgresql]']
     79: end

    Compiled Resource:
    ------------------
    # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/postgresql/recipes/bin.rb:61:in `from_file'

    ruby_block("Link postgresql bin files to the correct version") do
      action [:run]
      default_guard_interpreter :default
      declared_type :ruby_block
      cookbook_name "postgresql"
      recipe_name "bin"
      block #<Proc:0x000000000516ada0@/opt/gitlab/embedded/cookbooks/cache/cookbooks/postgresql/recipes/bin.rb:62>
      block_name "Link postgresql bin files to the correct version"
      only_if { #code block }
    end

    System Info:
    ------------
    chef_version=15.9.17
    platform=centos
    platform_version=7.6.1810
    ruby=ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux]
    program_name=/opt/gitlab/embedded/bin/chef-client
    executable=/opt/gitlab/embedded/bin/chef-client


Running handlers:
There was an error running gitlab-ctl reconfigure:

ruby_block[Link postgresql bin files to the correct version] (postgresql::bin line 61) had an error: RuntimeError: Could not find PostgreSQL binaries

Running handlers complete
Chef Infra Client failed. 1 resources updated in 34 seconds
Edited by Teo Cheng Lim