Investigate infrequent "advisory lock" upgrade issue on environment builds
Sometimes environment builds fail with the below error when migrating db data. Likely a timing issue in the builder. Task is to investigate and fix.
fatal: [gitlab-qa-10k-gitlab-rails-1]: FAILED! => changed=true cmd: gitlab-ctl reconfigure delta: '0:00:26.691282' end: '2020-02-21 04:27:23.773153' msg: non-zero return code rc: 1 start: '2020-02-21 04:26:57.081871' stderr: |- There was an error running gitlab-ctl reconfigure: bash[migrate gitlab-rails database] (gitlab::database_migrations line 54) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with , but received '1' ---- Begin output of "bash" "/tmp/chef-script20200221-18276-1lmrlc6" ---- STDOUT: WARNING: you don't own a lock of type ExclusiveLock rake aborted! ActiveRecord::ConcurrentMigrationError: Failed to release advisory lock /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:49:in `block (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:in `load' /opt/gitlab/embedded/bin/bundle:23:in `<main>' Tasks: TOP => db:migrate