Skip to content

Investigate infrequent "advisory lock" upgrade issue on environment builds

Sometimes environment builds fail with the below error when migrating db data. Likely a timing issue in the builder. Task is to investigate and fix.

 fatal: [gitlab-qa-10k-gitlab-rails-1]: FAILED! => changed=true 
   cmd: gitlab-ctl reconfigure
   delta: '0:00:26.691282'
   end: '2020-02-21 04:27:23.773153'
   msg: non-zero return code
   rc: 1
   start: '2020-02-21 04:26:57.081871'
   stderr: |-
     There was an error running gitlab-ctl reconfigure:
   
     bash[migrate gitlab-rails database] (gitlab::database_migrations line 54) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
     ---- Begin output of "bash"  "/tmp/chef-script20200221-18276-1lmrlc6" ----
     STDOUT: WARNING:  you don't own a lock of type ExclusiveLock
     rake aborted!
     ActiveRecord::ConcurrentMigrationError:
   
     Failed to release advisory lock
   
     /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:49:in `block (3 levels) in <top (required)>'
     /opt/gitlab/embedded/bin/bundle:23:in `load'
     /opt/gitlab/embedded/bin/bundle:23:in `<main>'
     Tasks: TOP => db:migrate