Gitlab monitor crash after 11.9 update

Summary

After Gitlab Omnibus upgrade from 11.8 to 11.9, it started to misbehave from the web interface, then I ran gitlab-ctl restart and continues to fail, then I tried to reconfigure it running gitlab-ctl reconfigure and fails on database migration.

Steps to reproduce

Upgrade from 11.8.x to 11.9, wait for restart.

What is the current bug behavior?

Error 500 on web interface, the database migration fails.

What is the expected correct behavior?

Gitlab up and running

Relevant logs

Relevant logs
There was an error running gitlab-ctl reconfigure:

bash[migrate gitlab-rails database] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1' ---- Begin output of "bash" "/tmp/chef-script20190325-7100-a584k0" ---- STDOUT: rake aborted! StandardError: An error has occurred, all later migrations canceled:

PG::UndefinedObject: ERROR: operator class "gin_trgm_ops" does not exist for access method "gin" : CREATE INDEX CONCURRENTLY "index_tags_on_name_trigram" ON "tags" USING gin ("name" gin_trgm_ops) /opt/gitlab/embedded/service/gitlab-rails/config/initializers/postgresql_opclasses_support.rb:149:in add_index' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:69:in block in add_concurrent_index' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:254:in disable_statement_timeout' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:68:in add_concurrent_index' /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20190206193120_add_index_to_tags.rb:12:in up' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:53:in block (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:in load' /opt/gitlab/embedded/bin/bundle:23:in '

Caused by: ActiveRecord::StatementInvalid: PG::UndefinedObject: ERROR: operator class "gin_trgm_ops" does not exist for access method "gin" : CREATE INDEX CONCURRENTLY "index_tags_on_name_trigram" ON "tags" USING gin ("name" gin_trgm_ops) /opt/gitlab/embedded/service/gitlab-rails/config/initializers/postgresql_opclasses_support.rb:149:in add_index' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:69:in block in add_concurrent_index' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:254:in disable_statement_timeout' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:68:in add_concurrent_index' /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20190206193120_add_index_to_tags.rb:12:in up' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:53:in block (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:in load' /opt/gitlab/embedded/bin/bundle:23:in '

Caused by: PG::UndefinedObject: ERROR: operator class "gin_trgm_ops" does not exist for access method "gin" /opt/gitlab/embedded/service/gitlab-rails/config/initializers/postgresql_opclasses_support.rb:149:in add_index' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:69:in block in add_concurrent_index' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:254:in disable_statement_timeout' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:68:in add_concurrent_index' /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20190206193120_add_index_to_tags.rb:12:in up' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:53:in block (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:in `load'

==> /var/log/gitlab/gitlab-monitor/current <== 2019-03-25_11:01:06.48317 2019-03-25 11:01:06 - NoMethodError - undefined method `probe_memory' for #GitLab::Monitor::ProcessProber:0x00007f8398ad7e10 2019-03-25_11:01:06.48321 Did you mean? probe_count:

Details of package version

Provide the package version installation details
gitlab-ee-11.9.0-ee.0.el7.x86_64

Environment details

  • Operating System: CentOS 7
  • Installation Target, remove incorrect values:
    • VM: Openstack
  • Installation Type, remove incorrect values:
    • Upgrade from version 11.8
  • Is there any other software running on the machine: No
  • Is this a single or multiple node installation? Single
  • Resources
    • CPU: 4
    • Memory total: 8Gb

Configuration details

Provide the relevant sections of `/etc/gitlab/gitlab.rb`
No changes from the previous version