"update_column_in_batches can not be run inside a transaction" during upgrade to 11.3
I tried to upgrade to gitlab 11.3 last night (from the lastest previous version of the omnibus packages) and got the following error:
> bash[migrate gitlab-rails database] (gitlab::database_migrations line 49) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
> ---- Begin output of "bash" "/tmp/chef-script20180922-24559-1vw59jc" ----
> STDOUT: rake aborted!
> StandardError: An error has occurred, this and all later migrations canceled:
>
> update_column_in_batches can not be run inside a transaction, you can disable transactions by calling disable_ddl_transaction! in the body of your migration class
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:325:in `update_column_in_batches'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:10:in `update_column_in_batches'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:126:in `block in remove_cached_html_for_projects'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:125:in `each'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:125:in `remove_cached_html_for_projects'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:39:in `rename_namespace_dependencies'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:31:in `rename_namespace'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:10:in `block in rename_namespaces'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:9:in `rename_namespaces'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1.rb:30:in `rename_root_paths'
> /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20180816193530_rename_login_root_namespaces.rb:11:in `block in up'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:254:in `disable_statement_timeout'
> /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20180816193530_rename_login_root_namespaces.rb:10:in `up'
> /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:52:in `block (3 levels) in <top (required)>'
> /opt/gitlab/embedded/bin/bundle:23:in `load'
> /opt/gitlab/embedded/bin/bundle:23:in `<main>'
>
> Caused by:
> update_column_in_batches can not be run inside a transaction, you can disable transactions by calling disable_ddl_transaction! in the body of your migration class
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:325:in `update_column_in_batches'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:10:in `update_column_in_batches'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:126:in `block in remove_cached_html_for_projects'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:125:in `each'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_base.rb:125:in `remove_cached_html_for_projects'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:39:in `rename_namespace_dependencies'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:31:in `rename_namespace'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:10:in `block in rename_namespaces'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_namespaces.rb:9:in `rename_namespaces'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/rename_reserved_paths_migration/v1.rb:30:in `rename_root_paths'
> /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20180816193530_rename_login_root_namespaces.rb:11:in `block in up'
> /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:254:in `disable_statement_timeout'
> /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20180816193530_rename_login_root_namespaces.rb:10:in `up'
> /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:52:in `block (3 levels) in <top (required)>'
> /opt/gitlab/embedded/bin/bundle:23:in `load'
> /opt/gitlab/embedded/bin/bundle:23:in `<main>'
> Tasks: TOP => db:migrate
> (See full trace by running task with --trace)
> == 20180101160629 CreatePrometheusMetrics: migrating ==========================
> -- create_table(:prometheus_metrics)
> -> 0.2996s
> == 20180101160629 CreatePrometheusMetrics: migrated (0.2997s) =================
>
> == 20180101160630 ChangeProjectIdForPrometheusMetrics: migrating ==============
> -- change_column_null(:prometheus_metrics, :project_id, true)
> -> 0.0003s
> == 20180101160630 ChangeProjectIdForPrometheusMetrics: migrated (0.0003s) =====
>
> == 20180228172924 AddIncludePrivateContributionsToUsers: migrating ============
> -- add_column(:users, :include_private_contributions, :boolean)
> -> 0.0004s
> == 20180228172924 AddIncludePrivateContributionsToUsers: migrated (0.0005s) ===
>
> == 20180308125206 AddUserInternalRegexToApplicationSetting: migrating =========
> -- add_column(:application_settings, :user_default_internal_regex, :string, {:null=>true})
> -> 0.0005s
> == 20180308125206 AddUserInternalRegexToApplicationSetting: migrated (0.0005s)
>
> == 20180607071808 AddPushEventsBranchFilterToWebHooks: migrating ==============
> -- add_column(:web_hooks, :push_events_branch_filter, :text)
> -> 0.0004s
> == 20180607071808 AddPushEventsBranchFilterToWebHooks: migrated (0.0004s) =====
>
> == 20180711103851 DropDuplicateProtectedTags: migrating =======================
> == 20180711103851 DropDuplicateProtectedTags: migrated (0.0351s) ==============
>
> == 20180711103922 AddProtectedTagsIndex: migrating ============================
> -- transaction_open?()
> -> 0.0000s
> -- index_exists?(:protected_tags, [:project_id, :name], {:unique=>true, :algorithm=>:concurrently})
> -> 0.0015s
> -- execute("SET statement_timeout TO 0")
> -> 0.0001s
> -- add_index(:protected_tags, [:project_id, :name], {:unique=>true, :algorithm=>:concurrently})
> -> 0.0065s
> -- execute("RESET ALL")
> -> 0.0001s
> == 20180711103922 AddProtectedTagsIndex: migrated (0.0083s) ===================
>
> == 20180720023512 AddReceiveMaxInputSizeToApplicationSettings: migrating ======
> -- add_column(:application_settings, :receive_max_input_size, :integer)
> -> 0.0715s
> == 20180720023512 AddReceiveMaxInputSizeToApplicationSettings: migrated (0.0715s)
>
> == 20180723130817 DeleteInconsistentInternalIdRecords: migrating ==============
> -- execute("SET statement_timeout TO 0")
> -> 0.0001s
> -- execute("RESET ALL")
> -> 0.0001s
> == 20180723130817 DeleteInconsistentInternalIdRecords: migrated (0.0165s) =====
>
> == 20180807153545 RemoveRedundantStatusIndexOnCiBuilds: migrating =============
> -- transaction_open?()
> -> 0.0000s
> -- select_one("SELECT current_setting('server_version_num') AS v")
> -> 0.0002s
> -- index_exists?(:ci_builds, :status, {:algorithm=>:concurrently})
> -> 0.0044s
> -- execute("SET statement_timeout TO 0")
> -> 0.0001s
> -- remove_index(:ci_builds, {:algorithm=>:concurrently, :column=>:status})
> -> 0.0315s
> -- execute("RESET ALL")
> -> 0.0002s
> == 20180807153545 RemoveRedundantStatusIndexOnCiBuilds: migrated (0.0366s) ====
>
> == 20180808162000 AddUserShowAddSshKeyMessageToApplicationSettings: migrating =
> -- transaction_open?()
> -> 0.0000s
> -- execute("SET statement_timeout TO 0")
> -> 0.0001s
> -- transaction()
> -- add_column(:application_settings, :user_show_add_ssh_key_message, :boolean, {:default=>nil})
> -> 0.0006s
> -- change_column_default(:application_settings, :user_show_add_ssh_key_message, true)
> -> 0.0071s
> -> 0.0081s
> -- transaction_open?()
> -> 0.0000s
> -- exec_query("SELECT COUNT(*) AS count FROM \"application_settings\"")
> -> 0.0003s
> -- exec_query("SELECT \"application_settings\".\"id\" FROM \"application_settings\" ORDER BY \"application_settings\".\"id\" ASC LIMIT 1")
> -> 0.0002s
> -- exec_query("SELECT \"application_settings\".\"id\" FROM \"application_settings\" WHERE \"application_settings\".\"id\" >= 1 ORDER BY \"application_settings\".\"id\" ASC LIMIT 1 OFFSET 1")
> -> 0.0002s
> -- execute("UPDATE \"application_settings\" SET \"user_show_add_ssh_key_message\" = 't' WHERE \"application_settings\".\"id\" >= 1")
> -> 0.0004s
> -- change_column_null(:application_settings, :user_show_add_ssh_key_message, false)
> -> 0.0003s
> -- execute("RESET ALL")
> -> 0.0001s
> == 20180808162000 AddUserShowAddSshKeyMessageToApplicationSettings: migrated (0.0167s)
>
> == 20180809195358 MigrateNullWikiAccessLevels: migrating ======================
> -- transaction()
> -- execute("SET LOCAL statement_timeout TO 0")
> -> 0.0001s
> -- execute("UPDATE site_statistics SET wikis_count = (SELECT COUNT(*) FROM project_features WHERE wiki_access_level != 0)")
> -> 0.0007s
> -> 0.0012s
> == 20180809195358 MigrateNullWikiAccessLevels: migrated (0.0036s) =============
>
> == 20180813101999 ChangeDefaultOfAutoDevopsInstanceWide: migrating ============
> -- change_column_default(:application_settings, :auto_devops_enabled, true)
> -> 0.0075s
> == 20180813101999 ChangeDefaultOfAutoDevopsInstanceWide: migrated (0.0075s) ===
>
> == 20180813102000 EnableAutoDevopsInstanceWideForEveryone: migrating ==========
> -- execute("UPDATE application_settings SET auto_devops_enabled = true")
> -> 0.0004s
> == 20180813102000 EnableAutoDevopsInstanceWideForEveryone: migrated (0.0005s) =
>
> == 20180815040323 AddAuthorizationTypeToClusterPlatformsKubernetes: migrating =
> -- add_column(:cluster_platforms_kubernetes, :authorization_type, :integer, {:limit=>2})
> -> 0.0003s
> == 20180815040323 AddAuthorizationTypeToClusterPlatformsKubernetes: migrated (0.0004s)
>
> == 20180815160409 AddFileLocationToCiJobArtifacts: migrating ==================
> -- add_column(:ci_job_artifacts, :file_location, :integer, {:limit=>2})
> -> 0.0003s
> == 20180815160409 AddFileLocationToCiJobArtifacts: migrated (0.0003s) =========
>
> == 20180815170510 AddPartialIndexToCiBuildsArtifactsFile: migrating ===========
> -- transaction_open?()
> -> 0.0000s
> -- index_exists?(:ci_builds, :id, {:where=>"artifacts_file <> ''", :name=>"partial_index_ci_builds_on_id_with_legacy_artifacts", :algorithm=>:concurrently})
> -> 0.0041s
> -- execute("SET statement_timeout TO 0")
> -> 0.0001s
> -- add_index(:ci_builds, :id, {:where=>"artifacts_file <> ''", :name=>"partial_index_ci_builds_on_id_with_legacy_artifacts", :algorithm=>:concurrently})
> -> 0.1258s
> -- execute("RESET ALL")
> -> 0.0001s
> == 20180815170510 AddPartialIndexToCiBuildsArtifactsFile: migrated (0.1303s) ==
>
> == 20180815175440 AddIndexOnListType: migrating ===============================
> -- transaction_open?()
> -> 0.0000s
> -- index_exists?(:lists, :list_type, {:algorithm=>:concurrently})
> -> 0.0018s
> -- execute("SET statement_timeout TO 0")
> -> 0.0001s
> -- add_index(:lists, :list_type, {:algorithm=>:concurrently})
> -> 0.0379s
> -- execute("RESET ALL")
> -> 0.0001s
> == 20180815175440 AddIndexOnListType: migrated (0.0401s) ======================
>
> == 20180816161409 MigrateLegacyArtifactsToJobArtifacts: migrating =============
> == 20180816161409 MigrateLegacyArtifactsToJobArtifacts: migrated (0.0324s) ====
>
> == 20180816193530 RenameLoginRootNamespaces: migrating ========================
> -- execute("SET statement_timeout TO 0")
> -> 0.0001s
> -- quote_string("LOGIN")
> -> 0.0000s
> -- quote_string("LOGIN/%")
> -> 0.0000s
> -- execute("UPDATE \"routes\" SET \"path\" = regexp_replace(\"routes\".\"path\", 'LOGIN', 'LOGIN0') WHERE routes.id IN ( SELECT routes.id FROM routes WHERE lower(routes.path) = lower('LOGIN') UNION SELECT routes.id FROM routes WHERE routes.path ILIKE 'LOGIN/%' )")
> -> 0.0107s
> -- tracked rename: rename:RenameLoginRootNamespaces:namespace: LOGIN -> LOGIN0
> -- transaction_open?()
> -> 0.0000s
> -- execute("RESET ALL")
> -> 0.0002s
> STDERR:
> ---- End output of "bash" "/tmp/chef-script20180922-24559-1vw59jc" ----
> Ran "bash" "/tmp/chef-script20180922-24559-1vw59jc" returned 1
adding disable_ddl_transaction!
in /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20180816193530_rename_login_root_namespaces.rb allowed me to proceed and upgrade gitlab properly.
Edited by Stan Hu