Upgrading from 14.1.3-ce.0 to 14.2.1-ce.0 failed with ci_job_artifacts column type change
My gitlab was at 13.6.x, so I followed this following path:
13.8.8 -> latest 13.12.Z -> latest 14.0.Z -> 14.1.Z -> latest 14.Y.Z
-> 13.8.8 -> 13.12.10 -> 14.0.8 -> 14.1.3 -> latest 14.2.1
The upgrades worked without any issues up to 14.1.3.
The upgrade to 14.2.1 failed. I don't have a he message now as the new terminal app I use actually clears console output when I issue command clear
.
So per the 14.0.0 suggestion, I re-ran command sudo gitlab-rake db:migrate
, and always end up with this error
== 20210706212710 FinalizeCiJobArtifactsBigintConversion: migrating ===========
rake aborted!
StandardError: An error has occurred, all later migrations canceled:
Expected batched background migration for the given configuration to be marked as 'finished', but it is 'active': {:job_class_name=>"CopyColumnUsingBackgroundMigrationJob", :table_name=>"ci_job_artifacts", :column_name=>"id", :job_arguments=>[["id", "job_id"], ["id_convert_to_bigint", "job_id_convert_to_bigint"]]}
Finalize it manualy by running
sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_job_artifacts,id,'[["id"\, "job_id"]\, ["id_convert_to_bigint"\, "job_id_convert_to_bigint"]]']
For more information, check the documentation
https://docs.gitlab.com/ee/user/admin_area/monitoring/background_migrations.html#database-migrations-failing-because-of-batched-background-migration-not-finished
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1129:in `ensure_batched_background_migration_is_finished'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210706212710_finalize_ci_job_artifacts_bigint_conversion.rb:14:in `up'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
Expected batched background migration for the given configuration to be marked as 'finished', but it is 'active': {:job_class_name=>"CopyColumnUsingBackgroundMigrationJob", :table_name=>"ci_job_artifacts", :column_name=>"id", :job_arguments=>[["id", "job_id"], ["id_convert_to_bigint", "job_id_convert_to_bigint"]]}
Finalize it manualy by running
sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_job_artifacts,id,'[["id"\, "job_id"]\, ["id_convert_to_bigint"\, "job_id_convert_to_bigint"]]']
For more information, check the documentation
https://docs.gitlab.com/ee/user/admin_area/monitoring/background_migrations.html#database-migrations-failing-because-of-batched-background-migration-not-finished
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:1129:in `ensure_batched_background_migration_is_finished'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210706212710_finalize_ci_job_artifacts_bigint_conversion.rb:14:in `up'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
So I just followed the instruction to downgrade to 14.1.3 but it failed:
Malformed configuration JSON file found at /opt/gitlab/embedded/nodes/site.com.json.
This usually happens when your last run of `gitlab-ctl reconfigure` didn't complete successfully.
This file is used to check if any of the unsupported configurations are enabled,
and hence require a working reconfigure before upgrading.
Please run `sudo gitlab-ctl reconfigure` to fix it and try again.
If I run sudo gitlab-ctl reconfigure
, I got the same error mentioned at the beginning of the issue.
So how do I get out of this loop?
Edited by Mike Wang