Failure during migration from 13.9.4 to 13.10.0
Summary
Migration seems to fail from 13.9.4 to 13.10.0 on arm64
Steps to reproduce
Unfortunately, I couldn't reproduce the error creating a fresh installation nor I could reproduce creating a fresh installation of 13.9.4 and immediately upgrading to 13.10.0. The only way I have to reproduce is to migrate my own installation from 13.9.4 to 13.10.0.
What is the current bug behavior?
Impossible to migrate to 13.10.0. Only possible to create a new fresh install.
What is the expected correct behavior?
Migration should succeed and the instance should run properly.
Relevant logs
docker logs
Recipe: gitlab::database_migrations
* ruby_block[check remote PG version] action nothing (skipped due to action :nothing)
* bash[migrate gitlab-rails database] action run
[execute] == 20200816133024 AddCveIdRequestProjectSetting: migrating ====================
-- add_column(:project_settings, :cve_id_request_enabled, :boolean, {:default=>true, :null=>false})
-> 0.0979s
== 20200816133024 AddCveIdRequestProjectSetting: migrated (0.0982s) ===========
== 20201112130715 ScheduleRecalculateUuidOnVulnerabilitiesOccurrences: migrating
-- Scheduling RecalculateVulnerabilitiesOccurrencesUuid jobs
-- Scheduled 0 RecalculateVulnerabilitiesOccurrencesUuid jobs with a maximum of 2500 records per batch and an interval of 120 seconds.
The migration is expected to take at least 0 seconds. Expect all jobs to have completed after 2021-03-23 20:20:05 +0100."
== 20201112130715 ScheduleRecalculateUuidOnVulnerabilitiesOccurrences: migrated (0.1624s)
== 20201209163113 RecreateIndexIssueEmailParticipantsOnIssueIdAndEmail: migrating
-- transaction_open?()
-> 0.0001s
-- index_exists?(:issue_email_participants, "issue_id, lower(email)", {:unique=>true, :name=>"index_issue_email_participants_on_issue_id_and_lower_email", :algorithm=>:concurrently})
-> 0.0136s
-- execute("SET statement_timeout TO 0")
-> 0.0008s
-- add_index(:issue_email_participants, "issue_id, lower(email)", {:unique=>true, :name=>"index_issue_email_participants_on_issue_id_and_lower_email", :algorithm=>:concurrently})
-> 0.0897s
-- execute("RESET ALL")
-> 0.0010s
-- transaction_open?()
-> 0.0000s
-- indexes(:issue_email_participants)
-> 0.0079s
-- execute("SET statement_timeout TO 0")
-> 0.0007s
-- remove_index(:issue_email_participants, {:algorithm=>:concurrently, :name=>"index_issue_email_participants_on_issue_id_and_email"})
-> 0.0638s
-- execute("RESET ALL")
-> 0.0010s
== 20201209163113 RecreateIndexIssueEmailParticipantsOnIssueIdAndEmail: migrated (0.1824s)
== 20201228110136 CreateIterationsCadence: migrating ==========================
-- create_table(:iterations_cadences, {})
-- quote_column_name(:title)
-> 0.0001s
-> 0.2331s
-- quote_table_name("check_fedff82d3b")
-> 0.0001s
-- quote_table_name(:iterations_cadences)
-> 0.0000s
-- execute("ALTER TABLE \"iterations_cadences\"\nADD CONSTRAINT \"check_fedff82d3b\" CHECK (char_length(\"title\") <= 255)\n")
-> 0.0036s
== 20201228110136 CreateIterationsCadence: migrated (0.2531s) =================
== 20201228110238 AddIterationsCadenceToSprints: migrating ====================
-- column_exists?(:sprints, :iterations_cadence_id)
-> 0.0077s
-- add_column(:sprints, :iterations_cadence_id, :integer)
-> 0.0031s
-- transaction_open?()
-> 0.0000s
-- index_exists?(:sprints, :iterations_cadence_id, {:name=>"index_sprints_iterations_cadence_id", :algorithm=>:concurrently})
-> 0.0200s
-- execute("SET statement_timeout TO 0")
-> 0.0007s
-- add_index(:sprints, :iterations_cadence_id, {:name=>"index_sprints_iterations_cadence_id", :algorithm=>:concurrently})
-> 0.0227s
-- execute("RESET ALL")
-> 0.0008s
-- transaction_open?()
-> 0.0000s
-- foreign_keys(:sprints)
-> 0.0525s
-- execute("ALTER TABLE sprints\nADD CONSTRAINT fk_365d1db505\nFOREIGN KEY (iterations_cadence_id)\nREFERENCES iterations_cadences (id)\nON DELETE CASCADE\nNOT VALID;\n")
-> 0.0033s
-- execute("SET statement_timeout TO 0")
-> 0.0007s
-- execute("ALTER TABLE sprints VALIDATE CONSTRAINT fk_365d1db505;")
-> 0.0738s
-- execute("RESET ALL")
-> 0.0011s
== 20201228110238 AddIterationsCadenceToSprints: migrated (0.1975s) ===========
== 20201231133921 ScheduleSetDefaultIterationCadences: migrating ==============
== 20201231133921 ScheduleSetDefaultIterationCadences: migrated (0.0001s) =====
== 20210106191305 RenameIndexesOnGitLabCom: migrating =========================
-- indexes(:ldap_group_links)
-> 0.0042s
-- current_schema()
-> 0.0008s
-- indexes(:emails)
-> 0.0096s
-- current_schema()
-> 0.0006s
-- indexes(:users)
-> 0.0422s
-- current_schema()
-> 0.0008s
-- indexes(:users)
-> 0.0374s
-- current_schema()
-> 0.0007s
-- indexes(:users)
-> 0.0382s
-- current_schema()
-> 0.0008s
-- indexes(:schema_migrations)
-> 0.0039s
-- current_schema()
-> 0.0009s
== 20210106191305 RenameIndexesOnGitLabCom: migrated (0.1923s) ================
== 20210111075206 ScheduleUuidPopulationForSecurityFindings2: migrating =======
== 20210111075206 ScheduleUuidPopulationForSecurityFindings2: migrated (0.0053s)
== 20210127152613 AddIterationsCadenceDateRangeConstraint: migrating ==========
-- execute("ALTER TABLE sprints\n ADD CONSTRAINT iteration_start_and_due_date_iterations_cadence_id_constraint\n EXCLUDE USING gist\n ( iterations_cadence_id WITH =,\n daterange(start_date, due_date, '[]') WITH &&\n )\n WHERE (group_id IS NOT NULL)\n")
-> 0.0367s
== 20210127152613 AddIterationsCadenceDateRangeConstraint: migrated (0.0427s) =
== 20210127202613 RemoveIterationGroupDateRangeConstraint: migrating ==========
-- execute("ALTER TABLE sprints\n DROP CONSTRAINT IF EXISTS iteration_start_and_due_daterange_group_id_constraint\n")
-> 0.0196s
== 20210127202613 RemoveIterationGroupDateRangeConstraint: migrated (0.0276s) =
== 20210128172149 CreateBackgroundMigrationTrackingTables: migrating ==========
-- create_table(:batched_background_migrations, {})
-- quote_column_name(:job_class_name)
-> 0.0001s
-- quote_column_name(:batch_class_name)
-> 0.0000s
-- quote_column_name(:table_name)
-> 0.0000s
-- quote_column_name(:column_name)
-> 0.0000s
-> 0.0338s
-- quote_table_name("check_e6c75b1e29")
-> 0.0001s
-- quote_table_name("check_fe10674721")
-> 0.0001s
-- quote_table_name("check_6b6a06254a")
-> 0.0003s
-- quote_table_name("check_5bb0382d6f")
-> 0.0001s
-- quote_table_name(:check_positive_min_value)
-> 0.0001s
-- quote_table_name(:check_max_value_in_range)
-> 0.0001s
-- quote_table_name(:check_positive_sub_batch_size)
-> 0.0001s
-- quote_table_name(:check_batch_size_in_range)
-> 0.0001s
-- quote_table_name(:batched_background_migrations)
-> 0.0000s
-- execute("ALTER TABLE \"batched_background_migrations\"\nADD CONSTRAINT \"check_e6c75b1e29\" CHECK (char_length(\"job_class_name\") <= 100),\nADD CONSTRAINT \"check_fe10674721\" CHECK (char_length(\"batch_class_name\") <= 100),\nADD CONSTRAINT \"check_6b6a06254a\" CHECK (char_length(\"table_name\") <= 63),\nADD CONSTRAINT \"check_5bb0382d6f\" CHECK (char_length(\"column_name\") <= 63),\nADD CONSTRAINT \"check_positive_min_value\" CHECK (min_value > 0),\nADD CONSTRAINT \"check_max_value_in_range\" CHECK (max_value >= min_value),\nADD CONSTRAINT \"check_positive_sub_batch_size\" CHECK (sub_batch_size > 0),\nADD CONSTRAINT \"check_batch_size_in_range\" CHECK (batch_size >= sub_batch_size)\n")
-> 0.0090s
-- create_table(:batched_background_migration_jobs)
-> 0.0251s
== 20210128172149 CreateBackgroundMigrationTrackingTables: migrated (0.0750s) =
== 20210201034649 AddActivePeriodsToOnCallRotations: migrating ================
-- add_column(:incident_management_oncall_rotations, :active_period_start, :time, {:null=>true})
-> 0.0185s
-- add_column(:incident_management_oncall_rotations, :active_period_end, :time, {:null=>true})
-> 0.0022s
== 20210201034649 AddActivePeriodsToOnCallRotations: migrated (0.0211s) =======
== 20210205134213 AddCreatorIdToCustomEmoji: migrating ========================
-- execute("DELETE FROM custom_emoji")
-> 0.0460s
-- add_reference(:custom_emoji, :creator, {:index=>true, :null=>false, :foreign_key=>false})
-> 0.0202s
== 20210205134213 AddCreatorIdToCustomEmoji: migrated (0.0666s) ===============
== 20210208103243 AddIssueCreatedAtToOnboardingProgress: migrating ============
-- add_column(:onboarding_progresses, :issue_created_at, :datetime_with_timezone)
-> 0.0040s
== 20210208103243 AddIssueCreatedAtToOnboardingProgress: migrated (0.0042s) ===
== 20210208200914 AddEndsAtToOncallRotations: migrating =======================
-- add_column(:incident_management_oncall_rotations, :ends_at, :datetime_with_timezone)
-> 0.0026s
== 20210208200914 AddEndsAtToOncallRotations: migrated (0.0028s) ==============
== 20210209110019 CreateExternalApprovalRules: migrating ======================
-- create_table(:external_approval_rules, {:if_not_exists=>true})WARNING: terminating connection because of crash of another server process
DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
HINT: In a moment you should be able to reconnect to the database and repeat your command.
rake aborted!
ActiveRecord::StatementInvalid: PG::ConnectionBad: PQconsumeInput() server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
PG::ConnectionBad: PQconsumeInput() server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
StandardError: An error has occurred, this and all later migrations canceled:
PG::UnableToSend: no connection to the server
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:342:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:341:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
ActiveRecord::StatementInvalid: PG::UnableToSend: no connection to the server
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:342:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:341:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
PG::UnableToSend: no connection to the server
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:342:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:341:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
ActiveRecord::StatementInvalid: PG::ConnectionBad: PQconsumeInput() server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20210218142626_change_finding_fingerprint_enum.rb:9:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:342:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:341:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
PG::ConnectionBad: PQconsumeInput() server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20210218142626_change_finding_fingerprint_enum.rb:9:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:342:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:341:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
-- quote_column_name(:external_url)
-> 0.0001s
-- quote_column_name(:name)
-> 0.0001s
-> 0.0863s
-- quote_table_name("check_b634ca168d")
-> 0.0001s
-- quote_table_name("check_1c64b53ea5")
-> 0.0001s
-- quote_table_name(:external_approval_rules)
-> 0.0000s
-- execute("ALTER TABLE \"external_approval_rules\"\nADD CONSTRAINT \"check_b634ca168d\" CHECK (char_length(\"external_url\") <= 255),\nADD CONSTRAINT \"check_1c64b53ea5\" CHECK (char_length(\"name\") <= 255)\n")
-> 0.0029s
-- create_table(:external_approval_rules_protected_branches)
-> 0.0614s
== 20210209110019 CreateExternalApprovalRules: migrated (0.1579s) =============
== 20210209232508 AddMarkdownSurroundSelectionToUserPreferences: migrating ====
-- add_column(:user_preferences, :markdown_surround_selection, :boolean, {:default=>true, :null=>false})
-> 0.0050s
== 20210209232508 AddMarkdownSurroundSelectionToUserPreferences: migrated (0.0111s)
== 20210210221006 CleanupProjectsWithBadHasExternalIssueTrackerData: migrating
-- transaction_open?()
-> 0.0001s
-- index_exists?(:projects, :id, {:where=>"\"projects\".\"has_external_issue_tracker\" = TRUE\nAND \"projects\".\"pending_delete\" = FALSE\n", :name=>"tmp_idx_projects_on_id_where_has_external_issue_tracker_is_true", :algorithm=>:concurrently})
-> 0.0824s
-- execute("SET statement_timeout TO 0")
-> 0.0008s
-- add_index(:projects, :id, {:where=>"\"projects\".\"has_external_issue_tracker\" = TRUE\nAND \"projects\".\"pending_delete\" = FALSE\n", :name=>"tmp_idx_projects_on_id_where_has_external_issue_tracker_is_true", :algorithm=>:concurrently})
-> 0.0193s
-- execute("RESET ALL")
-> 0.0007s
-- transaction_open?()
-> 0.0000s
-- indexes(:projects)
-> 0.0790s
-- execute("SET statement_timeout TO 0")
-> 0.0007s
-- remove_index(:projects, {:algorithm=>:concurrently, :name=>"tmp_idx_projects_on_id_where_has_external_issue_tracker_is_true"})
-> 0.0063s
-- execute("RESET ALL")
-> 0.0009s
== 20210210221006 CleanupProjectsWithBadHasExternalIssueTrackerData: migrated (0.5100s)
== 20210212153934 MakeTheGeoOauthApplicationTrustedByDefault: migrating =======
-- execute("UPDATE oauth_applications SET confidential = true, trusted = true WHERE id IN (SELECT oauth_application_id FROM geo_nodes);")
-> 0.0542s
== 20210212153934 MakeTheGeoOauthApplicationTrustedByDefault: migrated (0.0545s)
== 20210212163231 AddMergeWhenPipelineSucceedsToNotificationSettings: migrating
-- add_column(:notification_settings, :merge_when_pipeline_succeeds, :boolean, {:default=>false, :null=>false})
-> 0.0063s
== 20210212163231 AddMergeWhenPipelineSucceedsToNotificationSettings: migrated (0.0065s)
== 20210214201118 AddDelayedProjectRemovalToNamespaceSettings: migrating ======
-- add_column(:namespace_settings, :delayed_project_removal, :boolean, {:default=>false, :null=>false})
-> 0.0046s
== 20210214201118 AddDelayedProjectRemovalToNamespaceSettings: migrated (0.0048s)
== 20210214205155 AddIndexToNamespacesDelayedProjectRemoval: migrating ========
-- transaction_open?()
-> 0.0001s
-- index_exists?(:namespaces, :id, {:name=>"tmp_idx_on_namespaces_delayed_project_removal", :where=>"delayed_project_removal = TRUE", :algorithm=>:concurrently})
-> 0.0328s
-- execute("SET statement_timeout TO 0")
-> 0.0007s
-- add_index(:namespaces, :id, {:name=>"tmp_idx_on_namespaces_delayed_project_removal", :where=>"delayed_project_removal = TRUE", :algorithm=>:concurrently})
-> 0.0215s
-- execute("RESET ALL")
-> 0.0009s
== 20210214205155 AddIndexToNamespacesDelayedProjectRemoval: migrated (0.0581s)
== 20210215095328 MigrateDelayedProjectRemovalFromNamespacesToNamespaceSettings: migrating
== 20210215095328 MigrateDelayedProjectRemovalFromNamespacesToNamespaceSettings: migrated (0.2093s)
== 20210215144909 MigrateUsageTrendsSidekiqQueue: migrating ===================
== 20210215144909 MigrateUsageTrendsSidekiqQueue: migrated (0.0010s) ==========
== 20210216122140 AddInProductMarketingEmailsEnabledSetting: migrating ========
-- add_column(:application_settings, :in_product_marketing_emails_enabled, :boolean, {:null=>false, :default=>true})
-> 0.0296s
== 20210216122140 AddInProductMarketingEmailsEnabledSetting: migrated (0.0299s)
== 20210216193620 AddDescriptionToClusterToken: migrating =====================
-- column_exists?(:cluster_agent_tokens, :description)
-> 0.0326s
-- add_column(:cluster_agent_tokens, :description, :text)
-> 0.0038s
-- transaction_open?()
-> 0.0001s
-- current_schema()
-> 0.0010s
-- execute("ALTER TABLE cluster_agent_tokens\nADD CONSTRAINT check_4e4ec5070a\nCHECK ( char_length(description) <= 1024 )\nNOT VALID;\n")
-> 0.0033s
-- current_schema()
-> 0.0009s
-- execute("SET statement_timeout TO 0")
-> 0.0009s
-- execute("ALTER TABLE cluster_agent_tokens VALIDATE CONSTRAINT check_4e4ec5070a;")
-> 0.0032s
-- execute("RESET ALL")
-> 0.0009s
== 20210216193620 AddDescriptionToClusterToken: migrated (0.0708s) ============
== 20210216223335 RemoveIndexOnIssuesWhereServiceDeskReplyToIsNotNull: migrating
-- transaction_open?()
-> 0.0000s
-- indexes(:issues)
-> 0.0485s
-- execute("SET statement_timeout TO 0")
-> 0.0007s
-- remove_index(:issues, {:algorithm=>:concurrently, :name=>"idx_on_issues_where_service_desk_reply_to_is_not_null"})
-> 0.0050s
-- execute("RESET ALL")
-> 0.0008s
== 20210216223335 RemoveIndexOnIssuesWhereServiceDeskReplyToIsNotNull: migrated (0.0586s)
== 20210217100728 MoveCreateReleaseEvidenceQueueOutOfCronjobNamespace: migrating
== 20210217100728 MoveCreateReleaseEvidenceQueueOutOfCronjobNamespace: migrated (0.0005s)
== 20210217101901 CreateEpicListUserPreferences: migrating ====================
-- create_table(:boards_epic_list_user_preferences)
-> 0.0382s
-- add_index(:boards_epic_list_user_preferences, [:user_id, :epic_list_id], {:unique=>true, :name=>"index_epic_board_list_preferences_on_user_and_list"})
-> 0.0111s
== 20210217101901 CreateEpicListUserPreferences: migrated (0.0497s) ===========
== 20210218040814 AddEnvironmentScopeToGroupVariables: migrating ==============
-- column_exists?(:ci_group_variables, :environment_scope)
-> 0.0064s
-- add_column(:ci_group_variables, :environment_scope, :text, {:null=>false, :default=>"*"})
-> 0.0052s
-- transaction_open?()
-> 0.0000s
-- index_exists?(:ci_group_variables, [:group_id, :key, :environment_scope], {:unique=>true, :name=>"index_ci_group_variables_on_group_id_and_key_and_environment", :algorithm=>:concurrently})
-> 0.0069s
-- execute("SET statement_timeout TO 0")
-> 0.0008s
-- add_index(:ci_group_variables, [:group_id, :key, :environment_scope], {:unique=>true, :name=>"index_ci_group_variables_on_group_id_and_key_and_environment", :algorithm=>:concurrently})
-> 0.0114s
-- execute("RESET ALL")
-> 0.0009s
-- transaction_open?()
-> 0.0000s
-- indexes(:ci_group_variables)
-> 0.0078s
-- execute("SET statement_timeout TO 0")
-> 0.0006s
-- remove_index(:ci_group_variables, {:algorithm=>:concurrently, :name=>"index_ci_group_variables_on_group_id_and_key"})
-> 0.0279s
-- execute("RESET ALL")
-> 0.0011s
== 20210218040814 AddEnvironmentScopeToGroupVariables: migrated (0.0728s) =====
== 20210218105431 RemoveDeprecatedCiBuildsColumns: migrating ==================
-- remove_column(:ci_builds, :artifacts_file)
-> 0.0278s
-- remove_column(:ci_builds, :artifacts_file_store)
-> 0.0038s
-- remove_column(:ci_builds, :artifacts_metadata)
-> 0.0018s
-- remove_column(:ci_builds, :artifacts_metadata_store)
-> 0.0017s
-- remove_column(:ci_builds, :artifacts_size)
-> 0.0232s
-- remove_column(:ci_builds, :commands)
-> 0.0029s
== 20210218105431 RemoveDeprecatedCiBuildsColumns: migrated (0.0720s) =========
== 20210218110552 RemoveDeprecatedCiRunnerColumn: migrating ===================
-- remove_column(:ci_runners, :is_shared)
-> 0.0297s
== 20210218110552 RemoveDeprecatedCiRunnerColumn: migrated (0.0441s) ==========
== 20210218142626 ChangeFindingFingerprintEnum: migrating =====================
-- change_column(:vulnerability_finding_fingerprints, :algorithm_type, :integer, {:limit=>2})
postgresql logs
2021-03-23_19:17:56.36489 LOG: starting PostgreSQL 12.5 on aarch64-unknown-linux-gnu, compiled by gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, 64-bit 2021-03-23_19:17:56.38664 LOG: listening on Unix socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432" 2021-03-23_19:17:56.51207 LOG: database system was interrupted; last known up at 2021-03-23 19:15:17 GMT 2021-03-23_19:17:57.15674 LOG: database system was not properly shut down; automatic recovery in progress 2021-03-23_19:17:57.21548 LOG: invalid record length at 0/31DBAA00: wanted 24, got 0 2021-03-23_19:17:57.21561 LOG: redo is not required 2021-03-23_19:17:57.36552 LOG: database system is ready to accept connections 2021-03-23_19:17:59.90387 LOG: no match in usermap "gitlab" for user "gitlab" authenticated as "root" 2021-03-23_19:17:59.90408 FATAL: Peer authentication failed for user "gitlab" 2021-03-23_19:17:59.90409 DETAIL: Connection matched pg_hba.conf line 70: "local all all peer map=gitlab" 2021-03-23_19:18:00.26820 LOG: no match in usermap "gitlab" for user "gitlab" authenticated as "root" 2021-03-23_19:18:00.26843 FATAL: Peer authentication failed for user "gitlab" 2021-03-23_19:18:00.26846 DETAIL: Connection matched pg_hba.conf line 70: "local all all peer map=gitlab" 2021-03-23_19:18:00.45099 LOG: no match in usermap "gitlab" for user "gitlab" authenticated as "root" 2021-03-23_19:18:00.45106 FATAL: Peer authentication failed for user "gitlab" 2021-03-23_19:18:00.45107 DETAIL: Connection matched pg_hba.conf line 70: "local all all peer map=gitlab" **2021-03-23_19:20:09.33060 LOG: server process (PID 724) was terminated by signal 11: Segmentation fault 2021-03-23_19:20:09.33069 DETAIL: Failed process was running: /*application:web*/ ALTER TABLE "vulnerability_finding_fingerprints" ALTER COLUMN "algorithm_type" TYPE smallint** 2021-03-23_19:20:09.33072 LOG: terminating any other active server processes 2021-03-23_19:20:09.33266 WARNING: terminating connection because of crash of another server process 2021-03-23_19:20:09.33275 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2021-03-23_19:20:09.33278 HINT: In a moment you should be able to reconnect to the database and repeat your command. 2021-03-23_19:20:09.33280 WARNING: terminating connection because of crash of another server process 2021-03-23_19:20:09.33281 DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory. 2021-03-23_19:20:09.33283 HINT: In a moment you should be able to reconnect to the database and repeat your command. 2021-03-23_19:20:09.33955 LOG: all server processes terminated; reinitializing 2021-03-23_19:20:09.38283 LOG: database system was interrupted; last known up at 2021-03-23 19:17:57 GMT 2021-03-23_19:20:09.87672 LOG: database system was not properly shut down; automatic recovery in progress 2021-03-23_19:20:09.92749 LOG: redo starts at 0/31DBAA78 2021-03-23_19:20:09.96708 LOG: redo done at 0/31EF9480 2021-03-23_19:20:10.29330 LOG: database system is ready to accept connections
Details of package version
Provide the package version installation details
Package is the official arm64 deb package: https://packages.gitlab.com/gitlab/gitlab-ce/packages/ubuntu/focal/gitlab-ce_13.10.0-ce.0_arm64.deb/download.debEnvironment details
- Operating System: docker ubuntu 20.04
- Installation Target, remove incorrect values:
- Other: docker container
- Installation Type, remove incorrect values:
- New Installation
- Upgrade from version 13.9.4
- Is there any other software running on the machine: gitlab is running in a docker image very similar to the one of the official amd64 image
- Is this a single or multiple node installation? single
- Resources
- CPU: arm64
- Memory total: 4GB (but other containers are running)
Configuration details
Some options from the configuration
gitlab_rails['db_connect_timeout'] = 12000 gitlab_rails['db_statement_timeout'] = 12000000 puma['worker_timeout'] = 120000 logging['logrotate_frequency'] = "daily" puma['worker_processes'] = 2 sidekiq['concurrency'] = 9 prometheus_monitoring['enable'] = false gitlab_rails['lfs_enabled'] = true [...]