Skip to content
Snippets Groups Projects

rspec migration pg14 single-db-ci-connection 7/26

Passed Started by
Simon Tomlinson
@stomlinson
Log timestamps in UTC.
121:21:49Running with gitlab-runner 17.4.0~pre.110.g27400594 (27400594)
221:21:49 on blue-8.private.runners-manager.gitlab.com/gitlab.com/gitlab-org t2_JMznMq, system ID: s_d955b1c3cfe6
321:21:49 feature flags: FF_NETWORK_PER_BUILD:true, FF_TIMESTAMPS:true
421:21:49Resolving secrets
5 21:21:49 Preparing the "docker+machine" executor 00:10
621:21:49Using Docker executor with image registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14:rubygems-3.5-git-2.45-lfs-2.9-chrome-123-yarn-1.22-graphicsmagick-1.3.36 ...
721:21:51Starting service registry.gitlab.com/gitlab-org/gitlab-build-images:postgres-14-pgvector-0.4.1...
821:21:51Authenticating with credentials from job payload (GitLab Registry)
921:21:51Pulling docker image registry.gitlab.com/gitlab-org/gitlab-build-images:postgres-14-pgvector-0.4.1 ...
1021:21:52Using docker image sha256:82f132c22567f8ce17cc132c14b2f9aa3eaf9bb5424d91c2d966912433f0257d for registry.gitlab.com/gitlab-org/gitlab-build-images:postgres-14-pgvector-0.4.1 with digest registry.gitlab.com/gitlab-org/gitlab-build-images@sha256:c55e1364ee4327b244a9cfa1750f19feea14fc774eaf792b725967d146eb45f5 ...
1121:21:52WARNING: Service registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12 is already created. Ignoring.
1221:21:52WARNING: Service registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12 is already created. Ignoring.
1321:21:52Starting service registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12...
1421:21:52Authenticating with credentials from job payload (GitLab Registry)
1521:21:52Pulling docker image registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12 ...
1621:21:53Using docker image sha256:5c5ba5b83d6c60cb0b225bd670b7a43be8c238cc72347d24af57e5027c3648f3 for registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12 with digest registry.gitlab.com/gitlab-org/gitlab-build-images@sha256:46aceab84c42d0e819c8e7940f2f98ce9e184785cd7b22d95872024a35f52407 ...
1721:21:53Starting service redis:6.2-alpine...
1821:21:53Pulling docker image redis:6.2-alpine ...
1921:21:54Using docker image sha256:4100b5bd17430263e85ad9d8ab851959b234acaaf378e093f1f98400f4998815 for redis:6.2-alpine with digest redis@sha256:eaba718fecd1196d88533de7ba49bf903ad33664a92debb24660a922ecd9cac8 ...
2021:21:55Waiting for services to be up and running (timeout 30 seconds)...
2121:21:58Authenticating with credentials from job payload (GitLab Registry)
2221:21:58Pulling docker image registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14:rubygems-3.5-git-2.45-lfs-2.9-chrome-123-yarn-1.22-graphicsmagick-1.3.36 ...
2321:21:59Using docker image sha256:e83f017c5ffd177beb4bc3b4bfabde989d741e04fef90cd764896cf9b670552f for registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14:rubygems-3.5-git-2.45-lfs-2.9-chrome-123-yarn-1.22-graphicsmagick-1.3.36 with digest registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14@sha256:770feb0a974e36f8b83d22a3cb18310d0cfeae11e02f6195b1577c73ce26b398 ...
24 21:21:59 Preparing environment 00:00
2521:21:59Running on runner-t2jmznmq-project-278964-concurrent-0 via runner-t2jmznmq-private-1731611017-4bbf378d...
26 21:21:59 Getting source from Git repository 00:01
2721:22:00Skipping Git repository setup
2821:22:00Skipping Git checkout
2921:22:00Skipping Git submodules setup
30 21:22:00 Restoring cache 00:17
3121:22:00Checking cache for ruby-gems-debian-bookworm-ruby-3.2.5-gemfile-Gemfile-20...
3221:22:00cache.zip is up to date
3321:22:11Successfully extracted cache
34 21:22:17 Downloading artifacts 00:46
3521:22:18Downloading artifacts for clone-gitlab-repo (8368443091)...
3621:22:22Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8368443091 responseStatus=200 OK token=glcbt-66
3721:22:23WARNING: Part of .git directory is on the list of files to extract
3821:22:23WARNING: This may introduce unexpected problems
3921:22:35Downloading artifacts for compile-test-assets (8368443337)...
4021:22:39Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8368443337 responseStatus=200 OK token=glcbt-66
4121:22:45Downloading artifacts for detect-tests (8368443460)...
4221:22:45Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8368443460 responseStatus=200 OK token=glcbt-66
4321:22:45Downloading artifacts for retrieve-tests-metadata (8368443480)...
4421:22:46Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8368443480 responseStatus=200 OK token=glcbt-66
4521:22:46Downloading artifacts for setup-test-env (8368443411)...
4621:22:50Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8368443411 responseStatus=200 OK token=glcbt-66
47 21:23:03 Executing "step_script" stage of the job script
4821:23:03Using docker image sha256:e83f017c5ffd177beb4bc3b4bfabde989d741e04fef90cd764896cf9b670552f for registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14:rubygems-3.5-git-2.45-lfs-2.9-chrome-123-yarn-1.22-graphicsmagick-1.3.36 with digest registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14@sha256:770feb0a974e36f8b83d22a3cb18310d0cfeae11e02f6195b1577c73ce26b398 ...
4921:23:03$ echo $FOSS_ONLY
5021:23:03$ [ "$FOSS_ONLY" = "1" ] && rm -rf ee/ qa/spec/ee/ qa/qa/specs/features/ee/ qa/qa/ee/ qa/qa/ee.rb
5121:23:03$ export GOPATH=$CI_PROJECT_DIR/.go
5221:23:03$ mkdir -p $GOPATH
5321:23:03$ source scripts/utils.sh
5421:23:03$ log_disk_usage "true"
5521:23:03*******************************************************
5621:23:03This runner currently has 76G free disk space.
5721:23:03*******************************************************
58 21:23:03 Disk usage detail 00:06
5921:23:03df -h
6021:23:03Filesystem Size Used Avail Use% Mounted on
6121:23:03overlay 95G 19G 76G 20% /
6221:23:03tmpfs 64M 0 64M 0% /dev
6321:23:03shm 64M 0 64M 0% /dev/shm
6421:23:03/dev/sda1 95G 19G 76G 20% /cache
6521:23:03tmpfs 3.9G 0 3.9G 0% /sys/devices/virtual/dmi/id
6621:23:03du -h -d 1
6721:23:037.4M ./workhorse
6821:23:03148K ./bin
6921:23:031.2M ./.gitlab
7021:23:03177M ./.git
7121:23:034.0K ./builds
7221:23:0328M ./lib
7321:23:038.0K ./.vscode
7421:23:03560K ./storybook
7521:23:0324K ./log
7621:23:031.8M ./data
7721:23:0311M ./qa
7821:23:033.2M ./.rubocop_todo
7921:23:03856M ./vendor
8021:23:031023M ./public
8121:23:0362M ./doc
8221:23:03484K ./danger
8321:23:0332K ./rspec
8421:23:03556K ./tooling
8521:23:04146M ./ee
8621:23:041.3M ./knapsack
8721:23:04300K ./patches
8821:23:044.3M ./changelogs
8921:23:0420K ./haml_lint
9021:23:0428K ./metrics_server
9121:23:04108K ./keeps
9221:23:0436K ./shared
9321:23:0444K ./generator_templates
9421:23:041.3M ./scripts
9521:23:04763M ./tmp
9621:23:04127M ./spec
9721:23:0420K ./file_hooks
9821:23:0432K ./crystalball
9921:23:0416K ./sidekiq_cluster
10021:23:0417M ./gems
10121:23:041.2M ./fixtures
10221:23:04964K ./rubocop
10321:23:044.0K ./.go
10421:23:0433M ./db
10521:23:0412K ./.github
10621:23:04129M ./locale
10721:23:0420M ./config
10821:23:0482M ./app
10921:23:0420K ./.lefthook
11021:23:043.5G .
11121:23:04$ source scripts/prepare_build.sh
112 21:23:04 Installing gems 00:21
14321:23:25Using two connections, single database config (config/database.yml.postgresql)
14421:23:25Geo DB won't be set up.
14521:23:25Embedding DB won't be set up.
146 21:23:25 Setting up DBs 00:56
15921:24:21$ source ./scripts/rspec_helpers.sh
16021:24:21$ run_timed_command "gem install knapsack --no-document"
16121:24:20$ gem install knapsack --no-document
16221:24:22Successfully installed knapsack-4.0.0
16321:24:221 gem installed
16421:24:21==> 'gem install knapsack --no-document' succeeded in 1 seconds.
16521:24:22$ section_start "gitaly-test-spawn" "Spawning Gitaly"; scripts/gitaly-test-spawn; section_end "gitaly-test-spawn"
166 21:24:22 Spawning Gitaly 00:04
17021:24:26$ export RSPEC_SKIPPED_TESTS_REPORT_PATH="rspec/skipped_tests-${CI_JOB_ID}.txt"
17121:24:26$ export RSPEC_RETRIED_TESTS_REPORT_PATH="rspec/retried_tests-${CI_JOB_ID}.txt"
17221:24:26$ tooling/bin/create_job_metrics_file || true
17321:24:29[job-metrics] Creating the job metrics file for the CI/CD job.
17421:24:29$ rspec_section rspec_parallelized_job "--fail-fast=${RSPEC_FAIL_FAST_THRESHOLD} --tag ~quarantine --tag ~zoekt --tag ~click_house"
175 21:24:29 RSpec 04:24
17621:24:29[21:24:29] Starting rspec_parallelized_job
17721:24:22RETRY_FAILED_TESTS_IN_NEW_PROCESS: true
17821:24:29KNAPSACK_GENERATE_REPORT: true
17921:24:29FLAKY_RSPEC_GENERATE_REPORT: true
18021:24:29KNAPSACK_TEST_FILE_PATTERN: spec/{migrations}{,/**/}*_spec.rb
18121:24:29KNAPSACK_LOG_LEVEL: debug
18221:24:29KNAPSACK_REPORT_PATH: knapsack/rspec_migration_pg14_single-db-ci-connection_7_26_278964_report.json
18321:24:29FLAKY_RSPEC_SUITE_REPORT_PATH: rspec/flaky/report-suite.json
18421:24:29FLAKY_RSPEC_REPORT_PATH: rspec/flaky/all_rspec_migration_pg14_single-db-ci-connection_7_26_278964_report.json
18521:24:29NEW_FLAKY_RSPEC_REPORT_PATH: rspec/flaky/new_rspec_migration_pg14_single-db-ci-connection_7_26_278964_report.json
18621:24:29RSPEC_SKIPPED_TESTS_REPORT_PATH: rspec/skipped_tests-8368446046.txt
18721:24:29CRYSTALBALL:
18821:24:29RSPEC_TESTS_MAPPING_ENABLED:
18921:24:29RSPEC_TESTS_FILTER_FILE:
19021:24:29Shell set options (set -o) enabled:
19121:24:29braceexpand on
19221:24:29hashall on
19321:24:29interactive-comments on
19421:24:29pipefail on
19521:24:36Knapsack report generator started!
19621:24:29/builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/io-event-1.6.5/lib/io/event/support.rb:27: warning: IO::Buffer is experimental and both the Ruby and C interface may change in the future!
19721:25:15DEPRECATION WARNING: Support for Rails versions < 7.1 is deprecated and will be removed from ViewComponent 4.0.0 (ViewComponent v4 will remove support for Rails versions < 7.1 no earlier than April 1, 2025) (called from <top (required)> at /builds/gitlab-org/gitlab/config/environment.rb:7)
19821:25:27warning: parser/current is loading parser/ruby32, which recognizes 3.2.6-compliant syntax, but you are running 3.2.5.
19921:25:27Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
20021:25:31Run options: exclude {:quarantine=>true, :zoekt=>true, :click_house=>true}
20121:25:31# [RSpecRunTime] Starting RSpec timer...
20221:25:31[TEST PROF INFO] EventProf enabled (sql.active_record)
20321:25:31unknown OID 2249: failed to recognize type of 'pg_stat_file'. It will be treated as String.
20421:25:38Test environment set up in 3.658291393 seconds
20521:25:38# [RSpecRunTime] Starting example group spec/migrations/lock_duo_features_enabled_update_spec.rb. Expected to take 36.63 seconds.
20621:25:38LockDuoFeaturesEnabledUpdate
20721:25:50main: == [advisory_lock_connection] object_id: 2451980, pg_backend_pid: 122
20821:25:50main: == 20240926061349 LockDuoFeaturesEnabledUpdate: migrating =====================
20921:25:50main: -- execute("UPDATE namespace_settings SET lock_duo_features_enabled = false WHERE duo_features_enabled = true AND lock_duo_features_enabled = true")
21021:25:50main: -> 0.0020s
21121:25:50main: == 20240926061349 LockDuoFeaturesEnabledUpdate: migrated (0.0123s) ============
21221:25:50main: == [advisory_lock_connection] object_id: 2451980, pg_backend_pid: 122
21321:25:54 correctly updates lock_duo_features_enabled
21421:25:55main: == [advisory_lock_connection] object_id: 3261800, pg_backend_pid: 124
21521:25:55main: == 20240926061349 LockDuoFeaturesEnabledUpdate: migrating =====================
21621:25:55main: -- execute("UPDATE namespace_settings SET lock_duo_features_enabled = false WHERE duo_features_enabled = true AND lock_duo_features_enabled = true")
21721:25:55main: -> 0.0021s
21821:25:55main: == 20240926061349 LockDuoFeaturesEnabledUpdate: migrated (0.0110s) ============
21921:25:55main: == [advisory_lock_connection] object_id: 3261800, pg_backend_pid: 124
22021:25:59 handles multiple records correctly
22121:26:00main: == [advisory_lock_connection] object_id: 3991140, pg_backend_pid: 126
22221:26:00main: == 20240926061349 LockDuoFeaturesEnabledUpdate: migrating =====================
22321:26:00main: -- execute("UPDATE namespace_settings SET lock_duo_features_enabled = false WHERE duo_features_enabled = true AND lock_duo_features_enabled = true")
22421:26:00main: -> 0.0020s
22521:26:00main: == 20240926061349 LockDuoFeaturesEnabledUpdate: migrated (0.0115s) ============
22621:26:00main: == [advisory_lock_connection] object_id: 3991140, pg_backend_pid: 126
22721:26:03 does not update records where lock_duo_features_enabled is already false
22821:26:04main: == [advisory_lock_connection] object_id: 4590700, pg_backend_pid: 129
22921:26:04main: == 20240926061349 LockDuoFeaturesEnabledUpdate: migrating =====================
23021:26:04main: -- execute("UPDATE namespace_settings SET lock_duo_features_enabled = false WHERE duo_features_enabled = true AND lock_duo_features_enabled = true")
23121:26:04main: -> 0.0015s
23221:26:04main: == 20240926061349 LockDuoFeaturesEnabledUpdate: migrated (0.0094s) ============
23321:26:04main: == [advisory_lock_connection] object_id: 4590700, pg_backend_pid: 129
23421:26:08 does not update records where duo_features_enabled is false
23521:26:08 #down
23621:26:12 does not modify any records
23721:26:34# [RSpecRunTime] Finishing example group spec/migrations/lock_duo_features_enabled_update_spec.rb. It took 56.02 seconds. Expected to take 36.63 seconds.
23821:26:35# [RSpecRunTime] RSpec elapsed time: 1 minute 3.4 seconds. Current RSS: ~1196M. Threads: 1. load average: 1.01 1.44 1.63 1/281 387
23921:26:35.
24021:26:35# [RSpecRunTime] Starting example group spec/migrations/20241028085044_update_invalid_ci_deleted_object_records_spec.rb. Expected to take 15.46 seconds.
24121:26:35UpdateInvalidCiDeletedObjectRecords
24221:26:36 #up
24321:26:37ci: == [advisory_lock_connection] object_id: 8764460, pg_backend_pid: 142
24421:26:37ci: == 20241028085044 UpdateInvalidCiDeletedObjectRecords: migrating ==============
24521:26:37ci: == 20241028085044 UpdateInvalidCiDeletedObjectRecords: migrated (0.0437s) =====
24621:26:37ci: == [advisory_lock_connection] object_id: 8764460, pg_backend_pid: 142
24721:26:41 sets project_id to -1 for records without a project_id
24821:26:42ci: == [advisory_lock_connection] object_id: 9754640, pg_backend_pid: 148
24921:26:42ci: == 20241028085044 UpdateInvalidCiDeletedObjectRecords: migrating ==============
25021:26:42ci: == 20241028085044 UpdateInvalidCiDeletedObjectRecords: migrated (0.0285s) =====
25121:26:42ci: == [advisory_lock_connection] object_id: 9754640, pg_backend_pid: 148
25221:26:45 does nothing on gitlab.com
25321:26:45# [RSpecRunTime] RSpec elapsed time: 1 minute 14.2 seconds. Current RSS: ~1196M. Threads: 1. load average: 1.01 1.43 1.62 1/281 388
25421:26:45.
25521:26:56# [RSpecRunTime] Finishing example group spec/migrations/20241028085044_update_invalid_ci_deleted_object_records_spec.rb. It took 21.25 seconds. Expected to take 15.46 seconds.
25621:26:56# [RSpecRunTime] RSpec elapsed time: 1 minute 24.71 seconds. Current RSS: ~1199M. Threads: 1. load average: 1.41 1.50 1.64 1/281 389
25721:26:56.
25821:26:56# [RSpecRunTime] Starting example group spec/migrations/20241029170331_queue_resync_has_vulnerabilities_spec.rb. Expected to take 8.24 seconds.
25921:26:56QueueResyncHasVulnerabilities
26021:26:58main: == [advisory_lock_connection] object_id: 10940960, pg_backend_pid: 158
26121:26:58main: == 20241029170331 QueueResyncHasVulnerabilities: migrating ====================
26221:26:59main: == 20241029170331 QueueResyncHasVulnerabilities: migrated (0.0705s) ===========
26321:26:59main: == [advisory_lock_connection] object_id: 10940960, pg_backend_pid: 158
26421:27:03 schedules a new batched migration
26521:27:13# [RSpecRunTime] Finishing example group spec/migrations/20241029170331_queue_resync_has_vulnerabilities_spec.rb. It took 17.48 seconds. Expected to take 8.24 seconds.
26621:27:13# [RSpecRunTime] RSpec elapsed time: 1 minute 42.24 seconds. Current RSS: ~1204M. Threads: 1. load average: 1.25 1.45 1.62 1/280 390
26721:27:13.
26821:27:13# [RSpecRunTime] Starting example group spec/migrations/20230929155123_migrate_disable_merge_trains_value_spec.rb. Missing expected duration from Knapsack report for spec/migrations/20230929155123_migrate_disable_merge_trains_value_spec.rb.
26921:27:13MigrateDisableMergeTrainsValue
27021:27:13 #up
27121:27:13 migrates the flag value into the setting value (PENDING: Skipping because migration MigrateDisableMergeTrainsValue is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
27221:27:13 #down
27321:27:13 reverts the migration (PENDING: Skipping because migration MigrateDisableMergeTrainsValue is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
27421:27:13# [RSpecRunTime] Finishing example group spec/migrations/20230929155123_migrate_disable_merge_trains_value_spec.rb. It took 0.03 second. Missing expected duration from Knapsack report for spec/migrations/20230929155123_migrate_disable_merge_trains_value_spec.rb.
27521:27:13# [RSpecRunTime] Starting example group spec/migrations/20231218092401_queue_backfill_partition_id_ci_pipeline_chat_data_spec.rb. Missing expected duration from Knapsack report for spec/migrations/20231218092401_queue_backfill_partition_id_ci_pipeline_chat_data_spec.rb.
27621:27:13QueueBackfillPartitionIdCiPipelineChatData
27721:27:13 #up
27821:27:13 with migration present
27921:27:13 when migration finished successfully
28021:27:13 does not raise exception (PENDING: Skipping because migration QueueBackfillPartitionIdCiPipelineChatData is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
28121:27:13 schedules background jobs for each batch of ci_pipeline_chat_data (PENDING: Skipping because migration QueueBackfillPartitionIdCiPipelineChatData is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
28221:27:13 #down
28321:27:13 deletes all batched migration records (PENDING: Skipping because migration QueueBackfillPartitionIdCiPipelineChatData is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
28421:27:13# [RSpecRunTime] Finishing example group spec/migrations/20231218092401_queue_backfill_partition_id_ci_pipeline_chat_data_spec.rb. It took 0.01 second. Missing expected duration from Knapsack report for spec/migrations/20231218092401_queue_backfill_partition_id_ci_pipeline_chat_data_spec.rb.
28521:27:13# [RSpecRunTime] Starting example group spec/migrations/20240223130548_queue_update_sbom_components_name_based_on_pep503_spec.rb. Missing expected duration from Knapsack report for spec/migrations/20240223130548_queue_update_sbom_components_name_based_on_pep503_spec.rb.
28621:27:13QueueUpdateSbomComponentsNameBasedOnPep503
28721:27:13 schedules a new batched migration (PENDING: Skipping because migration QueueUpdateSbomComponentsNameBasedOnPep503 is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
28821:27:13# [RSpecRunTime] Finishing example group spec/migrations/20240223130548_queue_update_sbom_components_name_based_on_pep503_spec.rb. It took 0.01 second. Missing expected duration from Knapsack report for spec/migrations/20240223130548_queue_update_sbom_components_name_based_on_pep503_spec.rb.
28921:27:13# [RSpecRunTime] Starting example group spec/migrations/20240502044605_remove_create_empty_embeddings_records_worker_spec.rb. Missing expected duration from Knapsack report for spec/migrations/20240502044605_remove_create_empty_embeddings_records_worker_spec.rb.
29021:27:13RemoveCreateEmptyEmbeddingsRecordsWorker
29121:27:13 when cron job exists
29221:27:13 deletes the cron job and enqueued jobs (PENDING: Skipping because migration RemoveCreateEmptyEmbeddingsRecordsWorker is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
29321:27:13 when cron job does not exist
29421:27:13 no-ops (PENDING: Skipping because migration RemoveCreateEmptyEmbeddingsRecordsWorker is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
29521:27:13# [RSpecRunTime] Finishing example group spec/migrations/20240502044605_remove_create_empty_embeddings_records_worker_spec.rb. It took 0.01 second. Missing expected duration from Knapsack report for spec/migrations/20240502044605_remove_create_empty_embeddings_records_worker_spec.rb.
29621:27:13# [RSpecRunTime] Starting example group spec/migrations/20240529112047_queue_backfill_wiki_page_slugs_project_id_spec.rb. Missing expected duration from Knapsack report for spec/migrations/20240529112047_queue_backfill_wiki_page_slugs_project_id_spec.rb.
29721:27:13QueueBackfillWikiPageSlugsProjectId
29821:27:13 schedules a new batched migration (PENDING: Skipping because migration QueueBackfillWikiPageSlugsProjectId is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
29921:27:13# [RSpecRunTime] Finishing example group spec/migrations/20240529112047_queue_backfill_wiki_page_slugs_project_id_spec.rb. It took 0.01 second. Missing expected duration from Knapsack report for spec/migrations/20240529112047_queue_backfill_wiki_page_slugs_project_id_spec.rb.
30021:27:13# [RSpecRunTime] Starting example group spec/migrations/20240607105705_queue_backfill_dast_site_validations_project_id_spec.rb. Missing expected duration from Knapsack report for spec/migrations/20240607105705_queue_backfill_dast_site_validations_project_id_spec.rb.
30121:27:13QueueBackfillDastSiteValidationsProjectId
30221:27:13 schedules a new batched migration (PENDING: Skipping because migration QueueBackfillDastSiteValidationsProjectId is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
30321:27:13# [RSpecRunTime] Finishing example group spec/migrations/20240607105705_queue_backfill_dast_site_validations_project_id_spec.rb. It took 0.01 second. Missing expected duration from Knapsack report for spec/migrations/20240607105705_queue_backfill_dast_site_validations_project_id_spec.rb.
30421:27:13# [RSpecRunTime] Starting example group spec/migrations/20240618123929_queue_backfill_epic_issues_namespace_id_spec.rb. Missing expected duration from Knapsack report for spec/migrations/20240618123929_queue_backfill_epic_issues_namespace_id_spec.rb.
30521:27:13QueueBackfillEpicIssuesNamespaceId
30621:27:13 schedules a new batched migration (PENDING: Skipping because migration QueueBackfillEpicIssuesNamespaceId is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
30721:27:13# [RSpecRunTime] Finishing example group spec/migrations/20240618123929_queue_backfill_epic_issues_namespace_id_spec.rb. It took 0.01 second. Missing expected duration from Knapsack report for spec/migrations/20240618123929_queue_backfill_epic_issues_namespace_id_spec.rb.
30821:27:13# [RSpecRunTime] Starting example group spec/migrations/20240708130252_remove_idx_project_statistics_wiki_size_and_project_id_sync_spec.rb. Missing expected duration from Knapsack report for spec/migrations/20240708130252_remove_idx_project_statistics_wiki_size_and_project_id_sync_spec.rb.
30921:27:13RemoveIdxProjectStatisticsWikiSizeAndProjectIdSync
31021:27:13 #up
31121:27:13 does nothing when not on gitlab.com (PENDING: Skipping because migration RemoveIdxProjectStatisticsWikiSizeAndProjectIdSync is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
31221:27:13 removes the index when on gitlab.com (PENDING: Skipping because migration RemoveIdxProjectStatisticsWikiSizeAndProjectIdSync is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
31321:27:13 #down
31421:27:13 does nothing when not on gitlab.com (PENDING: Skipping because migration RemoveIdxProjectStatisticsWikiSizeAndProjectIdSync is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
31521:27:13 re-adds the index when on gitlab.com (PENDING: Skipping because migration RemoveIdxProjectStatisticsWikiSizeAndProjectIdSync is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
31621:27:13# [RSpecRunTime] Finishing example group spec/migrations/20240708130252_remove_idx_project_statistics_wiki_size_and_project_id_sync_spec.rb. It took 0.01 second. Missing expected duration from Knapsack report for spec/migrations/20240708130252_remove_idx_project_statistics_wiki_size_and_project_id_sync_spec.rb.
31721:27:13# [RSpecRunTime] Starting example group spec/migrations/20240801195253_queue_backfill_reserved_storage_bytes_spec.rb. Missing expected duration from Knapsack report for spec/migrations/20240801195253_queue_backfill_reserved_storage_bytes_spec.rb.
31821:27:13QueueBackfillReservedStorageBytes
31921:27:13 schedules a new batched migration (PENDING: Skipping because migration QueueBackfillReservedStorageBytes is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
32021:27:13# [RSpecRunTime] Finishing example group spec/migrations/20240801195253_queue_backfill_reserved_storage_bytes_spec.rb. It took 0.04 second. Missing expected duration from Knapsack report for spec/migrations/20240801195253_queue_backfill_reserved_storage_bytes_spec.rb.
32121:27:14# [RSpecRunTime] Starting example group spec/migrations/20240829160537_queue_backfill_p_ci_runner_machine_builds_project_id_spec.rb. Missing expected duration from Knapsack report for spec/migrations/20240829160537_queue_backfill_p_ci_runner_machine_builds_project_id_spec.rb.
32221:27:14QueueBackfillPCiRunnerMachineBuildsProjectId
32321:27:14 schedules a new batched migration (PENDING: Skipping because migration QueueBackfillPCiRunnerMachineBuildsProjectId is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
32421:27:14 when executed on .com
32521:27:14 schedules a new batched migration (PENDING: Skipping because migration QueueBackfillPCiRunnerMachineBuildsProjectId is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
32621:27:14# [RSpecRunTime] Finishing example group spec/migrations/20240829160537_queue_backfill_p_ci_runner_machine_builds_project_id_spec.rb. It took 0.02 second. Missing expected duration from Knapsack report for spec/migrations/20240829160537_queue_backfill_p_ci_runner_machine_builds_project_id_spec.rb.
32721:27:14# [RSpecRunTime] Starting example group spec/migrations/db/post_migrate/20240704184634_finalize_resync_basic_epic_fields_to_work_item_spec.rb. Missing expected duration from Knapsack report for spec/migrations/db/post_migrate/20240704184634_finalize_resync_basic_epic_fields_to_work_item_spec.rb.
32821:27:14FinalizeResyncBasicEpicFieldsToWorkItem
32921:27:14 #up
33021:27:14 ensures the migration is completed for self-managed instances (PENDING: Skipping because migration FinalizeResyncBasicEpicFieldsToWorkItem is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
33121:27:14# [RSpecRunTime] Finishing example group spec/migrations/db/post_migrate/20240704184634_finalize_resync_basic_epic_fields_to_work_item_spec.rb. It took 0.01 second. Missing expected duration from Knapsack report for spec/migrations/db/post_migrate/20240704184634_finalize_resync_basic_epic_fields_to_work_item_spec.rb.
33221:27:17auto_explain log contains 846 entries for main, writing to /builds/gitlab-org/gitlab/auto_explain/rspec-migration-pg14-single-db-ci-connection-7-26.342.main.ndjson.gz
33321:27:17took 3.186210408
33421:27:19auto_explain log contains 847 entries for ci, writing to /builds/gitlab-org/gitlab/auto_explain/rspec-migration-pg14-single-db-ci-connection-7-26.342.ci.ndjson.gz
33521:27:19took 1.847018718
33621:27:19[TEST PROF INFO] EventProf results for sql.active_record
33721:27:19Total time: 00:49.507 of 01:35.082 (52.07%)
33821:27:19Total events: 48227
33921:27:19Top 5 slowest suites (by time):
34021:27:19LockDuoFeaturesEnabledUpdate (./spec/migrations/lock_duo_features_enabled_update_spec.rb:6) – 00:29.729 (27685 / 5) of 00:56.089 (53.0%)
34121:27:19UpdateInvalid...dObjectRecords (./spec/migrations/20241028085044_update_invalid_ci_deleted_object_records_spec.rb:6) – 00:10.849 (11561 / 2) of 00:21.288 (50.96%)
34221:27:19QueueResyncHasVulnerabilities (./spec/migrations/20241029170331_queue_resync_has_vulnerabilities_spec.rb:6) – 00:08.899 (8904 / 1) of 00:17.520 (50.79%)
34321:27:19MigrateDisableMergeTrainsValue (./spec/migrations/20230929155123_migrate_disable_merge_trains_value_spec.rb:6) – 00:00.003 (7 / 2) of 00:00.026 (13.48%)
34421:27:19QueueBackfill...uildsProjectId (./spec/migrations/20240829160537_queue_backfill_p_ci_runner_machine_builds_project_id_spec.rb:6) – 00:00.003 (7 / 2) of 00:00.014 (20.31%)
34521:27:19Knapsack report was generated. Preview:
34621:27:19{
34721:27:19 "spec/migrations/lock_duo_features_enabled_update_spec.rb": 34.59716269599994,
34821:27:19 "spec/migrations/20241028085044_update_invalid_ci_deleted_object_records_spec.rb": 10.800435160001143,
34921:27:19 "spec/migrations/20241029170331_queue_resync_has_vulnerabilities_spec.rb": 7.299031680000553
35021:27:19}
35121:27:19Knapsack global time execution for tests: 52s
35221:27:19Pending: (Failures listed here are expected and do not affect your suite's status)
35321:27:19 1) MigrateDisableMergeTrainsValue#up migrates the flag value into the setting value
35421:27:19 # Skipping because migration MigrateDisableMergeTrainsValue is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
35521:27:19 # ./spec/migrations/20230929155123_migrate_disable_merge_trains_value_spec.rb:61
35621:27:19 2) MigrateDisableMergeTrainsValue#down reverts the migration
35721:27:19 # Skipping because migration MigrateDisableMergeTrainsValue is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
35821:27:19 # ./spec/migrations/20230929155123_migrate_disable_merge_trains_value_spec.rb:72
35921:27:19 3) QueueBackfillPartitionIdCiPipelineChatData#up with migration present when migration finished successfully does not raise exception
36021:27:19 # Skipping because migration QueueBackfillPartitionIdCiPipelineChatData is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
36121:27:19 # ./spec/migrations/20231218092401_queue_backfill_partition_id_ci_pipeline_chat_data_spec.rb:29
36221:27:19 4) QueueBackfillPartitionIdCiPipelineChatData#up with migration present when migration finished successfully schedules background jobs for each batch of ci_pipeline_chat_data
36321:27:19 # Skipping because migration QueueBackfillPartitionIdCiPipelineChatData is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
36421:27:19 # ./spec/migrations/20231218092401_queue_backfill_partition_id_ci_pipeline_chat_data_spec.rb:33
36521:27:19 5) QueueBackfillPartitionIdCiPipelineChatData#down deletes all batched migration records
36621:27:19 # Skipping because migration QueueBackfillPartitionIdCiPipelineChatData is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
36721:27:19 # ./spec/migrations/20231218092401_queue_backfill_partition_id_ci_pipeline_chat_data_spec.rb:49
36821:27:19 6) QueueUpdateSbomComponentsNameBasedOnPep503 schedules a new batched migration
36921:27:19 # Skipping because migration QueueUpdateSbomComponentsNameBasedOnPep503 is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
37021:27:19 # ./spec/migrations/20240223130548_queue_update_sbom_components_name_based_on_pep503_spec.rb:10
37121:27:19 7) RemoveCreateEmptyEmbeddingsRecordsWorker when cron job exists deletes the cron job and enqueued jobs
37221:27:19 # Skipping because migration RemoveCreateEmptyEmbeddingsRecordsWorker is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
37321:27:19 # ./spec/migrations/20240502044605_remove_create_empty_embeddings_records_worker_spec.rb:43
37421:27:19 8) RemoveCreateEmptyEmbeddingsRecordsWorker when cron job does not exist no-ops
37521:27:19 # Skipping because migration RemoveCreateEmptyEmbeddingsRecordsWorker is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
37621:27:19 # ./spec/migrations/20240502044605_remove_create_empty_embeddings_records_worker_spec.rb:54
37721:27:19 9) QueueBackfillWikiPageSlugsProjectId schedules a new batched migration
37821:27:19 # Skipping because migration QueueBackfillWikiPageSlugsProjectId is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
37921:27:19 # ./spec/migrations/20240529112047_queue_backfill_wiki_page_slugs_project_id_spec.rb:9
38021:27:19 10) QueueBackfillDastSiteValidationsProjectId schedules a new batched migration
38121:27:19 # Skipping because migration QueueBackfillDastSiteValidationsProjectId is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
38221:27:19 # ./spec/migrations/20240607105705_queue_backfill_dast_site_validations_project_id_spec.rb:9
38321:27:19 11) QueueBackfillEpicIssuesNamespaceId schedules a new batched migration
38421:27:19 # Skipping because migration QueueBackfillEpicIssuesNamespaceId is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
38521:27:19 # ./spec/migrations/20240618123929_queue_backfill_epic_issues_namespace_id_spec.rb:9
38621:27:19 12) RemoveIdxProjectStatisticsWikiSizeAndProjectIdSync#up does nothing when not on gitlab.com
38721:27:19 # Skipping because migration RemoveIdxProjectStatisticsWikiSizeAndProjectIdSync is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
38821:27:19 # ./spec/migrations/20240708130252_remove_idx_project_statistics_wiki_size_and_project_id_sync_spec.rb:13
38921:27:19 13) RemoveIdxProjectStatisticsWikiSizeAndProjectIdSync#up removes the index when on gitlab.com
39021:27:19 # Skipping because migration RemoveIdxProjectStatisticsWikiSizeAndProjectIdSync is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
39121:27:19 # ./spec/migrations/20240708130252_remove_idx_project_statistics_wiki_size_and_project_id_sync_spec.rb:19
39221:27:19 14) RemoveIdxProjectStatisticsWikiSizeAndProjectIdSync#down does nothing when not on gitlab.com
39321:27:19 # Skipping because migration RemoveIdxProjectStatisticsWikiSizeAndProjectIdSync is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
39421:27:19 # ./spec/migrations/20240708130252_remove_idx_project_statistics_wiki_size_and_project_id_sync_spec.rb:27
39521:27:19 15) RemoveIdxProjectStatisticsWikiSizeAndProjectIdSync#down re-adds the index when on gitlab.com
39621:27:19 # Skipping because migration RemoveIdxProjectStatisticsWikiSizeAndProjectIdSync is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
39721:27:19 # ./spec/migrations/20240708130252_remove_idx_project_statistics_wiki_size_and_project_id_sync_spec.rb:33
39821:27:19 16) QueueBackfillReservedStorageBytes schedules a new batched migration
39921:27:19 # Skipping because migration QueueBackfillReservedStorageBytes is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
40021:27:19 # ./spec/migrations/20240801195253_queue_backfill_reserved_storage_bytes_spec.rb:9
40121:27:19 17) QueueBackfillPCiRunnerMachineBuildsProjectId schedules a new batched migration
40221:27:19 # Skipping because migration QueueBackfillPCiRunnerMachineBuildsProjectId is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
40321:27:19 # ./spec/migrations/20240829160537_queue_backfill_p_ci_runner_machine_builds_project_id_spec.rb:10
40421:27:19 18) QueueBackfillPCiRunnerMachineBuildsProjectId when executed on .com schedules a new batched migration
40521:27:19 # Skipping because migration QueueBackfillPCiRunnerMachineBuildsProjectId is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
40621:27:19 # ./spec/migrations/20240829160537_queue_backfill_p_ci_runner_machine_builds_project_id_spec.rb:35
40721:27:19 19) FinalizeResyncBasicEpicFieldsToWorkItem#up ensures the migration is completed for self-managed instances
40821:27:19 # Skipping because migration FinalizeResyncBasicEpicFieldsToWorkItem is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
40921:27:19 # ./spec/migrations/db/post_migrate/20240704184634_finalize_resync_basic_epic_fields_to_work_item_spec.rb:8
41021:27:19Finished in 1 minute 47.54 seconds (files took 58.54 seconds to load)
41121:27:1927 examples, 0 failures, 19 pending
41221:27:19Randomized with seed 62851
41321:27:27[TEST PROF INFO] Time spent in factories: 00:00.141 (0.13% of total time)
41421:27:27RSpec exited with 0.
41521:27:27No examples to retry, congrats!
416 21:27:27 Running after_script 00:15
41721:27:28Running after script...
41821:27:28$ source scripts/utils.sh
41921:27:28$ log_disk_usage
42021:27:28*******************************************************
42121:27:28This runner currently has 75G free disk space.
42221:27:28*******************************************************
423 21:27:28 Disk usage detail 00:06
47921:27:34$ bundle exec gem list gitlab_quality-test_tooling
48021:27:37gitlab_quality-test_tooling (2.1.0)
48121:27:37$ section_start "failed-test-issues" "Report test failures" # collapsed multi-line command
482 21:27:37 Report test failures 00:00
48421:27:37$ section_start "flaky-test-issues" "Report test flakiness" # collapsed multi-line command
485 21:27:37 Report test flakiness 00:00
48721:27:37$ section_start "slow-test-issues" "Report test slowness" # collapsed multi-line command
488 21:27:37 Report test slowness 00:00
49021:27:37$ section_start "slow-test-merge-request-report-note" "Report test slowness in MR note" # collapsed multi-line command
491 21:27:37 Report test slowness in MR note 00:02
49421:27:39$ section_start "knapsack-report-issues" "Report test files close to timing out" # collapsed multi-line command
495 21:27:39 Report test files close to timing out 00:02
49821:27:41$ tooling/bin/push_job_metrics || true
49921:27:42[job-metrics] Pushing job metrics file for the CI/CD job.
50021:27:42[job-metrics] Pushed 4 CI job metric entries to InfluxDB.
501 21:27:42 Saving cache for successful job 00:01
50221:27:43Not uploading cache ruby-gems-debian-bookworm-ruby-3.2.5-gemfile-Gemfile-20 due to policy
503 21:27:43 Uploading artifacts for successful job 00:05
50421:27:44Uploading artifacts...
50521:27:44auto_explain/: found 3 matching artifact files and directories
50621:27:44coverage/: found 5 matching artifact files and directories
50721:27:44crystalball/: found 2 matching artifact files and directories
50821:27:44WARNING: deprecations/: no matching files. Ensure that the artifact path is relative to the working directory (/builds/gitlab-org/gitlab)
50921:27:44knapsack/: found 4 matching artifact files and directories
51021:27:44rspec/: found 16 matching artifact files and directories
51121:27:44WARNING: tmp/capybara/: no matching files. Ensure that the artifact path is relative to the working directory (/builds/gitlab-org/gitlab)
51221:27:44log/*.log: found 13 matching artifact files and directories
51321:27:44WARNING: Upload request redirected location=https://gitlab.com/api/v4/jobs/8368446046/artifacts?artifact_format=zip&artifact_type=archive&expire_in=31d new-url=https://gitlab.com
51421:27:44WARNING: Retrying... context=artifacts-uploader error=request redirected
51521:27:47Uploading artifacts as "archive" to coordinator... 201 Created id=8368446046 responseStatus=201 Created token=glcbt-66
51621:27:47Uploading artifacts...
51721:27:47rspec/rspec-*.xml: found 1 matching artifact files and directories
51821:27:47WARNING: Upload request redirected location=https://gitlab.com/api/v4/jobs/8368446046/artifacts?artifact_format=gzip&artifact_type=junit&expire_in=31d new-url=https://gitlab.com
51921:27:47WARNING: Retrying... context=artifacts-uploader error=request redirected
52021:27:48Uploading artifacts as "junit" to coordinator... 201 Created id=8368446046 responseStatus=201 Created token=glcbt-66
521 21:27:48 Cleaning up project directory and file based variables 00:01
52221:27:57Job succeeded