Skip to content
Snippets Groups Projects

rspec background_migration pg14 single-db 6/17

Passed Started by
Simon Tomlinson
@stomlinson
Log timestamps in UTC.
122:12:10Running with gitlab-runner 17.4.0~pre.110.g27400594 (27400594)
222:12:10 on blue-4.private.runners-manager.gitlab.com/gitlab.com/gitlab-org tDjA3Sp3, system ID: s_96a8ae3d1050
322:12:10 feature flags: FF_NETWORK_PER_BUILD:true, FF_TIMESTAMPS:true
422:12:10Resolving secrets
5 22:12:10 Preparing the "docker+machine" executor 01:29
622:12:10Using Docker executor with image registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14:rubygems-3.5-git-2.45-lfs-2.9-chrome-123-yarn-1.22-graphicsmagick-1.3.36 ...
722:12:14Starting service registry.gitlab.com/gitlab-org/gitlab-build-images:postgres-14-pgvector-0.4.1...
822:12:14Authenticating with credentials from job payload (GitLab Registry)
922:12:14Pulling docker image registry.gitlab.com/gitlab-org/gitlab-build-images:postgres-14-pgvector-0.4.1 ...
1022:12:21Using docker image sha256:82f132c22567f8ce17cc132c14b2f9aa3eaf9bb5424d91c2d966912433f0257d for registry.gitlab.com/gitlab-org/gitlab-build-images:postgres-14-pgvector-0.4.1 with digest registry.gitlab.com/gitlab-org/gitlab-build-images@sha256:c55e1364ee4327b244a9cfa1750f19feea14fc774eaf792b725967d146eb45f5 ...
1122:12:25WARNING: Service registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12 is already created. Ignoring.
1222:12:25WARNING: Service registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12 is already created. Ignoring.
1322:12:25Starting service registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12...
1422:12:25Authenticating with credentials from job payload (GitLab Registry)
1522:12:25Pulling docker image registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12 ...
1622:12:29Using docker image sha256:5c5ba5b83d6c60cb0b225bd670b7a43be8c238cc72347d24af57e5027c3648f3 for registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12 with digest registry.gitlab.com/gitlab-org/gitlab-build-images@sha256:46aceab84c42d0e819c8e7940f2f98ce9e184785cd7b22d95872024a35f52407 ...
1722:12:30Starting service redis:6.2-alpine...
1822:12:30Pulling docker image redis:6.2-alpine ...
1922:12:33Using docker image sha256:4100b5bd17430263e85ad9d8ab851959b234acaaf378e093f1f98400f4998815 for redis:6.2-alpine with digest redis@sha256:eaba718fecd1196d88533de7ba49bf903ad33664a92debb24660a922ecd9cac8 ...
2022:12:34Waiting for services to be up and running (timeout 30 seconds)...
2122:12:35Authenticating with credentials from job payload (GitLab Registry)
2222:12:35Pulling docker image registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14:rubygems-3.5-git-2.45-lfs-2.9-chrome-123-yarn-1.22-graphicsmagick-1.3.36 ...
2322:13:39Using docker image sha256:e83f017c5ffd177beb4bc3b4bfabde989d741e04fef90cd764896cf9b670552f for registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14:rubygems-3.5-git-2.45-lfs-2.9-chrome-123-yarn-1.22-graphicsmagick-1.3.36 with digest registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14@sha256:770feb0a974e36f8b83d22a3cb18310d0cfeae11e02f6195b1577c73ce26b398 ...
24 22:13:39 Preparing environment 00:01
2522:13:40Running on runner-tdja3sp3-project-13083-concurrent-0 via runner-tdja3sp3-private-1731535887-19b8939b...
26 22:13:40 Getting source from Git repository 00:00
2722:13:40Skipping Git repository setup
2822:13:40Skipping Git checkout
2922:13:40Skipping Git submodules setup
30 22:13:40 Restoring cache 00:17
3122:13:41Checking cache for ruby-gems-debian-bookworm-ruby-3.2.5-gemfile-Gemfile-3...
3222:13:41Downloading cache from https://storage.googleapis.com/gitlab-com-runners-cache/project/13083/ruby-gems-debian-bookworm-ruby-3.2.5-gemfile-Gemfile-3 ETag="2f01fc71854036e0b450342e02ca5726"
3322:13:51Successfully extracted cache
34 22:13:57 Downloading artifacts 00:39
3522:13:58Downloading artifacts for clone-gitlab-repo (8353300418)...
3622:14:02Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8353300418 responseStatus=200 OK token=glcbt-66
3722:14:02WARNING: Part of .git directory is on the list of files to extract
3822:14:02WARNING: This may introduce unexpected problems
3922:14:10Downloading artifacts for compile-test-assets (8353300423)...
4022:14:15Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8353300423 responseStatus=200 OK token=glcbt-66
4122:14:20Downloading artifacts for detect-tests (8353300433)...
4222:14:20Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8353300433 responseStatus=200 OK token=glcbt-66
4322:14:20Downloading artifacts for retrieve-tests-metadata (8353300434)...
4422:14:21Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8353300434 responseStatus=200 OK token=glcbt-66
4522:14:21Downloading artifacts for setup-test-env (8353300427)...
4622:14:26Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8353300427 responseStatus=200 OK token=glcbt-66
47 22:14:36 Executing "step_script" stage of the job script
4822:14:36Using docker image sha256:e83f017c5ffd177beb4bc3b4bfabde989d741e04fef90cd764896cf9b670552f for registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14:rubygems-3.5-git-2.45-lfs-2.9-chrome-123-yarn-1.22-graphicsmagick-1.3.36 with digest registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14@sha256:770feb0a974e36f8b83d22a3cb18310d0cfeae11e02f6195b1577c73ce26b398 ...
4922:14:38$ echo $FOSS_ONLY
5022:14:38$ [ "$FOSS_ONLY" = "1" ] && rm -rf ee/ qa/spec/ee/ qa/qa/specs/features/ee/ qa/qa/ee/ qa/qa/ee.rb
5122:14:38$ export GOPATH=$CI_PROJECT_DIR/.go
5222:14:38$ mkdir -p $GOPATH
5322:14:38$ source scripts/utils.sh
5422:14:38$ log_disk_usage "true"
5522:14:38*******************************************************
5622:14:38This runner currently has 80G free disk space.
5722:14:38*******************************************************
58 22:14:38 Disk usage detail 00:01
5922:14:38df -h
6022:14:38Filesystem Size Used Avail Use% Mounted on
6122:14:38overlay 95G 15G 80G 16% /
6222:14:38tmpfs 64M 0 64M 0% /dev
6322:14:38shm 64M 0 64M 0% /dev/shm
6422:14:38/dev/sda1 95G 15G 80G 16% /cache
6522:14:38tmpfs 3.9G 0 3.9G 0% /sys/devices/virtual/dmi/id
6622:14:38du -h -d 1
6722:14:387.4M ./workhorse
6822:14:38148K ./bin
6922:14:381.2M ./.gitlab
7022:14:38121M ./.git
7122:14:384.0K ./builds
7222:14:3828M ./lib
7322:14:38560K ./storybook
7422:14:3824K ./log
7522:14:381.8M ./data
7622:14:387.1M ./qa
7722:14:383.2M ./.rubocop_todo
7822:14:38856M ./vendor
7922:14:38858M ./public
8022:14:3862M ./doc
8122:14:38484K ./danger
8222:14:3832K ./rspec
8322:14:38556K ./tooling
8422:14:381.3M ./knapsack
8522:14:38300K ./patches
8622:14:383.9M ./changelogs
8722:14:3820K ./haml_lint
8822:14:3828K ./metrics_server
8922:14:38108K ./keeps
9022:14:3836K ./shared
9122:14:3844K ./generator_templates
9222:14:381.3M ./scripts
9322:14:38641M ./tmp
9422:14:38127M ./spec
9522:14:3820K ./file_hooks
9622:14:3832K ./crystalball
9722:14:3816K ./sidekiq_cluster
9822:14:3817M ./gems
9922:14:381.2M ./fixtures
10022:14:38964K ./rubocop
10122:14:384.0K ./.go
10222:14:3833M ./db
10322:14:3812K ./.github
10422:14:38129M ./locale
10522:14:3820M ./config
10622:14:3882M ./app
10722:14:3820K ./.lefthook
10822:14:383.0G .
10922:14:38$ source scripts/prepare_build.sh
110 22:14:38 Installing gems 00:38
15122:15:17Using two connections, single database config (config/database.yml.postgresql)
15222:15:17Disabling ci connection in config/database.yml
15322:15:16Geo DB won't be set up.
15422:15:17Embedding DB won't be set up.
155 22:15:17 Setting up DBs 01:00
18022:16:17$ source ./scripts/rspec_helpers.sh
18122:16:17$ run_timed_command "gem install knapsack --no-document"
18222:16:16$ gem install knapsack --no-document
18322:16:18Successfully installed knapsack-4.0.0
18422:16:181 gem installed
18522:16:17==> 'gem install knapsack --no-document' succeeded in 1 seconds.
18622:16:18$ section_start "gitaly-test-spawn" "Spawning Gitaly"; scripts/gitaly-test-spawn; section_end "gitaly-test-spawn"
187 22:16:18 Spawning Gitaly 00:05
19122:16:23$ export RSPEC_SKIPPED_TESTS_REPORT_PATH="rspec/skipped_tests-${CI_JOB_ID}.txt"
19222:16:23$ export RSPEC_RETRIED_TESTS_REPORT_PATH="rspec/retried_tests-${CI_JOB_ID}.txt"
19322:16:23$ tooling/bin/create_job_metrics_file || true
19422:16:26[job-metrics] Feature disabled because CI_JOB_METRICS_ENABLED is not set to true.
19522:16:26$ rspec_section rspec_parallelized_job "--fail-fast=${RSPEC_FAIL_FAST_THRESHOLD} --tag ~quarantine --tag ~zoekt --tag ~click_house"
196 22:16:26 RSpec 29:35
19722:16:26[22:16:26] Starting rspec_parallelized_job
19822:16:18RETRY_FAILED_TESTS_IN_NEW_PROCESS: true
19922:16:26KNAPSACK_GENERATE_REPORT: true
20022:16:26FLAKY_RSPEC_GENERATE_REPORT: true
20122:16:26KNAPSACK_TEST_FILE_PATTERN: spec/{lib/gitlab/background_migration,lib/ee/gitlab/background_migration}{,/**/}*_spec.rb
20222:16:26KNAPSACK_LOG_LEVEL: debug
20322:16:26KNAPSACK_REPORT_PATH: knapsack/rspec_background_migration_pg14_single-db_6_17_13083_report.json
20422:16:26FLAKY_RSPEC_SUITE_REPORT_PATH: rspec/flaky/report-suite.json
20522:16:26FLAKY_RSPEC_REPORT_PATH: rspec/flaky/all_rspec_background_migration_pg14_single-db_6_17_13083_report.json
20622:16:26NEW_FLAKY_RSPEC_REPORT_PATH: rspec/flaky/new_rspec_background_migration_pg14_single-db_6_17_13083_report.json
20722:16:26RSPEC_SKIPPED_TESTS_REPORT_PATH: rspec/skipped_tests-8353301043.txt
20822:16:26CRYSTALBALL:
20922:16:26RSPEC_TESTS_MAPPING_ENABLED:
21022:16:26RSPEC_TESTS_FILTER_FILE:
21122:16:26Shell set options (set -o) enabled:
21222:16:26braceexpand on
21322:16:26hashall on
21422:16:26interactive-comments on
21522:16:26pipefail on
21622:16:32Knapsack report generator started!
21722:16:26/builds/gitlab-org/gitlab-foss/vendor/ruby/3.2.0/gems/io-event-1.6.5/lib/io/event/support.rb:27: warning: IO::Buffer is experimental and both the Ruby and C interface may change in the future!
21822:16:41 ██  ██  █████  ██████  ███  ██ ██ ███  ██  ██████ 
21922:16:41 ██  ██ ██   ██ ██   ██ ████  ██ ██ ████  ██ ██      
22022:16:41 ██  █  ██ ███████ ██████  ██ ██  ██ ██ ██ ██  ██ ██  ███ 
22122:16:41 ██ ███ ██ ██   ██ ██   ██ ██  ██ ██ ██ ██  ██ ██ ██  ██ 
22222:16:41  ███ ███  ██  ██ ██  ██ ██   ████ ██ ██   ████  ██████  
22322:16:41******************************************************************************
22422:16:41 Your database has a single connection, and single connections were
22522:16:41 deprecated in GitLab 15.9 https://docs.gitlab.com/ee/update/deprecations.html#single-database-connection-is-deprecated.
22622:16:41 In GitLab 17.0 and later, you must have the two main: and ci: sections in your database.yml.
22722:16:41 Please add a :ci section to your database, following these instructions:
22822:16:41 https://docs.gitlab.com/ee/install/installation.html#configure-gitlab-db-settings.
22922:16:41******************************************************************************
23022:17:07DEPRECATION WARNING: Support for Rails versions < 7.1 is deprecated and will be removed from ViewComponent 4.0.0 (ViewComponent v4 will remove support for Rails versions < 7.1 no earlier than April 1, 2025) (called from <top (required)> at /builds/gitlab-org/gitlab-foss/config/environment.rb:7)
23122:17:15warning: parser/current is loading parser/ruby32, which recognizes 3.2.6-compliant syntax, but you are running 3.2.5.
23222:17:15Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
23322:17:24/builds/gitlab-org/gitlab-foss/vendor/ruby/3.2.0/gems/io-event-1.6.5/lib/io/event/support.rb:27: warning: IO::Buffer is experimental and both the Ruby and C interface may change in the future!
23422:17:29 ██  ██  █████  ██████  ███  ██ ██ ███  ██  ██████ 
23522:17:29 ██  ██ ██   ██ ██   ██ ████  ██ ██ ████  ██ ██      
23622:17:29 ██  █  ██ ███████ ██████  ██ ██  ██ ██ ██ ██  ██ ██  ███ 
23722:17:29 ██ ███ ██ ██   ██ ██   ██ ██  ██ ██ ██ ██  ██ ██ ██  ██ 
23822:17:29  ███ ███  ██  ██ ██  ██ ██   ████ ██ ██   ████  ██████  
23922:17:29******************************************************************************
24022:17:29 Your database has a single connection, and single connections were
24122:17:29 deprecated in GitLab 15.9 https://docs.gitlab.com/ee/update/deprecations.html#single-database-connection-is-deprecated.
24222:17:29 In GitLab 17.0 and later, you must have the two main: and ci: sections in your database.yml.
24322:17:29 Please add a :ci section to your database, following these instructions:
24422:17:29 https://docs.gitlab.com/ee/install/installation.html#configure-gitlab-db-settings.
24522:17:29******************************************************************************
24622:17:35DEPRECATION WARNING: Support for Rails versions < 7.1 is deprecated and will be removed from ViewComponent 4.0.0 (ViewComponent v4 will remove support for Rails versions < 7.1 no earlier than April 1, 2025) (called from <main> at /builds/gitlab-org/gitlab-foss/config/environment.rb:7)
24722:17:51Run options: exclude {:quarantine=>true, :zoekt=>true, :click_house=>true}
24822:17:51# [RSpecRunTime] Starting RSpec timer...
24922:17:51[TEST PROF INFO] EventProf enabled (sql.active_record)
25022:17:51unknown OID 2249: failed to recognize type of 'pg_stat_file'. It will be treated as String.
25122:17:58Test environment set up in 2.499704167 seconds
25222:17:59# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/destroy_invalid_members_spec.rb. Expected to take 2 minutes 39.98 seconds.
25322:17:59Gitlab::BackgroundMigration::DestroyInvalidMembers
25422:19:58 tracks timings of queries
25522:20:17 logs IDs of deleted records
25622:20:38 removes invalid memberships but keeps valid ones
25722:21:04INFO: analyzing "public.p_ci_pipeline_variables" inheritance tree
25822:21:04INFO: analyzing "public.ci_pipeline_variables"
25922:21:04INFO: "ci_pipeline_variables": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
26022:21:06INFO: analyzing "public.p_ci_pipeline_variables" inheritance tree
26122:21:06INFO: analyzing "public.ci_pipeline_variables"
26222:21:06INFO: "ci_pipeline_variables": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
26322:21:11INFO: analyzing "public.p_ci_job_artifacts" inheritance tree
26422:21:11INFO: analyzing "public.ci_job_artifacts"
26522:21:11INFO: "ci_job_artifacts": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
26622:21:14INFO: analyzing "public.p_ci_stages" inheritance tree
26722:21:14INFO: analyzing "public.ci_stages"
26822:21:14INFO: "ci_stages": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
26922:22:15INFO: analyzing "public.p_ci_pipelines" inheritance tree
27022:22:15INFO: analyzing "public.ci_pipelines"
27122:22:15INFO: "ci_pipelines": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
27222:22:47# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/destroy_invalid_members_spec.rb. It took 4 minutes 48.1 seconds. Expected to take 2 minutes 39.98 seconds.
27322:22:47# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/backfill_merge_request_diffs_project_id_spec.rb. Expected to take 1 minute 54.68 seconds.
27422:22:47Gitlab::BackgroundMigration::BackfillMergeRequestDiffsProjectId
27522:24:37 backfills the missing project_id for the batch
27622:24:53INFO: analyzing "public.p_ci_pipeline_variables" inheritance tree
27722:24:53INFO: analyzing "public.ci_pipeline_variables"
27822:24:53INFO: "ci_pipeline_variables": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
27922:24:55INFO: analyzing "public.p_ci_pipeline_variables" inheritance tree
28022:24:55INFO: analyzing "public.ci_pipeline_variables"
28122:24:55INFO: "ci_pipeline_variables": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
28222:25:00INFO: analyzing "public.p_ci_job_artifacts" inheritance tree
28322:25:00INFO: analyzing "public.ci_job_artifacts"
28422:25:00INFO: "ci_job_artifacts": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
28522:25:03INFO: analyzing "public.p_ci_stages" inheritance tree
28622:25:03INFO: analyzing "public.ci_stages"
28722:25:03INFO: "ci_stages": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
28822:26:05INFO: analyzing "public.p_ci_pipelines" inheritance tree
28922:26:05INFO: analyzing "public.ci_pipelines"
29022:26:05INFO: "ci_pipelines": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
29122:26:37# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/backfill_merge_request_diffs_project_id_spec.rb. It took 3 minutes 50.31 seconds. Expected to take 1 minute 54.68 seconds.
29222:26:37# [RSpecRunTime] RSpec elapsed time: 8 minutes 45.62 seconds. Current RSS: ~935M. Threads: 1. load average: 0.94 0.97 0.77 1/262 411
29322:26:37.
29422:26:37# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/backfill_default_organization_owners_spec.rb. Expected to take 1 minute 37.4 seconds.
29522:26:37Gitlab::BackgroundMigration::BackfillDefaultOrganizationOwners
29622:27:55 #perform
29722:28:06 runs without error for no-op
29822:28:06# [RSpecRunTime] RSpec elapsed time: 10 minutes 14.76 seconds. Current RSS: ~933M. Threads: 1. load average: 0.92 0.96 0.79 1/263 412
29922:28:06.
30022:28:08INFO: analyzing "public.p_ci_pipeline_variables" inheritance tree
30122:28:08INFO: analyzing "public.ci_pipeline_variables"
30222:28:08INFO: "ci_pipeline_variables": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
30322:28:10INFO: analyzing "public.p_ci_pipeline_variables" inheritance tree
30422:28:10INFO: analyzing "public.ci_pipeline_variables"
30522:28:10INFO: "ci_pipeline_variables": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
30622:28:15INFO: analyzing "public.p_ci_job_artifacts" inheritance tree
30722:28:15INFO: analyzing "public.ci_job_artifacts"
30822:28:15INFO: "ci_job_artifacts": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
30922:28:19INFO: analyzing "public.p_ci_stages" inheritance tree
31022:28:19INFO: analyzing "public.ci_stages"
31122:28:19INFO: "ci_stages": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
31222:29:18INFO: analyzing "public.p_ci_pipelines" inheritance tree
31322:29:18INFO: analyzing "public.ci_pipelines"
31422:29:18INFO: "ci_pipelines": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
31522:29:50# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/backfill_default_organization_owners_spec.rb. It took 3 minutes 12.52 seconds. Expected to take 1 minute 37.4 seconds.
31622:29:50# [RSpecRunTime] RSpec elapsed time: 11 minutes 58.54 seconds. Current RSS: ~938M. Threads: 1. load average: 1.00 0.98 0.82 2/263 413
31722:29:50.
31822:29:50# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/backfill_draft_notes_project_id_spec.rb. Expected to take 1 minute 16.96 seconds.
31922:29:50Gitlab::BackgroundMigration::BackfillDraftNotesProjectId
32022:30:45 performs without error
32122:30:52 constructs a valid query
32222:31:29INFO: analyzing "public.p_ci_pipelines" inheritance tree
32322:31:29INFO: analyzing "public.ci_pipelines"
32422:31:29INFO: "ci_pipelines": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
32522:32:00# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/backfill_draft_notes_project_id_spec.rb. It took 2 minutes 9.81 seconds. Expected to take 1 minute 16.96 seconds.
32622:32:00# [RSpecRunTime] RSpec elapsed time: 14 minutes 8.38 seconds. Current RSS: ~936M. Threads: 1. load average: 0.97 0.98 0.84 1/263 414
32722:32:00.
32822:32:00# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/backfill_ml_experiment_metadata_project_id_spec.rb. Expected to take 1 minute 14.26 seconds.
32922:32:00Gitlab::BackgroundMigration::BackfillMlExperimentMetadataProjectId
33022:32:55 performs without error
33122:33:02 constructs a valid query
33222:33:38INFO: analyzing "public.p_ci_pipelines" inheritance tree
33322:33:38INFO: analyzing "public.ci_pipelines"
33422:33:38INFO: "ci_pipelines": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
33522:34:11# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/backfill_ml_experiment_metadata_project_id_spec.rb. It took 2 minutes 10.8 seconds. Expected to take 1 minute 14.26 seconds.
33622:34:11# [RSpecRunTime] RSpec elapsed time: 16 minutes 19.21 seconds. Current RSS: ~936M. Threads: 1. load average: 1.13 1.01 0.87 1/261 415
33722:34:11.
33822:34:11# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/backfill_dora_daily_metrics_project_id_spec.rb. Expected to take 1 minute 10.65 seconds.
33922:34:11Gitlab::BackgroundMigration::BackfillDoraDailyMetricsProjectId
34022:35:06 constructs a valid query
34122:35:13 performs without error
34222:35:51INFO: analyzing "public.p_ci_pipelines" inheritance tree
34322:35:51INFO: analyzing "public.ci_pipelines"
34422:35:51INFO: "ci_pipelines": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
34522:36:22# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/backfill_dora_daily_metrics_project_id_spec.rb. It took 2 minutes 11.67 seconds. Expected to take 1 minute 10.65 seconds.
34622:36:22# [RSpecRunTime] RSpec elapsed time: 18 minutes 30.91 seconds. Current RSS: ~937M. Threads: 1. load average: 1.09 1.06 0.92 1/263 416
34722:36:22.
34822:36:22# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/backfill_vulnerability_flags_project_id_spec.rb. Expected to take 59.03 seconds.
34922:36:22Gitlab::BackgroundMigration::BackfillVulnerabilityFlagsProjectId
35022:37:08 performs without error
35122:37:13 constructs a valid query
35222:37:39INFO: analyzing "public.p_ci_pipelines" inheritance tree
35322:37:39INFO: analyzing "public.ci_pipelines"
35422:37:39INFO: "ci_pipelines": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
35522:38:11# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/backfill_vulnerability_flags_project_id_spec.rb. It took 1 minute 48.16 seconds. Expected to take 59.03 seconds.
35622:38:11# [RSpecRunTime] RSpec elapsed time: 20 minutes 19.11 seconds. Current RSS: ~935M. Threads: 1. load average: 0.97 1.00 0.91 1/263 417
35722:38:11.
35822:38:11# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/backfill_vulnerability_external_issue_links_project_id_spec.rb. Expected to take 57.2 seconds.
35922:38:11Gitlab::BackgroundMigration::BackfillVulnerabilityExternalIssueLinksProjectId
36022:38:55 constructs a valid query
36122:39:01 performs without error
36222:39:26INFO: analyzing "public.p_ci_pipelines" inheritance tree
36322:39:26INFO: analyzing "public.ci_pipelines"
36422:39:26INFO: "ci_pipelines": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
36522:39:58# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/backfill_vulnerability_external_issue_links_project_id_spec.rb. It took 1 minute 47.81 seconds. Expected to take 57.2 seconds.
36622:39:58# [RSpecRunTime] RSpec elapsed time: 22 minutes 6.95 seconds. Current RSS: ~931M. Threads: 1. load average: 1.00 1.00 0.93 1/262 418
36722:39:58.
36822:39:58# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/backfill_protected_environment_approval_rules_protected_environment_project_id_spec.rb. Expected to take 47.69 seconds.
36922:39:58Gitlab::BackgroundMigration::BackfillProtectedEnvironmentApprovalRulesProtectedEnvironmentProjectId
37022:40:35 performs without error
37122:40:40 constructs a valid query
37222:40:55INFO: analyzing "public.p_ci_pipelines" inheritance tree
37322:40:55INFO: analyzing "public.ci_pipelines"
37422:40:55INFO: "ci_pipelines": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
37522:41:27# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/backfill_protected_environment_approval_rules_protected_environment_project_id_spec.rb. It took 1 minute 28.28 seconds. Expected to take 47.69 seconds.
37622:41:27# [RSpecRunTime] RSpec elapsed time: 23 minutes 35.27 seconds. Current RSS: ~934M. Threads: 1. load average: 0.93 0.96 0.92 1/262 419
37722:41:27.
37822:41:27# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/backfill_ml_candidate_params_project_id_spec.rb. Expected to take 38.92 seconds.
37922:41:27Gitlab::BackgroundMigration::BackfillMlCandidateParamsProjectId
38022:41:55 constructs a valid query
38122:41:58 performs without error
38222:42:02INFO: analyzing "public.p_ci_pipelines" inheritance tree
38322:42:02INFO: analyzing "public.ci_pipelines"
38422:42:02INFO: "ci_pipelines": scanned 0 of 0 pages, containing 0 live rows and 0 dead rows; 0 rows in sample, 0 estimated total rows
38522:42:34# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/backfill_ml_candidate_params_project_id_spec.rb. It took 1 minute 6.8 seconds. Expected to take 38.92 seconds.
38622:42:34# [RSpecRunTime] RSpec elapsed time: 24 minutes 42.1 seconds. Current RSS: ~937M. Threads: 1. load average: 1.05 0.99 0.93 1/262 420
38722:42:34.
38822:42:34# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/backfill_pages_deployment_states_project_id_spec.rb. Expected to take 25.44 seconds.
38922:42:34Gitlab::BackgroundMigration::BackfillPagesDeploymentStatesProjectId
39022:42:50 constructs a valid query
39122:42:53 performs without error
39222:43:17# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/backfill_pages_deployment_states_project_id_spec.rb. It took 42.97 seconds. Expected to take 25.44 seconds.
39322:43:17# [RSpecRunTime] RSpec elapsed time: 25 minutes 25.1 seconds. Current RSS: ~935M. Threads: 1. load average: 0.92 0.96 0.92 1/262 421
39422:43:17.
39522:43:17# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/set_correct_vulnerability_state_spec.rb. Expected to take 18.65 seconds.
39622:43:17Gitlab::BackgroundMigration::SetCorrectVulnerabilityState
39722:43:17 #filter_batch
39822:43:21 filters out vulnerabilities where dismissed_at is null
39922:43:21# [RSpecRunTime] RSpec elapsed time: 25 minutes 29.96 seconds. Current RSS: ~927M. Threads: 1. load average: 0.93 0.96 0.92 1/262 422
40022:43:21.
40122:43:21 #perform
40222:43:25 changes vulnerability state to `dismissed` when dismissed_at is not nil
40322:43:29 does not change the state when dismissed_at is nil
40422:43:29# [RSpecRunTime] RSpec elapsed time: 25 minutes 37.15 seconds. Current RSS: ~926M. Threads: 1. load average: 0.86 0.95 0.92 1/262 423
40522:43:29.
40622:43:33# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/set_correct_vulnerability_state_spec.rb. It took 16.46 seconds. Expected to take 18.65 seconds.
40722:43:33# [RSpecRunTime] RSpec elapsed time: 25 minutes 41.6 seconds. Current RSS: ~928M. Threads: 1. load average: 0.79 0.93 0.91 1/262 424
40822:43:33.
40922:43:33# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/deduplicate_lfs_objects_projects_spec.rb. Expected to take 13.78 seconds.
41022:43:33Gitlab::BackgroundMigration::DeduplicateLfsObjectsProjects
41122:43:34 #perform
41222:43:34 with duplicates
41322:43:38 deduplicates lfs_objects_projects by lfs_object_id, repository_type and project_id
41422:43:38# [RSpecRunTime] RSpec elapsed time: 25 minutes 46.7 seconds. Current RSS: ~934M. Threads: 1. load average: 0.81 0.93 0.91 1/262 425
41522:43:38.
41622:43:38 without duplicates
41722:43:42 does not delete any records
41822:43:42# [RSpecRunTime] RSpec elapsed time: 25 minutes 50.11 seconds. Current RSS: ~929M. Threads: 1. load average: 0.81 0.93 0.91 2/262 426
41922:43:42.
42022:43:48# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/deduplicate_lfs_objects_projects_spec.rb. It took 14.76 seconds. Expected to take 13.78 seconds.
42122:43:48# [RSpecRunTime] RSpec elapsed time: 25 minutes 56.39 seconds. Current RSS: ~934M. Threads: 1. load average: 0.84 0.94 0.91 1/262 427
42222:43:48.
42322:43:48# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/fix_namespace_id_for_work_item_parent_links_spec.rb. Expected to take 6.82 seconds.
42422:43:48Gitlab::BackgroundMigration::FixNamespaceIdForWorkItemParentLinks
42522:43:49 #perform
42622:43:52 fixes the namespace_id for work item parent links
42722:43:52# [RSpecRunTime] RSpec elapsed time: 26 minutes 0.57 second. Current RSS: ~932M. Threads: 1. load average: 0.85 0.94 0.91 1/262 428
42822:43:52.
42922:43:57# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/fix_namespace_id_for_work_item_parent_links_spec.rb. It took 8.67 seconds. Expected to take 6.82 seconds.
43022:43:57# [RSpecRunTime] RSpec elapsed time: 26 minutes 5.09 seconds. Current RSS: ~930M. Threads: 1. load average: 0.85 0.94 0.91 1/262 429
43122:43:57.
43222:43:57# [RSpecRunTime] Starting example group spec/lib/gitlab/background_migration/encrypt_ci_trigger_token_spec.rb. Expected to take 6.18 seconds.
43322:43:57Gitlab::BackgroundMigration::EncryptCiTriggerToken
43422:44:00 ensures all unencrypted tokens are encrypted
43522:44:05# [RSpecRunTime] Finishing example group spec/lib/gitlab/background_migration/encrypt_ci_trigger_token_spec.rb. It took 8.37 seconds. Expected to take 6.18 seconds.
43622:44:05# [RSpecRunTime] RSpec elapsed time: 26 minutes 13.5 seconds. Current RSS: ~936M. Threads: 1. load average: 0.79 0.92 0.91 1/262 430
43722:44:05.
43822:44:05[TEST PROF INFO] EventProf results for sql.active_record
43922:44:05Total time: 14:46.448 of 26:06.313 (56.59%)
44022:44:05Total events: 667388
44122:44:05Top 5 slowest suites (by time):
44222:44:05Gitlab::Backg...InvalidMembers (./spec/lib/gitlab/background_migration/destroy_invalid_members_spec.rb:6) – 02:31.331 (115069 / 3) of 04:48.096 (52.53%)
44322:44:05Gitlab::Backg...DiffsProjectId (./spec/lib/gitlab/background_migration/backfill_merge_request_diffs_project_id_spec.rb:5) – 02:18.466 (103955 / 1) of 03:50.698 (60.02%)
44422:44:05Gitlab::Backg...nizationOwners (./spec/lib/gitlab/background_migration/backfill_default_organization_owners_spec.rb:5) – 01:58.732 (92365 / 1) of 03:12.556 (61.66%)
44522:44:05Gitlab::Backg...tricsProjectId (./spec/lib/gitlab/background_migration/backfill_dora_daily_metrics_project_id_spec.rb:5) – 01:15.725 (54636 / 2) of 02:11.706 (57.5%)
44622:44:05Gitlab::Backg...adataProjectId (./spec/lib/gitlab/background_migration/backfill_ml_experiment_metadata_project_id_spec.rb:5) – 01:15.168 (54364 / 2) of 02:10.829 (57.46%)
44722:44:05Knapsack report was generated. Preview:
44822:44:05{
44922:44:05 "spec/lib/gitlab/background_migration/destroy_invalid_members_spec.rb": 159.26324221,
45022:44:05 "spec/lib/gitlab/background_migration/backfill_merge_request_diffs_project_id_spec.rb": 110.17629155400004,
45122:44:05 "spec/lib/gitlab/background_migration/backfill_default_organization_owners_spec.rb": 88.78659623400006,
45222:44:05 "spec/lib/gitlab/background_migration/backfill_draft_notes_project_id_spec.rb": 62.112849847999996,
45322:44:05 "spec/lib/gitlab/background_migration/backfill_ml_experiment_metadata_project_id_spec.rb": 62.1120903420001,
45422:44:05 "spec/lib/gitlab/background_migration/backfill_dora_daily_metrics_project_id_spec.rb": 62.34864502900018,
45522:44:05 "spec/lib/gitlab/background_migration/backfill_vulnerability_flags_project_id_spec.rb": 51.109984625999914,
45622:44:05 "spec/lib/gitlab/background_migration/backfill_vulnerability_external_issue_links_project_id_spec.rb": 50.20331572000009,
45722:44:05 "spec/lib/gitlab/background_migration/backfill_protected_environment_approval_rules_protected_environment_project_id_spec.rb": 41.32344927200006,
45822:44:05 "spec/lib/gitlab/background_migration/backfill_ml_candidate_params_project_id_spec.rb": 31.45378810299985,
45922:44:05 "spec/lib/gitlab/background_migration/backfill_pages_deployment_states_project_id_spec.rb": 19.635333366000168,
46022:44:05 "spec/lib/gitlab/background_migration/set_correct_vulnerability_state_spec.rb": 12.054261199999928,
46122:44:05 "spec/lib/gitlab/background_migration/deduplicate_lfs_objects_projects_spec.rb": 8.51899354200009,
46222:44:05 "spec/lib/gitlab/background_migration/fix_namespace_id_for_work_item_parent_links_spec.rb": 4.180245577000051,
46322:44:05 "spec/lib/gitlab/background_migration/encrypt_ci_trigger_token_spec.rb": 3.9024314189998677
46422:44:05}
46522:44:05Knapsack global time execution for tests: 12m 47s
46622:44:05Finished in 26 minutes 14 seconds (files took 1 minute 22.64 seconds to load)
46722:44:0528 examples, 0 failures
46822:44:05Randomized with seed 40722
46922:44:11RSpec exited with 0.
47022:44:11No examples to retry, congrats!
471 22:44:11 Running after_script 00:04
47222:44:12Running after script...
47322:44:12$ source scripts/utils.sh
47422:44:12$ log_disk_usage
47522:44:12*******************************************************
47622:44:12This runner currently has 80G free disk space.
47722:44:12*******************************************************
478 22:44:12 Disk usage detail 00:01
53122:44:13$ bundle exec gem list gitlab_quality-test_tooling
53222:44:14gitlab_quality-test_tooling (2.1.0)
53322:44:14$ section_start "failed-test-issues" "Report test failures" # collapsed multi-line command
534 22:44:14 Report test failures 00:00
53622:44:14$ section_start "flaky-test-issues" "Report test flakiness" # collapsed multi-line command
537 22:44:14 Report test flakiness 00:00
53922:44:14$ section_start "slow-test-issues" "Report test slowness" # collapsed multi-line command
540 22:44:14 Report test slowness 00:00
54222:44:14$ section_start "slow-test-merge-request-report-note" "Report test slowness in MR note" # collapsed multi-line command
543 22:44:14 Report test slowness in MR note 00:00
54522:44:14$ section_start "knapsack-report-issues" "Report test files close to timing out" # collapsed multi-line command
546 22:44:14 Report test files close to timing out 00:00
54822:44:14$ tooling/bin/push_job_metrics || true
54922:44:15[job-metrics] Feature disabled because CI_JOB_METRICS_ENABLED is not set to true.
550 22:44:15 Saving cache for successful job 00:01
55122:44:16Not uploading cache ruby-gems-debian-bookworm-ruby-3.2.5-gemfile-Gemfile-3 due to policy
552 22:44:16 Uploading artifacts for successful job 00:05
55322:44:16Uploading artifacts...
55422:44:16WARNING: auto_explain/: no matching files. Ensure that the artifact path is relative to the working directory (/builds/gitlab-org/gitlab-foss)
55522:44:16coverage/: found 5 matching artifact files and directories
55622:44:16crystalball/: found 2 matching artifact files and directories
55722:44:16WARNING: deprecations/: no matching files. Ensure that the artifact path is relative to the working directory (/builds/gitlab-org/gitlab-foss)
55822:44:16knapsack/: found 4 matching artifact files and directories
55922:44:16rspec/: found 16 matching artifact files and directories
56022:44:16WARNING: tmp/capybara/: no matching files. Ensure that the artifact path is relative to the working directory (/builds/gitlab-org/gitlab-foss)
56122:44:16log/*.log: found 13 matching artifact files and directories
56222:44:16WARNING: Upload request redirected location=https://gitlab.com/api/v4/jobs/8353301043/artifacts?artifact_format=zip&artifact_type=archive&expire_in=31d new-url=https://gitlab.com
56322:44:16WARNING: Retrying... context=artifacts-uploader error=request redirected
56422:44:19Uploading artifacts as "archive" to coordinator... 201 Created id=8353301043 responseStatus=201 Created token=glcbt-66
56522:44:19Uploading artifacts...
56622:44:19rspec/rspec-*.xml: found 1 matching artifact files and directories
56722:44:19WARNING: Upload request redirected location=https://gitlab.com/api/v4/jobs/8353301043/artifacts?artifact_format=gzip&artifact_type=junit&expire_in=31d new-url=https://gitlab.com
56822:44:19WARNING: Retrying... context=artifacts-uploader error=request redirected
56922:44:20Uploading artifacts as "junit" to coordinator... 201 Created id=8353301043 responseStatus=201 Created token=glcbt-66
570 22:44:21 Cleaning up project directory and file based variables 00:00
57122:44:28Job succeeded