Skip to content
Snippets Groups Projects

rspec-ee migration pg14 single-db-ci-connection 1/2

Passed Started by
Simon Tomlinson
@stomlinson
Log timestamps in UTC.
121:22:44Running with gitlab-runner 17.4.0~pre.110.g27400594 (27400594)
221:22:44 on blue-3.private.runners-manager.gitlab.com/gitlab.com/gitlab-org 6io2xoDD, system ID: s_8f94f7760223
321:22:44 feature flags: FF_NETWORK_PER_BUILD:true, FF_TIMESTAMPS:true
421:22:44Resolving secrets
5 21:22:44 Preparing the "docker+machine" executor 03:12
621:22:44Using Docker executor with image registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14:rubygems-3.5-git-2.45-lfs-2.9-chrome-123-yarn-1.22-graphicsmagick-1.3.36 ...
721:22:49Starting service registry.gitlab.com/gitlab-org/gitlab-build-images:postgres-14-pgvector-0.4.1...
821:22:49Authenticating with credentials from job payload (GitLab Registry)
921:22:49Pulling docker image registry.gitlab.com/gitlab-org/gitlab-build-images:postgres-14-pgvector-0.4.1 ...
1021:22:57Using docker image sha256:82f132c22567f8ce17cc132c14b2f9aa3eaf9bb5424d91c2d966912433f0257d for registry.gitlab.com/gitlab-org/gitlab-build-images:postgres-14-pgvector-0.4.1 with digest registry.gitlab.com/gitlab-org/gitlab-build-images@sha256:c55e1364ee4327b244a9cfa1750f19feea14fc774eaf792b725967d146eb45f5 ...
1121:23:01WARNING: Service registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12 is already created. Ignoring.
1221:23:01WARNING: Service registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12 is already created. Ignoring.
1321:23:01Starting service registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12...
1421:23:01Authenticating with credentials from job payload (GitLab Registry)
1521:23:01Pulling docker image registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12 ...
1621:23:05Using docker image sha256:5c5ba5b83d6c60cb0b225bd670b7a43be8c238cc72347d24af57e5027c3648f3 for registry.gitlab.com/gitlab-org/gitlab-build-images:redis-cluster-6.2.12 with digest registry.gitlab.com/gitlab-org/gitlab-build-images@sha256:46aceab84c42d0e819c8e7940f2f98ce9e184785cd7b22d95872024a35f52407 ...
1721:23:06Starting service redis:6.2-alpine...
1821:23:06Pulling docker image redis:6.2-alpine ...
1921:23:10Using docker image sha256:4100b5bd17430263e85ad9d8ab851959b234acaaf378e093f1f98400f4998815 for redis:6.2-alpine with digest redis@sha256:eaba718fecd1196d88533de7ba49bf903ad33664a92debb24660a922ecd9cac8 ...
2021:23:11WARNING: Service registry.gitlab.com/gitlab-org/gitlab-build-images:zoekt-ci-image-1.8 is already created. Ignoring.
2121:23:11WARNING: Service registry.gitlab.com/gitlab-org/gitlab-build-images:zoekt-ci-image-1.8 is already created. Ignoring.
2221:23:11Starting service registry.gitlab.com/gitlab-org/gitlab-build-images:zoekt-ci-image-1.8...
2321:23:11Authenticating with credentials from job payload (GitLab Registry)
2421:23:11Pulling docker image registry.gitlab.com/gitlab-org/gitlab-build-images:zoekt-ci-image-1.8 ...
2521:23:13Using docker image sha256:df819140e3df737cdc7b8d6fb73ce804b05058c790e025b5f52840956feb1734 for registry.gitlab.com/gitlab-org/gitlab-build-images:zoekt-ci-image-1.8 with digest registry.gitlab.com/gitlab-org/gitlab-build-images@sha256:7bdf349fc3dd9da64bc6581d44c0e906bffd38977ddbd2668ba34adb5dedfe38 ...
2621:23:14Starting service elasticsearch:7.17.6...
2721:23:14Pulling docker image elasticsearch:7.17.6 ...
2821:23:28Using docker image sha256:5fad10241ffd65d817ed0ddfaf6e87eee1f7dc2a7db33db1047835560ea71fda for elasticsearch:7.17.6 with digest elasticsearch@sha256:6c128de5d01c0c130a806022d6bd99b3e4c27a9af5bfc33b6b81861ae117d028 ...
2921:23:29Starting service registry.gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/model-gateway:v1.18.0...
3021:23:29Authenticating with credentials from job payload (GitLab Registry)
3121:23:29Pulling docker image registry.gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/model-gateway:v1.18.0 ...
3221:24:03Using docker image sha256:eb97d149d5b7fa6f8d1c3551036054839115ce4ae8b1896d5b4cc1563b098719 for registry.gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/model-gateway:v1.18.0 with digest registry.gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/model-gateway@sha256:d8fb36abcbce56db6ebaaff6504a26ff7345d5f463f273b9f993e0cf65c85818 ...
3321:24:06Starting service ghcr.io/berriai/litellm:main-latest...
3421:24:06Pulling docker image ghcr.io/berriai/litellm:main-latest ...
3521:24:39Using docker image sha256:006af10d9c70ad67274d5937788e42ce2cddb97f5496178c013ed1eac59ab6ae for ghcr.io/berriai/litellm:main-latest with digest ghcr.io/berriai/litellm@sha256:e7a60e9428c4946ae4b6036206bcf98b2cdb8b74e54f75793b51592d1fc005cd ...
3621:24:45Waiting for services to be up and running (timeout 30 seconds)...
3721:24:51Authenticating with credentials from job payload (GitLab Registry)
3821:24:51Pulling docker image registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14:rubygems-3.5-git-2.45-lfs-2.9-chrome-123-yarn-1.22-graphicsmagick-1.3.36 ...
3921:25:56Using docker image sha256:e83f017c5ffd177beb4bc3b4bfabde989d741e04fef90cd764896cf9b670552f for registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14:rubygems-3.5-git-2.45-lfs-2.9-chrome-123-yarn-1.22-graphicsmagick-1.3.36 with digest registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14@sha256:770feb0a974e36f8b83d22a3cb18310d0cfeae11e02f6195b1577c73ce26b398 ...
40 21:25:56 Preparing environment 00:01
4121:25:56Running on runner-6io2xodd-project-278964-concurrent-0 via runner-6io2xodd-private-1731619320-72719b9f...
42 21:25:57 Getting source from Git repository 00:01
4321:25:58Skipping Git repository setup
4421:25:58Skipping Git checkout
4521:25:58Skipping Git submodules setup
46 21:25:58 Restoring cache 00:17
4721:25:58Checking cache for ruby-gems-debian-bookworm-ruby-3.2.5-gemfile-Gemfile-20...
4821:25:59Downloading cache from https://storage.googleapis.com/gitlab-com-runners-cache/project/278964/ruby-gems-debian-bookworm-ruby-3.2.5-gemfile-Gemfile-20 ETag="19021ff660774ac886a027bf8fc2c15e"
4921:26:11Successfully extracted cache
50 21:26:15 Downloading artifacts 00:46
5121:26:16Downloading artifacts for clone-gitlab-repo (8368443091)...
5221:26:20Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8368443091 responseStatus=200 OK token=glcbt-66
5321:26:20WARNING: Part of .git directory is on the list of files to extract
5421:26:20WARNING: This may introduce unexpected problems
5521:26:34Downloading artifacts for compile-test-assets (8368443337)...
5621:26:38Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8368443337 responseStatus=200 OK token=glcbt-66
5721:26:44Downloading artifacts for detect-tests (8368443460)...
5821:26:44Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8368443460 responseStatus=200 OK token=glcbt-66
5921:26:44Downloading artifacts for retrieve-tests-metadata (8368443480)...
6021:26:45Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8368443480 responseStatus=200 OK token=glcbt-66
6121:26:45Downloading artifacts for setup-test-env (8368443411)...
6221:26:50Downloading artifacts from coordinator... ok host=storage.googleapis.com id=8368443411 responseStatus=200 OK token=glcbt-66
63 21:27:01 Executing "step_script" stage of the job script
6421:27:01Using docker image sha256:e83f017c5ffd177beb4bc3b4bfabde989d741e04fef90cd764896cf9b670552f for registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14:rubygems-3.5-git-2.45-lfs-2.9-chrome-123-yarn-1.22-graphicsmagick-1.3.36 with digest registry.gitlab.com/gitlab-org/gitlab-build-images/debian-bookworm-ruby-3.2.5-golang-1.23-rust-1.73-node-20.12-postgresql-14@sha256:770feb0a974e36f8b83d22a3cb18310d0cfeae11e02f6195b1577c73ce26b398 ...
6521:27:03$ echo $FOSS_ONLY
6621:27:03$ [ "$FOSS_ONLY" = "1" ] && rm -rf ee/ qa/spec/ee/ qa/qa/specs/features/ee/ qa/qa/ee/ qa/qa/ee.rb
6721:27:03$ export GOPATH=$CI_PROJECT_DIR/.go
6821:27:03$ mkdir -p $GOPATH
6921:27:03$ source scripts/utils.sh
7021:27:03$ log_disk_usage "true"
7121:27:03*******************************************************
7221:27:03This runner currently has 77G free disk space.
7321:27:03*******************************************************
74 21:27:03 Disk usage detail 00:00
7521:27:03df -h
7621:27:03Filesystem Size Used Avail Use% Mounted on
7721:27:03overlay 95G 18G 77G 19% /
7821:27:03tmpfs 64M 0 64M 0% /dev
7921:27:03shm 64M 0 64M 0% /dev/shm
8021:27:03/dev/sda1 95G 18G 77G 19% /cache
8121:27:03tmpfs 3.9G 0 3.9G 0% /sys/devices/virtual/dmi/id
8221:27:03du -h -d 1
8321:27:037.4M ./workhorse
8421:27:03148K ./bin
8521:27:031.2M ./.gitlab
8621:27:03177M ./.git
8721:27:034.0K ./builds
8821:27:0328M ./lib
8921:27:038.0K ./.vscode
9021:27:03560K ./storybook
9121:27:0324K ./log
9221:27:031.8M ./data
9321:27:0311M ./qa
9421:27:033.2M ./.rubocop_todo
9521:27:03856M ./vendor
9621:27:031023M ./public
9721:27:0362M ./doc
9821:27:03484K ./danger
9921:27:0332K ./rspec
10021:27:03556K ./tooling
10121:27:03146M ./ee
10221:27:031.3M ./knapsack
10321:27:03300K ./patches
10421:27:034.3M ./changelogs
10521:27:0320K ./haml_lint
10621:27:0328K ./metrics_server
10721:27:03108K ./keeps
10821:27:0336K ./shared
10921:27:0344K ./generator_templates
11021:27:031.3M ./scripts
11121:27:03763M ./tmp
11221:27:03127M ./spec
11321:27:0320K ./file_hooks
11421:27:0332K ./crystalball
11521:27:0316K ./sidekiq_cluster
11621:27:0317M ./gems
11721:27:031.2M ./fixtures
11821:27:03964K ./rubocop
11921:27:034.0K ./.go
12021:27:0333M ./db
12121:27:0312K ./.github
12221:27:03129M ./locale
12321:27:0320M ./config
12421:27:0382M ./app
12521:27:0320K ./.lefthook
12621:27:033.5G .
12721:27:03$ source scripts/prepare_build.sh
128 21:27:03 Installing gems 00:29
15921:27:32Using two connections, single database config (config/database.yml.postgresql)
16021:27:32Geo DB will be set up.
16121:27:32Embedding DB will be set up.
162 21:27:32 Setting up DBs 01:36
18321:29:08$ source ./scripts/rspec_helpers.sh
18421:29:08$ run_timed_command "gem install knapsack --no-document"
18521:29:07$ gem install knapsack --no-document
18621:29:09Successfully installed knapsack-4.0.0
18721:29:091 gem installed
18821:29:08==> 'gem install knapsack --no-document' succeeded in 1 seconds.
18921:29:09$ section_start "gitaly-test-spawn" "Spawning Gitaly"; scripts/gitaly-test-spawn; section_end "gitaly-test-spawn"
190 21:29:09 Spawning Gitaly 00:06
19421:29:15$ export RSPEC_SKIPPED_TESTS_REPORT_PATH="rspec/skipped_tests-${CI_JOB_ID}.txt"
19521:29:15$ export RSPEC_RETRIED_TESTS_REPORT_PATH="rspec/retried_tests-${CI_JOB_ID}.txt"
19621:29:15$ tooling/bin/create_job_metrics_file || true
19721:29:18[job-metrics] Creating the job metrics file for the CI/CD job.
19821:29:18$ rspec_section rspec_parallelized_job "--fail-fast=${RSPEC_FAIL_FAST_THRESHOLD} --tag ~quarantine --tag ~zoekt --tag ~click_house"
199 21:29:18 RSpec 06:24
20021:29:18[21:29:18] Starting rspec_parallelized_job
20121:29:09RETRY_FAILED_TESTS_IN_NEW_PROCESS: true
20221:29:19KNAPSACK_GENERATE_REPORT: true
20321:29:19FLAKY_RSPEC_GENERATE_REPORT: true
20421:29:19KNAPSACK_TEST_FILE_PATTERN: {ee/}spec/{migrations}{,/**/}*_spec.rb
20521:29:19KNAPSACK_LOG_LEVEL: debug
20621:29:19KNAPSACK_REPORT_PATH: knapsack/rspec-ee_migration_pg14_single-db-ci-connection_1_2_278964_report.json
20721:29:19FLAKY_RSPEC_SUITE_REPORT_PATH: rspec/flaky/report-suite.json
20821:29:19FLAKY_RSPEC_REPORT_PATH: rspec/flaky/all_rspec-ee_migration_pg14_single-db-ci-connection_1_2_278964_report.json
20921:29:19NEW_FLAKY_RSPEC_REPORT_PATH: rspec/flaky/new_rspec-ee_migration_pg14_single-db-ci-connection_1_2_278964_report.json
21021:29:19RSPEC_SKIPPED_TESTS_REPORT_PATH: rspec/skipped_tests-8368447959.txt
21121:29:19CRYSTALBALL:
21221:29:19RSPEC_TESTS_MAPPING_ENABLED:
21321:29:19RSPEC_TESTS_FILTER_FILE:
21421:29:19Shell set options (set -o) enabled:
21521:29:19braceexpand on
21621:29:19hashall on
21721:29:19interactive-comments on
21821:29:19pipefail on
21921:29:28Knapsack report generator started!
22021:29:19/builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/io-event-1.6.5/lib/io/event/support.rb:27: warning: IO::Buffer is experimental and both the Ruby and C interface may change in the future!
22121:30:53DEPRECATION WARNING: Support for Rails versions < 7.1 is deprecated and will be removed from ViewComponent 4.0.0 (ViewComponent v4 will remove support for Rails versions < 7.1 no earlier than April 1, 2025) (called from <top (required)> at /builds/gitlab-org/gitlab/config/environment.rb:7)
22221:31:05warning: parser/current is loading parser/ruby32, which recognizes 3.2.6-compliant syntax, but you are running 3.2.5.
22321:31:05Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
22421:31:20/builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/io-event-1.6.5/lib/io/event/support.rb:27: warning: IO::Buffer is experimental and both the Ruby and C interface may change in the future!
22521:31:34DEPRECATION WARNING: Support for Rails versions < 7.1 is deprecated and will be removed from ViewComponent 4.0.0 (ViewComponent v4 will remove support for Rails versions < 7.1 no earlier than April 1, 2025) (called from <main> at /builds/gitlab-org/gitlab/config/environment.rb:7)
22621:31:58Run options: exclude {:quarantine=>true, :zoekt=>true, :click_house=>true}
22721:31:58# [RSpecRunTime] Starting RSpec timer...
22821:31:58[TEST PROF INFO] EventProf enabled (sql.active_record)
22921:31:58unknown OID 2249: failed to recognize type of 'pg_stat_file'. It will be treated as String.
23021:32:06Test environment set up in 4.356800436 seconds
23121:32:07# [RSpecRunTime] Starting example group ee/spec/migrations/geo/migrate_ci_job_artifacts_to_separate_registry_spec.rb. Expected to take 24.72 seconds.
23221:32:07MigrateCiJobArtifactsToSeparateRegistry
23321:32:10 #up
23421:32:10geo: == [advisory_lock_connection] object_id: 881980, pg_backend_pid: 186
23521:32:10== 20180322062741 MigrateCiJobArtifactsToSeparateRegistry: migrating ==========
23621:32:10== 20180322062741 MigrateCiJobArtifactsToSeparateRegistry: migrated (0.0395s) =
23721:32:10geo: == [advisory_lock_connection] object_id: 881980, pg_backend_pid: 186
23821:32:14 migrates all job artifacts to its own data table
23921:32:14geo: == [advisory_lock_connection] object_id: 1585320, pg_backend_pid: 193
24021:32:14== 20180322062741 MigrateCiJobArtifactsToSeparateRegistry: migrating ==========
24121:32:14== 20180322062741 MigrateCiJobArtifactsToSeparateRegistry: migrated (0.0349s) =
24221:32:14geo: == [advisory_lock_connection] object_id: 1585320, pg_backend_pid: 193
24321:32:18 creates a new artifact with the trigger
24421:32:18geo: == [advisory_lock_connection] object_id: 2282300, pg_backend_pid: 199
24521:32:18== 20180322062741 MigrateCiJobArtifactsToSeparateRegistry: migrating ==========
24621:32:18== 20180322062741 MigrateCiJobArtifactsToSeparateRegistry: migrated (0.0372s) =
24721:32:18geo: == [advisory_lock_connection] object_id: 2282300, pg_backend_pid: 199
24821:32:22 updates a new artifact with the trigger
24921:32:22geo: == [advisory_lock_connection] object_id: 2988000, pg_backend_pid: 205
25021:32:22== 20180322062741 MigrateCiJobArtifactsToSeparateRegistry: migrating ==========
25121:32:22== 20180322062741 MigrateCiJobArtifactsToSeparateRegistry: migrated (0.0400s) =
25221:32:22geo: == [advisory_lock_connection] object_id: 2988000, pg_backend_pid: 205
25321:32:26 creates a new artifact using the next ID
25421:32:26 #down
25521:32:26geo: == [advisory_lock_connection] object_id: 3626900, pg_backend_pid: 211
25621:32:26== 20180322062741 MigrateCiJobArtifactsToSeparateRegistry: migrating ==========
25721:32:26== 20180322062741 MigrateCiJobArtifactsToSeparateRegistry: migrated (0.0568s) =
25821:32:26geo: == [advisory_lock_connection] object_id: 3626900, pg_backend_pid: 211
25921:32:31 rolls back data properly
26021:32:31# [RSpecRunTime] RSpec elapsed time: 32.88 seconds. Current RSS: ~1195M. Threads: 1. load average: 0.97 1.42 0.99 1/401 408
26121:32:31.
26221:32:42# [RSpecRunTime] Finishing example group ee/spec/migrations/geo/migrate_ci_job_artifacts_to_separate_registry_spec.rb. It took 35.17 seconds. Expected to take 24.72 seconds.
26321:32:42# [RSpecRunTime] RSpec elapsed time: 44.19 seconds. Current RSS: ~1191M. Threads: 1. load average: 0.98 1.41 0.99 1/401 409
26421:32:42.
26521:32:42# [RSpecRunTime] Starting example group ee/spec/migrations/geo/fix_state_column_in_file_registry_spec.rb. Expected to take 4.8 seconds.
26621:32:42FixStateColumnInFileRegistry
26721:32:43geo: == [advisory_lock_connection] object_id: 4702320, pg_backend_pid: 252
26821:32:43== 20210818111211 FixStateColumnInFileRegistry: migrating =====================
26921:32:43-- exec_query("SELECT COUNT(*) AS count FROM \"file_registry\" WHERE \"file_registry\".\"success\" = TRUE")
27021:32:43 -> 0.0024s
27121:32:43-- exec_query("SELECT \"file_registry\".\"id\" FROM \"file_registry\" WHERE \"file_registry\".\"success\" = TRUE ORDER BY \"file_registry\".\"id\" ASC LIMIT 1")
27221:32:43 -> 0.0020s
27321:32:43-- exec_query("SELECT \"file_registry\".\"id\" FROM \"file_registry\" WHERE \"file_registry\".\"id\" >= 23 AND \"file_registry\".\"success\" = TRUE ORDER BY \"file_registry\".\"id\" ASC LIMIT 1 OFFSET 1")
27421:32:43 -> 0.0020s
27521:32:43-- transaction(nil)
27621:32:43-- execute("UPDATE \"file_registry\" SET \"state\" = 2 WHERE \"file_registry\".\"id\" >= 23 AND \"file_registry\".\"id\" < 25 AND \"file_registry\".\"success\" = TRUE")
27721:32:43 -> 0.0033s
27821:32:43 -> 0.0047s
27921:32:43-- exec_query("SELECT \"file_registry\".\"id\" FROM \"file_registry\" WHERE \"file_registry\".\"id\" >= 25 AND \"file_registry\".\"success\" = TRUE ORDER BY \"file_registry\".\"id\" ASC LIMIT 1 OFFSET 1")
28021:32:43 -> 0.0018s
28121:32:43-- transaction(nil)
28221:32:43-- execute("UPDATE \"file_registry\" SET \"state\" = 2 WHERE \"file_registry\".\"id\" >= 25 AND \"file_registry\".\"success\" = TRUE")
28321:32:43 -> 0.0025s
28421:32:43 -> 0.0037s
28521:32:43== 20210818111211 FixStateColumnInFileRegistry: migrated (0.0191s) ============
28621:32:43geo: == [advisory_lock_connection] object_id: 4702320, pg_backend_pid: 252
28721:32:47 correctly sets registry state value
28821:32:57# [RSpecRunTime] Finishing example group ee/spec/migrations/geo/fix_state_column_in_file_registry_spec.rb. It took 15.02 seconds. Expected to take 4.8 seconds.
28921:32:57# [RSpecRunTime] RSpec elapsed time: 59.26 seconds. Current RSS: ~1192M. Threads: 1. load average: 0.90 1.37 0.99 1/401 410
29021:32:57.
29121:32:57# [RSpecRunTime] Starting example group ee/spec/migrations/geo/fix_state_column_in_lfs_object_registry_spec.rb. Expected to take 4.76 seconds.
29221:32:57FixStateColumnInLfsObjectRegistry
29321:32:58geo: == [advisory_lock_connection] object_id: 5683580, pg_backend_pid: 289
29421:32:58== 20210624160455 FixStateColumnInLfsObjectRegistry: migrating ================
29521:32:58-- exec_query("SELECT COUNT(*) AS count FROM \"lfs_object_registry\" WHERE \"lfs_object_registry\".\"success\" = TRUE")
29621:32:58 -> 0.0024s
29721:32:58-- exec_query("SELECT \"lfs_object_registry\".\"id\" FROM \"lfs_object_registry\" WHERE \"lfs_object_registry\".\"success\" = TRUE ORDER BY \"lfs_object_registry\".\"id\" ASC LIMIT 1")
29821:32:58 -> 0.0020s
29921:32:58-- exec_query("SELECT \"lfs_object_registry\".\"id\" FROM \"lfs_object_registry\" WHERE \"lfs_object_registry\".\"id\" >= 2 AND \"lfs_object_registry\".\"success\" = TRUE ORDER BY \"lfs_object_registry\".\"id\" ASC LIMIT 1 OFFSET 1")
30021:32:58 -> 0.0020s
30121:32:58-- transaction(nil)
30221:32:58-- execute("UPDATE \"lfs_object_registry\" SET \"state\" = 2 WHERE \"lfs_object_registry\".\"id\" >= 2 AND \"lfs_object_registry\".\"id\" < 4 AND \"lfs_object_registry\".\"success\" = TRUE")
30321:32:58 -> 0.0029s
30421:32:58 -> 0.0044s
30521:32:58-- exec_query("SELECT \"lfs_object_registry\".\"id\" FROM \"lfs_object_registry\" WHERE \"lfs_object_registry\".\"id\" >= 4 AND \"lfs_object_registry\".\"success\" = TRUE ORDER BY \"lfs_object_registry\".\"id\" ASC LIMIT 1 OFFSET 1")
30621:32:58 -> 0.0020s
30721:32:58-- transaction(nil)
30821:32:58-- execute("UPDATE \"lfs_object_registry\" SET \"state\" = 2 WHERE \"lfs_object_registry\".\"id\" >= 4 AND \"lfs_object_registry\".\"success\" = TRUE")
30921:32:58 -> 0.0027s
31021:32:58 -> 0.0040s
31121:32:58== 20210624160455 FixStateColumnInLfsObjectRegistry: migrated (0.0189s) =======
31221:32:58geo: == [advisory_lock_connection] object_id: 5683580, pg_backend_pid: 289
31321:33:02 correctly sets registry state value
31421:33:12# [RSpecRunTime] Finishing example group ee/spec/migrations/geo/fix_state_column_in_lfs_object_registry_spec.rb. It took 14.67 seconds. Expected to take 4.76 seconds.
31521:33:12# [RSpecRunTime] RSpec elapsed time: 1 minute 13.99 seconds. Current RSS: ~1195M. Threads: 1. load average: 0.98 1.37 0.99 1/401 411
31621:33:12.
31721:33:12# [RSpecRunTime] Starting example group ee/spec/migrations/20230901120000_queue_update_workspaces_config_version_spec.rb. Missing expected duration from Knapsack report for ee/spec/migrations/20230901120000_queue_update_workspaces_config_version_spec.rb.
31821:33:12QueueUpdateWorkspacesConfigVersion
31921:33:12 schedules a new batched migration (PENDING: Skipping because migration QueueUpdateWorkspacesConfigVersion is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
32021:33:12# [RSpecRunTime] Finishing example group ee/spec/migrations/20230901120000_queue_update_workspaces_config_version_spec.rb. It took 0.02 second. Missing expected duration from Knapsack report for ee/spec/migrations/20230901120000_queue_update_workspaces_config_version_spec.rb.
32121:33:12# [RSpecRunTime] Starting example group ee/spec/migrations/20240104085448_queue_update_workspaces_config_version3_spec.rb. Missing expected duration from Knapsack report for ee/spec/migrations/20240104085448_queue_update_workspaces_config_version3_spec.rb.
32221:33:12QueueUpdateWorkspacesConfigVersion3
32321:33:12 schedules a new batched migration (PENDING: Skipping because migration QueueUpdateWorkspacesConfigVersion3 is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
32421:33:12# [RSpecRunTime] Finishing example group ee/spec/migrations/20240104085448_queue_update_workspaces_config_version3_spec.rb. It took 0.05 second. Missing expected duration from Knapsack report for ee/spec/migrations/20240104085448_queue_update_workspaces_config_version3_spec.rb.
32521:33:12# [RSpecRunTime] Starting example group ee/spec/migrations/geo/resync_direct_upload_job_artifact_registry_spec.rb. Missing expected duration from Knapsack report for ee/spec/migrations/geo/resync_direct_upload_job_artifact_registry_spec.rb.
32621:33:12ResyncDirectUploadJobArtifactRegistry
32721:33:12 #up
32821:33:12 when direct upload is enabled for job artifacts
32921:33:12 when the site is a Geo secondary
33021:33:12 when sync_object_storage is enabled for the current Geo site
33121:33:12 with job artifact registry rows
33221:33:12 marks pending if synced between 2023-06-22 and 2024-02-03 (PENDING: Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
33321:33:12 does not update if not synced (PENDING: Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
33421:33:12 does not update if synced before 2023-06-22 (PENDING: Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
33521:33:12 does not update if synced after 2024-02-03 (PENDING: Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
33621:33:12 when sync_object_storage is disabled for the current Geo site
33721:33:12 does not update job artifact registry (PENDING: Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
33821:33:12 when the site is a Geo primary
33921:33:12 does not update job artifact registry (PENDING: Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
34021:33:12 when direct upload is disabled for job artifacts
34121:33:12 does not update job artifact registry (PENDING: Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
34221:33:12 when object storage is disabled for job artifacts
34321:33:12 does not update job artifact registry (PENDING: Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.)
34421:33:12# [RSpecRunTime] Finishing example group ee/spec/migrations/geo/resync_direct_upload_job_artifact_registry_spec.rb. It took 0.02 second. Missing expected duration from Knapsack report for ee/spec/migrations/geo/resync_direct_upload_job_artifact_registry_spec.rb.
34521:33:15auto_explain log contains 799 entries for main, writing to /builds/gitlab-org/gitlab/auto_explain/rspec-ee-migration-pg14-single-db-ci-connection-1-2.344.main.ndjson.gz
34621:33:15took 2.918513219
34721:33:16auto_explain log contains 800 entries for ci, writing to /builds/gitlab-org/gitlab/auto_explain/rspec-ee-migration-pg14-single-db-ci-connection-1-2.344.ci.ndjson.gz
34821:33:16took 1.54479011
34921:33:16[TEST PROF INFO] EventProf results for sql.active_record
35021:33:16Total time: 00:35.978 of 01:05.116 (55.25%)
35121:33:16Total events: 43652
35221:33:16Top 5 slowest suites (by time):
35321:33:16MigrateCiJobA...parateRegistry (./ee/spec/migrations/geo/migrate_ci_job_artifacts_to_separate_registry_spec.rb:6) – 00:18.421 (22452 / 5) of 00:35.226 (52.3%)
35421:33:16FixStateColumnInFileRegistry (./ee/spec/migrations/geo/fix_state_column_in_file_registry_spec.rb:7) – 00:09.006 (10575 / 1) of 00:15.070 (59.76%)
35521:33:16FixStateColum...ObjectRegistry (./ee/spec/migrations/geo/fix_state_column_in_lfs_object_registry_spec.rb:7) – 00:08.534 (10601 / 1) of 00:14.725 (57.96%)
35621:33:16QueueUpdateWo...sConfigVersion (./ee/spec/migrations/20230901120000_queue_update_workspaces_config_version_spec.rb:6) – 00:00.005 (8 / 1) of 00:00.022 (23.94%)
35721:33:16QueueUpdateWo...ConfigVersion3 (./ee/spec/migrations/20240104085448_queue_update_workspaces_config_version3_spec.rb:6) – 00:00.005 (8 / 1) of 00:00.050 (10.15%)
35821:33:16Knapsack report was generated. Preview:
35921:33:16{
36021:33:16 "ee/spec/migrations/geo/migrate_ci_job_artifacts_to_separate_registry_spec.rb": 24.390303711,
36121:33:16 "ee/spec/migrations/geo/fix_state_column_in_file_registry_spec.rb": 5.226750314000014,
36221:33:16 "ee/spec/migrations/geo/fix_state_column_in_lfs_object_registry_spec.rb": 5.034278290999964
36321:33:16}
36421:33:16Knapsack global time execution for tests: 34s
36521:33:16Pending: (Failures listed here are expected and do not affect your suite's status)
36621:33:16 1) QueueUpdateWorkspacesConfigVersion schedules a new batched migration
36721:33:16 # Skipping because migration QueueUpdateWorkspacesConfigVersion is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
36821:33:16 # ./ee/spec/migrations/20230901120000_queue_update_workspaces_config_version_spec.rb:9
36921:33:16 2) QueueUpdateWorkspacesConfigVersion3 schedules a new batched migration
37021:33:16 # Skipping because migration QueueUpdateWorkspacesConfigVersion3 is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
37121:33:16 # ./ee/spec/migrations/20240104085448_queue_update_workspaces_config_version3_spec.rb:9
37221:33:16 3) ResyncDirectUploadJobArtifactRegistry#up when direct upload is enabled for job artifacts when the site is a Geo secondary when sync_object_storage is enabled for the current Geo site with job artifact registry rows marks pending if synced between 2023-06-22 and 2024-02-03
37321:33:16 # Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
37421:33:16 # ./ee/spec/migrations/geo/resync_direct_upload_job_artifact_registry_spec.rb:34
37521:33:16 4) ResyncDirectUploadJobArtifactRegistry#up when direct upload is enabled for job artifacts when the site is a Geo secondary when sync_object_storage is enabled for the current Geo site with job artifact registry rows does not update if not synced
37621:33:16 # Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
37721:33:16 # ./ee/spec/migrations/geo/resync_direct_upload_job_artifact_registry_spec.rb:44
37821:33:16 5) ResyncDirectUploadJobArtifactRegistry#up when direct upload is enabled for job artifacts when the site is a Geo secondary when sync_object_storage is enabled for the current Geo site with job artifact registry rows does not update if synced before 2023-06-22
37921:33:16 # Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
38021:33:16 # ./ee/spec/migrations/geo/resync_direct_upload_job_artifact_registry_spec.rb:54
38121:33:16 6) ResyncDirectUploadJobArtifactRegistry#up when direct upload is enabled for job artifacts when the site is a Geo secondary when sync_object_storage is enabled for the current Geo site with job artifact registry rows does not update if synced after 2024-02-03
38221:33:16 # Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
38321:33:16 # ./ee/spec/migrations/geo/resync_direct_upload_job_artifact_registry_spec.rb:63
38421:33:16 7) ResyncDirectUploadJobArtifactRegistry#up when direct upload is enabled for job artifacts when the site is a Geo secondary when sync_object_storage is disabled for the current Geo site does not update job artifact registry
38521:33:16 # Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
38621:33:16 # ./ee/spec/migrations/geo/resync_direct_upload_job_artifact_registry_spec.rb:77
38721:33:16 8) ResyncDirectUploadJobArtifactRegistry#up when direct upload is enabled for job artifacts when the site is a Geo primary does not update job artifact registry
38821:33:16 # Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
38921:33:16 # ./ee/spec/migrations/geo/resync_direct_upload_job_artifact_registry_spec.rb:96
39021:33:16 9) ResyncDirectUploadJobArtifactRegistry#up when direct upload is disabled for job artifacts does not update job artifact registry
39121:33:16 # Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
39221:33:16 # ./ee/spec/migrations/geo/resync_direct_upload_job_artifact_registry_spec.rb:117
39321:33:16 10) ResyncDirectUploadJobArtifactRegistry#up when object storage is disabled for job artifacts does not update job artifact registry
39421:33:16 # Skipping because migration ResyncDirectUploadJobArtifactRegistry is outside the test window. Run with `RUN_ALL_MIGRATION_TESTS=1 bundle exec rspec <spec> if you need to run this spec.
39521:33:16 # ./ee/spec/migrations/geo/resync_direct_upload_job_artifact_registry_spec.rb:137
39621:33:16Finished in 1 minute 18.72 seconds (files took 2 minutes 35.7 seconds to load)
39721:33:1617 examples, 0 failures, 10 pending
39821:33:23[TEST PROF INFO] Time spent in factories: 00:00.092 (0.07% of total time)
39921:33:24RSpec exited with 0.
40021:33:24No examples to retry, congrats!
401 21:33:25 Running after_script 00:09
40221:33:26Running after script...
40321:33:26$ source scripts/utils.sh
40421:33:26$ log_disk_usage
40521:33:26*******************************************************
40621:33:26This runner currently has 76G free disk space.
40721:33:26*******************************************************
408 21:33:26 Disk usage detail 00:00
46421:33:26$ bundle exec gem list gitlab_quality-test_tooling
46521:33:28gitlab_quality-test_tooling (2.1.0)
46621:33:28$ section_start "failed-test-issues" "Report test failures" # collapsed multi-line command
467 21:33:28 Report test failures 00:00
46921:33:28$ section_start "flaky-test-issues" "Report test flakiness" # collapsed multi-line command
470 21:33:28 Report test flakiness 00:00
47221:33:28$ section_start "slow-test-issues" "Report test slowness" # collapsed multi-line command
473 21:33:28 Report test slowness 00:00
47521:33:28$ section_start "slow-test-merge-request-report-note" "Report test slowness in MR note" # collapsed multi-line command
476 21:33:28 Report test slowness in MR note 00:03
47921:33:31$ section_start "knapsack-report-issues" "Report test files close to timing out" # collapsed multi-line command
480 21:33:31 Report test files close to timing out 00:02
48321:33:33$ tooling/bin/push_job_metrics || true
48421:33:34[job-metrics] Pushing job metrics file for the CI/CD job.
48521:33:34[job-metrics] Pushed 4 CI job metric entries to InfluxDB.
486 21:33:34 Saving cache for successful job 00:01
48721:33:35Not uploading cache ruby-gems-debian-bookworm-ruby-3.2.5-gemfile-Gemfile-20 due to policy
488 21:33:35 Uploading artifacts for successful job 00:06
48921:33:36Uploading artifacts...
49021:33:36auto_explain/: found 3 matching artifact files and directories
49121:33:36coverage/: found 5 matching artifact files and directories
49221:33:36crystalball/: found 2 matching artifact files and directories
49321:33:36WARNING: deprecations/: no matching files. Ensure that the artifact path is relative to the working directory (/builds/gitlab-org/gitlab)
49421:33:36knapsack/: found 4 matching artifact files and directories
49521:33:36rspec/: found 16 matching artifact files and directories
49621:33:36WARNING: tmp/capybara/: no matching files. Ensure that the artifact path is relative to the working directory (/builds/gitlab-org/gitlab)
49721:33:36log/*.log: found 13 matching artifact files and directories
49821:33:36WARNING: Upload request redirected location=https://gitlab.com/api/v4/jobs/8368447959/artifacts?artifact_format=zip&artifact_type=archive&expire_in=31d new-url=https://gitlab.com
49921:33:36WARNING: Retrying... context=artifacts-uploader error=request redirected
50021:33:39Uploading artifacts as "archive" to coordinator... 201 Created id=8368447959 responseStatus=201 Created token=glcbt-66
50121:33:39Uploading artifacts...
50221:33:39rspec/rspec-*.xml: found 1 matching artifact files and directories
50321:33:39WARNING: Upload request redirected location=https://gitlab.com/api/v4/jobs/8368447959/artifacts?artifact_format=gzip&artifact_type=junit&expire_in=31d new-url=https://gitlab.com
50421:33:39WARNING: Retrying... context=artifacts-uploader error=request redirected
50521:33:41Uploading artifacts as "junit" to coordinator... 201 Created id=8368447959 responseStatus=201 Created token=glcbt-66
506 21:33:41 Cleaning up project directory and file based variables 00:00
50721:33:49Job succeeded