`update` verify job timing out in CI
Overview
Multiple update verify jobs are failing with timeouts, blocking MRs from getting merged.
Example stack trace:
Job #4776347492 failed for 996b86b5:
--------------------------------------------------------------------------------
Ensuring necessary data services are running
--------------------------------------------------------------------------------
gdk start rails-migration-dependencies
ok: run: /home/gdk/gdk/services/postgresql: (pid 1081) 1565s, normally down
ok: run: /home/gdk/gdk/services/praefect: (pid 1301) 1564s, normally down
ok: run: /home/gdk/gdk/services/praefect-gitaly-0: (pid 1300) 1564s, normally down
ok: run: /home/gdk/gdk/services/redis: (pid 1188) 1565s, normally down
rake gitlab-db-migrate
--------------------------------------------------------------------------------
Processing gitlab rails DB migrations
--------------------------------------------------------------------------------
DEBUG: command=[bundle exec rake db:migrate db:test:prepare], opts=[{:chdir=>#<Pathname:/home/gdk/gdk/gitlab>}], display_output=[true], retry_attempts=[0]
ERROR: Failed to cleanup volumes
ERROR: Job failed: execution took longer than 1h30m0s seconds
Example jobs:
- https://gitlab.com/gitlab-org/gitlab-development-kit/-/jobs/4776347492
- https://gitlab.com/gitlab-org/gitlab-development-kit/-/jobs/4775326797
- https://gitlab.com/gitlab-org/gitlab-development-kit/-/jobs/4775027586
- https://gitlab.com/gitlab-org/gitlab-development-kit/-/jobs/4774562864
- https://gitlab.com/gitlab-org/gitlab-development-kit/-/jobs/4773463853
- https://gitlab.com/gitlab-org/gitlab-development-kit/-/jobs/4774633727
- https://gitlab.com/gitlab-org/gitlab-development-kit/-/jobs/4770806806
- https://gitlab.com/gitlab-org/gitlab-development-kit/-/jobs/4770131093
- https://gitlab.com/gitlab-org/gitlab-development-kit/-/jobs/4769184567
Impacted categories
The following categories relate to this issue:
-
gdk-reliability - e.g. When a GDK action fails to complete. -
gdk-usability - e.g. Improvements or suggestions around how the GDK functions. -
gdk-performance - e.g. When a GDK action is slow or times out.
Steps to replicate (optional)
Proposal (optional)
Environment (optional)
- Operating system name:
<!-- output of `uname -a` command --> - Architecture:
<!-- output of `arch` command --> - The contents of your
gdk.yml(if any) - Ruby version:
<!-- output of `ruby --version` command --> - GDK version:
<!-- output of `git rev-parse --short HEAD` command -->