Investigate more than usual job timeouts since May 17
There are a lot of examples, but I'm linking the ones I came across today:
date | job | branch | relevant data from job log | runner info |
---|---|---|---|---|
May 17, 2024 at 7:40:14 PM UTC | rspec unit pg14 29/32 | master | see comment | Running with gitlab-runner 17.0.0~pre.88.g761ae5dd (761ae5dd) on green-4.private.runners-manager.gitlab.com/gitlab.com/gitlab-org rpvz2FF9, system ID: s_d704414ba02a |
May 19, 2024 at 1:49:55 AM UTC | rspec unit pg16 29/32 | master | see comment | Running with gitlab-runner 17.0.0~pre.88.g761ae5dd (761ae5dd) on green-4.private.runners-manager.gitlab.com/gitlab.com/gitlab-org rpvz2FF9, system ID: s_d704414ba02a |
May 21, 2024 at 3:44:29 PM UTC | rspec unit pg14 20/32 |
cherry-pick-734e8efc-to-16-8 (high priority backport MR) |
see comment | Running with gitlab-runner 17.0.0~pre.88.g761ae5dd (761ae5dd) on green-1.private.runners-manager.gitlab.com/gitlab.com/gitlab-org 4bq1s9yM, system ID: s_9d15f3e4e7ae |
May 21, 2024 at 4:34:08 PM UTC | rspec unit pg14 18/32 | only-check-added-ff-files-with-danger |
see comment | Running with gitlab-runner 17.0.0~pre.88.g761ae5dd (761ae5dd) on green-3.private.runners-manager.gitlab.com/gitlab.com/gitlab-org QuQPoFsC, system ID: s_fc023362bf2f |
May 21, 2024 at 7:23:17 PM UTC | rspec unit pg14 18/32 | smh-reapply-tx-jobs |
See comment | Running with gitlab-runner 17.0.0~pre.88.g761ae5dd (761ae5dd) on green-4.private.runners-manager.gitlab.com/gitlab.com/gitlab-org rpvz2FF9, system ID: s_d704414ba02a |
May 21, 2024 at 8:02:08 PM UTC | rspec unit pg14 18/32 |
j_lar/move_spec_to_ee (MR to fix priority1 master:broken ) |
see 2nd section of this comment | Running with gitlab-runner 17.0.0~pre.88.g761ae5dd (761ae5dd)on green-2.private.runners-manager.gitlab.com/gitlab.com/gitlab-org GaSD-S1F, system ID: s_5651e5b5643b |
Investigation
I have only managed to collect data at this point and perform preliminary data analysis. See Data Analysis section for findings so far
Edited by Jennifer Li