Log threads in rspec_run_time
What does this MR do and why?
As a part of the corrective actions mentioned in #462928 (closed), let's print the number of threads in rspec_run_time
throughout rspec execution to help us troubleshoot thread leaking problems.
Tested the output in https://gitlab.com/gitlab-org/gitlab/-/jobs/6927949980#L833 by reverting our fix to the base_exporter_spec
# [RSpecRunTime] RSpec elapsed time: 6.43 seconds. Current RSS: ~1189M. Threads: 7. Open files: 0. load average: 1.18 0.65 0.59 2/304 390
. Although later I have learned from Stan's comment that thread leaking by itself didn't cause our timeout, so one can argue that this is not entirely helpful for the issue in question. Still, I think this is a good data to keep track of in the jobs.
On a side note, I noticed that the number of threads can actually be much higher, such as in https://gitlab.com/gitlab-org/gitlab/-/jobs/6927949718:
# [RSpecRunTime] RSpec elapsed time: 22 minutes 23.81 seconds. Current RSS: ~1462M. Threads: 26. Open files: 0. load average: 1.60 1.14 0.99 1/488 8249
and no evidence suggests that they are ever being cleared at the end of the job execution so this might be interesting to watch for.
Please ignore the open files
output because I realized our image doesn't have lsof
so I will leave this out of this MR.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
Before | After |
---|---|
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.