Long standing failure/slow test issues cause automation to fail
If an issue that tooling uses as medium to store history for a particular test exists long enough and accumulates enough updates, it will start failing the tooling with EOFError when trying to fetch notes via api. This is probably a problem with the root cause that gitlab issues are simply not designed to keep track of history long term.
Example job with such failure: https://gitlab.com/gitlab-org/gitlab/-/jobs/11002514703#L5000
[Aug 13 2025 07:58:51 UTC (Gitlab QA)] INFO -- => Reporting slow test to existing issue: https://gitlab.com/gitlab-org/gitlab/-/issues/503476
Posting Slack message to channel: e2e-run-master
bundler: failed to load command: slow-test-issues (/builds/gitlab-org/gitlab/vendor/ruby/3.2.0/bin/slow-test-issues)
/builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:237:in `rbuf_fill': end of file reached (EOFError)
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:199:in `readuntil'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-protocol-0.2.2/lib/net/protocol.rb:209:in `readline'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-http-0.6.0/lib/net/http/response.rb:625:in `read_chunked'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-http-0.6.0/lib/net/http/response.rb:595:in `block in read_body_0'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-http-0.6.0/lib/net/http/response.rb:570:in `inflater'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-http-0.6.0/lib/net/http/response.rb:593:in `read_body_0'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-http-0.6.0/lib/net/http/response.rb:363:in `read_body'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-http-0.6.0/lib/net/http/response.rb:401:in `body'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-http-0.6.0/lib/net/http/response.rb:321:in `reading_body'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-http-0.6.0/lib/net/http.rb:2430:in `transport_request'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-http-0.6.0/lib/net/http.rb:2384:in `request'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-http-0.6.0/lib/net/http.rb:2377:in `block in request'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-http-0.6.0/lib/net/http.rb:1632:in `start'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/net-http-0.6.0/lib/net/http.rb:2375:in `request'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/httparty-0.23.1/lib/httparty/request.rb:157:in `perform'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/httparty-0.23.1/lib/httparty.rb:621:in `perform_request'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/httparty-0.23.1/lib/httparty.rb:531:in `get'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab-4.19.0/lib/gitlab/request.rb:54:in `block (2 levels) in <class:Request>'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:91:in `next_page'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:40:in `each_page'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:50:in `each'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:50:in `each'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:50:in `each'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:50:in `each'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:50:in `to_a'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab-4.19.0/lib/gitlab/paginated_response.rb:50:in `auto_paginate'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/gitlab_client/issues_client.rb:71:in `block in find_issue_notes'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/gitlab_client/gitlab_client.rb:20:in `handle_gitlab_client_exceptions'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/gitlab_client/issues_client.rb:70:in `find_issue_notes'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/report/health_problem_reporter.rb:224:in `existing_reports_note'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/report/health_problem_reporter.rb:179:in `add_report_to_issue'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/report/health_problem_reporter.rb:174:in `block in update_reports'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/report/health_problem_reporter.rb:172:in `each'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/report/health_problem_reporter.rb:172:in `update_reports'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/report/health_problem_reporter.rb:147:in `create_or_update_test_health_issues'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/report/health_problem_reporter.rb:123:in `block in process_issues_for_tests'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/report/health_problem_reporter.rb:120:in `each'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/report/health_problem_reporter.rb:120:in `process_issues_for_tests'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/report/health_problem_reporter.rb:107:in `process_test_results'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/report/health_problem_reporter.rb:99:in `block in run!'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/test_results/builder.rb:28:in `block in test_results_per_file'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/test_results/builder.rb:15:in `each'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/test_results/builder.rb:15:in `test_results_per_file'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/report/health_problem_reporter.rb:96:in `run!'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/lib/gitlab_quality/test_tooling/report/report_as_issue.rb:23:in `invoke!'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/gitlab_quality-test_tooling-2.18.0/exe/slow-test-issues:50:in `<top (required)>'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/bin/slow-test-issues:25:in `load'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/bin/slow-test-issues:25:in `<top (required)>'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/bundler-2.7.1/lib/bundler/cli/exec.rb:59:in `load'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/bundler-2.7.1/lib/bundler/cli/exec.rb:59:in `kernel_load'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/bundler-2.7.1/lib/bundler/cli/exec.rb:23:in `run'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/bundler-2.7.1/lib/bundler/cli.rb:451:in `exec'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/bundler-2.7.1/lib/bundler/vendor/thor/lib/thor/command.rb:28:in `run'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/bundler-2.7.1/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/bundler-2.7.1/lib/bundler/vendor/thor/lib/thor.rb:538:in `dispatch'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/bundler-2.7.1/lib/bundler/cli.rb:35:in `dispatch'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/bundler-2.7.1/lib/bundler/vendor/thor/lib/thor/base.rb:584:in `start'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/bundler-2.7.1/lib/bundler/cli.rb:29:in `start'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/bundler-2.7.1/exe/bundle:28:in `block in <top (required)>'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/bundler-2.7.1/lib/bundler/friendly_errors.rb:118:in `with_friendly_errors'
from /builds/gitlab-org/gitlab/vendor/ruby/3.2.0/gems/bundler-2.7.1/exe/bundle:20:in `<top (required)>'
from /usr/local/bin/bundle:25:in `load'
from /usr/local/bin/bundle:25:in `<main>'