CI: Print most recent log errors

What does this merge request do and why?

This makes inspect logfile artifacts less likely and spotting errors easier.

Contributes to #3037 (closed).

How to set up and validate locally

  1. gdk rake gdk:recent_log_errors
  2. MAX_LINES=10 gdk rake gdk:recent_log_errors
$ gdk rake gdk:recent_log_errors
Searching for errors in 45 logfiles...

-------------------- START gitlab/log/application_json.log ---------------------
{"severity":"ERROR","time":"2025-10-13T07:36:27.344Z","message":"Projects::CreateService import_project failed to be reported to collector at 127.0.0.1:9091"}
{"severity":"ERROR","time":"2025-10-13T07:36:27.344Z","message":"Projects::CreateService import_project failed to be reported to collector at 127.0.0.1:9091"}
{"severity":"ERROR","time":"2025-10-13T07:36:27.344Z","message":"Projects::CreateService import_project failed to be reported to collector at 127.0.0.1:9091"}
{"severity":"ERROR","time":"2025-10-13T07:36:27.344Z","message":"Projects::CreateService import_project failed to be reported to collector at 127.0.0.1:9091"}
{"severity":"ERROR","time":"2025-10-13T07:36:27.344Z","message":"Projects::CreateService import_project failed to be reported to collector at 127.0.0.1:9091"}

[...]


------------------- START log/rails-background-jobs/current --------------------
2025-10-13_07:49:43.52497 rails-background-jobs   : E, [2025-10-13T09:49:43.524808 #848657] ERROR -- : Exception rescued in listen-run_thread:
2025-10-13_07:49:43.52498 rails-background-jobs   : INotify::QueueOverflowError: inotify event queue has overflowed.
2025-10-13_07:49:43.52502 rails-background-jobs   : --- Thread.new ---
2025-10-13_11:43:54.02734 rails-background-jobs   :     from /home/peter/mise/installs/ruby/3.3.9/lib/ruby/3.3.0/bundled_gems.rb:69:in `block (2 levels) in replace_require'
2025-10-13_11:43:54.03047 rails-background-jobs   :     from /home/peter/mise/installs/ruby/3.3.9/lib/ruby/3.3.0/bundled_gems.rb:69:in `block (2 levels) in replace_require'

... 342 lines omitted. See log/rails-background-jobs/current for the full log.
-------------------- END log/rails-background-jobs/current ---------------------

------------------------- START log/rails-web/current --------------------------
2025-10-13_07:49:43.74026 rails-web               : INotify::QueueOverflowError: inotify event queue has overflowed.
2025-10-13_07:49:43.85544 rails-web               : INotify::QueueOverflowError: inotify event queue has overflowed.
2025-10-13_07:49:45.42415 rails-web               : INotify::QueueOverflowError: inotify event queue has overflowed.
2025-10-13_16:52:56.42633 rails-web               : {"timestamp":"2025-10-13T16:52:56.426Z","pid":848404,"message":"! Terminating timed out worker (Worker 0 failed to check in within 60 seconds): 853590"}
2025-10-14_08:03:16.16947 rails-web               : {"timestamp":"2025-10-14T08:03:16.169Z","pid":848404,"message":"! Terminating timed out worker (Worker 1 failed to check in within 60 seconds): 853594"}

... 521 lines omitted. See log/rails-web/current for the full log.
-------------------------- END log/rails-web/current ---------------------------
$ gdk rake gdk:recent_log_errors | grep START
-------------------- START gitlab/log/application_json.log ---------------------
----------------------- START gitlab/log/development.log -----------------------
--------------------- START gitlab/log/exceptions_json.log ---------------------
----------------------- START gitlab/log/gitaly-test.log -----------------------
---------------------- START gitlab/log/gitaly2-test.log -----------------------
---------------------- START gitlab/log/praefect-test.log ----------------------
------------------------- START gitlab/log/sidekiq.log -------------------------
-------------------------- START gitlab/log/test.log ---------------------------
---------------- START log/clickhouse/clickhouse-server.err.log ----------------
------------------ START log/clickhouse/clickhouse-server.log ------------------
------------------------- START log/clickhouse/current -------------------------
------------------ START log/gitlab-topology-service/current -------------------
-------------------------- START log/openbao/current ---------------------------
------------------------ START log/openbao/service.log -------------------------
------------------------- START log/postgresql/current -------------------------
--------------------- START log/praefect-gitaly-0/current ----------------------
-------------------------- START log/praefect/current --------------------------
------------------- START log/rails-background-jobs/current --------------------
------------------------- START log/rails-web/current --------------------------

Impacted categories

The following categories relate to this merge request:

Merge request checklist

  • This MR references an issue describing the change.
  • This change is backward compatible. If not, include steps to communicate to users.
  • Tests added for new functionality. If not, raise an issue to follow-up.
  • Observability added/updated (logging, metrics, tracing).
  • Documentation added/updated.
  • Announcement added for notable changes.
  • gdk doctor test added.
Edited by Peter Leitzen

Merge request reports

Loading