Only return failure notifications in JSON formatter
Context
In !124438 (merged), we decided to stop an RSpec process early if we detect PG::QueryCanceled errors. When stopping the RSpec process early, there is an error that is being displayed in JSONFormatter (see example):
/builds/gitlab-org/gitlab/spec/support/formatters/json_formatter.rb:30:in `block (3 levels) in stop': undefined method `message_lines' for #<struct RSpec::Core::Notifications::ExampleNotification example=#<RSpec::Core::Example "shows a tab for All jobs and count">> (NoMethodError)
from /builds/gitlab-org/gitlab/spec/support/formatters/json_formatter.rb:26:in `map'
from /builds/gitlab-org/gitlab/spec/support/formatters/json_formatter.rb:26:in `block (2 levels) in stop'
from <internal:kernel>:90:in `tap'
from /builds/gitlab-org/gitlab/spec/support/formatters/json_formatter.rb:22:in `block in stop'
from /builds/gitlab-org/gitlab/spec/support/formatters/json_formatter.rb:21:in `map'
from /builds/gitlab-org/gitlab/spec/support/formatters/json_formatter.rb:21:in `stop'
from /builds/gitlab-org/gitlab/vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:209:in `block in notify'
from /builds/gitlab-org/gitlab/vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:208:in `each'
from /builds/gitlab-org/gitlab/vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:208:in `notify'
from /builds/gitlab-org/gitlab/vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:202:in `stop'
from /builds/gitlab-org/gitlab/vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:175:in `block in finish'
from /builds/gitlab-org/gitlab/vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:194:in `close_after'
from /builds/gitlab-org/gitlab/vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:174:in `finish'
from /builds/gitlab-org/gitlab/vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:76:in `ensure in report'
from /builds/gitlab-org/gitlab/vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/reporter.rb:76:in `report'
from /builds/gitlab-org/gitlab/vendor/ruby/3.0.0/gems/rspec-core-3.12.0/lib/rspec/core/runner.rb:115:in `run_specs'
See this slack discussion (internal link).
What does this MR do and why?
As advised in Slack, we filter the RSpec Notifications from all notifications to only failure notifications.
Does it work?
I added a test commit: !124596 (03ba5378).
This will trigger the fail-early mechanism introduced in !124438 (merged), and indeed, the original error is gone
* Listening on unix:///tmp/20230626-324-1au18r
********************************************************************************************
********************************************************************************************
********************************************************************************************
* *
* We have detected a PG::QueryCanceled error in the specs, so we're failing early. *
* Please retry this job. *
* *
* See https://gitlab.com/gitlab-org/gitlab/-/issues/402915 for more info. *
* *
********************************************************************************************
********************************************************************************************
********************************************************************************************
Finished in 29.91 seconds (files took 1 minute 6.62 seconds to load)
1 example, 0 failures, 1 error occurred outside of examples
[TEST PROF INFO] Time spent in factories: 00:01.201 (2.04% of total time)
Stopped processing SimpleCov as a previous error not related to SimpleCov has been detected
Failed to write to log, write log/workhorse-test.log: file already closed
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by David Dieulivol