Surface per-plan errors and a high-level run summary

!infra-branch: per-plan-error-reason-and-logs

Previously a useful top-level result.summary was produced only when a plan crashed (an exception was raised). When tmt ran to completion but reported error/failed plan results, the request finished with no summary and no per-plan explanation. This reports failures at two levels.

Per-plan (xUnit <testsuite>):

  • Crashed plans (entry state == ERROR) carry an <error> with a stage-aware reason (e.g. guest setup failed: ...) plus a link to the log of the failing guest-setup phase.
  • Plans where tmt completed but reported an error result (result == ERROR, state stays OK) carry <error message="tmt errored out"/> linked to the tmt-log.
  • A crashed plan always carries an <error>; an empty exception string falls back to a clear message instead of being mislabeled.

Top level (result.summary):

  • Build a short summary such as 3 plans failed, 2 plans errored out from the per-plan results, reported when the pipeline completes without raising.
  • The crash path now raises the same count-based summary via summarize_schedule_results(), so the top-level summary has one consistent format regardless of crash vs. clean completion; per-plan detail lives in the <error> elements.

Supporting changes:

  • Add error_message and error_guest_setup_stage to TestScheduleEntry, recorded in _on_job_error.
  • Add Results.summary, wired into testing-farm-request-state-reporter (_get_summary falls back to it when there is no failure).
  • Add error, error_log_name and error_log_href to TestSuite, serialized as <error message=… name=… href=…> at the plan level.
  • Remove the unused plan-level <failure> element; only <error> is emitted for a suite.

Testing Farm CI sanity test runs against infrastructure branch per-plan-error-reason-and-logs (infrastructure!1486 (merged)), which updates the tmt-multihost-invalid-provision-how expected results.xml for the new <error> element.

Assisted-by: Claude Code Signed-off-by: Miroslav Vadkerti mvadkert@redhat.com

Edited by Miroslav Vadkerti

Merge request reports

Loading