logdetective : Implement helper for Log Detective API

Adds a log-detective gluetool module that automatically analyzes failed tmt tests and stores AI explanations in their test results.

  • Log Detective integration: Adds the log-detective module to request AI failure analysis through the /analyze API analyzing failing tmt testcases separately, including failed, error, and needs_inspection outcomes.
  • Plan-level fallback: Analyzes eligible plans without failing testcases, including infrastructure errors where the test result remains undefined. With synthetic error text when suitable log URLs are unavailable so that Log Detective has something to work with. This is a stop gap, until specialized Log Detective agent for Testing Farm is finished.
  • Artifact selection: Prioritizes testcase output and journal logs, filters auxiliary files for text content with a configurable artifact limit, defaulting to 15.
  • Artifact availability checks: Checks HTTP(S) log URLs before submission, with streamed GET fallback when HEAD is unsupported. Submits reachable logs by URL. This may be an overkill, if we could guarantee availability without it, this check could be removed.
  • Publication before analysis: Stops parallel archiving and publishes logs before final analysis. Redacts temporary copies so open log writers can continue writing, while preserving configured destinations and directory layout.
  • Final report publication: Uploads the completed results.xml, including AI summaries, during normal archive teardown.
  • Structured analysis results: Adds explanations, suggested solutions, a no-issue-found flag, and relevant snippets with source filenames and line numbers to testcase or suite elements. This format may somewhat change with move to Log Detective Testing Farm agent.
  • Analysis caching: Retains summaries across report regeneration and keeps results associated with their schedule entry and testcase.
  • Authentication and redaction: Registers optional API tokens during execute(). Redacts inline error text, artifact names, and metadata; skips authenticated requests if token registration fails.
  • Retries and time budgets: Supports asynchronous polling, transient-error retries, Retry-After, and UUID-based idempotent submissions. Defaults to 120 seconds per analysis and 300 seconds across the run; archive publication has its own timing. We can tune these depending on performance of Log Detective server.
  • Failure isolation and dry-run support: Analysis failures leave test outcomes unchanged. Progress reporting and dry runs do not submit analysis requests.
  • XML-safe output: LLM may produce characters that would break final XML. So we remove XML-invalid characters from analysis text before serialization.

The module documentation and tests cover log selection, URL availability, provisional results, synthetic fallback, caching, retries, and failure handling.

Edited by Jiri Podivin

Merge request reports

Loading
Loading