Decouple failure analysis data from generic webhook export workflow

Context

Currently, failure analysis relies on the full webhook payload stored in the database. However, the actual analysis only requires a minimal subset of fields:

  • project_id
  • project_path
  • job_name
  • job_status
  • ref
  • created_at
  • job_id (if needed)

Proposal

Consider storing these specific fields directly in the failure analysis tables instead of referencing the full webhook payload.

Benefits:

  • Decouple failure analysis from the generic webhook export workflow
  • Reduce dependencies between two increasingly independent workflows
  • Potentially simplify queries and improve performance
  • Make the failure analysis data model more explicit and self-contained

Tradeoffs:

  • Introduces some data duplication
  • Requires refactoring effort
  • Need to maintain consistency between webhook ingestion and failure analysis storage

Related