Skip to content

Extract failed job trace for RCA

Hongtao Yang requested to merge hyang/rca-data into main

What does this merge request do and why?

A poc of extracting the job error trace into BQ

The resulting data can be found in dev-ai-research-0e2f8974.root_cause_analysis.hyang_error_trace_poc

Remaining tasks:

  • Support for pagination
  • Extract n jobs per MR, instead of n latest jobs of a projects

Ref: #254 (closed)

How to set up and validate locally

  1. Ensure GCP access and GitLab token are setup.
  2. Check out to this merge request's branch.
  3. Run the follow command to kick off the extraction.
    poetry run promptlib root-cause-analysis extract-data --output-table dev-ai-research-0e2f8974.root_cause_analysis.<your_output_table_name>
  4. Confirm the data in the BigQuery table.

Merge request checklist

  • I've ran the affected pipeline(s) to validate that nothing is broken.
  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Hongtao Yang

Merge request reports

Loading