Extract failed job trace for RCA
What does this merge request do and why?
A poc of extracting the job error trace into BQ
The resulting data can be found in dev-ai-research-0e2f8974.root_cause_analysis.hyang_error_trace_poc
Remaining tasks:
-
Support for pagination -
Extract n
jobs per MR, instead ofn
latest jobs of a projects
Ref: #254 (closed)
How to set up and validate locally
- Ensure GCP access and GitLab token are setup.
- Check out to this merge request's branch.
- Run the follow command to kick off the extraction.
poetry run promptlib root-cause-analysis extract-data --output-table dev-ai-research-0e2f8974.root_cause_analysis.<your_output_table_name>
- Confirm the data in the BigQuery table.
Merge request checklist
-
I've ran the affected pipeline(s) to validate that nothing is broken. -
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Edited by Hongtao Yang