Skip to content

Added created_at column to code-suggestions output table

Hongtao Yang requested to merge hotfix/code-suggestions-schema-created_at into main

What does this merge request do and why?

All of our code-suggestions pipeline will fail now because there is no created_at column in the schema, but we changed the WriteDatasetToBigQuery function to always have a created_at column a while back in this MR.

RuntimeError: BigQuery job beam_bq_job_LOAD_AUTOMATIC_JOB_NAME_LOAD_STEP_94db77a955c347318cad18d4b9b67c32_3b1f14611004e2236db15e2145aa432b_pane0_partition0 failed. Error Result: <ErrorProto
 location: 
'gs://prompt-library/tmp/bq_load/99f4e6f120dc441a80d321158cddde3d/dev-ai-research-0e2f8974.code_suggestion_experiments.tle_eval_gecko_go_tree_sitter/04f55a8d-7b63-4195-83a9-3a09380dd934'
 message: 'Error while reading data, error message: JSON table encountered too many errors, giving up. Rows: 1; errors: 1. Please look into the errors[] collection for more details. File: 
gs://prompt-library/tmp/bq_load/99f4e6f120dc441a80d321158cddde3d/dev-ai-research-0e2f8974.code_suggestion_experiments.tle_eval_gecko_go_tree_sitter/04f55a8d-7b63-4195-83a9-3a09380dd934'
 reason: 'invalid'> [while running 'Write results/WriteToBigQuery/BigQueryBatchFileLoads/TriggerLoadJobsWithoutTempTables/ParDo(TriggerLoadJobs)']

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Merge request checklist

  • I've ran the affected pipeline(s) to validate that nothing is broken.
  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Tan Le

Merge request reports