Migrate Centralised Evaluation daily runs from Google Data Pipelines to GitLab Pipeline Schedules
Problem to solve
We have been observing missing jobs in Google Data Pipelines ( #176 (closed)). Google Data Pipelines is also not visible to other team members outside of groupai model validation.
Proposal
We should explore migrating Centralised Evaluation daily runs from Google Data Pipelines to GitLab Pipeline Schedules.
- Provide better reliability.
- Dogfooding GitLab features.
- More visibility to schedules.
For the evaluation pipelines, we should only run daily on the GitLab AI feature models. For foundational models, we should generate answers and evaluations once because they don't change.
Further details
The investigation ongoing is documented in the issue linked above : #176 (closed)
Links / references
Edited by Tan Le