Draft: feat: add cli param to run dataset splits
What does this merge request do and why?
Resolves #22
TODO:
-
Add class for only existing splits -
See what happens running multiple -
Change name to singular?
How to set up and validate locally
Existing splits are listed at the top of the code-suggestions-input-testcases-v1
dataset:
poetry run eli5 code-suggestions evaluate \
--dataset="code-suggestions-input-testcases-v1" \
--source=gitlab \
--limit=10 \
--offset=0 \
--experiment-prefix=splits-test \
--intent=completion \
--rate-limit=29 \
--splits=lang=php
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Edited by Missy Davies