Move code-suggestions eval logic to the eli5 package
This MR moves the existing code-suggestions evaluation logic the eli5
package.
The idea is to consolidate our development following the Python standards.
Please, note this MR doesn't change the logic and provides the same CLI API using the typer
package.
How to run:
- Set the required env variables by creating a
.env
file in the root project folder or manually:ELI5_LANGCHAIN_API_KEY="your_langsmith_api_key" # renamed from LANGCHAIN_API_KEY
- Get help by running:
poetry install # install all required dependencies poetry run eli5 code-suggestions --help poetry run eli5 code-suggestions evaluate --help
- Run the evaluation process:
poetry run eli5 code-suggestions evaluate \ --dataset="<your dataset>" \ --source=gitlab \ --num-tests=0 \ --evaluate-with-llm \ --experiment-prefix=experiment
Edited by Alexander Chueshev