Blueprint for AI Evaluation through Prompt Library
Proposal
https://docs.google.com/drawings/d/1ozZsH2EouHeuWSoPt_yd-CSUo8IjLFDIGs0pj48xNZo/edit
Further details
- Overview @mray2020
- Curating the CodeBase through GitLab Projects
- Chunking Algorithms
- Similarity Score
- Prompt Library as a benchmark for Pre and Post prompt engineering
- Findings and Recommendations
- Limitations
- Pipelines
- Eval Metrics
- Use-case Customization ; @srayner
- How to use Prompt Library for Prompt Experimentation;
Links / references
Edited by Mon Ray