Restructure codebase to allow easy addition of new usecases

Problem to solve

Currently different usecases that prompt-library supports has lots of duplication because they roughly follow the same logical flow and the same code structure. But they do have subtle differences that merge them all if not ideal.

Proposal

We should extract the common repeated logic/pattern into the common module to allow minimal duplication of new usecases while allow the flexibility for them to be different.

These are some areas where we see duplication:

• write results
• config
• apply prompt templates
• batch request LLM
• generic judges
• prepare testcases
• inference models

Further details

Links / references