Customer-Facing Validation for Customized Features - RAG Implementations
What kind of validation can we offer to customer's as they customize their Duo Features?
A future area of exploration for custom models is allowing user's to customize prompt injections associated with Duo Features for their own use cases.
Ideas:
- a smoke test of sorts, with 10-20 prompts for each use case allowing users to manually view before/after of their prompt changes
- allowing users to insert their own prompts/ground truths
- allow access to the GL CEF pipeline for already established use cases - with what limits?
- Vertex AI SxS
Meta Questions:
- What is the minimally viable 'validation' that we can offer customization customers
- how can we identify and flag if prompt changes may degrade performance without using the entire CEF?
Edited by Susie Bitters