Skip to content

Document developer workflow to enable efficient evaluations

Problem to solve

It can be hard for engineers to know when and how to evaluate AI features. The AI feature development playbook has some information, but it's incomplete and does not necessarily provide the relevant information in an easy-to-follow, practical style.

At minimum, engineers need to know what tools to use, but also when and how it's appropriate to use which tools. This requires at least some understanding of how evaluation fits within the overall software development workflow.

Proposal

  • Collect the documentation on evaluation tools and processes, and organize them into a set of guidelines covering which tools to use, and how to use them.
  • Link to the guidelines (and from there to existing detailed instructions) from the testing/evaluation phase outlined in #743 (closed).
Edited by Mark Lapierre