Skip to content

Support running ELI5 service

David O'Regan requested to merge oregand-main-patch-38ba into main

Following https://gitlab.slack.com/archives/C2Z9A056E/p1717509057369009, we want to do something similar to Resolve "Include AI Gateway in GDK" (!3646 - merged) • Ash McKenzie • 16.11 to support running https://gitlab.com/gitlab-org/ai-powered/eli5 via a gdk command i.e. gdk evaluate chat, and then providing the dataset name to be evaluated which are stored in https://smith.langchain.com/

ToDo:

  1. Enable dynamic evaluation selection i.e. select feature and evaluation to be performed
  2. Support dataset size selection i.e. splitting datasets

Merge request reports