Add the script to pull the aggregated vote data
Pull the vote data from the command line
Move into the eval code viewer directory
cd eval-code-viewer
Ensure the dependencies are installed through yarn.
Run the script to get the aggregate data:
▶ node tasks/get_evaluations.js
{
data: [
{
model: 'anthropic',
language: 'javascript',
total_vote_count: '14',
thumbs_up_count: '6',
thumbs_down_count: '8',
prompt_type: generation,
avg_response_time: 0.9
},
{
model: 'anthropic_fast',
language: 'javascript',
total_vote_count: '12',
thumbs_up_count: '7',
thumbs_down_count: '5',
prompt_type: generation,
avg_response_time: 0.9
},
...
],
message: 'Successfully retrieved aggregated votes'
}
Run the script that fetches the raw data:
node tasks/get_evaluations.js raw
{
data: [
...
{
id: 104,
prompt_result_id: 'c3462496-6912-4fa7-9814-254c5ce37576',
language: 'javascript',
model: 'vertex_text_bison',
option: 'thumbsDown'
},
{
id: 105,
prompt_result_id: 'be68d7d6-4b73-4082-86f8-99f66df80295',
language: 'javascript',
model: 'anthropic_fast',
option: 'thumbsUp'
},
... 107 more items
],
message: 'Successfully retrieved raw votes'
}
Edited by Allison Browne