API throttling blocking specific runners

I see that we rolled out new API throttling rules in the last few hours. A number of users have reported issues:

/cc: @northrup

Edited by Stan Hu