Rate limiting in gitlab-runner
Description
Using docker+machine with the Amazon EC2 driver, 3 defined runners, and IdleCount = 6, we encounter API rate limiting which prevents any machines from being spun up for the third defined runner when we start gitlab-runner. We could ease this by lowering the IdleCount, but that optimizes for less than 1% of the expected lifetime of the gitlab-runner. What would be better is if gitlab-runner could rate-limit itself.
Proposal
Creation of two new options for the docker+machine executor: a "burst" option and a "rate" option. The "burst" option would allow up to that number of runner machines to be started within the "rate" interval; thereafter, only 1 machine per rate interval could be created, until no new machines have been created within, say, 3 times the rate interval. I envision using the leaky bucket algorithm to accomplish this. No rate limiting need be done if these options are not set.
This would allow us to work around the AWS API rate limiting while still getting all runners online. That is, I envision setting "burst" to 3 for all three runners, allowing them to get a total of 9 runner machines online; thereafter, at, say, a 1 minute interval, each would spin up an additional runner machine until IdleCount was satisfied. The same could happen when spinning up additional runner machines after the closure of an off-peak period, which is likely to occur more frequently than the initial runner start-up.