Wait for IdleTime before destroying an autoscaled runner with IdleCount = 0

Context / use case

I am configuring a gitlab runner with autoscaling enabled for a project with only a few commits a day, for an organization tight on money. I would like to stop all runners when there is no job to be built, but I would also like to keep a runner some time after it has been used in case of a new job comming to avoid waiting the creation of a new one.

Problem

Today, my configuration looks like this:

concurrent = 2

[[runners]]
  name = "My autoscale runner"
  url = "https://gitlab.com/ci"
  token = "XXXXXXXXXXXXXXXXXXXXXXXXXX"
  executor = "docker+machine"
  limit = 2
  [runners.machine]
    IdleCount = 0
    IdleTime = 1800
    MaxBuilds = 100
    MachineName = "gitlab-runner-autoscale-%s"
    MachineDriver = "amazonec2"
    MachineOptions = [
        "amazonec2-access-key=XXXXXXXXXXXXXXXXXXXXXXXXX",
        "amazonec2-secret-key=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
        "amazonec2-region=eu-west-1",
        "amazonec2-instance-type=c4.large",
        "amazonec2-request-spot-instance=true",
        "amazonec2-spot-price=0.03"
    ]

The problem is that, as I have IdleCount set to 0, the autoscaled runners are terminated just after finishing the job, even if there are other jobs in the queue.

Suggested solution

In case of IdleCount=0, wait for IdleTime before actually terminating the runner, or provide an option that allows this behavior. Also, #1374 (closed) seems a very good idea for optimizing the use of runners and lower their cost.