docker+machine: MaxAge for idle machines
What does this MR do?
Adds MaxAge to [runners.machine]: idle machines older than the configured age (seconds) are removed and replaced, including machines held by the IdleCount floor. Machines running a job are never touched. Default 0 changes nothing.
Why is this MR needed?
Idle removal requires both IdleTime exceeded and more idle machines than IdleCount, so the machines that make up the idle floor are never removed for age: on a low-throughput runner they live until a job consumes them, which can be weeks.
That breaks external cleanup of leaked VMs. docker-machine loses instances on crashes and API errors, and a cleanup script can't tell a three-week-old idle machine from a leak, so it has to use a threshold long enough to be safe: GitLab's own cleaner uses 3 days, leaving leaked VMs running and billing for days. With MaxAge set, no legitimate machine is older than the limit, and cleanup can act within hours. Rotating old idle machines also picks up fresh images.
Context: production-engineering#29599.
What's the best way to test this MR?
Unit tests cover the new check: exceeded within the floor, not reached, disabled. On a live runner, set MaxAge = 3600 on a config with IdleCount > 0 and low throughput, and watch idle machines get replaced after an hour with removal reason "max age exceeded".