Add cleanup support for provisioned Docker Machines
Description
When gitlab-ci-multi-runner is configured to use Docker Machine, it provisions instances on startup in order to match the configured IdleCount. Unfortunately when the service stops or the runner instance is terminated those provisioned machines are not destroyed, resulting in orphaned instances. We have our runners in an AWS autoscaling group for self-healing purposes, and we end up with orphaned machines of which we dont have the ssh key pair because it was stored exclusively on the runner. This means we need to determine which instances are orphaned based on their machine name containing known runner tokens and then terminate them manually.
My current workaround for this on Amazon Linux is to update the stop() function in /etc/init.d/gitlab-runner to additionally unregister the runner and then run a /usr/local/bin/docker-machine rm -f $(/usr/local/bin/docker-machine ls -q)
Proposal
It would be great if gitlab-ci-multi-runner could be configured to cleanup after itself, performing a docker-machine rm -f on any machines that it has created. It probably shouldn't simply be on service stop (the workaround I provided above) since there are scenarios where you may not want to destroy your instances on a service stop (such as just restarting the service in place). I think the best idea would be to either add a --cleanup parameter to unregister or a setting in config.toml that will allow it to happen automatically on unregister. This way I can simply unregister the runner on shutdown and know that it has cleaned up any provisioned resources.
Links to related issues and merge requests / references
It looks like the golang-kardianos-service library that is used for managing services is not very configurable to run additional commands during a stop, so I'm guessing that I would have to still add the unregister command myself.