Skip to content

digitalocean: Error creating machine: Error in driver during machine creation: too many tries (60)

We have an autoscaled gitlab runner setup in DO. It was working fine for many months until the last week. We didn't change anything on our side, but now we have "Error in driver during machine creation: too many tries (60)" every time we try to create a new machine. Manually or with gitlab-runner, doesn't matter. I can create a new droplet with plain API request. I have tried to reach DO tech support, but all they can told me is "something wrong with docker-machine, use terraform".

My config.toml:

concurrent = 10
check_interval = 10

[session_server]
  session_timeout = 1800

[[runners]]
  name = "do-autoscale"
  limit = 5
  output_limit = 10000
  url = "https://gitlab.com/"
  token = "XXX"
  executor = "docker+machine"
  [runners.custom_build_dir]
  [runners.cache]
    Type = "s3"
    Path = "cache"
    Shared = true
    [runners.cache.s3]
      ServerAddress = "XXX.sfo2.digitaloceanspaces.com"
      AccessKey = "XXX"
      SecretKey = "XXX"
      BucketName = "XXX"
  [runners.docker]
    host = "tcp://docker:2375"
    tls_verify = true
    image = "alpine"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache", "/certs/client"]
    cache_dir = "/cache"
    pull_policy = ["always"]
    shm_size = 0
  [runners.machine]
    IdleCount = 0
    IdleTime = 600
    MachineDriver = "digitalocean"
    MachineName = "gitlab-runner-autoscale-%s"
    MachineOptions = ["digitalocean-image=ubuntu-18-04-x64", "digitalocean-ssh-user=root", "digitalocean-access-token=XXX", "digitalocean-region=sfo2", "digitalocean-size=s-2vcpu-4gb", "digitalocean-tags=gitlab-runner-autoscale", "digitalocean-private-networking=true", "digitalocean-monitoring=false", "engine-install-url=https://releases.rancher.com/install-docker/19.03.9.sh"]

Manual:

root@nft-grb:/etc/gitlab-runner# docker-machine create --driver digitalocean --digitalocean-image ubuntu-20-04-x64 --digitalocean-access-token $DOTOKEN --digitalocean-region fra1 --engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" docker-ubuntu-2004
Running pre-create checks...
Creating machine...
(docker-ubuntu-2004) Creating SSH key...
(docker-ubuntu-2004) Creating Digital Ocean droplet...
(docker-ubuntu-2004) Waiting for IP address to be assigned to the Droplet...
(docker-ubuntu-2004) Droplet creation failed on Digital Ocean, removing...
Error creating machine: Error in driver during machine creation: too many tries (60)

I use latest docker-machine:

root@nft-grb:~# docker-machine version
docker-machine version 0.16.2-gitlab.13, build 8b93db11

Any ideas what might be wrong?