GitLab runner with docker+machine can't spawn runners

Summary

I set up a GitLab runner on AWS to use with our GitLab.com account following this guide. I built a basic pipeline for a repository as a chance to demo GitLab CI. The job runs just fine on a shared runner.

When I turn off shared runners and leave only my specific runner, the job starts but just hangs. When I log into the runner instance, I notice that there's an issue with docker-machine - I've posted the error below. My security group is completely open to all traffic - so it's not an ACL issue. The AWS IAM user I set up also has full access to EC2 and S3.

Any ideas on how to at least troubleshoot the issue, or what might be causing this? My runner was configured almost exactly as the guide described and I've included all config below. Please let me know if there's any more information that might make this easier to solve.

Error message in docker-machine:

root@****:~# docker-machine ls
NAME                                                        ACTIVE   DRIVER      STATE   URL   SWARM   DOCKER    ERRORS
runner-2fbdcf3e-gitlab-docker-machine-******   -        amazonec2   Error                 Unknown   MissingParameter: The request must contain the parameter InstanceId
                                                            status code: 400, request id: 2425d8be-cf93-487c-****

Actual behavior

The runner/job hangs and times out. The job seems to be picked up by my bastion instance, but the runner doesn't seem to be spawned correctly.

Expected behavior

I would expect docker-machine to spawn a runner instance and complete the job.

Environment description

Running on Ubuntu 1604 on an AWS t2.micro instance.

Here's my runner config:

concurrent = 10
check_interval = 0

[[runners]]
  name = "ec2-runner"
  url = "https://gitlab.com/"
  token = "****"
  executor = "docker+machine"
  limit = 10
  [runners.docker]
    image = "alpine:latest"
    privileged = true
    disable_cache = true
  [runners.cache]
    Type = "s3"
    ServerAddress = "s3.amazonaws.com"
    AccessKey = "****"
    SecretKey = "****"
    BucketName = "****"
    BucketLocation = "us-west-2"
    Shared = true
  [runners.machine]
    IdleCount = 0
    IdleTime = 1800
    MaxBuilds = 10
    OffPeakPeriods = [
      "* * 0-9,18-23 * * mon-fri *",
      "* * * * * sat,sun *"
    ]
    OffPeakIdleCount = 0
    OffPeakIdleTime = 1200
    MachineDriver = "amazonec2"
    MachineName = "gitlab-docker-machine-%s"
    OffPeakTimezone = "US/Mountain"
    MachineOptions = [
      "amazonec2-access-key=****",
      "amazonec2-secret-key=****",
      "amazonec2-region=us-west-2",
      "amazonec2-vpc-id=****",
      "amazonec2-subnet-id=****",
      "amazonec2-use-private-address=true",
      "amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true",
      "amazonec2-security-group=gitlab-runner",
      "amazonec2-instance-type=m4.xlarge",
    ]

Used GitLab Runner version

Version:      10.6.0
Git revision: a3543a27
Git branch:   10-6-stable
GO version:   go1.9.4
Built:        2018-03-22T08:34:11+00:00
OS/Arch:      linux/amd64

Docker version 18.03.0-ce, build 0520e24

docker-machine version 0.14.0, build 89b8332

Solution

#3222 (comment 126924393)

#3222 (comment 129064926)