GitLab runner with docker+machine can't spawn runners
Summary
I set up a GitLab runner on AWS to use with our GitLab.com account following this guide. I built a basic pipeline for a repository as a chance to demo GitLab CI. The job runs just fine on a shared runner.
When I turn off shared runners and leave only my specific runner, the job starts but just hangs. When I log into the runner instance, I notice that there's an issue with docker-machine - I've posted the error below. My security group is completely open to all traffic - so it's not an ACL issue. The AWS IAM user I set up also has full access to EC2 and S3.
Any ideas on how to at least troubleshoot the issue, or what might be causing this? My runner was configured almost exactly as the guide described and I've included all config below. Please let me know if there's any more information that might make this easier to solve.
Error message in docker-machine:
root@****:~# docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
runner-2fbdcf3e-gitlab-docker-machine-****** - amazonec2 Error Unknown MissingParameter: The request must contain the parameter InstanceId
status code: 400, request id: 2425d8be-cf93-487c-****
Actual behavior
The runner/job hangs and times out. The job seems to be picked up by my bastion instance, but the runner doesn't seem to be spawned correctly.
Expected behavior
I would expect docker-machine to spawn a runner instance and complete the job.
Environment description
Running on Ubuntu 1604 on an AWS t2.micro instance.
Here's my runner config:
concurrent = 10
check_interval = 0
[[runners]]
name = "ec2-runner"
url = "https://gitlab.com/"
token = "****"
executor = "docker+machine"
limit = 10
[runners.docker]
image = "alpine:latest"
privileged = true
disable_cache = true
[runners.cache]
Type = "s3"
ServerAddress = "s3.amazonaws.com"
AccessKey = "****"
SecretKey = "****"
BucketName = "****"
BucketLocation = "us-west-2"
Shared = true
[runners.machine]
IdleCount = 0
IdleTime = 1800
MaxBuilds = 10
OffPeakPeriods = [
"* * 0-9,18-23 * * mon-fri *",
"* * * * * sat,sun *"
]
OffPeakIdleCount = 0
OffPeakIdleTime = 1200
MachineDriver = "amazonec2"
MachineName = "gitlab-docker-machine-%s"
OffPeakTimezone = "US/Mountain"
MachineOptions = [
"amazonec2-access-key=****",
"amazonec2-secret-key=****",
"amazonec2-region=us-west-2",
"amazonec2-vpc-id=****",
"amazonec2-subnet-id=****",
"amazonec2-use-private-address=true",
"amazonec2-tags=runner-manager-name,gitlab-aws-autoscaler,gitlab,true,gitlab-runner-autoscale,true",
"amazonec2-security-group=gitlab-runner",
"amazonec2-instance-type=m4.xlarge",
]
Used GitLab Runner version
Version: 10.6.0
Git revision: a3543a27
Git branch: 10-6-stable
GO version: go1.9.4
Built: 2018-03-22T08:34:11+00:00
OS/Arch: linux/amd64
Docker version 18.03.0-ce, build 0520e24
docker-machine version 0.14.0, build 89b8332