Skip to content

Gitlab runner fails to start jobs on DigitalOcean

Summary

Steps to reproduce

.gitlab-ci.yml 
---
# yamllint disable rule:line-length
image: docker:stable

services:
  - docker:dind

stages:
  - build
  - deploy

build:
  stage: build
  image: registry.gitlab.com/danihodovic/jobs_aggregator:gitlab_runner
  script:
    - echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
    - task build-images
    - task push-images
    - docker-compose pull
    - task lint-py
    - task lint-js
    - task test

deploy:
  stage: deploy
  only:
    refs:
      - master
  image: danihodovic/ansible
  environment:
    name: production
    url: https://findwork.dev
  script:
    - eval $(ssh-agent -s)
    - echo "$SSH_KEY" | base64 -d | tr -d '\r' | ssh-add - > /dev/null
    - DOCKER_IMAGE=honeylogic/jobs-aggregator:$(git rev-parse --short HEAD)
    - mkdir ~/.ssh/ && echo "StrictHostKeyChecking no" > ~/.ssh/config
    - cd infra/ansible
    - ansible-galaxy install -r requirements.yml
    - echo $PROD_VAULT_PASSWORD > .prod_vault
    - export ANSIBLE_CONFIG=./ansible.cfg  # https://github.com/ansible/ansible/issues/42388
    - ansible-playbook deploy_app.yml -i environments/prod -u deployer -e app_image=$DOCKER_IMAGE

Actual behavior

No jobs can run.

Expected behavior

Jobs should run.

Relevant logs and/or screenshots

About to remove runner-lwspsn3x-auto-scale-1558453586-086a50dc  name=runner-lwspsn3x-auto-scale-1558453586-086a50dc operation=remove
WARNING: This action will delete both local reference and remote instance.  name=runner-lwspsn3x-auto-scale-1558453586-086a50dc operation=remove
ERROR: Error removing host "runner-lwspsn3x-auto-scale-1558453586-086a50dc": unexpected EOF  name=runner-lwspsn3x-auto-scale-1558453586-086a50dc operation=remove
WARNING: Stopping machine                           created=40h35m51.695630665s name=runner-lwspsn3x-auto-scale-1558453586-086a50dc reason=Failed to create used=40h25m32.76524001s usedCount=0
Stopping "runner-lwspsn3x-auto-scale-1558453586-086a50dc"...  name=runner-lwspsn3x-auto-scale-1558453586-086a50dc operation=stop
ERROR: id is invalid because cannot be less than 1  name=runner-lwspsn3x-auto-scale-1558453586-086a50dc operation=stop
WARNING: Error while stopping machine               created=40h35m51.742360073s error=exit status 1 name=runner-lwspsn3x-auto-scale-1558453586-086a50dc reason=Failed to create used=40h25m32.811969421s usedCount=0
WARNING: Removing machine                           created=40h35m51.742449358s name=runner-lwspsn3x-auto-scale-1558453586-086a50dc reason=Failed to create used=40h25m32.812058602s usedCount=0
About to remove runner-lwspsn3x-auto-scale-1558453586-086a50dc  name=runner-lwspsn3x-auto-scale-1558453586-086a50dc operation=remove
WARNING: This action will delete both local reference and remote instance.  name=runner-lwspsn3x-auto-scale-1558453586-086a50dc operation=remove
ERROR: Error removing host "runner-lwspsn3x-auto-scale-1558453586-086a50dc": unexpected EOF  name=runner-lwspsn3x-auto-scale-1558453586-086a50dc operation=remove

image

job log 
Running with gitlab-runner 11.10.1 (1f513601)
  on autoscale-runner LWSPSN3X

Environment description

I'm using both Gitlab.com shared runners and my own autoscale runners on DigitalOcean.

config.toml contents 
concurrent = 5

[[runners]]
  url = "https://gitlab.com"
  token = "{{ runner_token }}" # set by Ansible
  name = "autoscale-runner"
  executor = "docker+machine"
  limit = 5
  [runners.docker]
    image = "registry.gitlab.com/danihodovic/jobs_aggregator:gitlab_runner"
    privileged = true
  [runners.machine]
    OffPeakPeriods = [
      "* * 0-9,19-0 * * * *"
    ]
    OffPeakTimezone = "Asia/Hong_Kong"
    OffPeakIdleCount = 0
    OffPeakIdleTime = 300
    IdleCount = 0
    IdleTime = 1200
    MaxBuilds = 500
    MachineName = "auto-scale-%s"
    MachineDriver = "digitalocean"
    MachineOptions = [
        "digitalocean-image=coreos-stable",
        "digitalocean-ssh-user=core",
        "digitalocean-access-token={{ digitalocean_token }}",
        "digitalocean-region=sgp1",
        "digitalocean-size=4gb",
        "digitalocean-private-networking"
    ]

Used GitLab Runner version

bash-4.4# gitlab-runner --version
Version:      11.10.1
Git revision: 1f513601
Git branch:   11-10-stable
GO version:   go1.8.7
Built:        2019-04-24T09:29:18+0000
OS/Arch:      linux/amd64

-->

Edited by Dani Hodovic