Can't connect to Gitlab CI service (postgres)

Summary

I'm using the Gitlab Runner with docker executor to build and test an image of my app. It's a Rails app and after building my docker image I want to run the tests in a container from said image. I'm using a postgres service in my .gitlab-ci.yml because my Rails app test environment relies on Postgres as a DB.

Building works fine but I can't run my tests because the app container that is started by the Gitlab Runner can't connect to the postgres service.

Steps to reproduce

Gitlab Version: 8.16.6

OS on Gitlab Runner Host: Ubuntu 16.04.1

Docker version of Gitlab Runner Host: 1.13.0

This is how I set everything up:

Set up Gitlab Runner on my VPS

docker run -d --name gitlab-runner --restart always \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /srv/gitlab-runner/config:/etc/gitlab-runner \
  gitlab/gitlab-runner:latest

Register Gitlab Runner. Note that I'm not using the Docker-in-Docker setup but instead socket binding like described in the docs: https://docs.gitlab.com/ce/ci/docker/using_docker_build.html#use-docker-socket-binding

docker exec -it gitlab-runner gitlab-runner register -n \
  --url https://mygitlab.com \
  --registration-token XXX \
  --executor docker \
  --description "Gitlab Runner Docker" \
  --docker-image "docker:latest" \
  --docker-volumes /var/run/docker.sock:/var/run/docker.sock

Setup .gitlab-ci.ymlin my app repo

image: docker:1.13.0

services:
  - postgres:latest

stages:
  - build
  - test
  - release

variables:
  CONTAINER_TEST_IMAGE: mygitlab.com:4567/jonas/rails-ci-test:$CI_BUILD_REF_NAME
  CONTAINER_RELEASE_IMAGE: mygitlab.com:4567/jonas/rails-ci-test:latest
  POSTGRES_DB: railstest
  POSTGRES_USER: jonas
  POSTGRES_PASSWORD: ""
  POSTGRES_HOST: postgres
  RAILS_ENV: test

before_script:
  - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN mygitlab.com:4567

build:
 stage: build
 script:
   - docker build --pull -t $CONTAINER_TEST_IMAGE .
   - docker push $CONTAINER_TEST_IMAGE

test:
  stage: test
  script:
    - docker pull $CONTAINER_TEST_IMAGE
    - docker -v
    - docker ps -a
    - docker run $CONTAINER_TEST_IMAGE cat /etc/hosts
    - docker run $CONTAINER_TEST_IMAGE env
    - docker run $CONTAINER_TEST_IMAGE bundle exec rake test

release-image:
  stage: release
  script:
    - docker pull $CONTAINER_TEST_IMAGE
    - docker tag $CONTAINER_TEST_IMAGE $CONTAINER_RELEASE_IMAGE
    - docker push $CONTAINER_RELEASE_IMAGE

DB config of my Rails app (database.yml)

[...]
test:
  <<: *default
  host:     <%= ENV['POSTGRES_HOST'] %>
  database: <%= ENV['POSTGRES_DB'] %>
  username: <%= ENV['POSTGRES_USER'] %>
  password: <%= ENV['POSTGRES_PASSWORD'] %>
[...]

Actual behavior

The build stage runs fine but the test stage fails. Here's the log:

Running with gitlab-ci-multi-runner 1.10.5 (8bebc16)
Using Docker executor with image docker:1.13.0 ...
Starting service postgres:latest ...
Pulling docker image postgres:latest ...
Waiting for services to be up and running...
Pulling docker image docker:1.13.0 ...
Running on runner-9e68e243-project-4-concurrent-0 via 4e1792aeb8f8...
Fetching changes...
HEAD is now at bca01ec wfewaef
Checking out bca01ec5 as master...
Skipping Git submodules setup
$ docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN mygitlab.com:4567
Login Succeeded
$ docker pull $CONTAINER_TEST_IMAGE
master: Pulling from jonas/rails-ci-test
Digest: sha256:93e4c24fa6e9cb1254b1bda6214ea3c18fb8346eff7a23f3523b602b9c18b4ce
Status: Image is up to date for mygitlab.com:4567/jonas/rails-ci-test:master
$ docker -v
Docker version 1.13.0, build 49bf474
$ docker ps -a
CONTAINER ID        IMAGE                         COMMAND                  CREATED             STATUS                         PORTS               NAMES
e75a3682b921        9aa3005db491                  "docker-entrypoint..."   8 seconds ago       Up 2 seconds                                       runner-9e68e243-project-4-concurrent-0-build
81537bb5063b        f8fd5b8959f5                  "gitlab-runner-build"    10 seconds ago      Exited (0) 3 seconds ago                           runner-9e68e243-project-4-concurrent-0-predefined
209172e67097        ecd991538a0f                  "docker-entrypoint..."   17 seconds ago      Up 15 seconds                  5432/tcp            runner-9e68e243-project-4-concurrent-0-postgres
$ docker run $CONTAINER_TEST_IMAGE echo $POSTGRES_HOST
postgres
$ docker run $CONTAINER_TEST_IMAGE cat /etc/hosts
127.0.0.1	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
172.17.0.5	f221f5f4ddde
$ docker run $CONTAINER_TEST_IMAGE env
PATH=/usr/local/bundle/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=17872db3b88b
RUBY_MAJOR=2.3
RUBY_VERSION=2.3.3
RUBY_DOWNLOAD_SHA256=1a4fa8c2885734ba37b97ffdb4a19b8fba0e8982606db02d936e65bac07419dc
RUBYGEMS_VERSION=2.6.10
BUNDLER_VERSION=1.14.4
GEM_HOME=/usr/local/bundle
BUNDLE_PATH=/usr/local/bundle
BUNDLE_BIN=/usr/local/bundle/bin
BUNDLE_SILENCE_ROOT_WARNING=1
BUNDLE_APP_CONFIG=/usr/local/bundle
LANG=C.UTF-8
HOME=/root
$ docker run $CONTAINER_TEST_IMAGE bundle exec rake test
rake aborted!
PG::ConnectionBad: could not translate host name "postgres" to address: Name or service not known
[...]
Tasks: TOP => test
(See full trace by running task with --trace)
ERROR: Build failed: exit code 1

You can see in the logs that the postgres service can't be resolved by the app container. Also the output of cat /etc/hosts on the built container suggests that the link between app container and postgres service container doesn't work. docker ps shows that the postgres container was started by the gitlab runner, so it's there but it's not reachable for the app container.