Accessing a Docker container service from another container service (hosts and network)

In GitLab CI, I want to be able to access a container listed under services from another container listed under services. Unfortunately, it doesn't seem possible yet because services hosts are not shared to the other services containers (see "Access a service host in another service" below"). And there isn't a way to define a network that the container should belong to with the services syntax.

Here are the avenues, trials, and tribulations I have explored... 🤔

---

If you are curious how I setup the Docker registry locally, see https://gitlab.com/snippets/73109#setup-the-container-registry

Perhaps you are curious about my SSL issues trying to get a secure registry. But I just ended up using the instructions just above to create an insecure registry.

---

Some docs

• https://docs.gitlab.com/runner/executors/docker.html#accessing-the-services
• https://docs.gitlab.com/ce/ci/docker/using_docker_images.html

Docker network per build (gitlab-runner#1042 (closed)) ❔

I've never used this new method yet but is exactly what is needed so probably works great.

See gitlab!43875 (diffs) for how to enable FF_NETWORK_PER_BUILD.

Share the /etc/hosts file to Docker image via services entrypoint ✔️

See gitlab-runner#1042 (comment 144420147)

Grab the IP from the /etc/hosts file ✔️

See https://gitlab.com/MadLittleMods/gitlab-ci-access-main-container-from-service-container

Access a service host in another service ❌

Update: Nowadays see #docker-network-per-build-httpsgitlabcomgitlab-orggitlab-runner-issues1042-grey_question just above

This does not work but would be the ideal scenario. Hosts, whether auto-generated or aliased, are not shared or accessible within another service.

Confirmed here, https://gitlab.slack.com/archives/C0SFP840G/p1506632880000377

ex. job: https://gitlab.com/gitlab-org/gitlab-selenium-server/-/jobs/34442764

Dockerfile

FROM node:boron
WORKDIR /app
Install app dependencies
COPY package.json package-lock.json ./
RUN npm install
Bundle app source
COPY . .
EXPOSE 4444
CMD ["npm", "start"]

.gitlab-ci.yml

image: node:boron
variables:
docker.for.mac.localhost hostname via https://stackoverflow.com/a/43541732/796832s
$DOCKER_REGISTRY = registry.gitlab.com or docker.for.mac.localhost:5063
$IMAGE_NAMESPACE = gitlab-org/gitlab-selenium-server or root/gitlab-selenium-server
We have to separate this instead of constructing from $DOCKER_REGISTRY and $IMAGE_NAMESPACE
because docker.for.mac.localhost:5063/root/gitlab-selenium-server:latest does
not work because the host should be localhost ¯_(ツ)_/¯
$IMAGE_FULL_NAME = registry.gitlab.com/gitlab-org/gitlab-selenium-server or localhost:5063/root/gitlab-selenium-server
stages:

build
test

build-docker-image:
stage: build
image: docker:latest
variables:
DOCKER_DRIVER: overlay2
script:
- docker info
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $DOCKER_REGISTRY
- docker build -t DOCKER_REGISTRY/IMAGE_NAMESPACE:test --file="Dockerfile" .
- docker push DOCKER_REGISTRY/IMAGE_NAMESPACE:test
services:
- name: docker:dind
command:
- "--insecure-registry"
# Unable to use $DOCKER_REGISTRY because it doesn't seem to work :/
# See https://gitlab.com/gitlab-org/gitlab-ce/issues/38346
- "docker.for.mac.localhost:5063"
tags:
- docker
- shared
THIS DOES NOT WORK.
HOSTS ARE NOT ACCESSIBLE IN ANOTHER SERVICE
test-docker-image:
stage: test
script:
# The GitLab Selenium Proxy fails to find the target http://actual-selenium-server:4444/wd/hub
- curl http://gitlab-selenium-proxy:4444/ || true
- curl http://gitlab-selenium-proxy:4444/wd/hub/status || true
- curl http://gitlab-selenium-proxy:4444/logs || true
variables:
SELENIUM_REMOTE_URL: http://gitlab-selenium-proxy:4444/wd/hub
GITLAB_TARGET_SELENIUM_REMOTE_URL: http://actual-selenium-server:4444/wd/hub
services:
# To figure out the host name of a service, see https://docs.gitlab.com/runner/executors/docker.html#accessing-the-services
# ---
#
# See https://github.com/SeleniumHQ/docker-selenium
- name: selenium/standalone-chrome:3.4.0-francium
# For reference, the default host is selenium__standalone-chrome:4444
alias: actual-selenium-server
- name: $IMAGE_FULL_NAME:test
# For reference, the default host is when using a local registry, localhost__root__gitlab-selenium-server__selenium-server:4444
alias: gitlab-selenium-proxy
tags:
- docker
- shared

Create an image with the service already inside ✔️

Another alternative is to create an image FROM selenium/standalone-chrome:3.4.0-francium and include everything it needs(like Node and npm) inside so you can just use it as a GitLab CI service.

This is probably the cleanest solution that actually works at the moment. BUT it bloats the image size, stuck to a single Selenium Server in this case Chrome(unless we want to pack things ourselves), and there isn't a slick way to merge two images together because there can't be two separate base images (ideally just merge node:boron and selenium/standalone-chrome:3.4.0-francium)

Dockerfile

# Image stack:
# selenium/standalone-chrome ->
# selenium/node-chrome -> https://github.com/SeleniumHQ/docker-selenium/blob/3fb76ba36cfa241a7927d432fc162553063259c3/StandaloneChrome/Dockerfile#L5
# selenium/node-base -> https://github.com/SeleniumHQ/docker-selenium/blob/3fb76ba36cfa241a7927d432fc162553063259c3/NodeChrome/Dockerfile#L5
# selenium/base -> https://github.com/SeleniumHQ/docker-selenium/blob/3fb76ba36cfa241a7927d432fc162553063259c3/NodeBase/Dockerfile#L5
# ubuntu:16.04 -> https://github.com/SeleniumHQ/docker-selenium/blob/b5262f1e70cc1927cd31622b668e67628adf4777/Base/Dockerfile#L1
FROM selenium/standalone-chrome:3.4.0-francium
USER root
WORKDIR /app
ENV NVM_DIR /usr/local/nvm
ENV NODE_VERSION 6.11.3
Install nvm
RUN wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.33.4/install.sh | bash
Install Node and npm
RUN . $NVM_DIR/nvm.sh 

&& nvm install $NODE_VERSION 

&& nvm alias default $NODE_VERSION 

&& nvm use default
Add Node and npm to path so the commands are available
ENV NODE_PATH NVM_DIR/vNODE_VERSION/lib/node_modules
ENV PATH NVM_DIR/versions/node/vNODE_VERSION/bin:$PATH
Confirm installation
RUN node -v
RUN npm -v
Install app dependencies
COPY package.json package-lock.json ./
RUN npm install
Bundle app source
COPY . .
EXPOSE 4545
We have to combine these because there can only be one CMD
and it is overridden from the base images.

Start the Selenium server
This is copied over https://github.com/SeleniumHQ/docker-selenium/blob/b5262f1e70cc1927cd31622b668e67628adf4777/StandaloneChrome/Dockerfile#L13
And is normally run as a CMD here, https://github.com/SeleniumHQ/docker-selenium/blob/b5262f1e70cc1927cd31622b668e67628adf4777/NodeBase/Dockerfile#L63

And then start the app
CMD ["/bin/bash", "-c", "/opt/bin/entry_point.sh & npm start"]

.gitlab-ci.yml

image: node:boron
variables:
docker.for.mac.localhost hostname via https://stackoverflow.com/a/43541732/796832s
$DOCKER_REGISTRY = registry.gitlab.com or docker.for.mac.localhost:5063
$IMAGE_NAMESPACE = gitlab-org/gitlab-selenium-server or root/gitlab-selenium-server
We have to separate this instead of constructing from $DOCKER_REGISTRY and $IMAGE_NAMESPACE
because docker.for.mac.localhost:5063/root/gitlab-selenium-server:latest does
not work because the host should be localhost ¯_(ツ)_/¯
$IMAGE_FULL_NAME = registry.gitlab.com/gitlab-org/gitlab-selenium-server or localhost:5063/root/gitlab-selenium-server
stages:

build
test

build-docker-image:
stage: build
image: docker:latest
variables:
DOCKER_DRIVER: overlay2
script:
- docker info
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $DOCKER_REGISTRY
- docker build -t DOCKER_REGISTRY/IMAGE_NAMESPACE:test --file="Dockerfile" .
- docker push DOCKER_REGISTRY/IMAGE_NAMESPACE:test
services:
- name: docker:dind
command:
- "--insecure-registry"
# Unable to use $DOCKER_REGISTRY because it doesn't seem to work :/
# See https://gitlab.com/gitlab-org/gitlab-ce/issues/38346
- "docker.for.mac.localhost:5063"
tags:
- docker
- shared
test-docker-image:
stage: test
script:
- curl http://gitlab-selenium-proxy:4545/ || true
- curl http://gitlab-selenium-proxy:4545/wd/hub/status || true
# Show the logs for the GitLab Selenium Server service
- curl http://gitlab-selenium-proxy:4545/logs || true
variables:
SELENIUM_REMOTE_URL: http://gitlab-selenium-proxy:4545/wd/hub
services:
- name: $IMAGE_FULL_NAME:test
# For reference, the default host is when using a local registry, localhost__root__gitlab-selenium-server__selenium-server:4545
alias: gitlab-selenium-proxy
tags:
- docker
- shared

Using Docker in Docker (dind) ✔️

Running our own Docker in Docker (dind) image, we can setup a network that will share the host names. But this is cumbersome and requires a lot of extra boilerplate. And from my testing, seems a bit flakey even with the explicit waits for running and sleep times.

ex. job: https://gitlab.com/gitlab-org/gitlab-selenium-server/-/jobs/34442765

Dockerfile

FROM node:boron
WORKDIR /app
Install app dependencies
COPY package.json package-lock.json ./
RUN npm install
Bundle app source
COPY . .
EXPOSE 4444
CMD ["npm", "start"]

.gitlab-ci.yml

image: node:boron
variables:
docker.for.mac.localhost hostname via https://stackoverflow.com/a/43541732/796832s
$DOCKER_REGISTRY = registry.gitlab.com or docker.for.mac.localhost:5063
$IMAGE_NAMESPACE = gitlab-org/gitlab-selenium-server or root/gitlab-selenium-server
We have to separate this instead of constructing from $DOCKER_REGISTRY and $IMAGE_NAMESPACE
because docker.for.mac.localhost:5063/root/gitlab-selenium-server:latest does
not work because the host should be localhost ¯_(ツ)_/¯
$IMAGE_FULL_NAME = registry.gitlab.com/gitlab-org/gitlab-selenium-server or localhost:5063/root/gitlab-selenium-server
stages:

build
test

build-docker-image:
stage: build
image: docker:latest
variables:
DOCKER_DRIVER: overlay2
script:
- docker info
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $DOCKER_REGISTRY
- docker build -t DOCKER_REGISTRY/IMAGE_NAMESPACE:test --file="Dockerfile" .
- docker push DOCKER_REGISTRY/IMAGE_NAMESPACE:test
services:
- name: docker:dind
command:
- "--insecure-registry"
# Unable to use $DOCKER_REGISTRY because it doesn't seem to work :/
# See https://gitlab.com/gitlab-org/gitlab-ce/issues/38346
- "docker.for.mac.localhost:5063"
tags:
- docker
- shared
test-docker-image:
stage: test
image: docker:latest
variables:
DOCKER_DRIVER: overlay
SELENIUM_REMOTE_URL: http://gitlab-selenium-proxy:4444/wd/hub
GITLAB_TARGET_SELENIUM_REMOTE_URL: http://actual-selenium-server:4444/wd/hub
script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $DOCKER_REGISTRY
- docker pull selenium/standalone-chrome:3.4.0-francium
- docker pull DOCKER_REGISTRY/IMAGE_NAMESPACE:test
- docker pull node:boron
- docker network create some-network
# We are running against, docker: poorly formatted environment: variable '-----END CERTIFICATE-----' has white spaces.
# So we are just getting rid of the whole DOCKER_ENV_CI_SERVER_TLS_CA_FILE entry,
# See related issue, https://github.com/mesosphere/marathon-lb/issues/433
- env | sed '/^DOCKER_ENV_CI_SERVER_TLS_CA_FILE=/,/^$/d' > .env
- docker run -d --name=actual-selenium-server --net=some-network -p 4545:4444 --env-file=.env selenium/standalone-chrome:3.4.0-francium
- docker run -d --name=gitlab-selenium-proxy --net=some-network -p 4444:4444 --env-file=.env DOCKER_REGISTRY/IMAGE_NAMESPACE:test
- docker run -d --name=node-land --net=some-network --volume "PWD":/root --env-file=.env --tty --interactive node:boron /bin/bash - until [ "`docker inspect -f {{.State.Running}} actual-selenium-server`" == "true" ]; do sleep 0.1; done; - until [ "`docker inspect -f {{.State.Running}} gitlab-selenium-proxy`" == "true" ]; do sleep 0.1; done; - until [ "`docker inspect -f {{.State.Running}} node-land`" == "true" ]; do sleep 0.1; done; # Doesn't seem to work without this :/ - sleep 5 - docker ps --all # Now we can actually run our commands *phew - docker exec node-land curl http://gitlab-selenium-proxy:4444/ || true - docker exec node-land curl http://gitlab-selenium-proxy:4444/wd/hub/status || true - docker exec node-land curl http://gitlab-selenium-proxy:4444/logs || true services: - name: docker:dind command: - "--insecure-registry" # Unable to use `DOCKER_REGISTRY` because it doesn't seem to work :/
# See https://gitlab.com/gitlab-org/gitlab-ce/issues/38346
- "docker.for.mac.localhost:5063"
tags:
- docker
- shared