Accessing a Docker container service from another container service (hosts and network)

In GitLab CI, I want to be able to access a container listed under services from another container listed under services. Unfortunately, it doesn't seem possible yet because services hosts are not shared to the other services containers (see "Access a service host in another service" below"). And there isn't a way to define a network that the container should belong to with the services syntax.

Here are the avenues, trials, and tribulations I have explored... 🤔

If you are curious how I setup the Docker registry locally, see https://gitlab.com/snippets/73109#setup-the-container-registry

Perhaps you are curious about my SSL issues trying to get a secure registry. But I just ended up using the instructions just above to create an insecure registry.

Some docs

Docker network per build (gitlab-runner#1042 (closed)) ❔

I've never used this new method yet but is exactly what is needed so probably works great.

See gitlab!43875 (diffs) for how to enable FF_NETWORK_PER_BUILD.

Share the `/etc/hosts` file to Docker image via `services` `entrypoint` ✔

See gitlab-runner#1042 (comment 144420147)

Grab the IP from the `/etc/hosts` file ✔

See https://gitlab.com/MadLittleMods/gitlab-ci-access-main-container-from-service-container

Access a service host in another service ❌

Update: Nowadays see #docker-network-per-build-httpsgitlabcomgitlab-orggitlab-runner-issues1042-grey_question just above

This does not work but would be the ideal scenario. Hosts, whether auto-generated or aliased, are not shared or accessible within another service.

Confirmed here, https://gitlab.slack.com/archives/C0SFP840G/p1506632880000377

ex. job: https://gitlab.com/gitlab-org/gitlab-selenium-server/-/jobs/34442764

Dockerfile


FROM node:boron
WORKDIR /app
Install app dependencies
COPY package.json package-lock.json ./
RUN npm install
Bundle app source
COPY . .
EXPOSE 4444
CMD ["npm", "start"]

.gitlab-ci.yml


image: node:boron
variables:

docker.for.mac.localhost hostname via https://stackoverflow.com/a/43541732/796832s

$DOCKER_REGISTRY = registry.gitlab.com or docker.for.mac.localhost:5063
$IMAGE_NAMESPACE = gitlab-org/gitlab-selenium-server or root/gitlab-selenium-server
We have to separate this instead of constructing from $DOCKER_REGISTRY and $IMAGE_NAMESPACE
because docker.for.mac.localhost:5063/root/gitlab-selenium-server:latest does
not work because the host should be localhost ¯_(ツ)_/¯
$IMAGE_FULL_NAME = registry.gitlab.com/gitlab-org/gitlab-selenium-server or localhost:5063/root/gitlab-selenium-server
stages:

build
test

build-docker-image:
stage: build
image: docker:latest
variables:
DOCKER_DRIVER: overlay2
script:
- docker info
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $DOCKER_REGISTRY
- docker build -t $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test --file="Dockerfile" .
- docker push $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test
services:
- name: docker:dind
command:
- "--insecure-registry"
# Unable to use $DOCKER_REGISTRY because it doesn't seem to work :/
# See https://gitlab.com/gitlab-org/gitlab-ce/issues/38346
- "docker.for.mac.localhost:5063"
tags:
- docker
- shared
THIS DOES NOT WORK.
HOSTS ARE NOT ACCESSIBLE IN ANOTHER SERVICE
test-docker-image:
stage: test
script:
# The GitLab Selenium Proxy fails to find the target http://actual-selenium-server:4444/wd/hub
- curl http://gitlab-selenium-proxy:4444/ || true
- curl http://gitlab-selenium-proxy:4444/wd/hub/status || true
- curl http://gitlab-selenium-proxy:4444/logs || true
variables:
SELENIUM_REMOTE_URL: http://gitlab-selenium-proxy:4444/wd/hub
GITLAB_TARGET_SELENIUM_REMOTE_URL: http://actual-selenium-server:4444/wd/hub
services:
# To figure out the host name of a service, see https://docs.gitlab.com/runner/executors/docker.html#accessing-the-services
# ---
#
# See https://github.com/SeleniumHQ/docker-selenium
- name: selenium/standalone-chrome:3.4.0-francium
# For reference, the default host is selenium__standalone-chrome:4444
alias: actual-selenium-server
- name: $IMAGE_FULL_NAME:test
# For reference, the default host is when using a local registry, localhost__root__gitlab-selenium-server__selenium-server:4444
alias: gitlab-selenium-proxy
tags:
- docker
- shared

Create an image with the service already inside ✔

Another alternative is to create an image FROM selenium/standalone-chrome:3.4.0-francium and include everything it needs(like Node and npm) inside so you can just use it as a GitLab CI service.

This is probably the cleanest solution that actually works at the moment. BUT it bloats the image size, stuck to a single Selenium Server in this case Chrome(unless we want to pack things ourselves), and there isn't a slick way to merge two images together because there can't be two separate base images (ideally just merge node:boron and selenium/standalone-chrome:3.4.0-francium)

Dockerfile


# Image stack:
# selenium/standalone-chrome ->
# selenium/node-chrome -> https://github.com/SeleniumHQ/docker-selenium/blob/3fb76ba36cfa241a7927d432fc162553063259c3/StandaloneChrome/Dockerfile#L5
# selenium/node-base -> https://github.com/SeleniumHQ/docker-selenium/blob/3fb76ba36cfa241a7927d432fc162553063259c3/NodeChrome/Dockerfile#L5
# selenium/base -> https://github.com/SeleniumHQ/docker-selenium/blob/3fb76ba36cfa241a7927d432fc162553063259c3/NodeBase/Dockerfile#L5
# ubuntu:16.04 -> https://github.com/SeleniumHQ/docker-selenium/blob/b5262f1e70cc1927cd31622b668e67628adf4777/Base/Dockerfile#L1
FROM selenium/standalone-chrome:3.4.0-francium
USER root
WORKDIR /app
ENV NVM_DIR /usr/local/nvm
ENV NODE_VERSION 6.11.3
Install nvm
RUN wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.33.4/install.sh | bash
Install Node and npm
RUN . $NVM_DIR/nvm.sh 
&& nvm install $NODE_VERSION 
&& nvm alias default $NODE_VERSION 
&& nvm use default
Add Node and npm to path so the commands are available
ENV NODE_PATH $NVM_DIR/v$NODE_VERSION/lib/node_modules
ENV PATH $NVM_DIR/versions/node/v$NODE_VERSION/bin:$PATH
Confirm installation
RUN node -v
RUN npm -v
Install app dependencies
COPY package.json package-lock.json ./
RUN npm install
Bundle app source
COPY . .
EXPOSE 4545
We have to combine these because there can only be one CMD
and it is overridden from the base images.

Start the Selenium server
This is copied over https://github.com/SeleniumHQ/docker-selenium/blob/b5262f1e70cc1927cd31622b668e67628adf4777/StandaloneChrome/Dockerfile#L13

And is normally run as a CMD here, https://github.com/SeleniumHQ/docker-selenium/blob/b5262f1e70cc1927cd31622b668e67628adf4777/NodeBase/Dockerfile#L63


And then start the app
CMD ["/bin/bash", "-c", "/opt/bin/entry_point.sh & npm start"]

.gitlab-ci.yml


image: node:boron
variables:

docker.for.mac.localhost hostname via https://stackoverflow.com/a/43541732/796832s

$DOCKER_REGISTRY = registry.gitlab.com or docker.for.mac.localhost:5063
$IMAGE_NAMESPACE = gitlab-org/gitlab-selenium-server or root/gitlab-selenium-server
We have to separate this instead of constructing from $DOCKER_REGISTRY and $IMAGE_NAMESPACE
because docker.for.mac.localhost:5063/root/gitlab-selenium-server:latest does
not work because the host should be localhost ¯_(ツ)_/¯
$IMAGE_FULL_NAME = registry.gitlab.com/gitlab-org/gitlab-selenium-server or localhost:5063/root/gitlab-selenium-server
stages:

build
test

build-docker-image:
stage: build
image: docker:latest
variables:
DOCKER_DRIVER: overlay2
script:
- docker info
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $DOCKER_REGISTRY
- docker build -t $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test --file="Dockerfile" .
- docker push $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test
services:
- name: docker:dind
command:
- "--insecure-registry"
# Unable to use $DOCKER_REGISTRY because it doesn't seem to work :/
# See https://gitlab.com/gitlab-org/gitlab-ce/issues/38346
- "docker.for.mac.localhost:5063"
tags:
- docker
- shared
test-docker-image:
stage: test
script:
- curl http://gitlab-selenium-proxy:4545/ || true
- curl http://gitlab-selenium-proxy:4545/wd/hub/status || true
# Show the logs for the GitLab Selenium Server service
- curl http://gitlab-selenium-proxy:4545/logs || true
variables:
SELENIUM_REMOTE_URL: http://gitlab-selenium-proxy:4545/wd/hub
services:
- name: $IMAGE_FULL_NAME:test
# For reference, the default host is when using a local registry, localhost__root__gitlab-selenium-server__selenium-server:4545
alias: gitlab-selenium-proxy
tags:
- docker
- shared

Using Docker in Docker (dind) ✔

Running our own Docker in Docker (dind) image, we can setup a network that will share the host names. But this is cumbersome and requires a lot of extra boilerplate. And from my testing, seems a bit flakey even with the explicit waits for running and sleep times.

ex. job: https://gitlab.com/gitlab-org/gitlab-selenium-server/-/jobs/34442765

Dockerfile


FROM node:boron
WORKDIR /app
Install app dependencies
COPY package.json package-lock.json ./
RUN npm install
Bundle app source
COPY . .
EXPOSE 4444
CMD ["npm", "start"]

.gitlab-ci.yml


image: node:boron
variables:

docker.for.mac.localhost hostname via https://stackoverflow.com/a/43541732/796832s

$DOCKER_REGISTRY = registry.gitlab.com or docker.for.mac.localhost:5063
$IMAGE_NAMESPACE = gitlab-org/gitlab-selenium-server or root/gitlab-selenium-server
We have to separate this instead of constructing from $DOCKER_REGISTRY and $IMAGE_NAMESPACE
because docker.for.mac.localhost:5063/root/gitlab-selenium-server:latest does
not work because the host should be localhost ¯_(ツ)_/¯
$IMAGE_FULL_NAME = registry.gitlab.com/gitlab-org/gitlab-selenium-server or localhost:5063/root/gitlab-selenium-server
stages:

build
test

build-docker-image:
stage: build
image: docker:latest
variables:
DOCKER_DRIVER: overlay2
script:
- docker info
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $DOCKER_REGISTRY
- docker build -t $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test --file="Dockerfile" .
- docker push $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test
services:
- name: docker:dind
command:
- "--insecure-registry"
# Unable to use $DOCKER_REGISTRY because it doesn't seem to work :/
# See https://gitlab.com/gitlab-org/gitlab-ce/issues/38346
- "docker.for.mac.localhost:5063"
tags:
- docker
- shared
test-docker-image:
stage: test
image: docker:latest
variables:
DOCKER_DRIVER: overlay
SELENIUM_REMOTE_URL: http://gitlab-selenium-proxy:4444/wd/hub
GITLAB_TARGET_SELENIUM_REMOTE_URL: http://actual-selenium-server:4444/wd/hub
script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $DOCKER_REGISTRY
- docker pull selenium/standalone-chrome:3.4.0-francium
- docker pull $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test
- docker pull node:boron
- docker network create some-network
# We are running against, docker: poorly formatted environment: variable '-----END CERTIFICATE-----' has white spaces.
# So we are just getting rid of the whole DOCKER_ENV_CI_SERVER_TLS_CA_FILE entry,
# See related issue, https://github.com/mesosphere/marathon-lb/issues/433
- env | sed '/^DOCKER_ENV_CI_SERVER_TLS_CA_FILE=/,/^$/d' > .env
- docker run -d --name=actual-selenium-server --net=some-network -p 4545:4444 --env-file=.env selenium/standalone-chrome:3.4.0-francium
- docker run -d --name=gitlab-selenium-proxy --net=some-network -p 4444:4444 --env-file=.env $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test
- docker run -d --name=node-land --net=some-network --volume "$PWD":/root --env-file=.env --tty --interactive node:boron /bin/bash
- until [ "docker inspect -f {{.State.Running}} actual-selenium-server" == "true" ]; do sleep 0.1; done;
- until [ "docker inspect -f {{.State.Running}} gitlab-selenium-proxy" == "true" ]; do sleep 0.1; done;
- until [ "docker inspect -f {{.State.Running}} node-land" == "true" ]; do sleep 0.1; done;
# Doesn't seem to work without this :/
- sleep 5
- docker ps --all
# Now we can actually run our commands *phew
- docker exec node-land curl http://gitlab-selenium-proxy:4444/ || true
- docker exec node-land curl http://gitlab-selenium-proxy:4444/wd/hub/status || true
- docker exec node-land curl http://gitlab-selenium-proxy:4444/logs || true
services:
- name: docker:dind
command:
- "--insecure-registry"
# Unable to use $DOCKER_REGISTRY because it doesn't seem to work :/
# See https://gitlab.com/gitlab-org/gitlab-ce/issues/38346
- "docker.for.mac.localhost:5063"
tags:
- docker
- shared

Edited Jan 05, 2021 by Eric Eastwood

Accessing a Docker container service from another container service (hosts and network)

Docker network per build (gitlab-runner#1042 (closed)) ❔

Share the /etc/hosts file to Docker image via services entrypoint ✔

Grab the IP from the /etc/hosts file ✔

Access a service host in another service ❌

Create an image with the service already inside ✔

Using Docker in Docker (dind) ✔

Share the `/etc/hosts` file to Docker image via `services` `entrypoint` ✔

Grab the IP from the `/etc/hosts` file ✔