Accessing a Docker container service from another container service (hosts and network)
In GitLab CI, I want to be able to access a container listed under services
from another container listed under services
. Unfortunately, it doesn't seem possible yet because services
hosts are not shared to the other services
containers (see "Access a service host in another service" below"). And there isn't a way to define a network
that the container
should belong to with the services
syntax.
Here are the avenues, trials, and tribulations I have explored...
If you are curious how I setup the Docker registry locally, see https://gitlab.com/snippets/73109#setup-the-container-registry
Perhaps you are curious about my SSL issues trying to get a secure registry. But I just ended up using the instructions just above to create an insecure registry.
Some docs
- https://docs.gitlab.com/runner/executors/docker.html#accessing-the-services
- https://docs.gitlab.com/ce/ci/docker/using_docker_images.html
gitlab-runner#1042 (closed)) ❔
Docker network per build (I've never used this new method yet but is exactly what is needed so probably works great.
See gitlab!43875 (diffs) for how to enable FF_NETWORK_PER_BUILD
.
/etc/hosts
file to Docker image via services
entrypoint
✔
Share the See gitlab-runner#1042 (comment 144420147)
/etc/hosts
file ✔
Grab the IP from the See https://gitlab.com/MadLittleMods/gitlab-ci-access-main-container-from-service-container
❌
Access a service host in another service Update: Nowadays see #docker-network-per-build-httpsgitlabcomgitlab-orggitlab-runner-issues1042-grey_question
just above
This does not work but would be the ideal scenario. Hosts, whether auto-generated or aliased, are not shared or accessible within another service.
Confirmed here, https://gitlab.slack.com/archives/C0SFP840G/p1506632880000377
ex. job: https://gitlab.com/gitlab-org/gitlab-selenium-server/-/jobs/34442764
Dockerfile
FROM node:boron
WORKDIR /app
Install app dependencies
COPY package.json package-lock.json ./
RUN npm install
Bundle app source
COPY . .
EXPOSE 4444
CMD ["npm", "start"]
.gitlab-ci.yml
image: node:boron
variables:
docker.for.mac.localhost hostname via https://stackoverflow.com/a/43541732/796832s
$DOCKER_REGISTRY = registry.gitlab.com or docker.for.mac.localhost:5063
$IMAGE_NAMESPACE = gitlab-org/gitlab-selenium-server or root/gitlab-selenium-server
We have to separate this instead of constructing from $DOCKER_REGISTRY and $IMAGE_NAMESPACE
because docker.for.mac.localhost:5063/root/gitlab-selenium-server:latest does
not work because the host should be localhost ¯_(ツ)_/¯
$IMAGE_FULL_NAME = registry.gitlab.com/gitlab-org/gitlab-selenium-server or localhost:5063/root/gitlab-selenium-server
stages:
build
test
build-docker-image:
stage: build
image: docker:latest
variables:
DOCKER_DRIVER: overlay2
script:
- docker info
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $DOCKER_REGISTRY
- docker build -t $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test --file="Dockerfile" .
- docker push $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test
services:
- name: docker:dind
command:
- "--insecure-registry"
# Unable to use $DOCKER_REGISTRY because it doesn't seem to work :/
# See https://gitlab.com/gitlab-org/gitlab-ce/issues/38346
- "docker.for.mac.localhost:5063"
tags:
- docker
- shared
THIS DOES NOT WORK.
HOSTS ARE NOT ACCESSIBLE IN ANOTHER SERVICE
test-docker-image:
stage: test
script:
# The GitLab Selenium Proxy fails to find the target http://actual-selenium-server:4444/wd/hub
- curl http://gitlab-selenium-proxy:4444/ || true
- curl http://gitlab-selenium-proxy:4444/wd/hub/status || true
- curl http://gitlab-selenium-proxy:4444/logs || true
variables:
SELENIUM_REMOTE_URL: http://gitlab-selenium-proxy:4444/wd/hub
GITLAB_TARGET_SELENIUM_REMOTE_URL: http://actual-selenium-server:4444/wd/hub
services:
# To figure out the host name of a service, see https://docs.gitlab.com/runner/executors/docker.html#accessing-the-services
# ---
#
# See https://github.com/SeleniumHQ/docker-selenium
- name: selenium/standalone-chrome:3.4.0-francium
# For reference, the default host is selenium__standalone-chrome:4444
alias: actual-selenium-server
- name: $IMAGE_FULL_NAME:test
# For reference, the default host is when using a local registry, localhost__root__gitlab-selenium-server__selenium-server:4444
alias: gitlab-selenium-proxy
tags:
- docker
- shared
✔
Create an image with the service already inside Another alternative is to create an image FROM selenium/standalone-chrome:3.4.0-francium
and include everything it needs(like Node and npm) inside so you can just use it as a GitLab CI service.
This is probably the cleanest solution that actually works at the moment.
BUT it bloats the image size, stuck to a single Selenium Server in this case Chrome(unless we want to pack things ourselves), and there isn't a slick way to merge two images together because there can't be two separate base images (ideally just merge node:boron
and selenium/standalone-chrome:3.4.0-francium
)
Dockerfile
# Image stack:
# selenium/standalone-chrome ->
# selenium/node-chrome -> https://github.com/SeleniumHQ/docker-selenium/blob/3fb76ba36cfa241a7927d432fc162553063259c3/StandaloneChrome/Dockerfile#L5
# selenium/node-base -> https://github.com/SeleniumHQ/docker-selenium/blob/3fb76ba36cfa241a7927d432fc162553063259c3/NodeChrome/Dockerfile#L5
# selenium/base -> https://github.com/SeleniumHQ/docker-selenium/blob/3fb76ba36cfa241a7927d432fc162553063259c3/NodeBase/Dockerfile#L5
# ubuntu:16.04 -> https://github.com/SeleniumHQ/docker-selenium/blob/b5262f1e70cc1927cd31622b668e67628adf4777/Base/Dockerfile#L1
FROM selenium/standalone-chrome:3.4.0-francium
USER root
WORKDIR /app
ENV NVM_DIR /usr/local/nvm
ENV NODE_VERSION 6.11.3
Install nvm
RUN wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.33.4/install.sh | bash
Install Node and npm
RUN . $NVM_DIR/nvm.sh
&& nvm install $NODE_VERSION
&& nvm alias default $NODE_VERSION
&& nvm use default
Add Node and npm to path so the commands are available
ENV NODE_PATH $NVM_DIR/v$NODE_VERSION/lib/node_modules
ENV PATH $NVM_DIR/versions/node/v$NODE_VERSION/bin:$PATH
Confirm installation
RUN node -v
RUN npm -v
Install app dependencies
COPY package.json package-lock.json ./
RUN npm install
Bundle app source
COPY . .
EXPOSE 4545
We have to combine these because there can only be one CMD
and it is overridden from the base images.
Start the Selenium server
This is copied over https://github.com/SeleniumHQ/docker-selenium/blob/b5262f1e70cc1927cd31622b668e67628adf4777/StandaloneChrome/Dockerfile#L13
And is normally run as a CMD here, https://github.com/SeleniumHQ/docker-selenium/blob/b5262f1e70cc1927cd31622b668e67628adf4777/NodeBase/Dockerfile#L63
And then start the app
CMD ["/bin/bash", "-c", "/opt/bin/entry_point.sh & npm start"]
.gitlab-ci.yml
image: node:boron
variables:
docker.for.mac.localhost hostname via https://stackoverflow.com/a/43541732/796832s
$DOCKER_REGISTRY = registry.gitlab.com or docker.for.mac.localhost:5063
$IMAGE_NAMESPACE = gitlab-org/gitlab-selenium-server or root/gitlab-selenium-server
We have to separate this instead of constructing from $DOCKER_REGISTRY and $IMAGE_NAMESPACE
because docker.for.mac.localhost:5063/root/gitlab-selenium-server:latest does
not work because the host should be localhost ¯_(ツ)_/¯
$IMAGE_FULL_NAME = registry.gitlab.com/gitlab-org/gitlab-selenium-server or localhost:5063/root/gitlab-selenium-server
stages:
build
test
build-docker-image:
stage: build
image: docker:latest
variables:
DOCKER_DRIVER: overlay2
script:
- docker info
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $DOCKER_REGISTRY
- docker build -t $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test --file="Dockerfile" .
- docker push $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test
services:
- name: docker:dind
command:
- "--insecure-registry"
# Unable to use $DOCKER_REGISTRY because it doesn't seem to work :/
# See https://gitlab.com/gitlab-org/gitlab-ce/issues/38346
- "docker.for.mac.localhost:5063"
tags:
- docker
- shared
test-docker-image:
stage: test
script:
- curl http://gitlab-selenium-proxy:4545/ || true
- curl http://gitlab-selenium-proxy:4545/wd/hub/status || true
# Show the logs for the GitLab Selenium Server service
- curl http://gitlab-selenium-proxy:4545/logs || true
variables:
SELENIUM_REMOTE_URL: http://gitlab-selenium-proxy:4545/wd/hub
services:
- name: $IMAGE_FULL_NAME:test
# For reference, the default host is when using a local registry, localhost__root__gitlab-selenium-server__selenium-server:4545
alias: gitlab-selenium-proxy
tags:
- docker
- shared
✔
Using Docker in Docker (dind) Running our own Docker in Docker (dind) image, we can setup a network that will share the host names. But this is cumbersome and requires a lot of extra boilerplate. And from my testing, seems a bit flakey even with the explicit waits for running
and sleep times.
ex. job: https://gitlab.com/gitlab-org/gitlab-selenium-server/-/jobs/34442765
Dockerfile
FROM node:boron
WORKDIR /app
Install app dependencies
COPY package.json package-lock.json ./
RUN npm install
Bundle app source
COPY . .
EXPOSE 4444
CMD ["npm", "start"]
.gitlab-ci.yml
image: node:boron
variables:
docker.for.mac.localhost hostname via https://stackoverflow.com/a/43541732/796832s
$DOCKER_REGISTRY = registry.gitlab.com or docker.for.mac.localhost:5063
$IMAGE_NAMESPACE = gitlab-org/gitlab-selenium-server or root/gitlab-selenium-server
We have to separate this instead of constructing from $DOCKER_REGISTRY and $IMAGE_NAMESPACE
because docker.for.mac.localhost:5063/root/gitlab-selenium-server:latest does
not work because the host should be localhost ¯_(ツ)_/¯
$IMAGE_FULL_NAME = registry.gitlab.com/gitlab-org/gitlab-selenium-server or localhost:5063/root/gitlab-selenium-server
stages:
build
test
build-docker-image:
stage: build
image: docker:latest
variables:
DOCKER_DRIVER: overlay2
script:
- docker info
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $DOCKER_REGISTRY
- docker build -t $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test --file="Dockerfile" .
- docker push $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test
services:
- name: docker:dind
command:
- "--insecure-registry"
# Unable to use $DOCKER_REGISTRY because it doesn't seem to work :/
# See https://gitlab.com/gitlab-org/gitlab-ce/issues/38346
- "docker.for.mac.localhost:5063"
tags:
- docker
- shared
test-docker-image:
stage: test
image: docker:latest
variables:
DOCKER_DRIVER: overlay
SELENIUM_REMOTE_URL: http://gitlab-selenium-proxy:4444/wd/hub
GITLAB_TARGET_SELENIUM_REMOTE_URL: http://actual-selenium-server:4444/wd/hub
script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $DOCKER_REGISTRY
- docker pull selenium/standalone-chrome:3.4.0-francium
- docker pull $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test
- docker pull node:boron
- docker network create some-network
# We are running against, docker: poorly formatted environment: variable '-----END CERTIFICATE-----' has white spaces.
# So we are just getting rid of the whole DOCKER_ENV_CI_SERVER_TLS_CA_FILE entry,
# See related issue, https://github.com/mesosphere/marathon-lb/issues/433
- env | sed '/^DOCKER_ENV_CI_SERVER_TLS_CA_FILE=/,/^$/d' > .env
- docker run -d --name=actual-selenium-server --net=some-network -p 4545:4444 --env-file=.env selenium/standalone-chrome:3.4.0-francium
- docker run -d --name=gitlab-selenium-proxy --net=some-network -p 4444:4444 --env-file=.env $DOCKER_REGISTRY/$IMAGE_NAMESPACE:test
- docker run -d --name=node-land --net=some-network --volume "$PWD":/root --env-file=.env --tty --interactive node:boron /bin/bash
- until [ "docker inspect -f {{.State.Running}} actual-selenium-server" == "true" ]; do sleep 0.1; done;
- until [ "docker inspect -f {{.State.Running}} gitlab-selenium-proxy" == "true" ]; do sleep 0.1; done;
- until [ "docker inspect -f {{.State.Running}} node-land" == "true" ]; do sleep 0.1; done;
# Doesn't seem to work without this :/
- sleep 5
- docker ps --all
# Now we can actually run our commands *phew
- docker exec node-land curl http://gitlab-selenium-proxy:4444/ || true
- docker exec node-land curl http://gitlab-selenium-proxy:4444/wd/hub/status || true
- docker exec node-land curl http://gitlab-selenium-proxy:4444/logs || true
services:
- name: docker:dind
command:
- "--insecure-registry"
# Unable to use $DOCKER_REGISTRY because it doesn't seem to work :/
# See https://gitlab.com/gitlab-org/gitlab-ce/issues/38346
- "docker.for.mac.localhost:5063"
tags:
- docker
- shared