Commit e3392bdf authored by Steve Azzopardi's avatar Steve Azzopardi

Merge branch 'master' into 'master'

Improve documentation concerning proxy setting in the case of docker-in-docker-executor

See merge request gitlab-org/gitlab-runner!1090
parents fed7fcfa ea98d62a
Pipeline #46677850 passed with stages
in 33 minutes and 16 seconds
......@@ -139,3 +139,72 @@ environment = ["HTTPS_PROXY=docker0_interface_ip:3128", "HTTP_PROXY=docker0_inte
Where `docker0_interface_ip` is the IP address of the `docker0` interface. You need to
be able to reach it from within the Docker containers, so it's important to set
it right.
## Proxy settings when using dind service
When using the [docker-in-docker executor](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker-executor) (dind),
it can be necessary to specify `docker:2375,docker:2376` in the `NO_PROXY`
environment variable. This is because the proxy intercepts the TCP connection between:
- `dockerd` from the dind container.
- `docker` from the client container.
The ports can be required because otherwise `docker push` will be blocked
as it originates from the IP mapped to docker. However, in that case, it is meant to go through the proxy.
When testing the communication between `dockerd` from dind and a `docker` client locally
(as described here: https://hub.docker.com/_/docker/),
`dockerd` from dind is initially started as a client on the host system by root,
and the proxy variables are taken from `/root/.docker/config.json`.
For example:
```json
{
"proxies": {
"default": {
"httpProxy": "http://proxy:8080",
"httpsProxy": "http://proxy:8080",
"noProxy": "docker:2375,docker:2376"
}
}
}
```
However, the container started for executing `.gitlab-ci.yml` scripts will have
the environment variables set by the settings of the `gitlab-runner` configuration (`/etc/gitlab-runner/config.toml`).
These are available as environment variables as is (in contrast to `.docker/config.json` of the local test above)
in the dind containers running `dockerd` as a service and `docker` client executing `.gitlab-ci.yml`.
In `.gitlab-ci.yml`, the environment variables will be picked up by any program honouring the proxy settings from default environment variables. For example,
`wget`, `apt`, `apk`, `docker info` and `docker pull` (but not by `docker run` or `docker build` as per:
https://github.com/moby/moby/issues/24697#issuecomment-366680499).
`docker run` or `docker build` executed inside the container of the docker executor
will look for the proxy settings in `$HOME/.docker/config.json`,
which is now inside the executor container (and initally empty).
Therefore, `docker run` or `docker build` executions will have no proxy settings. In order to pass on the settings,
a `$HOME/.docker/config.json` needs to be created in the executor container. For example:
```yaml
before_script:
- mkdir -p $HOME/.docker/
- 'echo "{ \"proxies\": { \"default\": { \"httpProxy\": \"$HTTP_PROXY\", \"httpsProxy\": \"$HTTPS_PROXY\", \"noProxy\": \"$NO_PROXY\" } } }" > $HOME/.docker/config.json'
```
Because it is confusing to add additional lines in a `.gitlab-ci.yml` file that are only needed in case of a proxy,
it is better to move the creation of the `$HOME/.docker/config.json` into the
configuration of the `gitlab-runner` (`/etc/gitlab-runner/config.toml`) that is actually affected:
```toml
[[runners]]
pre_build_script = "mkdir -p $HOME/.docker/ && echo \"{ \\\"proxies\\\": { \\\"default\\\": { \\\"httpProxy\\\": \\\"$HTTP_PROXY\\\", \\\"httpsProxy\\\": \\\"$HTTPS_PROXY\\\", \\\"noProxy\\\": \\\"$NO_PROXY\\\" } } }\" > $HOME/.docker/config.json"
```
NOTE: **Note:**
An additional level of escaping `"` is needed here because this is the creation of a
JSON file with a shell specified as a single string inside a TOML file.
Because it is not YAML anymore, do not escape the `:`.
Note that if the `NO_PROXY` list needs to be extended, wildcards `*` only work for suffixes
but not for prefixes or CIDR notation.
For more information, see
<https://github.com/moby/moby/issues/9145>
and
<https://unix.stackexchange.com/questions/23452/set-a-network-range-in-the-no-proxy-environment-variable>.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment