Docker container names exceed maximum allowed DNS hostname length
Summary
Generated docker container names by Gitlab Runner exceed the maximum allowed DNS hostname length (63 characters) which results in reverse DNS queries failing.
RFC1034 states that each part of a DNS name must be between 0 and 63 characters long, and the total DNS name must not exceed 253 characters.
Gitlab runner has generated the following container name:
runner-jl9klc-project-30504-concurrent-0-fd6217607b395033-build-3.runner-jl9klc-project-30504-concurrent-0-job-1297211-network
(host part 65 characters, network part 60 characters)
When attempting to do forward/reverse DNS lookups, I observe the following:
nslookup build
Server: 127.0.0.11
Address: 127.0.0.11#53
Non-authoritative answer:
Name: build
Address: 172.18.0.2
nslookup 172.18.0.2
;; connection timed out; no servers could be reached
The docker daemon log outputs the following error:
Apr 13 08:20:06 test-docker.novalocal dockerd[15371]: time="2021-04-13T08:20:06.417125309Z" level=debug msg="[resolver] lookup for IP 2.0.19.172: name runner-jl9klc-project-30504-concurrent-0-fd6217607b395033-build-3.runner-jl9klc-project-30504-concurrent-0-job-1297211-network"
Apr 13 08:20:06 test-docker.novalocal dockerd[15371]: time="2021-04-13T08:20:06.417150733Z" level=error msg="[resolver] error writing resolver resp, dns: bad rdata"
Steps to reproduce
Sample .gitlab-ci.yml below which reproduces the issue
.gitlab-ci.yml
image: tutum/dnsutils
variables:
FF_NETWORK_PER_BUILD: 1
MYSQL_ALLOW_EMPTY_PASSWORD: "yes"
services:
- name: mariadb:latest
build:
stage: build
script:
- dig mariadb
- nslookup $(dig mariadb +short)
Actual behavior
IP addresses should be able to be resolved into hostnames using nslookup
or dig -x
Expected behavior
Docker daemon DNS throws an error and no DNS response is received.
Relevant logs and/or screenshots
job log
[0;m[32;1m$ dig mariadb[0;m
; <<>> DiG 9.9.5-3ubuntu0.2-Ubuntu <<>> mariadb
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30065
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;mariadb. IN A
;; ANSWER SECTION:
mariadb. 600 IN A 172.24.0.2
;; Query time: 0 msec
;; SERVER: 127.0.0.11#53(127.0.0.11)
;; WHEN: Tue Apr 13 09:37:27 UTC 2021
;; MSG SIZE rcvd: 48
[32;1m$ nslookup $(dig mariadb +short)[0;m
;; connection timed out; no servers could be reached
section_end:1618306662:step_script
Environment description
Host OS CentOS 7
Server: Docker Engine - Community
Engine:
Version: 20.10.6
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: 8728dd2
Built: Fri Apr 9 22:43:57 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.4
GitCommit: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
runc:
Version: 1.0.0-rc93
GitCommit: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
docker-init:
Version: 0.19.0
GitCommit: de40ad0
config.toml contents
concurrent = 1
check_interval = 0
[session_server]
session_timeout = 1800
[[runners]]
name = "dev-cit-2"
url = "https://xxx/"
token = "xxx"
executor = "docker"
pre_clone_script = "git config --global http.proxy proxy:8080; git config --global https.proxy proxy:8080"
[runners.feature_flags]
FF_NETWORK_PER_BUILD = true
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "maven:3-jdk-11"
privileged = false
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/cache"]
shm_size = 0
Used GitLab Runner version
Running with gitlab-runner 13.10.0 (54944146)
on dev-cit-2 _JL9kL_c
feature flags: FF_NETWORK_PER_BUILD:true
Possible fixes
Rethink naming strategy of containers.
It seems that the project projectUniqueName
it potentially too large as it contains a lot of information, leaving very little room for the container image name.
Additionally when generating projectUniqueName
the result is passed through dns.MakeRFC1123Compatible()
which correctly limits the size, but this is not invoked again after appending the image name to the end.