using_docker_build.md 13.5 KB
Newer Older
1 2
# Using Docker Build

3
GitLab CI allows you to use Docker Engine to build and test docker-based projects.
4 5 6

**This also allows to you to use `docker-compose` and other docker-enabled tools.**

7
One of the new trends in Continuous Integration/Deployment is to:
8

9 10
1. create an application image,
1. run tests against the created image,
11 12
1. push image to a remote registry, and
1. deploy to a server from the pushed image.
13

14
It's also useful when your application already has the `Dockerfile` that can be used to create and test an image:
15

16 17 18 19 20 21 22
```bash
$ docker build -t my-image dockerfiles/
$ docker run my-docker-image /script/to/run/tests
$ docker tag my-image my-registry:5000/my-image
$ docker push my-registry:5000/my-image
```

23
This requires special configuration of GitLab Runner to enable `docker` support during jobs.
24

25 26
## Runner Configuration

27
There are three methods to enable the use of `docker build` and `docker run` during jobs; each with their own tradeoffs.
28 29

### Use shell executor
30 31

The simplest approach is to install GitLab Runner in `shell` execution mode.
32
GitLab Runner then executes job scripts as the `gitlab-runner` user.
33 34 35

1. Install [GitLab Runner](https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/#installation).

36
1. During GitLab Runner installation select `shell` as method of executing job scripts or use command:
37 38

    ```bash
39
    sudo gitlab-ci-multi-runner register -n \
40
      --url https://gitlab.com/ \
Mark Pundsack's avatar
Mark Pundsack committed
41
      --registration-token REGISTRATION_TOKEN \
42
      --executor shell \
43 44 45
      --description "My Runner"
    ```

46
2. Install Docker Engine on server.
47

48 49
    For more information how to install Docker Engine on different systems
    checkout the [Supported installations](https://docs.docker.com/engine/installation/).
50 51

3. Add `gitlab-runner` user to `docker` group:
52

53
    ```bash
54
    sudo usermod -aG docker gitlab-runner
55 56 57
    ```

4. Verify that `gitlab-runner` has access to Docker:
58

59
    ```bash
60
    sudo -u gitlab-runner -H docker info
61
    ```
62

63
    You can now verify that everything works by adding `docker info` to `.gitlab-ci.yml`:
64

65 66 67
    ```yaml
    before_script:
      - docker info
68

69 70 71 72 73 74 75 76
    build_image:
      script:
        - docker build -t my-docker-image .
        - docker run my-docker-image /script/to/run/tests
    ```

5. You can now use `docker` command and install `docker-compose` if needed.

77
> **Note:**
Mark Pundsack's avatar
Mark Pundsack committed
78 79
* By adding `gitlab-runner` to the `docker` group you are effectively granting `gitlab-runner` full root permissions.
For more information please read [On Docker security: `docker` group considered harmful](https://www.andreas-jung.com/contents/on-docker-security-docker-group-considered-harmful).
80

81
### Use docker-in-docker executor
82

83 84
The second approach is to use the special docker-in-docker (dind)
[Docker image](https://hub.docker.com/_/docker/) with all tools installed
85
(`docker` and `docker-compose`) and run the job script in context of that
86 87
image in privileged mode.

Mark Pundsack's avatar
Mark Pundsack committed
88
In order to do that, follow the steps:
89

90
1. Install [GitLab Runner](https://docs.gitlab.com/runner/install).
91

92 93
1. Register GitLab Runner from the command line to use `docker` and `privileged`
   mode:
94 95

    ```bash
Mark Pundsack's avatar
Mark Pundsack committed
96
    sudo gitlab-ci-multi-runner register -n \
97
      --url https://gitlab.com/ \
Mark Pundsack's avatar
Mark Pundsack committed
98
      --registration-token REGISTRATION_TOKEN \
99 100
      --executor docker \
      --description "My Docker Runner" \
101
      --docker-image "docker:latest" \
102 103
      --docker-privileged
    ```
104

105 106 107 108 109
    The above command will register a new Runner to use the special
    `docker:latest` image which is provided by Docker. **Notice that it's using
    the `privileged` mode to start the build and service containers.** If you
    want to use [docker-in-docker] mode, you always have to use `privileged = true`
    in your Docker containers.
110 111 112 113 114

    The above command will create a `config.toml` entry similar to this:

    ```
    [[runners]]
115
      url = "https://gitlab.com/"
116 117 118 119 120 121 122 123 124 125 126 127
      token = TOKEN
      executor = "docker"
      [runners.docker]
        tls_verify = false
        image = "docker:latest"
        privileged = true
        disable_cache = false
        volumes = ["/cache"]
      [runners.cache]
        Insecure = false
    ```

128 129
1. You can now use `docker` in the build script (note the inclusion of the
   `docker:dind` service):
130

131
    ```yaml
132 133
    image: docker:latest

134 135 136 137 138
    # When using dind, it's wise to use the overlayfs driver for
    # improved performance.
    variables:
      DOCKER_DRIVER: overlay

139 140 141
    services:
    - docker:dind

142
    before_script:
143
    - docker info
144 145 146

    build:
      stage: build
147
      script:
148 149
      - docker build -t my-docker-image .
      - docker run my-docker-image /script/to/run/tests
150 151
    ```

152 153 154 155 156 157 158 159
Docker-in-Docker works well, and is the recommended configuration, but it is
not without its own challenges:

- By enabling `--docker-privileged`, you are effectively disabling all of
  the security mechanisms of containers and exposing your host to privilege
  escalation which can lead to container breakout. For more information, check
  out the official Docker documentation on
  [Runtime privilege and Linux capabilities][docker-cap].
160 161
- When using docker-in-docker, each job is in a clean environment without the past
  history. Concurrent jobs work fine because every build gets it's own
162
  instance of Docker engine so they won't conflict with each other. But this
163
  also means jobs can be slower because there's no caching of layers.
164 165 166
- By default, `docker:dind` uses `--storage-driver vfs` which is the slowest
  form offered. To use a different driver, see
  [Using the overlayfs driver](#using-the-overlayfs-driver).
167 168

An example project using this approach can be found here: https://gitlab.com/gitlab-examples/docker.
169

170
### Use Docker socket binding
171 172 173

The third approach is to bind-mount `/var/run/docker.sock` into the container so that docker is available in the context of that image.

Mark Pundsack's avatar
Mark Pundsack committed
174
In order to do that, follow the steps:
175

176
1. Install [GitLab Runner](https://docs.gitlab.com/runner/install).
177

Mark Pundsack's avatar
Mark Pundsack committed
178
1. Register GitLab Runner from the command line to use `docker` and share `/var/run/docker.sock`:
179 180

    ```bash
Mark Pundsack's avatar
Mark Pundsack committed
181
    sudo gitlab-ci-multi-runner register -n \
182
      --url https://gitlab.com/ \
Mark Pundsack's avatar
Mark Pundsack committed
183
      --registration-token REGISTRATION_TOKEN \
184 185 186 187 188 189 190 191
      --executor docker \
      --description "My Docker Runner" \
      --docker-image "docker:latest" \
      --docker-volumes /var/run/docker.sock:/var/run/docker.sock
    ```

    The above command will register a new Runner to use the special
    `docker:latest` image which is provided by Docker. **Notice that it's using
192 193 194
    the Docker daemon of the Runner itself, and any containers spawned by docker
    commands will be siblings of the Runner rather than children of the runner.**
    This may have complications and limitations that are unsuitable for your workflow.
195 196 197 198 199

    The above command will create a `config.toml` entry similar to this:

    ```
    [[runners]]
200
      url = "https://gitlab.com/"
201
      token = REGISTRATION_TOKEN
202 203 204 205 206 207
      executor = "docker"
      [runners.docker]
        tls_verify = false
        image = "docker:latest"
        privileged = false
        disable_cache = false
208
        volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
209 210 211 212
      [runners.cache]
        Insecure = false
    ```

213 214
1. You can now use `docker` in the build script (note that you don't need to
   include the `docker:dind` service as when using the Docker in Docker executor):
215 216 217 218 219 220 221 222 223 224 225 226 227 228

    ```yaml
    image: docker:latest

    before_script:
    - docker info

    build:
      stage: build
      script:
      - docker build -t my-docker-image .
      - docker run my-docker-image /script/to/run/tests
    ```

229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245
While the above method avoids using Docker in privileged mode, you should be
aware of the following implications:

- By sharing the docker daemon, you are effectively disabling all
  the security mechanisms of containers and exposing your host to privilege
  escalation which can lead to container breakout. For example, if a project
  ran `docker rm -f $(docker ps -a -q)` it would remove the GitLab Runner
  containers.
- Concurrent jobs may not work; if your tests
  create containers with specific names, they may conflict with each other.
- Sharing files and directories from the source repo into containers may not
  work as expected since volume mounting is done in the context of the host
  machine, not the build container, e.g.:

    ```
    docker run --rm -t -i -v $(pwd)/src:/home/app/src test-image:latest run_app_tests
    ```
Mark Pundsack's avatar
Mark Pundsack committed
246

247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280
## Using the OverlayFS driver

By default, when using `docker:dind`, Docker uses the `vfs` storage driver which
copies the filesystem on every run. This is a very disk-intensive operation
which can be avoided if a different driver is used, for example `overlay`.

1. Make sure a recent kernel is used, preferably `>= 4.2`.
1. Check whether the `overlay` module is loaded:

    ```
    sudo lsmod | grep overlay
    ```

    If you see no result, then it isn't loaded. To load it use:

    ```
    sudo modprobe overlay
    ```

    If everything went fine, you need to make sure module is loaded on reboot.
    On Ubuntu systems, this is done by editing `/etc/modules`. Just add the
    following line into it:

    ```
    overlay
    ```

1. Use the driver by defining a variable at the top of your `.gitlab-ci.yml`:

    ```
    variables:
      DOCKER_DRIVER: overlay
    ```

281 282
## Using the GitLab Container Registry

283 284
> **Notes:**
- This feature requires GitLab 8.8 and GitLab Runner 1.2.
285 286 287
- Starting from GitLab 8.12, if you have [2FA] enabled in your account, you need
  to pass a [personal access token][pat] instead of your password in order to
  login to GitLab's Container Registry.
288

289 290 291 292
Once you've built a Docker image, you can push it up to the built-in
[GitLab Container Registry](../../user/project/container_registry.md). For example,
if you're using docker-in-docker on your runners, this is how your `.gitlab-ci.yml`
could look like:
293 294

```yaml
295
 build:
Mark Pundsack's avatar
Mark Pundsack committed
296
   image: docker:latest
297 298
   services:
   - docker:dind
299 300
   stage: build
   script:
301
     - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN registry.example.com
302 303
     - docker build -t registry.example.com/group/project/image:latest .
     - docker push registry.example.com/group/project/image:latest
304 305
```

306 307
You have to use the special `gitlab-ci-token` user created for you in order to
push to the Registry connected to your project. Its password is provided in the
308
`$CI_JOB_TOKEN` variable. This allows you to automate building and deployment
309
of your Docker images.
310

311 312 313 314 315 316 317
You can also make use of [other variables](../variables/README.md) to avoid hardcoding:

```yaml
services:
  - docker:dind

variables:
318
  IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME
319 320

before_script:
321
  - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
322 323 324 325 326 327 328 329 330

build:
  stage: build
  script:
    - docker build -t $IMAGE_TAG .
    - docker push $IMAGE_TAG
```

Here, `$CI_REGISTRY_IMAGE` would be resolved to the address of the registry tied
331
to this project, and `$CI_COMMIT_REF_NAME` would be resolved to the branch or
332 333 334
tag name for this particular job. We also declare our own variable, `$IMAGE_TAG`,
combining the two to save us some typing in the `script` section.

335
Here's a more elaborate example that splits up the tasks into 4 pipeline stages,
336
including two tests that run in parallel. The `build` is stored in the container
337 338 339 340 341
registry and used by subsequent stages, downloading the image
when needed. Changes to `master` also get tagged as `latest` and deployed using
an application-specific deploy script:

```yaml
Mark Pundsack's avatar
Mark Pundsack committed
342
image: docker:latest
343 344 345 346 347 348 349 350 351 352
services:
- docker:dind

stages:
- build
- test
- release
- deploy

variables:
353
  CONTAINER_TEST_IMAGE: registry.example.com/my-group/my-project/my-image:$CI_COMMIT_REF_NAME
354
  CONTAINER_RELEASE_IMAGE: registry.example.com/my-group/my-project/my-image:latest
355 356

before_script:
357
  - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN registry.example.com
358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393

build:
  stage: build
  script:
    - docker build --pull -t $CONTAINER_TEST_IMAGE .
    - docker push $CONTAINER_TEST_IMAGE

test1:
  stage: test
  script:
    - docker pull $CONTAINER_TEST_IMAGE
    - docker run $CONTAINER_TEST_IMAGE /script/to/run/tests

test2:
  stage: test
  script:
    - docker pull $CONTAINER_TEST_IMAGE
    - docker run $CONTAINER_TEST_IMAGE /script/to/run/another/test

release-image:
  stage: release
  script:
    - docker pull $CONTAINER_TEST_IMAGE
    - docker tag $CONTAINER_TEST_IMAGE $CONTAINER_RELEASE_IMAGE
    - docker push $CONTAINER_RELEASE_IMAGE
  only:
    - master

deploy:
  stage: deploy
  script:
    - ./deploy.sh
  only:
    - master
```

394
Some things you should be aware of when using the Container Registry:
395 396

- You must log in to the container registry before running commands. Putting
397
  this in `before_script` will run it before each job.
398 399 400 401 402 403
- Using `docker build --pull` makes sure that Docker fetches any changes to base
  images before building just in case your cache is stale. It takes slightly
  longer, but means you don’t get stuck without security patches to base images.
- Doing an explicit `docker pull` before each `docker run` makes sure to fetch
  the latest image that was just built. This is especially important if you are
  using multiple runners that cache images locally. Using the git SHA in your
404
  image tag makes this less necessary since each job will be unique and you
405 406
  shouldn't ever have a stale image, but it's still possible if you re-build a
  given commit after a dependency has changed.
407
- You don't want to build directly to `latest` in case there are multiple jobs
408
  happening simultaneously.
Mark Pundsack's avatar
Mark Pundsack committed
409

410
[docker-in-docker]: https://blog.docker.com/2013/09/docker-can-now-run-within-docker/
411
[docker-cap]: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities
412 413
[2fa]: ../../user/profile/account/two_factor_authentication.md
[pat]: ../../user/profile/personal_access_tokens.md