Split-application containers

Description

The current Gitlab omnibus container runs an Upstart environment with several services, including:

• Gitlab/Rails
• Git-shell/ssh
• nginx
• Postgres
• Redis

In running this, it is possible to use a compose file such as the following:

gitlab:
  image: gitlab/gitlab-ce:8.14.2-ce.0
  ports:
    - "80:80"
    - "2022:22"
  log_opt:
    max-size: "100M"
    max-file: "4"
  cap_drop:
    - net_raw
    - mknod
    #these break ssh
    #- setpcap
    #- setfcap
    #- sys_chroot
    #- audit_write
    # these break gitlab
    #- fowner
    #- dac_override
  volumes:
    - /opt/docker/gitlab/data/gitlab/:/var/opt/gitlab/
    - /opt/docker/opt/gitlab/conf:/etc/gitlab
    - /etc/localtime:/etc/localtime:ro
  restart:always

As you might notice, many cap_drop capabilities break gitlab or ssh. Dropping net_raw has real-world implications, sometimes.

Proposal

A modification to the entrypoint script (/assets/wrapper) to accept a command (argument) would allow selection of individual services. This means we can avoid bringing up Rails, and instead bring up only ssh, redis, or nginx. Default behavior would not change.

In such a case, the docker-compose.yml could look as below. This is a quite-comprehensive configuration which initiates separate containers for separate tasks, rotates the console logs, and drops unnecessary capabilities. This example also incorporates (as an example) the official Redis and Postgres containers.

gitlab:
  image: gitlab/gitlab-ce:8.14.2-ce.0
  command: "gitlab"
  log_opt:
    max-size: "100M"
    max-file: "4"
  cap_drop:
    - net_raw
    - mknod
    - setpcap
    - setfcap
    - sys_chroot
    - audit_write
    # these break gitlab
    #- fowner
    #- dac_override
  volumes:
    - /opt/docker/gitlab/data/gitlab/:/var/opt/gitlab/
    - /opt/docker/gitlab/conf:/etc/gitlab
    - /etc/localtime:/etc/localtime:ro
    # Gitlab socket, listening for nginx
    - /var/opt/gitlab/gitlab-workhorse/
  links:
    - db
    - redis
  restart:always

git-shell:
  image: gitlab/gitlab-ce:8.14.2-ce.0
  command: "git-shell"
  ports:
    - "2022:2022"
  log_opt:
    max-size: "100M"
    max-file: "4"
  cap_drop:
    - net_raw
    - mknod
    # Evaluate these
    #- fowner
    #- dac_override
  volumes_from:
    - gitlab
  restart: always

# nginx uses the gitlab socket file exposed as a volume
nginx:
  image: gitlab/gitlab-ce:8.14.2-ce.0
  command: "nginx"
  ports:
    - "80:80"
    - "443:443"
  log_opt:
    max-size: "100M"
    max-file: "4"
  cap_drop:
    - net_raw
    - mknod
    - setpcap
    - setfcap
    - fowner
    - dac_override
    - sys_chroot
    - audit_write
  volumes_from:
    - gitlab
  restart: always

# Redis 3 official image
# Runs as user 'redis', no root
# Drops tons of capabilities
redis:
  image: redis:3
  user: redis
  expose:
    - 6379
  log_opt:
    max-size: "100M"
    max-file: "4"
  cap_drop:
    - net_raw
    - mknod
    - setpcap
    - setfcap
    - fowner
    - dac_override
    - setgid
    - setuid
    - sys_chroot
    - audit_write
    - chown
  volumes:
    - /etc/localtime:/etc/localtime:ro
  restart: always

# Official postgres 9.5
db:
  image:  postgres:9.5
  environment:
    POSTGRES_USER: "gitlab"
    POSTGRES_PASSWORD: "gitlab"
  expose:
    - 5432
  log_opt:
    max-size: "100M"
    max-file: "4"
  cap_drop:
    - net_raw
    - mknod
    - setpcap
    - setfcap
    - sys_chroot
    - audit_write
    # The below are required
    #- fowner
    #- dac_override
    #- chown
    #- setgid
    #- setuid
  volumes:
    - /opt/docker/gitlab/data/db:/var/lib/postgresql/data
  restart: always

Above, we see that many capabilities can be dropped from nginx and git-shell. Redis and Postgres can readily run as separate containers as-is, and can drop many capabilities; Postgres needs some attention upstream if further capabilities are to be removed, and this is not a concern for Gitlab.

Absent a command directive, the Gitlab image would simply bring up gitlab, git-shell, nginx, postgres, and redis as it does now.

Links / references

• af_packet race condition mitigated by dropping net_raw capability