Integrate Docker Registry

We should add Docker Registry as part of Omnibus Package.

Proxy or not-to proxy

First we need to decide:

1. If registry will live on separate port (ex. :5000),
2. Run behind Nginx.

I personally would prefer to be behind Nginx. There's one downside, because it requires then to have separate TLS certificate for this host. The upside is that we will be talking to GitLab registry with nicer address: docker pull registry.gitlab.com/user/project:latest.

Changes

The changes required to be done in Omnibus:

1. Compile registry binary from https://github.com/docker/distribution: Exactly to execute: go get github.com/docker/distribution/cmd/registry. As for today we should use v2.3.1 tag.

2. Generate a shared HTTP secret for registry. 64 hex should be fine. The secret should be stored in /etc/gitlab/gitlab-secrets.json

3. Generate a RSA certificate pair. The key pair should also be stored in gitlab-secrets.json, but the separate files with certificate and key should be generated in /var/opt/gitlab/registry/{certificate,key}.pem.

4. Create a persistent storage in /var/opt/gitlab/gitlab-rails/shared/registry.

5. Generate a registry configuration in /var/opt/gitlab/registry/config.yml:

version: 0.1
log:
  fields:
    service: registry
storage:
    cache:
        blobdescriptor: inmemory
    filesystem:
        rootdirectory: /var/opt/gitlab/gitlab-rails/shared/registry
http:
    addr: :5000
    secret: "SHARED_HTTP_SECRET"
    headers:
        X-Content-Type-Options: [nosniff]
auth:
  token:
    realm: https://gitlab.example.com/api/v3/auth/token
    service: docker
    issuer: omnibus-issuer
    rootcertbundle: /var/opt/gitlab/registry/certificate.pem
health:
  storagedriver:
    enabled: true
    interval: 10s
    threshold: 3

6. Create a runsv service that will run the registry:

exec /opt/gitlab/bin/embedded/registry /var/opt/gitlab/registry/config.yml

7. Add options to config/gitlab.yml:

registry:
  enabled: true
  external_host: https://registry.example.com/ # this will be front facing address
  internal_host: http://localhost:5000/ # internal address to the registry, will be used by GitLab to directly communicate with API
  path: /var/opt/gitlab/gitlab-rails/shared/registry
  key: /var/opt/gitlab/registry/key.pem

8. Add a following options to gitlab.rb:

    registry_external_url "https://registry.example.com/"
   
    gitlab_rails['registry_path'] = "/mnt/storage/registry"
   
    # gitlab_registry['enable'] = false
    # gitlab_registry['external_host'] = nil # set to the value of registry_external_url
    # gitlab_registry['internal_host'] = nil # by default set to http://localhost:5000
    # gitlab_registry['dir'] = "/var/opt/gitlab/registry"
    # gitlab_registry['log_directory'] = "/var/log/gitlab/registry"
   
    # registry_nginx['enable'] = false
    # registry_nginx['redirect_http_to_https'] = false
    # registry_nginx['redirect_http_to_https_port'] = 80
    # registry_nginx['ssl_certificate'] = "/etc/gitlab/ssl/#{node['fqdn']}.crt"
    # registry_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/#{node['fqdn']}.key"
    # registry_nginx['ssl_ciphers'] = "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256"
    # registry_nginx['ssl_prefer_server_ciphers'] = "on"
    # registry_nginx['ssl_protocols'] = "TLSv1 TLSv1.1 TLSv1.2" # recommended by https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html & https://cipherli.st/
    # registry_nginx['ssl_session_cache'] = "builtin:1000  shared:SSL:10m" # recommended in http://nginx.org/en/docs/http/ngx_http_ssl_module.html
    # registry_nginx['ssl_session_timeout'] = "5m" # default according to http://nginx.org/en/docs/http/ngx_http_ssl_module.html
    # registry_nginx['ssl_dhparam'] = nil # Path to ci_dhparams.pem, eg. /etc/gitlab/ssl/ci_dhparams.pem
    # registry_nginx['listen_addresses'] = ['*']
    # registry_nginx['listen_port'] = nil # override only if you use a reverse proxy: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/nginx.md#setting-the-nginx-listen-port
    # registry_nginx['listen_https'] = nil # override only if your reverse proxy internally communicates over HTTP: https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/nginx.md#supporting-proxied-ssl
    # registry_nginx['custom_gitlab_server_config'] = "location ^~ /foo-namespace/bar-project/raw/ {\n deny all;\n}\n"
   
    ## Advanced settings
    # registry_nginx['dir'] = "/var/opt/gitlab/nginx"
    # registry_nginx['log_directory'] = "/var/log/gitlab/nginx"

9. Ideally we should run registry under it's own user. We need to make sure that the access to shared/registry folder is possible.