Use structured logging for GitLab.com Registry

In gitlab-org/omnibus-gitlab!2911 (merged), @alejandro added the option to allow structured logging for docker/distribution aka GitLab Registry logs.

Presently, the logs in Kibana are not yet JSON formatted, and appear as follows:

{"message":"10.216.5.3 - - [13/Mar/2019:11:05:44 +0000] \"GET /v2/ HTTP/1.1\" 401 87 \"\" \"docker/17.03.2-ce go/go1.10.3 git-commit/f5ec1e2 kernel/4.14.65+ os/linux arch/amd64 UpstreamClient(Go-http-client/1.1)\"","tag":"registry","environment":"gprd","hostname":"registry-01-sv-gprd","fqdn":"registry-01-sv-gprd.c.gitlab-production.internal"}

This is difficult to parse and difficult to do analysis on.

Since gitlab-org/omnibus-gitlab!2911 (merged) makes this easy, we should:

  1. Switch from text to json for the log formatter. Details in https://github.com/docker/distribution/blob/master/docs/configuration.md
  2. Parse the logs as JSON and not text (in fluentd/tdagent) and send to ELK as such

Once we've done that this will open up options for

  1. Analysis on Cloud Spend by individual projects and accounts
  2. Abuse reporting
  3. Diagnostics on the regular registry outages