Support external object storage

Summary

Only reconciles Minio resources when global.minio.enabled=true.

This included putting the bucket creation Job into the reconcileMinio method to ensure it was only reconciled when Minio was enabled. This follows the pattern from the Chart - we only run the bucket creation job if the included MinIO is enabled.

Closes #137 (closed)

Testing

1. Deploy external object storage

I first tried to deploy the MinIO operator, but it seemed to require a minimum of 4 nodes. I then moved on to the MinIO helm chart, but it's been deprecated for the one that deploys the MinIO operator :)

So instead I just deployed the GitLab Helm chart off master with the defaults in a miniotest namespace, and then deleted everything that wasn't related to MinIO. This means that only MinIO, NGINX, and CertManager were running (so it could be accessed over HTTPS).

Edit: https://github.com/bitnami/charts/tree/master/bitnami/minio will likely work, as Jason pointed out. I'll give this a try as well if I get a chance.

2. Deploy the GitLab Operator

$ TAG=137-support-external-object-storage make build_operator

$ kubectl create ns gitlab-system

$ kubectl apply -f .build/operator.yaml

3. Prepare MinIO resources

First, I grabbed the accesskey and secretkey from $releaaseName-minio-secret from the miniotest namespace.

Then I created a file named secret.yaml with the following contents:

provider: AWS
region: us-east-1
aws_access_key_id: <redacted>
aws_secret_access_key: <redacted>
aws_signature_version: 4
host: minio.mnielsen.k8s-ft.win
endpoint: "https://minio.mnielsen.k8s-ft.win"
path_style: true

I then created the secret: kubectl -n gitlab-system create secret generic gitlab-external-storage --from-file=connection=secret.yaml

Then I created a file named task-runner-secret.yaml with the following contents:

[default]
access_key = <redacted>
secret_key = <redacted>
bucket_location = us-east-1
multipart_chunk_size_mb = 128

I think created the secret: kubectl -n gitlab-system create secret generic s3cmd-config --from-file=config=task-runner-secret.yaml

Note: I tried calling this something else and then specifying the Secret name in my GitLab CR, but the task-runner pod kept failing to come up because it was waiting for s3cmd-config secret. So I just used that name in the meantime - I wonder if our charts aren't checking for that setting correctly?

4. Deploy a GitLab instance

I created a file named mygitlab.yaml:

apiVersion: apps.gitlab.com/v1beta1
kind: GitLab
metadata:
  name: gitlab
spec:
  chart:
    version: "5.1.1"
    values:
      certmanager:
        install: false
      certmanager-issuer:
        email: mnielsen@gitlab.com
      global:
        hosts:
          domain: mnielsen.k8s-ft.win
        ingress:
          configureCertmanager: true
        minio:
          enabled: false
        appConfig:
          object_store:
            enabled: true
            connection:
              secret: gitlab-external-storage
              key: connection
      gitlab:
        task-runner:
          backups:
            objectStorage:
              config:
                secret: s3cmd-config # tried a different value here, wasn't picked up
                key: config

When all of the pods were up and running, I confirmed that it had the correct object storage settings by running cat /etc/gitlab/objectstorage/object_store in one of the webservice pods.

Finally, I created a new issue and pasted a screenshot to validate that storage was connected:

Here's the file in MinIO: