Skip to content

backup-utility ignores external S3 config and always backups locally

Summary

Can't properly backup to external S3 storage with non-helpful error.

Steps to reproduce

  1. Deploy GitLab chart with a working backup.s3cfg (tested with s3cmd):
[default]
access_key = XXX
secret_key = XXX
host_base = s3.gra.cloud.ovh.net
host_bucket = %(bucket).s3.gra.cloud.ovh.net
bucket_location = gra
use_https = True

And the following relevant values:

global:
  appConfig:
    backups:
      bucket: gitlab-foobar-io
      tmpBucket: gitlab-tmp-foobar-io
backups:
  objectStorage:
    backend: s3
    config:
      secret: gitlab-backup-config
      key: backup.s3cfg

With the following relevant secret (generated with kustomize):

secretGenerator:
  - name: gitlab-backup-config
    files:
      - backup.s3cfg
generatorOptions:
  disableNameSuffixHash: true
  1. Attempt a backup
kubectl exec -it gitlab-task-runner-688b4f8f85-mdj28 --namespace "gitlab" --  backup-utility

Configuration used

global:
  appConfig:
    backups:
      bucket: gitlab-foobar-io
      tmpBucket: gitlab-tmp-foobar-io
backups:
  objectStorage:
    backend: s3
    config:
      secret: gitlab-backup-config
      key: backup.s3cfg

Current behavior

Failure to upload the backup:

[...]
Dumping terraform_state ...
empty
Packing up backup tar
WARNING: Module python-magic is not available. Guessing MIME types based on file extensions.
ERROR: S3 error: 404 (NoSuchBucket): The specified bucket does not exist
command terminated with exit code 12
make: *** [backup] Error 12

Unfortunately the message does not help to fix the error as I don't know which bucket backup-utility is trying to access. The utility is not verbose enough and that would be nice to improve, maybe even provide a test action to test/debug the connection.

Tried to create default bucket names (tmp & gitlab-backups) but no more luck.

Any idea?

Expected behavior

Working backup

Versions

  • Chart: gitlab-runner-0.19.1
  • Platform:
    • Self-hosted: kubeadm
  • Kubernetes: (kubectl version)
    • Client: v1.18.6
    • Server: v1.18.6
  • Helm: (helm version)
    • Client: v3.2.4
    • Server: n/a
Edited by Olivier Louvignes