Backup cronjob fails to start after upgrade

Summary

I recently upgraded our gitlab from chart 4.2.x to 4.3.x; it was a minor upgrade. After the upgrade, the gitlab-task-runner-backup stopped working. It no longer even starts a job and when I try to manually run the job, Kubernetes gives me this;

Job.batch "gitlab-task-runner-backup-manual-vlr9t" is invalid:
[spec.template.spec.volumes[0].projected.sources[0].configMap.name: Required value,
spec.template.spec.volumes[2].projected.sources[0].secret.name: Required value, 
spec.template.spec.volumes[2].projected.sources[1].secret.name: Required value, 
spec.template.spec.volumes[2].projected.sources[2].secret.name: Required value, 
spec.template.spec.volumes[2].projected.sources[3].secret.name: Required value, 
spec.template.spec.volumes[2].projected.sources[4].secret.name: Required value, 
spec.template.spec.volumes[2].projected.sources[5].secret.name: Required value, 
spec.template.spec.volumes[2].projected.sources[6].secret.name: Required value, 
spec.template.spec.volumes[2].projected.sources[7].secret.name: Required value, 
spec.template.spec.volumes[2].projected.sources[8].secret.name: Required value, 
spec.template.spec.volumes[2].projected.sources[9].secret.name: Required value, 
spec.template.spec.volumes[2].projected.sources[10].secret.name: Required value, 
spec.template.spec.volumes[2].projected.sources[11].secret.name: Required value, 
spec.template.spec.volumes[2].projected.sources[12].secret.name: Required value, 
spec.template.spec.volumes[5].projected.sources[0].secret.name: Required value, 
spec.template.spec.containers[0].volumeMounts[0].name: Not found: "task-runner-config", 
spec.template.spec.initContainers[0].volumeMounts[1].name: Not found: "custom-ca-certificates", 
spec.template.spec.initContainers[1].volumeMounts[0].name: Not found: "task-runner-config", 
spec.template.spec.initContainers[1].volumeMounts[1].name: Not found: "init-task-runner-secrets"]
Tracking number: d6901123946513935

As far as I can see, all those fields exist, the secrets exist, the rest of gitlab runs absolutely fine. The backup cronjob was running fine before and now it won't even spawn. I have no idea what caused this, or why.

Steps to reproduce

Upgrade gitlab from 4.2.x to 4.3.x, with backup cron enabled and then try to run it.

Configuration used

global:
  appConfig:
    backups:
      bucket: xxxx
      tmpBucket: xxxx

gitlab:
  task-runner:
    backups:
      cron:
        enabled: true
        schedule: "0 3 * * *"
        extraArgs: "--skip registry --skip artifacts --skip packages"
      objectStorage:
        backend: gcs
        config:
          secret: gitlab-storage-sa
          key: gcs-application-credentials-file
          gcpProject: xxxx

Current behavior

The backup cron job simply won't run.

Expected behavior

The backup cron job to run.

Versions

  • Chart: 4.3.5
  • Platform:
    • Cloud: GKE
  • Kubernetes: (kubectl version)
    • Client:
    • Server:
  • Helm: (helm version)
    • Client: v1.17.3
    • Server: v1.16.13-gke.1
Edited by Gerard