Skip to content

Switch Gitaly Deployment to a StatefulSet

DJ Mountney requested to merge gitaly-stateful into master

Closes: https://gitlab.com/charts/gitlab/issues/399

Note: This is a breaking change, as it removes the previous gitaly PVC, and uses a new one, created from a template.

Support for multiple gitaly pods

This change only supports a single gitaly replica in the statefulset, but will allow us to add support for multiple in the future without introducing an additional breaking change.

During testing that we could do this in the future, I did test a multi-gitaly chart by making manual changes to the chart:

  • increasing gitaly replias to 2
  • Introducing a erb ruby line to the gitaly config that chose the gitlab storage key based on which statefulset index the pod was. (I used 'default' for 0, and 'secondary' for 1
  • Added both storage key to unicorn/sidekiqs gitlab.yml, with references to their appropriate gitaly DNS.

I think we can provide the multi-gitaly in a followup by allowing the user to provide the the storage-key to gitaly address mapping manually, with added support for using the ordinal/index in the mapping instead of an address, and saving that information into its own entry in a configmap that can be mounted by gitaly and unicorn/sidekiq. The gitaly pods would reference the file to determine what their storage key name was, and unicorn and sidekiq would use the file to configure their storage.

How storage works

You are able to configure persistence the exact same way as the rest of deployments, with one exception:

You cannot provide a volumeName to connect to a pre-existing volume. Because the statefulset template is configured to support multple replicas, each with their own storage, you can't assign a single volume.

You can still use storageClass and labelSelectors to select specific disks for use. Allowing you to still manually create volumes if needed.

Additionally, to workaround the lack of a volumeName option, you can instead manually create the PVC in addition to the volume. If you manually create a PVC with the syntax repo-data-{release-name}-gitaly-{index} (repo-data-gitlab-gitaly-0), that PVC will be used by the pod with the corresponding statefulset index.

Edited by DJ Mountney

Merge request reports