GitLab HA Storage
One of the core components to consider with GitLab HA is the storage requirements. Today, we require a shared storage service like NFS to properly function. We presently do not provide one, and we require the customer to bring their own.
While this can work in some on-premise situations, it is a key blocker when trying to automate an HA deployment on a cloud provider like AWS or GCP. AWS does have EFS, however it is not suitable for Git traffic due to the high latency and inability to purchase provisioned IOPS.
Given that we want to be able to deliver an automated HA installation either via CloudFormation, Deployment Manager, or Terraform this is a problem we need to solve.
There are a few different areas that store content that must be shared across nodes:
- Repositories: being addressed with gitaly
- LFS: Object Storage with https://gitlab.com/gitlab-org/gitlab-ee/issues/2841
- Artifacts: Object Storage as of 9.5
- Uploads: Object Storage with https://gitlab.com/gitlab-org/gitlab-ce/issues/35733
- Build Logs: Object Storage with https://gitlab.com/gitlab-org/gitlab-ce/issues/29409
- Registry: Object Storage (already supports)
- Pages:
Aside from the repositories which is being handled with Gitaly, we are largely moving to support Object Storage for all other storage types. This will allow us to significantly reduce the need on a shared storage service, for deployments on cloud providers. There are some on-premise software products that can achieve this as well, although will remain a requirement of a third party product.
The other consideration is gitlab secrets, which we would need to determine a way to share between nodes. Perhaps we can use Consul for this, or consider k8s Secrets however this would mean different code paths between on-prem and cloud.