Bring Object Storage to CE
As part of the larger cloud native effort, there is a requirement to solve the shared storage requirement for NFS. NFS like storage solutions are not natively available within Kubernetes, and having one as a requirement is not desirable (gitlab-com/migration#23 (closed)).
To address this we are working towards:
- Adding object storage support to all types of storage except for repos. (gitlab-com/migration#23 (closed))
- Removing any need for temporary shared storage prior to upload to object storage (gitlab-com/migration#23 (closed))
However, object storage support in general is an EEP feature. We should being this to CE for a few reasons:
- As noted above, NFS is fragile and an undesirable solution especially for a cloud native architecture. (gitlab-com/migration#23 (closed))
- Object storage is much more commonly available in cloud providers than an NFS like solution.
- Shared storage is not supported within Kubernetes natively, and the only solution we can find to provide is, is marked as alpha. (rook.io) This compares to minio which can provide object storage, and is fairly stable.
- Supporting two different architectures and storage solutions between EEP and CE/EES would be overly complex and confusion. It would also make more upgrades difficult as well.
It is important to note that our packaged Postgres and Redis HA products remain EEP+.
Current status
Most of the work will be done in !17358 (merged) and gitlab-ee!4736 but some feature are still missing
| Missing EE MR name | EE link | CE backport MR |
|---|---|---|
| Store lfs on object storage by default | gitlab-org/gitlab-ee!4794 | !17752 (merged) |
| lfs ~"technical debt" | gitlab-ee!4932 | !17752 (merged) |
| Bump fog-google to 1.3.3 | gitlab-ee!4999 | !17752 (merged) |
| Fix proxy_download support for lfs controller | gitlab-ee!4894 | !17752 (merged) |
Edited by Alessio Caiazza