GitLab on CephFS
Deprecated issue, Ceph issues are in https://gitlab.com/gitlab-com/infrastructure/issues?scope=all&sort=updated_desc&state=opened&utf8=%E2%9C%93&label_name%5B%5D=ceph&label-name=
Using a virtual server in the cloud as an NFS server will only take you so far in terms of storage capacity. We want to see if CephFS can be used to store Git repository data (and all other file data managed by GitLab). If you are currently evaluating or running Ceph with GitLab please comment below. If you prefer to comment in private please email sytse@ our domain.
Status is:
-
Ongoing conversations with providers of support - https://gitlab.com/gitlab-com/operations/issues/202 -
Metrics in out current NFS backend server show that IOPS are not as high and latencies are not as low as initially thought #1 (comment 4795469) -
We need to add metrics on the NFS from the client perspective to get a grasp of how this solution behaves. -
Plan ahead using GitLab GEO
Relevant information/talks to watch:
-
Ceph at CERN: A Year in the Life of a Petabyte-Scale Block Storage Service
- Follow up conversation by email with Daniel: #1 (comment 4518151)
- Notes on the CERN video: #1 (comment 4626461)
- Building Your First Ceph Cluster for OpenStack— Fighting for Performance, Solving Tradeoffs
-
Possible candidate as a distributed filesystem: Lustre FS
- http://www.eofs.eu/fileadmin/lad2015/slides/07_Andreas_Dilger_LAD2015-Lustre2_9andBeyond.pdf
- http://lustre.ornl.gov/ecosystem/documents/tutorials/Stearman-LLNL-ZFS.pdf
- As a pro, it is mature and has been in use for quite a long time.
Production work happening over in: