The Direct to Object Storage Task Force
(Spawned from the GCP Migration / Cloud Native / Platform / Product meeting on 16 Nov 2017, attended by @andrewn @mydigitalself @marin @bikebilly @jramsay @DouweM @ayufan @victorwu @smcgivern @joshlambert).
The problem with shared file-systems is documented in #23 (closed).
Currently, GitLab has multiple codepaths that rely on shared file-system access in order to work properly.
Even the current implementation of object storage in GitLab EEP relies on a shared-file systems in order to work correctly.
This issue is spread across multiple components of the GitLab application (ie, Platform, Discussion & CI) but, from our discussions, it seems like a solution could be rolled out in a fairly consistent manner, across the code-base.
In the meeting we agreed to:
- Come up with an estimate as to the amount of work involved in solving the problem
- Each Product Manager would consider the effect of dedicating a single developer from their area to work on the problem
- The group of product managers and leads would take this proposal to our respective line-management for approval
- Each team would devote a single developer to a task force (dubbed "the Avengers") to work on the solution.
Additionally, it was pointed out that, while attachments do not support object storage yet, moving attachments to object storage can actually be done very easily, allowing this task to commence early and taking it off the critical path, and allowing it be removed from the dataset required to be geo-replicated for the GCP Migration (which would be good news for @nick.thomas, @ernstvn and the geo team)
Related Issues
Platform
- LFS Object Storage support
- Import/Export
- gitlab-org/gitlab-ee#4157
Discussion
-
Move Attachments to Object Storage
-
This may no be sufficient as the current implementation writes to local disk first, the uses a cron to move the assets to object storage
CI
-
Artifacts Direct to CI
-
Build Logs on Object Storage
- Add multiple artifacts per-build: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/14367
- Async upload artifacts to object storage: https://gitlab.com/gitlab-org/gitlab-ee/issues/3346
- Migrate traces to be artifacts: https://gitlab.com/gitlab-org/gitlab-ce/issues/34317#note_38120715
- Make Runner to upload artifacts directly to Object Storage: https://gitlab.com/gitlab-org/gitlab-ee/issues/2348