Hashed Storage as the only storage
This epic is a follow up on &21 and &75. Hashed Storage effort started as a necessity for Geo, but also as a way to fix the ~"technical debt" in how GitLab stores repositories and blobs. By adopting a immutable location, we avoid many performance penalties when renaming/moving large containers of data like namespaces.
Immutability is also a desirable feature when building a distributed system with replication like Geo and Gitaly object deduplication feature.
Hashed Storage is out in GA for several months, and is also the default since %12.0. The remaining effort was to make the migration from Legacy Storage as frictionless as possible, while making sure we never risk any data-loss.
The migration code was made to be as conservative as possible to make sure we are not putting any data at risk. This means that whenever we find data in any state that may indicate a data-loss situation, we always skip. In GitLab dot com we found several corner cases that our initial implementations were not considering. You can check the migration summary in https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/4772 with a list of cases we found and what we've learned from it.
---------------------
# The Plan
We are aiming at disabling the possibility of any installation to opt for the Legacy Storage in %13.0. To make the transition to the major release frictionless, we should still keep the migration code available for a few other releases, but we will remove the possibility to create new projects with the Legacy Storage.
~~We should also look at automatically triggering a background migration in either %"13.0" .0 or %"13.1" to move existing legacy storage projects to hashed storage. By not removing the migration code yet, we allow any corner case to be manually handled.~~ Automatic migration is scheduled for %"13.3" based on the deprecation warning and full support removal in %"14.0"
Ideally customers will trigger the migration before %13.0, and we should encourage them to do so.
We need to start communicating this plan on our release notes and coordinate with Customer Success
epic