Manage shards (for repositories) in the admin panel instead of config/gitlab.yml
The following discussion from !28084 should be addressed:
- [ ] @nick.thomas started a [discussion](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/28084#note_311954873): (+6 comments)
> @rymai in production, the shards table must be updated every time the `config/gitlab.yml` file is updated - and ideally, it'd be the `config/gitlab.yml` file on every machine in a HA setup. I get the feeling this isn't the case if we call it from here?
>
> When I introduced the `shards` table, I think the idea was that over time we'd *remove* the shards from `config/gitlab.yml` over time, in favour of managing them in the admin panel, but that hasn't happened yet.
There are multiple problems with just filling the `shards` table from `config/gitlab.yml`, not least that different machines can have different sets of shards configured, and that changes are only picked up on process restart.
If we introduce some UI for adding shards directly to the database, we can then deprecate `config/gitlab.yml` in %13.0 as a source of these. Throughout 13.x, we can work on making the database-backed form canonical, then in 14.0 we can stop consulting this flag in `config/gitlab.yml` at all.
There is a minor interaction with Geo, but since Geo already requires that primaries and secondaries share shard *names*, this represents an improvement - there's no need to ensure the `repository_storages` section of `config/gitlab.yml` is the same on both primaries and secondaries.
There is an M:N relationship between shards and gitaly servers. I'm unsure offhand whether we need to tackle the mapping as part of this effort.
The current initializer approach only *adds* shards to the database. It never removes a shard, or modifies the name of an existing shard, so I think we can defer that functionality for now as well.
cc @m_gill @danielgruesso
issue