Compatible and conflicting features for namespace sharding

I would like to summarize the strategy we are currently discussing around topics like "database sharding", "service extraction" and "application sharding" in a rather cohesive doc in the handbook.

I'll use this issue as a scratch pad.

"If we shard by root namespace, people only care about people in their organization."

I believe the common ground for "how to scale GitLab" (the application) is the idea that everything belongs to a namespace. This is a great summary: https://www.youtube.com/watch?v=TGulb4sGJ9g&feature=youtu.be&t=1324

This idea has motivated the thinking around root-namespace sharding because so many GitLab features and their data reside in a namespace. A problem is that GitLab (the application) doesn't have a notion of an "organization". Your user can be active in many (root) namespaces and this is what makes it difficult to shard like this at the moment: How do you check which issues your user is assigned to (as an example)? The scope of this is global - the access-pattern is "by user", which doesn't benefit from namespace-based sharding (you end up checking all namespaces). Similar issue with /explore , user dashboards, TODOs. CI scheduling is site-wide/global AFAIK.

For now, I'd like to compile an overview of features that conflict with namespace-based sharding. This should be informative for conversations about feasibility of implementing sharding in GitLab.

Conflicting Features

Feature Conflict
Audit Events 75% of the data is not associated with a namespace.
Runners Can be created for a project or group and then assigned to another.
Merge Requests Can be created from forks from a different namespace
Admin Console Configures instance wide behavior. Examples kubernetes clusters, runners, etc
Issue_Links Issue can relate issues across namespaces
Management (Ownership) of Kubernetes clusters These can move across namespaces
Multi project pipelines https://docs.gitlab.com/ee/ci/multi_project_pipelines.html
User dashboards Issues and MRs search by assignee, author, etc
Search by user across namespace Similar to dashboards
Global snippets
Instance wide audit log
User profile Contribution log, events, etc
Todos
Group -> Group sharing https://docs.gitlab.com/ee/user/group/#sharing-a-group-with-another-group
Group - movement You can transfer a group or a project 'anywhere': https://docs.gitlab.com/ee/user/project/settings/#transferring-an-existing-project-into-another-namespace / https://docs.gitlab.com/ee/user/group/#transferring-groups
Issue - movement You can also move an issue across any two projects, as long as you have sufficient permissions: https://docs.gitlab.com/ee/user/project/issues/managing_issues.html#moving-issues

Easily Shardable Features (by Namespace)

Feature Conflict
Edited by Craig Gomes