New installations are decomposed (two databases) by default
Problem to solve
New installations of self-managed customers are still utilizing only a single database. To ensure future scalability and ease of migrations, new installations should utilize two databases, ci
and main
by default
Proposal
New installations in 16.0 should create two separate logical DBs by default. This would then allow for easier future migrations to separate physical DBs via migration tooling.
Designs
- Show closed items
Is blocked by
- omnibus-gitlab #6638Next 1-3 releases
Relates to
Activity
-
Newest first Oldest first
-
Show all activity Show comments only Show history only
- Fabian Zimmer added to epic &7509
added to epic &7509
- Fabian Zimmer changed milestone to %16.0
changed milestone to %16.0
- Fabian Zimmer changed the description
Compare with previous version changed the description
- Fabian Zimmer added grouptenant scale typefeature labels
added grouptenant scale typefeature labels
- Fabian Zimmer mentioned in epic &7509
mentioned in epic &7509
- 🤖 GitLab Bot 🤖 added [deprecated] Accepting merge requests label
added [deprecated] Accepting merge requests label
- Maintainer
Setting label(s) ~"Category:Pods (formerly Sharding)" ~"devops::data_stores" sectionenablement based on ~"group::pods".
- 🤖 GitLab Bot 🤖 added devopsdata stores sectioncore platform labels
added devopsdata stores sectioncore platform labels
- Maintainer
Setting label(s) ~"Category:Pods (formerly Sharding)" based on ~"group::pods".
- Maintainer
Setting label(s) ~"Category:Pods (formerly Sharding)" based on ~"group::pods".
- Maintainer
Setting label(s) ~"Category:Pods (formerly Sharding)" based on ~"group::pods".
- Maintainer
Setting label(s) ~"Category:Pods (formerly Sharding)" based on ~"group::pods".
- Maintainer
Setting label(s) ~"Category:Pods (formerly Sharding)" based on ~"group::pods".
- Maintainer
Setting label(s) ~"Category:Pods (formerly Sharding)" based on ~"group::pods".
- Maintainer
Setting label(s) ~"Category:Pods (formerly Sharding)" based on ~"group::pods".
- Maintainer
Setting label(s) ~"Category:Pods (formerly Sharding)" based on ~"group::pods".
- Maintainer
Setting label(s) ~"Category:Pods (formerly Sharding)" based on ~"group::pods".
- Maintainer
Setting label(s) ~"Category:Pods" based on ~"group::pods".
- 🤖 GitLab Bot 🤖 added Category:Cell label
added Category:Cell label
- Thong Kuah added groupdistribution label and removed grouptenant scale label
added groupdistribution label and removed grouptenant scale label
- 🤖 GitLab Bot 🤖 added devopssystems label and removed devopsdata stores label
added devopssystems label and removed devopsdata stores label
- Developer
- Dilan Orrino mentioned in epic &8735 (closed)
mentioned in epic &8735 (closed)
- Thong Kuah changed title from New installations are decomposed by default to New installations are decomposed (two databases) by default
changed title from New installations are decomposed by default to New installations are decomposed (two databases) by default
- Maintainer
/cc @OmarQunsulGitlab @gitlabbarry
This is the major piece where we will need help from groupdistribution. We have two options:
- Implement this ourselves with support from Distribution
- Wait for Distribution's availability
There are multiple installation methods, so I expect we will promote this to a sub-epic. The installation methods I know of to change to decomposed (two databases) by default are :
- Omnibus https://docs.gitlab.com/omnibus/. Project: https://gitlab.com/gitlab-org/omnibus-gitlab
- Helm chart https://docs.gitlab.com/charts/. Project: https://gitlab.com/gitlab-org/charts/gitlab
- From source https://docs.gitlab.com/ee/install/installation.html
Collapse replies - Maintainer
Before we even achieve this we need to do some work prior to 16.0
- Establish if we can detect "new installations". Asking in https://gitlab.slack.com/archives/C1FCTU4BE/p1667982049482999
- Allow installations to opt into decomposed mode prior to 16.0 so that we can iron out any problems prior to GA in %16.0. Opened #382026 (closed)
Edited by Thong Kuah - Developer
When we implement omnibus-gitlab#6638 (closed), we might be able to look at the problem from a different angle.
With that issue, we will hopefully start creating additional databases (like
ci
) and related objects if they are specified ingitlab_rails['databases']
hash. This can essentially become the opt-in phase (well, the second opt-in phase, as the original one was when we added support for multiple databases in database.yml, but expected users to create them manually).Then in 16.0, we start automatically adding
ci
database to the hash for new installs, so that the automation added in omnibus-gitlab#6638 (closed) will just do the needful.Need to think how to do this for Charts, though.
1
- Thong Kuah added workflowplanning breakdown label
added workflowplanning breakdown label
- Thong Kuah added 1 deleted label
added 1 deleted label
- Thong Kuah added grouptenant scale label and removed groupdistribution label
added grouptenant scale label and removed groupdistribution label
- 🤖 GitLab Bot 🤖 added devopsdata stores label and removed devopssystems label
added devopsdata stores label and removed devopssystems label
- Thong Kuah marked #370798 (closed) as a duplicate of this issue
marked #370798 (closed) as a duplicate of this issue
- Thong Kuah marked this issue as related to #370798 (closed)
marked this issue as related to #370798 (closed)
- Thong Kuah marked this issue as blocked by omnibus-gitlab#6638 (closed)
marked this issue as blocked by omnibus-gitlab#6638 (closed)
- Thong Kuah marked this issue as related to #382026 (closed)
marked this issue as related to #382026 (closed)
- Thong Kuah mentioned in issue omnibus-gitlab#6638 (closed)
mentioned in issue omnibus-gitlab#6638 (closed)
- Thong Kuah mentioned in issue #368727 (closed)
mentioned in issue #368727 (closed)
- Thong Kuah mentioned in merge request omnibus-gitlab!5492 (merged)
mentioned in merge request omnibus-gitlab!5492 (merged)
- Thong Kuah mentioned in issue #385685 (closed)
mentioned in issue #385685 (closed)
- Thong Kuah changed milestone to %Backlog
changed milestone to %Backlog
- Thong Kuah changed epic to &9628
changed epic to &9628
- Maintainer
Close as not needed
- Thong Kuah closed
closed
- Thong Kuah added won't do label
added won't do label
- Nick Nguyen removed 1 deleted label
removed 1 deleted label
added devopstenant scale groupcells infrastructure sectioninfrastructure platforms labels and removed devopsdata stores grouptenant scale [DEPRECATED] sectioncore platform labels