Add MainClusterwide::ApplicationRecord and new connection
NOTE: This MR should be a no-op for most environments as the new connection won't be configured in GDK or production. We don't expect any issues but if something goes wrong it should be safe to revert this MR.
What does this MR do and why?
Adds a new main_clusterwide
connection option that can be configured in config/database.yml
. This connection is part of the Pods initiative (soon to be renamed to "Silos").
This connection won't be configured in production or even GDK any time soon and as such we've opted to have these code paths untested in CI. Trying to test this new connection in CI would mean having a separate config/database.yml
that ran all RSpec tests again which is just too expensive for this early stage of the project.
The main_clusterwide
connection will eventually be used to have a shared Postgres database that is used by many separate GitLab pods (silos/instances). We need to share some data like settings, users, licenses across all these pods while separating pod-specific data like projects, notes, merge_requests . This allows us to horizontally scale most of the data but still users can login and access data in any pod.
When this connection is not configured then the main
connection will be used (as before) so this change should be a no-op for anyone that doesn't explicitly go in and add main_clusterwide
to their config/database.yml
.
There is a lot more detail on the architecture at https://docs.gitlab.com/ee/architecture/blueprints/pods/ but the main point is that this connection is for "shared" stuff and we're starting with application_settings
and licenses
as these are the simplest tables to decouple.
Screenshots or screen recordings
How to set up and validate locally
Update your config/database.yml
file and copy/paste the main
database configuration section and name the other one main_clusterwide
. Add database_tasks: false
underneath main_clusterwide
. You can leave the pointing to the same database for now as we only care to test the behaviour when it uses a different database connection. Then GitLab should all work the same though there are a few RSpec tests that may fail that are depending on there only being 2 connections configured.
Here is what my config/database.yml
looks like when I tested it:
development:
main:
adapter: postgresql
encoding: unicode
database: gitlabhq_development
host: /Users/dylan/workspace/gitlab-development-kit/postgresql
port: 5432
pool: 10
prepared_statements: false
variables:
statement_timeout: 120s
main_clusterwide:
database_tasks: false
adapter: postgresql
encoding: unicode
database: gitlabhq_development
host: /Users/dylan/workspace/gitlab-development-kit/postgresql
port: 5432
pool: 10
prepared_statements: false
variables:
statement_timeout: 120s
ci:
adapter: postgresql
encoding: unicode
database: gitlabhq_development_ci
host: /Users/dylan/workspace/gitlab-development-kit/postgresql
port: 5432
pool: 10
prepared_statements: false
variables:
statement_timeout: 120s
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #388276 (closed)