Skip to content

Add MainClusterwide::ApplicationRecord and new connection

NOTE: This MR should be a no-op for most environments as the new connection won't be configured in GDK or production. We don't expect any issues but if something goes wrong it should be safe to revert this MR.

What does this MR do and why?

Adds a new main_clusterwide connection option that can be configured in config/database.yml. This connection is part of the Pods initiative (soon to be renamed to "Silos").

This connection won't be configured in production or even GDK any time soon and as such we've opted to have these code paths untested in CI. Trying to test this new connection in CI would mean having a separate config/database.yml that ran all RSpec tests again which is just too expensive for this early stage of the project.

The main_clusterwide connection will eventually be used to have a shared Postgres database that is used by many separate GitLab pods (silos/instances). We need to share some data like settings, users, licenses across all these pods while separating pod-specific data like projects, notes, merge_requests . This allows us to horizontally scale most of the data but still users can login and access data in any pod.

When this connection is not configured then the main connection will be used (as before) so this change should be a no-op for anyone that doesn't explicitly go in and add main_clusterwide to their config/database.yml.

There is a lot more detail on the architecture at https://docs.gitlab.com/ee/architecture/blueprints/pods/ but the main point is that this connection is for "shared" stuff and we're starting with application_settings and licenses as these are the simplest tables to decouple.

Screenshots or screen recordings

How to set up and validate locally

Update your config/database.yml file and copy/paste the main database configuration section and name the other one main_clusterwide. Add database_tasks: false underneath main_clusterwide. You can leave the pointing to the same database for now as we only care to test the behaviour when it uses a different database connection. Then GitLab should all work the same though there are a few RSpec tests that may fail that are depending on there only being 2 connections configured.

Here is what my config/database.yml looks like when I tested it:

development:
  main:
    adapter: postgresql
    encoding: unicode
    database: gitlabhq_development
    host: /Users/dylan/workspace/gitlab-development-kit/postgresql
    port: 5432
    pool: 10
    prepared_statements: false
    variables:
      statement_timeout: 120s
  main_clusterwide:
    database_tasks: false
    adapter: postgresql
    encoding: unicode
    database: gitlabhq_development
    host: /Users/dylan/workspace/gitlab-development-kit/postgresql
    port: 5432
    pool: 10
    prepared_statements: false
    variables:
      statement_timeout: 120s
  ci:
    adapter: postgresql
    encoding: unicode
    database: gitlabhq_development_ci
    host: /Users/dylan/workspace/gitlab-development-kit/postgresql
    port: 5432
    pool: 10
    prepared_statements: false
    variables:
      statement_timeout: 120s

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #388276 (closed)

Edited by Dylan Griffith

Merge request reports

Loading