Support `database_tasks:` for database configuration

TL;DR

If the main: and ci: point to the same primary database GitLab will fail to load schema or run migrations in some cases.

To properly support this mode of operation we need to support database_tasks: false for ci: as a way to indicate that such database should not be migrated as it is already being managed.

Problematic configuration

# main/ci points to the same database
production:
  main:
    host: postgres
    database: gitlabhq_main
  ci:
    host: postgres
    database: gitlabhq_main
    database_tasks: false

This config without database_tasks: false would fail to run the bin/rake db:drop db:setup due to inability to create schema on CI as it was already provisioned. This problem is described in more detail in context of DB migrations: #355014 (closed).

Solution

Configure database_tasks: false as a way to indicate that this DB configuration should be skipped for all DB rake tasks (create, drop, migrations).

Requirements

This is now Rails 7.x feature. A backport is open to allow this to be configured in GitLab: !82902 (merged)
If the database_tasks: false is misconfigured (for a case where ci: is in fact another DB not connected with main:) we might not execute migrations and we might end-up with misconfigured environment. We should guard database configuration to ensure that database_tasks: is properly used by discovering if DB is shared. Example of such task is implemented in here: https://gitlab.com/gitlab-org/gitlab/-/blob/d99889c0ef69b4706befedd8271949df0e4ac81e/lib/tasks/gitlab/db.rake
We should allow the database_tasks: to be configurable in:
- GitLab: Backport `database_tasks:` from Rails 7.x (!82902 - merged)
- Omnibus: Expose `db_database_tasks` attribute (omnibus-gitlab!5982 - merged)
- CNG: Add `databaseTasks: boolean` to `gitlab.psql.` ... (gitlab-org/charts/gitlab!2450 - merged)
- GDK: Set database_tasks to false when CI uses main d... (gitlab-development-kit!2472 - merged)
- GCK: (likely not needed since this is not configured today, by default)
The Phase 3 and Phase 4 configured on staging and production should use database_tasks: false for ci: in CNG configuration to indicate that DB tasks should be skipped for CI (as the primary is being shared)

1. Omnibus

We need to update database.yml.erb to include database_tasks: flag (set to true by default): https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/files/gitlab-cookbooks/gitlab/templates/default/database.yml.erb#L5

2. CNG

We need to update _database.yml.tpl to include database_tasks: flag (set to true by default): https://gitlab.com/gitlab-org/charts/gitlab/-/blob/master/charts/gitlab/templates/_database.yml.tpl#L30

3. GDK

We need to update GDK to emit config/database.yml:

If gitlab.rails.databases.ci.enabled == true
And if gitlab.rails.databases.ci.use_main_database == true
The ci: should contain configuration with database_tasks: false

Ref.: gitlab-development-kit!2464 (diffs)

Edited Mar 23, 2022 by Jason Plum