Support `database_tasks:` for database configuration
TL;DR
If the main:
and ci:
point to the same primary database
GitLab will fail to load schema or run migrations in some cases.
To properly support this mode of operation we need to support database_tasks: false
for ci:
as a way to indicate that such database should not be migrated
as it is already being managed.
Problematic configuration
# main/ci points to the same database
production:
main:
host: postgres
database: gitlabhq_main
ci:
host: postgres
database: gitlabhq_main
database_tasks: false
This config without database_tasks: false
would fail to run the bin/rake db:drop db:setup
due to inability to create schema on CI as it was already provisioned. This problem is described in more detail in context of DB migrations: #355014 (closed).
Solution
Configure database_tasks: false
as a way to indicate that this DB configuration should be skipped
for all DB rake tasks (create, drop, migrations).
Requirements
- This is now Rails 7.x feature. A backport is open to allow this to be configured in GitLab: !82902 (merged)
- If the
database_tasks: false
is misconfigured (for a case whereci:
is in fact another DB not connected withmain:
) we might not execute migrations and we might end-up with misconfigured environment. We should guard database configuration to ensure thatdatabase_tasks:
is properly used by discovering if DB is shared. Example of such task is implemented in here: https://gitlab.com/gitlab-org/gitlab/-/blob/d99889c0ef69b4706befedd8271949df0e4ac81e/lib/tasks/gitlab/db.rake - We should allow the
database_tasks:
to be configurable in:-
GitLab: Backport `database_tasks:` from Rails 7.x (!82902 - merged) -
Omnibus: Expose `db_database_tasks` attribute (omnibus-gitlab!5982 - merged) -
CNG: Add `databaseTasks: boolean` to `gitlab.psql.` ... (gitlab-org/charts/gitlab!2450 - merged) -
GDK: Set database_tasks to false when CI uses main d... (gitlab-development-kit!2472 - merged) -
GCK: (likely not needed since this is not configured today, by default)
-
- The Phase 3 and Phase 4 configured on staging and production should use
database_tasks: false
forci:
in CNG configuration to indicate that DB tasks should be skipped for CI (as the primary is being shared)
1. Omnibus
We need to update database.yml.erb
to include database_tasks:
flag (set to true by default): https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/files/gitlab-cookbooks/gitlab/templates/default/database.yml.erb#L5
2. CNG
We need to update _database.yml.tpl
to include database_tasks:
flag (set to true by default): https://gitlab.com/gitlab-org/charts/gitlab/-/blob/master/charts/gitlab/templates/_database.yml.tpl#L30
3. GDK
We need to update GDK to emit config/database.yml
:
- If
gitlab.rails.databases.ci.enabled == true
- And if
gitlab.rails.databases.ci.use_main_database == true
- The
ci:
should contain configuration withdatabase_tasks: false
Ref.: gitlab-development-kit!2464 (diffs)