Many databases can be configured in Omnibus
Following our guidelines many databases are supported by our Rails stack: https://docs.gitlab.com/ee/development/database/multiple_databases.html#multiple-databases.
Current
Currently config/database.yml
as generated is in a form of:
development:
adapter: postgresql
encoding: unicode
database: gitlabhq_development
host: /path/to/gdk/postgresql
pool: 10
prepared_statements: false
variables:
statement_timeout: 120s
Proposed change
Perform transformation of config/database.yml
to do the following:
- A database configuration is transformed into
many-database
syntax usingmain
as a name of main database - An additional database for CI can be configured
many-databases
1. Transformed into This is backward/forward-compatible change, that does not impact application after the gitlab!65054 (merged) is merged.
This requires that the config/database.yml
config looks the following:
development:
main:
adapter: postgresql
encoding: unicode
database: gitlabhq_development
host: /path/to/gdk/postgresql
pool: 10
prepared_statements: false
variables:
statement_timeout: 120s
2. Allow additional databases to be configured
We expect to have many additional databases used over time, with the CI
being the first.
We need a scalable design to allow us to define all parameters required for such change.
A configured Omnibus allows to enable ci:
database and define all parameters
of such database, initially pointing to exactly the same database server, but different database name.
We don't fully understand yet how we gonna run provisioning of such database, so, we expect
in initial phase to just be able to emit config/database.yml
with the CI
entry added,
but without any other automated provisioning unless done automatically by commands like db:create
:
development:
main:
adapter: postgresql
encoding: unicode
database: gitlabhq_development
host: /path/to/gdk/postgresql
pool: 10
prepared_statements: false
variables:
statement_timeout: 120s
ci:
adapter: postgresql
encoding: unicode
database: gitlabhq_development_ci
...
migration_paths: db/ci_migrate
The Omnibus ideally should be configurable with set off gitlab.rb
settings. Probably in a similar form:
### GitLab database settings
###! Docs: https://docs.gitlab.com/omnibus/settings/database.html
###! **Only needed if you use an external database.**
# gitlab_rails['db_adapter'] = "postgresql"
# gitlab_rails['db_encoding'] = "unicode"
# gitlab_rails['db_collation'] = nil
# gitlab_rails['db_database'] = "gitlabhq_production"
# gitlab_rails['db_username'] = "gitlab"
# ...
gitlab_rails['db_ci']['enable'] = true
gitlab_rails['db_ci']['db_database'] = "gitlabhq_production_ci"
gitlab_rails['db_ci']['db_adapter'] = # inherits gitlab_rails['db_adapter']
... and so on as for the inheritance of parameters
... we assume that in most cases only a `db_database` will be different
Tradeoffs
- We are aiming at exposing only configuration aspect for purpose of using that on GitLab.com
- We don't expect all our automation to work in this BETA phase for on-premise customers, the follow-ups should be created to handle all supported workflows related to migration unless they are covered automatically by Rails