Skip to content

Support many database configuration

Robert Marshall requested to merge 2841-support-decomposed-many-databases into master

What does this MR do?

Implement configuration for database decomposition.

Does

Implement charts/gitlab/templates/_databaseDatamodel.tpl and consuming this model throughout charts that consume various gitlab.psql.* for ConfigMaps, Secrets, Volumes, and VolumeMounts.

Does not

  • Address the problem of PostgresSQL certificate communication, due to bug in ActiveRecord
  • implement multiple instances of PostgreSQL databases (with or without instances)

Goal

  1. Transparently implement database decomposition capability without impacting users who do not make use of it. That is, because most users will not currently care about decomposition and will have no need to for now, we should ensure this is a seamless and transparent upgrade path for the 99% of users who won't need to alter configuration for this added functionality.
  2. Implement this functionality in a maintainable, straight forward, easy to comprehend manner.

Related issues

Closes #2841 (closed)

Testing

Several cases to account for:

  1. Default values result in the appropriate rendering of main stanza within database.yml.erb
  2. When providing a knownDecomposition (e.g. ci), a stanza is present within database.yml.erb representing that configuration.
  3. Properties are inherited from x.psql into x.psql.y, where y is any entry in knownDecompositions

Example database.yml.erb when ci is presented:

$ helm get values gitlab | yq '.global.psql'
{
  "ci": {
    "host": "bogus"
  }
}
$ kubectl get configmap gitlab-task-runner -oyaml | yq -r '.items[0].data."database.yml.erb"'
production:
  ci:
    adapter: postgresql
    encoding: unicode
    database: gitlabhq_production
    username: gitlab
    password: "<%= File.read("/etc/gitlab/postgres/psql-password-ci").strip.dump[1..-2] %>"
    host: "bogus"
    port: 5432
    connect_timeout: 
    keepalives: 
    keepalives_idle: 
    keepalives_interval: 
    keepalives_count: 
    tcp_user_timeout: 
    application_name: 
    prepared_statements: false
    
    
  main:
    adapter: postgresql
    encoding: unicode
    database: gitlabhq_production
    username: gitlab
    password: "<%= File.read("/etc/gitlab/postgres/psql-password-main").strip.dump[1..-2] %>"
    host: "gitlab-postgresql.default.svc"
    port: 5432
    connect_timeout: 
    keepalives: 
    keepalives_idle: 
    keepalives_interval: 
    keepalives_count: 
    tcp_user_timeout: 
    application_name: 
    prepared_statements: false

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

  • Merge Request Title and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes
  • Documentation created/updated
  • Tests added
  • Integration tests added to GitLab QA
  • Equivalent MR/issue for omnibus-gitlab opened
Edited by Jason Plum

Merge request reports