Draft: POC Supporting Internal Decomposed Database in Helm Charts

What does this MR do?

This was a time-capped POC, that was done a while ago, to experiment with what needed to be done to address #4082 (closed)

What was the goal of this MR?

When a user deploys GitLab charts with postgresql.install set to true, which the default value, it's going to create a PostgreSQL pod with the main database gitabhq_development. Creating the ci database, either as a separate Kubernetes Pod, or as a separate database on the same pod, was not supported. And because we wanted to support multiple databases by default, we needed to modify the chart to support having the ci database as well.

Some more context: GitLab uses Bitnami Subcharts to create the PostgreSQL database Kubernetes Pod, and also the database gitlabhq_development on it. It uses version 8.9.4. Update: That's not longer the case, because it the bitnami subchart was updated to a newer version.

There was two ways to implement this:

1. (Not done in this MR) Automatically creating gitlabhq_development_ci as another database on the same pod.

This required custom script to create and configure the database, because Bitnami creates the first database gitlabhq_development as part of the configuration.

2. Creating another PostgreSQL pod also using bitnami

This is the solution that I was experimenting with here. By having two sub-charts of Bitnami that creates two database pods. The problem that the old version of this chart didn't support configuring the Kubernetes Service Name (might be a bug). And because of this, it caused a conflict on the K8S Service name. That's why I upgraded the sub-chart of Bitnami to 12.1.3, and it solved the problem of conflicting K8S Resources names, because we can override the names.

Creating the Pods & Databases worked well. But for some reason, the application couldn't create the database tables on the pods. And this is when I stopped the work.

Why did we put this work on hold?

• We switched priority by working on two-connections for release %16.0 instead. And postponing the multiple-databases to (probably) release %17.0. This still need to be decided.
• Customers who needed two databases on Kubernetes most probably are configuring external databases instead of internally in Kubernetes. And we have already covered this in the documentation. !2890 (merged). See here
• I wasn't sure that's the right approach for solving the problem. That's why I would love to have some feedback on this approach, before we proceed again in the future.

Related issues

#4082 (closed)

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion.

Required

• Merge Request Title and Description are up to date, accurate, and descriptive
• MR targeting the appropriate branch
• MR has a green pipeline on GitLab.com
• When ready for review, MR is labeled "~workflow::ready for review" per the Distribution MR workflow

Expected (please provide an explanation if not completing)

• Test plan indicating conditions for success has been posted and passes
• Documentation created/updated
• Tests added
• Integration tests added to GitLab QA
• Equivalent MR/issue for omnibus-gitlab opened
• Validate potential values for new configuration settings. Formats such as integer 10, duration 10s, URI scheme://user:passwd@host:port may require quotation or other special handling when rendered in a template and written to a configuration file.