Container registry on PostgreSQL
The container registry team is working to put the registry's metadata into a PostgreSQL database (to implement efficient online garbage collection: gitlab-org&2313 (closed)).
This poses the question where to put this PostgreSQL data. There is a general discussion ongoing to define a strategy how to deal with services separate from gitlab-rails and their database needs and whether or not we should be putting separate components/services into their own (logical) databases.
Leaving that on the side, the question here is: Given container registry is going to need a PostgreSQL database soon - how would we run this on GitLab.com?
Would we run this on the same patroni cluster as gitlab-rails connects to and let container registry connect to the same cluster (likely through pgbouncer)?
In terms of resource usage, the database size is expected to be at about 120 GB for starters and growing (gitlab-org/container-registry#61 (comment 332663736)). We currently seem to be running about 30 pods for the container registry and throughput peaks at about 700 requests/s. Depending on how connection pooling is implemented in container registry, we may need a few postgres connections per running instance (I'm guessing, @jdrpereira @hswimelar please correct if we know more about this already).
I'm not looking for a definitive answer here but rather for opinions and a tendency to inform the more general strategy discussion.
cc @albertoramos @Finotto @jdrpereira @hswimelar @craig-gomes