Skip to content

Introduce jobs with clusterwide connection, but with same db as `main`

Manoj M J requested to merge 410705-testing-gitlab-with-clusterwide-connection-but-with-same-database-as-main-in-ci-pipelines into master

What does this MR do and why?

This is the first step towards #410705 (closed):

Requirement:

As we are moving towards building out Cells, we will have both clusterwide and cell specific databases.

Currently, we only have 2 databases and 2 connections: main and ci. As we build cells, we will also need a main_clusterwide database.

As we are in the process of building this out, it is also important that we have the necessary setup to support Cells development in our CI pipelines and this MR is the first step towards the same:

In this MR, we make changes to pipelines such that:

  • Only if the label pipeline:run-clusterwide-db is present in an MR:
    • We setup databases with a new connection main_clusterwide, but pointing to the same database as main. This means that we will have a new ActiveRecord connection, but internally connecting to the same database as main.

And using this new setup, we will run unit, integration, system, migration, post migration jobs for both EE and FOSS on PG 14.

To note:

  • Some of the jobs running in this new setup will not always succeed, because the specs haven't yet been adapted to the new database setup. But grouptenant scale feels this is OK as these jobs will not disrupt regular workflows of other developers as it is invoked only when label pipeline:run-clusterwide-db is added to an MR. We would like to start introducing this setup and go forward by fixing the problems in the pipeline one by one. If we were to merge pipeline changes only after getting everything to green, that would take a long time and we currently do not have the bandwidth for it.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #410705 (closed)

Edited by Manoj M J

Merge request reports