gitlab_pm/gitlab_main_clusterwide/gitlab_main_cell tables are not locked on the CI database

As part of the database decomposition project we made a copy of the database on a new cluster. We used the first database cluster, called main for all the database tables that belong to the gitlab_main gitlab schema. And used the new database cluster ci for all the tables that belong to the gitlab_schema gitlab_ci. While the gitlab_shared tables are still used on both clusters

After the decomposition we use gitlab_main and gitlab_shared tables on the database cluster main. And we use the gitlab_ci and gitlab_shared tables on the database cluster ci. But the database schema (not to be confused with gitlab_schema) is still the same on both database clusters. We are keeping it now like this for simplicity. That means we still have some legacy tables, that we have been truncated, that the application should not write to. These tables have been locked for writes, and we automatically lock writes on any newly created tables on the other database where the table is not used.

For some reason, the automatic locking of writes on any newly created tables doesn't include gitlab_pm tables. Which at the moment are located on the main database. But they should be locked on the ci database in a decomposed database setup.

As part of this issue:

  1. Make sure that rake gitlab:db:lock_writes locks gitlab_pm / gitlab_main_clusterwide / gitlab_main_cell tables on the ci database. Maybe that's already the case, but verify.
  2. Confirmed to be broken, make sure that any newly created gitlab_pm table is locked on the ci database, and kept unlocked on the main database.. This is done by fixing the code in the file https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/database/migration_helpers/automatic_lock_writes_on_tables.rb

CC: @tkuah @rutgerwessels

Edited by Omar Qunsul