AutoRollbackService causes CrossDatabaseModificationAcrossUnsupportedTablesError
Per https://docs.gitlab.com/ee/development/database/multiple_databases.html#removing-cross-database-transactions we do not allow you to write to 2 different databases within the context of a single transaction. When calling Deployments::AutoRollbackService we're opening a transaction and writing to ci_builds as well as ci_builds_metadata and deployments which violates this rule:
$ bin/rspec ./ee/spec/services/deployments/auto_rollback_service_spec.rb
Failures:
1) Deployments::AutoRollbackService#execute successfully roll back a deployment
Failure/Error: raise CrossDatabaseModificationAcrossUnsupportedTablesError, message
Gitlab::Database::QueryAnalyzers::PreventCrossDatabaseModification::CrossDatabaseModificationAcrossUnsupportedTablesError:
Cross-database data modification of 'gitlab_ci, gitlab_main' were detected within a transaction modifying the 'ci_builds, ci_builds_metadata, deployments' tables.Please refer to https://docs.git
lab.com/ee/development/database/multiple_databases.html#removing-cross-database-transactions for details on how to resolve this exception. For now we have added this violation to the allowlist https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/database/cross-database-modification-allowlist.yml .
Possible solutions
The cleanup could be moved to a sidekiq worker or it could be done after commit.