Skip to content

Extend PreventCrossDB with information about queries to ease debugging

What does this MR do and why?

The lib/gitlab/database/query_analyzers/prevent_cross_database_modification.rb does only show the current query that violated transaction boundaries. This is especially problematic when developing (tests or development environment) where the lack of context of previous queries complicates on finding what was exectued prior to this.

Currently only this message will be outlined:

"Cross-database data modification of '#{schemas.to_a.join(", ")}' were detected within " \
"a transaction modifying the '#{all_tables.to_a.join(", ")}' tables. " \
"Please refer to https://docs.gitlab.com/ee/development/database/multiple_databases.html#removing-cross database-transactions for details on how to resolve this exception "

In this MR, we:

  • Capture generated SQL always in dev and test
  • Clear it across transactions
  • Print out a log of last 10 queries, since it will be enough to see the execution context

Screenshots or screen recordings

In test environments, on executing:

Project.transaction do
  project.touch
  pipeline.touch
end

the output comes out as:

Gitlab::Database::QueryAnalyzers::PreventCrossDatabaseModification::CrossDatabaseModificationAcrossUnsupportedTablesError: Cross-database data modification of 'gitlab_main, gitlab_ci' were detected within a transaction modifying the 'projects, ci_pipelines' tables. Please refer to https://docs.gitlab.com/ee/development/database/multiple_databases.html#removing-cross-database-transactions for details on how to resolve this exception.

0: UPDATE "projects" SET "updated_at" = $1 WHERE "projects"."id" = $2 /*application:test,correlation_id:588d36a5b5c9751ab36b5573244644e0,db_config_name:main,line:/spec/lib/gitlab/database/query_analyzers/prevent_cross_database_modification_spec.rb:106:in `run_queries'*/
1: UPDATE "ci_pipelines" SET "updated_at" = $1, "lock_version" = $2 WHERE "ci_pipelines"."id" = $3 AND "ci_pipelines"."lock_version" = $4 /*application:test,correlation_id:588d36a5b5c9751ab36b5573244644e0,db_config_name:ci,line:/spec/lib/gitlab/database/query_analyzers/prevent_cross_database_modification_spec.rb:107:in `run_queries'*/

In development environment, we raise the error:

Screenshot_2023-05-25_at_4.25.52_PM

How to set up and validate locally

  • In development, set the analyzer to enabled here, by adding return true, as the most easy method to enable it.
  • Add a cross-database query to any controller action, I chose to add it in RootController#index
  def index
    project = Project.first
    pipeline = Ci::Pipeline.first

    Project.transaction do
      project.touch
      pipeline.touch
    end
    
    # n+1: https://gitlab.com/gitlab-org/gitlab-foss/issues/40260
    Gitlab::GitalyClient.allow_n_plus_1_calls do
      projects
      super
    end
  end
  • hit, http://localhost:3000/ after logging in, and observe
  • you should be able to see the error page as in the screenshot above, describing that you have hit Gitlab::Database::QueryAnalyzers::PreventCrossDatabaseModification::CrossDatabaseModificationAcrossUnsupportedTablesError error, along with the SQL queries made within the transaction.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #409838 (closed)

Edited by Manoj M J

Merge request reports