Extend PreventCrossDB with information about queries to ease debugging
requested to merge 409838-cells-iteration-2-extend-in-tests-preventcrossdb-with-information-about-queries-registered into master
What does this MR do and why?
The lib/gitlab/database/query_analyzers/prevent_cross_database_modification.rb
does only show the current query that violated transaction boundaries. This is especially problematic when developing (tests or development environment) where the lack of context of previous queries complicates on finding what was exectued prior to this.
Currently only this message will be outlined:
"Cross-database data modification of '#{schemas.to_a.join(", ")}' were detected within " \
"a transaction modifying the '#{all_tables.to_a.join(", ")}' tables. " \
"Please refer to https://docs.gitlab.com/ee/development/database/multiple_databases.html#removing-cross database-transactions for details on how to resolve this exception "
In this MR, we:
- Capture generated SQL always in dev and test
- Clear it across transactions
- Print out a log of last 10 queries, since it will be enough to see the execution context
Screenshots or screen recordings
In test environments, on executing:
Project.transaction do
project.touch
pipeline.touch
end
the output comes out as:
Gitlab::Database::QueryAnalyzers::PreventCrossDatabaseModification::CrossDatabaseModificationAcrossUnsupportedTablesError: Cross-database data modification of 'gitlab_main, gitlab_ci' were detected within a transaction modifying the 'projects, ci_pipelines' tables. Please refer to https://docs.gitlab.com/ee/development/database/multiple_databases.html#removing-cross-database-transactions for details on how to resolve this exception.
0: UPDATE "projects" SET "updated_at" = $1 WHERE "projects"."id" = $2 /*application:test,correlation_id:588d36a5b5c9751ab36b5573244644e0,db_config_name:main,line:/spec/lib/gitlab/database/query_analyzers/prevent_cross_database_modification_spec.rb:106:in `run_queries'*/
1: UPDATE "ci_pipelines" SET "updated_at" = $1, "lock_version" = $2 WHERE "ci_pipelines"."id" = $3 AND "ci_pipelines"."lock_version" = $4 /*application:test,correlation_id:588d36a5b5c9751ab36b5573244644e0,db_config_name:ci,line:/spec/lib/gitlab/database/query_analyzers/prevent_cross_database_modification_spec.rb:107:in `run_queries'*/
In development environment, we raise the error:
How to set up and validate locally
- In development, set the analyzer to enabled here, by adding
return true
, as the most easy method to enable it. - Add a cross-database query to any controller action, I chose to add it in
RootController#index
def index
project = Project.first
pipeline = Ci::Pipeline.first
Project.transaction do
project.touch
pipeline.touch
end
# n+1: https://gitlab.com/gitlab-org/gitlab-foss/issues/40260
Gitlab::GitalyClient.allow_n_plus_1_calls do
projects
super
end
end
- hit, http://localhost:3000/ after logging in, and observe
- you should be able to see the error page as in the screenshot above, describing that you have hit
Gitlab::Database::QueryAnalyzers::PreventCrossDatabaseModification::CrossDatabaseModificationAcrossUnsupportedTablesError
error, along with the SQL queries made within the transaction.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #409838 (closed)
Edited by Manoj M J