Follow-up from "Fix n plus one issue ci build dast profile"
The following discussion from !75448 (merged) should be addressed:
-
@brytannia started a discussion: (+4 comments) Looking at the pipeline error I noticed an extra query to the projects table during the permission check for the
dast_site_profile_policy
.
Can we avoid the project load here or is it okay to add it to the accepted_n_plus_ones
list?
Follow below the query_recorder_bebug
outuput for this query:
Project Load (0.7ms) SELECT "projects"."id", "projects"."name", "projects"."path", "projects"."description", "projects"."created_at", "projects"."updated_at", "projects"."creator_id", "projects"."namespace_id", "projects"."last_activity_at", "projects"."import_url", "projects"."visibility_level", "projects"."archived", "projects"."avatar", "projects"."merge_requests_template", "projects"."star_count", "projects"."merge_requests_rebase_enabled", "projects"."import_type", "projects"."import_source", "projects"."approvals_before_merge", "projects"."reset_approvals_on_push", "projects"."merge_requests_ff_only_enabled", "projects"."issues_template", "projects"."mirror", "projects"."mirror_user_id", "projects"."shared_runners_enabled", "projects"."runners_token", "projects"."build_coverage_regex", "projects"."build_allow_git_fetch", "projects"."build_timeout", "projects"."mirror_trigger_builds", "projects"."pending_delete", "projects"."public_builds", "projects"."last_repository_check_failed", "projects"."last_repository_check_at", "projects"."only_allow_merge_if_pipeline_succeeds", "projects"."has_external_issue_tracker", "projects"."repository_storage", "projects"."repository_read_only", "projects"."request_access_enabled", "projects"."has_external_wiki", "projects"."ci_config_path", "projects"."lfs_enabled", "projects"."description_html", "projects"."only_allow_merge_if_all_discussions_are_resolved", "projects"."repository_size_limit", "projects"."printing_merge_request_link_enabled", "projects"."auto_cancel_pending_pipelines", "projects"."service_desk_enabled", "projects"."cached_markdown_version", "projects"."delete_error", "projects"."last_repository_updated_at", "projects"."disable_overriding_approvers_per_merge_request", "projects"."storage_version", "projects"."resolve_outdated_diff_discussions", "projects"."remote_mirror_available_overridden", "projects"."only_mirror_protected_branches", "projects"."pull_mirror_available_overridden", "projects"."jobs_cache_index", "projects"."external_authorization_classification_label", "projects"."mirror_overwrites_diverged_branches", "projects"."pages_https_only", "projects"."external_webhook_token", "projects"."packages_enabled", "projects"."merge_requests_author_approval", "projects"."pool_repository_id", "projects"."runners_token_encrypted", "projects"."bfg_object_map", "projects"."detected_repository_languages", "projects"."merge_requests_disable_committers_approval", "projects"."require_password_to_approve", "projects"."emails_disabled", "projects"."max_pages_size", "projects"."max_artifacts_size", "projects"."remove_source_branch_after_merge", "projects"."marked_for_deletion_at", "projects"."marked_for_deletion_by_user_id", "projects"."autoclose_referenced_issues", "projects"."suggestion_commit_message", "projects"."project_namespace_id" FROM "projects" WHERE "projects"."id" = 3 LIMIT 1 /*application:test,correlation_id:e8aba0d43ca681524c69f845df1e0d53,db_config_name:main*/
↳ ee/app/policies/dast_site_profile_policy.rb:4:in `block in <class:DastSiteProfilePolicy>'
QueryRecorder SQL: --> SELECT "projects"."id", "projects"."name", "projects"."path", "projects"."description", "projects"."created_at", "projects"."updated_at", "projects"."creator_id", "projects"."namespace_id", "projects"."last_activity_at", "projects"."import_url", "projects"."visibility_level", "projects"."archived", "projects"."avatar", "projects"."merge_requests_template", "projects"."star_count", "projects"."merge_requests_rebase_enabled", "projects"."import_type", "projects"."import_source", "projects"."approvals_before_merge", "projects"."reset_approvals_on_push", "projects"."merge_requests_ff_only_enabled", "projects"."issues_template", "projects"."mirror", "projects"."mirror_user_id", "projects"."shared_runners_enabled", "projects"."runners_token", "projects"."build_coverage_regex", "projects"."build_allow_git_fetch", "projects"."build_timeout", "projects"."mirror_trigger_builds", "projects"."pending_delete", "projects"."public_builds", "projects"."last_repository_check_failed", "projects"."last_repository_check_at", "projects"."only_allow_merge_if_pipeline_succeeds", "projects"."has_external_issue_tracker", "projects"."repository_storage", "projects"."repository_read_only", "projects"."request_access_enabled", "projects"."has_external_wiki", "projects"."ci_config_path", "projects"."lfs_enabled", "projects"."description_html", "projects"."only_allow_merge_if_all_discussions_are_resolved", "projects"."repository_size_limit", "projects"."printing_merge_request_link_enabled", "projects"."auto_cancel_pending_pipelines", "projects"."service_desk_enabled", "projects"."cached_markdown_version", "projects"."delete_error", "projects"."last_repository_updated_at", "projects"."disable_overriding_approvers_per_merge_request", "projects"."storage_version", "projects"."resolve_outdated_diff_discussions", "projects"."remote_mirror_available_overridden", "projects"."only_mirror_protected_branches", "projects"."pull_mirror_available_overridden", "projects"."jobs_cache_index", "projects"."external_authorization_classification_label", "projects"."mirror_overwrites_diverged_branches", "projects"."pages_https_only", "projects"."external_webhook_token", "projects"."packages_enabled", "projects"."merge_requests_author_approval", "projects"."pool_repository_id", "projects"."runners_token_encrypted", "projects"."bfg_object_map", "projects"."detected_repository_languages", "projects"."merge_requests_disable_committers_approval", "projects"."require_password_to_approve", "projects"."emails_disabled", "projects"."max_pages_size", "projects"."max_artifacts_size", "projects"."remove_source_branch_after_merge", "projects"."marked_for_deletion_at", "projects"."marked_for_deletion_by_user_id", "projects"."autoclose_referenced_issues", "projects"."suggestion_commit_message", "projects"."project_namespace_id" FROM "projects" WHERE "projects"."id" = 3 LIMIT 1 /*application:test,correlation_id:e8aba0d43ca681524c69f845df1e0d53,db_config_name:main*/
QueryRecorder DURATION: --> 0.007223
QueryRecorder backtrace: --> spec/support/helpers/query_recorder.rb:84:in `callback'
QueryRecorder backtrace: --> ee/app/policies/dast_site_profile_policy.rb:4:in `block in <class:DastSiteProfilePolicy>'
QueryRecorder backtrace: --> app/models/ability.rb:79:in `allowed?'
QueryRecorder backtrace: --> lib/gitlab/allowable.rb:6:in `can?'
QueryRecorder backtrace: --> ee/app/services/app_sec/dast/profiles/create_associations_service.rb:31:in `has_permission?'
QueryRecorder backtrace: --> ee/app/services/app_sec/dast/profiles/create_associations_service.rb:57:in `block in prepare_batch_inserts'
QueryRecorder backtrace: --> ee/app/services/app_sec/dast/profiles/create_associations_service.rb:51:in `prepare_batch_inserts'
QueryRecorder backtrace: --> ee/app/services/app_sec/dast/profiles/create_associations_service.rb:15:in `execute'
QueryRecorder backtrace: --> ee/lib/ee/gitlab/ci/pipeline/chain/create_cross_database_associations.rb:36:in `create_dast_associations'
QueryRecorder backtrace: --> ee/lib/ee/gitlab/ci/pipeline/chain/create_cross_database_associations.rb:14:in `perform!'
QueryRecorder backtrace: --> lib/gitlab/ci/pipeline/chain/sequence.rb:20:in `block in build!'
QueryRecorder backtrace: --> lib/gitlab/ci/pipeline/chain/sequence.rb:16:in `each'
QueryRecorder backtrace: --> lib/gitlab/ci/pipeline/chain/sequence.rb:16:in `build!'
QueryRecorder backtrace: --> app/services/ci/create_pipeline_service.rb:95:in `execute'
QueryRecorder backtrace: --> ee/spec/services/ci/create_pipeline_service/dast_configuration_spec.rb:234:in `execute_service'
QueryRecorder backtrace: --> spec/support/shared_examples/ci/create_pipeline_service_shared_examples.rb:22:in `block (3 levels) in <main>'
QueryRecorder backtrace: --> spec/support/helpers/query_recorder.rb:23:in `block in record'
QueryRecorder backtrace: --> spec/support/helpers/query_recorder.rb:22:in `record'
QueryRecorder backtrace: --> spec/support/helpers/query_recorder.rb:17:in `initialize'
QueryRecorder backtrace: --> spec/support/matchers/exceed_query_limit.rb:68:in `new'
QueryRecorder backtrace: --> spec/support/matchers/exceed_query_limit.rb:68:in `recorder'
QueryRecorder backtrace: --> spec/support/matchers/exceed_query_limit.rb:51:in `actual_count'
QueryRecorder backtrace: --> spec/support/matchers/exceed_query_limit.rb:162:in `verify_count'
QueryRecorder backtrace: --> spec/support/matchers/exceed_query_limit.rb:307:in `block (2 levels) in <main>'
QueryRecorder backtrace: --> spec/support/shared_examples/ci/create_pipeline_service_shared_examples.rb:19:in `block (2 levels) in <main>'
QueryRecorder backtrace: --> spec/spec_helper.rb:411:in `block (3 levels) in <top (required)>'
QueryRecorder backtrace: --> spec/support/sidekiq_middleware.rb:9:in `with_sidekiq_server_middleware'
QueryRecorder backtrace: --> spec/spec_helper.rb:402:in `block (2 levels) in <top (required)>'
QueryRecorder backtrace: --> spec/spec_helper.rb:398:in `block (3 levels) in <top (required)>'
QueryRecorder backtrace: --> lib/gitlab/application_context.rb:31:in `with_raw_context'
QueryRecorder backtrace: --> spec/spec_helper.rb:398:in `block (2 levels) in <top (required)>'
QueryRecorder backtrace: --> spec/spec_helper.rb:376:in `block (3 levels) in <top (required)>'
QueryRecorder backtrace: --> lib/gitlab/with_request_store.rb:17:in `enabling_request_store'
QueryRecorder backtrace: --> lib/gitlab/with_request_store.rb:10:in `with_request_store'
QueryRecorder backtrace: --> spec/spec_helper.rb:376:in `block (2 levels) in <top (required)>'
QueryRecorder backtrace: --> spec/support/system_exit_detected.rb:7:in `block (2 levels) in <main>'
QueryRecorder backtrace: --> spec/support/database/prevent_cross_joins.rb:106:in `block (3 levels) in <main>'
QueryRecorder backtrace: --> spec/support/database/prevent_cross_joins.rb:60:in `with_cross_joins_prevented'
QueryRecorder backtrace: --> spec/support/database/prevent_cross_joins.rb:106:in `block (2 levels) in <main>'
QueryRecorder backtrace: --> spec/support/caching.rb:41:in `block (3 levels) in <main>'
QueryRecorder backtrace: --> spec/support/caching.rb:40:in `block (2 levels) in <main>'
CACHE Project Load (0.0ms) SELECT "projects"."id", "projects"."name", "projects"."path", "projects"."description", "projects"."created_at", "projects"."updated_at", "projects"."creator_id", "projects"."namespace_id", "projects"."last_activity_at", "projects"."import_url", "projects"."visibility_level", "projects"."archived", "projects"."avatar", "projects"."merge_requests_template", "projects"."star_count", "projects"."merge_requests_rebase_enabled", "projects"."import_type", "projects"."import_source", "projects"."approvals_before_merge", "projects"."reset_approvals_on_push", "projects"."merge_requests_ff_only_enabled", "projects"."issues_template", "projects"."mirror", "projects"."mirror_user_id", "projects"."shared_runners_enabled", "projects"."runners_token", "projects"."build_coverage_regex", "projects"."build_allow_git_fetch", "projects"."build_timeout", "projects"."mirror_trigger_builds", "projects"."pending_delete", "projects"."public_builds", "projects"."last_repository_check_failed", "projects"."last_repository_check_at", "projects"."only_allow_merge_if_pipeline_succeeds", "projects"."has_external_issue_tracker", "projects"."repository_storage", "projects"."repository_read_only", "projects"."request_access_enabled", "projects"."has_external_wiki", "projects"."ci_config_path", "projects"."lfs_enabled", "projects"."description_html", "projects"."only_allow_merge_if_all_discussions_are_resolved", "projects"."repository_size_limit", "projects"."printing_merge_request_link_enabled", "projects"."auto_cancel_pending_pipelines", "projects"."service_desk_enabled", "projects"."cached_markdown_version", "projects"."delete_error", "projects"."last_repository_updated_at", "projects"."disable_overriding_approvers_per_merge_request", "projects"."storage_version", "projects"."resolve_outdated_diff_discussions", "projects"."remote_mirror_available_overridden", "projects"."only_mirror_protected_branches", "projects"."pull_mirror_available_overridden", "projects"."jobs_cache_index", "projects"."external_authorization_classification_label", "projects"."mirror_overwrites_diverged_branches", "projects"."pages_https_only", "projects"."external_webhook_token", "projects"."packages_enabled", "projects"."merge_requests_author_approval", "projects"."pool_repository_id", "projects"."runners_token_encrypted", "projects"."bfg_object_map", "projects"."detected_repository_languages", "projects"."merge_requests_disable_committers_approval", "projects"."require_password_to_approve", "projects"."emails_disabled", "projects"."max_pages_size", "projects"."max_artifacts_size", "projects"."remove_source_branch_after_merge", "projects"."marked_for_deletion_at", "projects"."marked_for_deletion_by_user_id", "projects"."autoclose_referenced_issues", "projects"."suggestion_commit_message", "projects"."project_namespace_id" FROM "projects" WHERE "projects"."id" = 3 LIMIT 1
↳ ee/app/policies/dast_scanner_profile_policy.rb:4:in `block in <class:DastScannerProfilePolicy>'
QueryRecorder SQL: --> SELECT "projects"."id", "projects"."name", "projects"."path", "projects"."description", "projects"."created_at", "projects"."updated_at", "projects"."creator_id", "projects"."namespace_id", "projects"."last_activity_at", "projects"."import_url", "projects"."visibility_level", "projects"."archived", "projects"."avatar", "projects"."merge_requests_template", "projects"."star_count", "projects"."merge_requests_rebase_enabled", "projects"."import_type", "projects"."import_source", "projects"."approvals_before_merge", "projects"."reset_approvals_on_push", "projects"."merge_requests_ff_only_enabled", "projects"."issues_template", "projects"."mirror", "projects"."mirror_user_id", "projects"."shared_runners_enabled", "projects"."runners_token", "projects"."build_coverage_regex", "projects"."build_allow_git_fetch", "projects"."build_timeout", "projects"."mirror_trigger_builds", "projects"."pending_delete", "projects"."public_builds", "projects"."last_repository_check_failed", "projects"."last_repository_check_at", "projects"."only_allow_merge_if_pipeline_succeeds", "projects"."has_external_issue_tracker", "projects"."repository_storage", "projects"."repository_read_only", "projects"."request_access_enabled", "projects"."has_external_wiki", "projects"."ci_config_path", "projects"."lfs_enabled", "projects"."description_html", "projects"."only_allow_merge_if_all_discussions_are_resolved", "projects"."repository_size_limit", "projects"."printing_merge_request_link_enabled", "projects"."auto_cancel_pending_pipelines", "projects"."service_desk_enabled", "projects"."cached_markdown_version", "projects"."delete_error", "projects"."last_repository_updated_at", "projects"."disable_overriding_approvers_per_merge_request", "projects"."storage_version", "projects"."resolve_outdated_diff_discussions", "projects"."remote_mirror_available_overridden", "projects"."only_mirror_protected_branches", "projects"."pull_mirror_available_overridden", "projects"."jobs_cache_index", "projects"."external_authorization_classification_label", "projects"."mirror_overwrites_diverged_branches", "projects"."pages_https_only", "projects"."external_webhook_token", "projects"."packages_enabled", "projects"."merge_requests_author_approval", "projects"."pool_repository_id", "projects"."runners_token_encrypted", "projects"."bfg_object_map", "projects"."detected_repository_languages", "projects"."merge_requests_disable_committers_approval", "projects"."require_password_to_approve", "projects"."emails_disabled", "projects"."max_pages_size", "projects"."max_artifacts_size", "projects"."remove_source_branch_after_merge", "projects"."marked_for_deletion_at", "projects"."marked_for_deletion_by_user_id", "projects"."autoclose_referenced_issues", "projects"."suggestion_commit_message", "projects"."project_namespace_id" FROM "projects" WHERE "projects"."id" = 3 LIMIT 1
QueryRecorder DURATION: --> 0.006692
QueryRecorder backtrace: --> spec/support/helpers/query_recorder.rb:84:in `callback'
QueryRecorder backtrace: --> ee/app/policies/dast_scanner_profile_policy.rb:4:in `block in <class:DastScannerProfilePolicy>'
QueryRecorder backtrace: --> app/models/ability.rb:79:in `allowed?'
QueryRecorder backtrace: --> lib/gitlab/allowable.rb:6:in `can?'
QueryRecorder backtrace: --> ee/app/services/app_sec/dast/profiles/create_associations_service.rb:31:in `has_permission?'
QueryRecorder backtrace: --> ee/app/services/app_sec/dast/profiles/create_associations_service.rb:63:in `block in prepare_batch_inserts'
QueryRecorder backtrace: --> ee/app/services/app_sec/dast/profiles/create_associations_service.rb:51:in `prepare_batch_inserts'
QueryRecorder backtrace: --> ee/app/services/app_sec/dast/profiles/create_associations_service.rb:15:in `execute'
QueryRecorder backtrace: --> ee/lib/ee/gitlab/ci/pipeline/chain/create_cross_database_associations.rb:36:in `create_dast_associations'
QueryRecorder backtrace: --> ee/lib/ee/gitlab/ci/pipeline/chain/create_cross_database_associations.rb:14:in `perform!'
QueryRecorder backtrace: --> lib/gitlab/ci/pipeline/chain/sequence.rb:20:in `block in build!'
QueryRecorder backtrace: --> lib/gitlab/ci/pipeline/chain/sequence.rb:16:in `each'
QueryRecorder backtrace: --> lib/gitlab/ci/pipeline/chain/sequence.rb:16:in `build!'
QueryRecorder backtrace: --> app/services/ci/create_pipeline_service.rb:95:in `execute'
QueryRecorder backtrace: --> ee/spec/services/ci/create_pipeline_service/dast_configuration_spec.rb:234:in `execute_service'
QueryRecorder backtrace: --> spec/support/shared_examples/ci/create_pipeline_service_shared_examples.rb:22:in `block (3 levels) in <main>'
QueryRecorder backtrace: --> spec/support/helpers/query_recorder.rb:23:in `block in record'
QueryRecorder backtrace: --> spec/support/helpers/query_recorder.rb:22:in `record'
QueryRecorder backtrace: --> spec/support/helpers/query_recorder.rb:17:in `initialize'
QueryRecorder backtrace: --> spec/support/matchers/exceed_query_limit.rb:68:in `new'
QueryRecorder backtrace: --> spec/support/matchers/exceed_query_limit.rb:68:in `recorder'
QueryRecorder backtrace: --> spec/support/matchers/exceed_query_limit.rb:51:in `actual_count'
QueryRecorder backtrace: --> spec/support/matchers/exceed_query_limit.rb:162:in `verify_count'
QueryRecorder backtrace: --> spec/support/matchers/exceed_query_limit.rb:307:in `block (2 levels) in <main>'
QueryRecorder backtrace: --> spec/support/shared_examples/ci/create_pipeline_service_shared_examples.rb:19:in `block (2 levels) in <main>'
QueryRecorder backtrace: --> spec/spec_helper.rb:411:in `block (3 levels) in <top (required)>'
QueryRecorder backtrace: --> spec/support/sidekiq_middleware.rb:9:in `with_sidekiq_server_middleware'
QueryRecorder backtrace: --> spec/spec_helper.rb:402:in `block (2 levels) in <top (required)>'
QueryRecorder backtrace: --> spec/spec_helper.rb:398:in `block (3 levels) in <top (required)>'
QueryRecorder backtrace: --> lib/gitlab/application_context.rb:31:in `with_raw_context'
QueryRecorder backtrace: --> spec/spec_helper.rb:398:in `block (2 levels) in <top (required)>'
QueryRecorder backtrace: --> spec/spec_helper.rb:376:in `block (3 levels) in <top (required)>'
QueryRecorder backtrace: --> lib/gitlab/with_request_store.rb:17:in `enabling_request_store'
QueryRecorder backtrace: --> lib/gitlab/with_request_store.rb:10:in `with_request_store'
QueryRecorder backtrace: --> spec/spec_helper.rb:376:in `block (2 levels) in <top (required)>'
QueryRecorder backtrace: --> spec/support/system_exit_detected.rb:7:in `block (2 levels) in <main>'
QueryRecorder backtrace: --> spec/support/database/prevent_cross_joins.rb:106:in `block (3 levels) in <main>'
QueryRecorder backtrace: --> spec/support/database/prevent_cross_joins.rb:60:in `with_cross_joins_prevented'
QueryRecorder backtrace: --> spec/support/database/prevent_cross_joins.rb:106:in `block (2 levels) in <main>'
QueryRecorder backtrace: --> spec/support/caching.rb:41:in `block (3 levels) in <main>'
QueryRecorder backtrace: --> spec/support/caching.rb:40:in `block (2 levels) in <main>'
Edited by Marcos Rocha