Skip to content

Avoid N+1 of group associations in Search

Dmitry Gruzd requested to merge fix-n-plus-1-for-project-search into master

What does this MR do?

This MR adds preloading of :group => [:ip_restrictions, :saml_provider] to eliminate N+1 for the scope=projects

Bullet output

USE eager loading detected: Group => [:ip_restrictions] Add to your finder: :includes => [:ip_restrictions]
  Group => [:ip_restrictions]
  Add to your finder: :includes => [:ip_restrictions]
Call stack
  /home/dgruzd/projects/gdk-ee/gitlab/ee/lib/gitlab/ip_restriction/enforcer.rb:31:in `allows_address?'
  /home/dgruzd/projects/gdk-ee/gitlab/ee/lib/gitlab/ip_restriction/enforcer.rb:21:in `allows_current_ip?'
  /home/dgruzd/projects/gdk-ee/gitlab/ee/app/policies/ee/project_policy.rb:376:in `block (2 levels) in <module:ProjectPolicy>'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/condition.rb:23:in `instance_eval'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/condition.rb:23:in `compute'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/condition.rb:44:in `block in pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/base.rb:303:in `cache'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/condition.rb:44:in `pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/rule.rb:81:in `pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/rule.rb:211:in `block in pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/rule.rb:211:in `all?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/rule.rb:211:in `pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/step.rb:81:in `pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:101:in `block in run'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:180:in `block in steps_by_score'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:149:in `loop'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:149:in `steps_by_score'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:82:in `run'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:60:in `pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/base.rb:234:in `block in allowed?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/base.rb:234:in `all?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/base.rb:234:in `allowed?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/base.rb:226:in `can?'
  /home/dgruzd/projects/gdk-ee/gitlab/app/models/ability.rb:72:in `allowed?'
  /home/dgruzd/projects/gdk-ee/gitlab/app/services/search_service.rb:81:in `visible_result?'
  /home/dgruzd/projects/gdk-ee/gitlab/app/services/search_service.rb:85:in `block in redact_unauthorized_results'
  /home/dgruzd/projects/gdk-ee/gitlab/app/services/search_service.rb:85:in `reject'
  /home/dgruzd/projects/gdk-ee/gitlab/app/services/search_service.rb:85:in `redact_unauthorized_results'
  /home/dgruzd/projects/gdk-ee/gitlab/app/services/search_service.rb:65:in `search_objects'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/search_controller.rb:35:in `show'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:546:in `block in allow_gitaly_ref_name_caching'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/gitaly_client.rb:325:in `allow_ref_name_caching'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:545:in `allow_gitaly_ref_name_caching'
  /home/dgruzd/projects/gdk-ee/gitlab/ee/lib/gitlab/ip_address_state.rb:10:in `with'
  /home/dgruzd/projects/gdk-ee/gitlab/ee/app/controllers/ee/application_controller.rb:44:in `set_current_ip_address'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:491:in `set_current_admin'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/session.rb:11:in `with_session'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:482:in `set_session_storage'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/i18n.rb:55:in `with_locale'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/i18n.rb:61:in `with_user_locale'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:476:in `set_locale'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/error_tracking.rb:50:in `with_context'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:541:in `sentry_context'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:469:in `block in set_current_context'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/application_context.rb:52:in `block in use'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/application_context.rb:52:in `use'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/application_context.rb:20:in `with_context'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:462:in `set_current_context'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/metrics/elasticsearch_rack_middleware.rb:24:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/metrics/redis_rack_middleware.rb:22:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/rails_queue_duration.rb:29:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/metrics/rack_middleware.rb:17:in `block in call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/metrics/transaction.rb:54:in `run'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/metrics/rack_middleware.rb:17:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/request_profiler/middleware.rb:17:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/query_limiting/middleware.rb:17:in `block in call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/query_limiting/transaction.rb:39:in `run'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/query_limiting/middleware.rb:16:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/ee/lib/gitlab/database/load_balancing/rack_middleware.rb:39:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/ee/lib/gitlab/jira/middleware.rb:19:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/go.rb:20:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/etag_caching/middleware.rb:13:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/multipart.rb:125:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/read_only/controller.rb:51:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/read_only.rb:18:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/same_site_cookies.rb:27:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/basic_health_check.rb:25:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/request_context.rb:23:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/config/initializers/fix_local_cache_middleware.rb:9:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/static.rb:11:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/webpack/dev_server_middleware.rb:27:in `perform_request'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/metrics/requests_rack_middleware.rb:60:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/release_env.rb:12:in `call'
USE eager loading detected: Group => [:saml_provider] Add to your finder: :includes => [:saml_provider]
  Group => [:saml_provider]
  Add to your finder: :includes => [:saml_provider]
Call stack
  /home/dgruzd/projects/gdk-ee/gitlab/ee/lib/gitlab/auth/group_saml/sso_enforcer.rb:35:in `group_access_restricted?'
  /home/dgruzd/projects/gdk-ee/gitlab/ee/app/policies/ee/project_policy.rb:372:in `block (2 levels) in <module:ProjectPolicy>'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/condition.rb:23:in `instance_eval'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/condition.rb:23:in `compute'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/condition.rb:44:in `block in pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/base.rb:303:in `cache'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/condition.rb:44:in `pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/rule.rb:81:in `pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/rule.rb:211:in `block in pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/rule.rb:211:in `all?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/rule.rb:211:in `pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/step.rb:81:in `pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:101:in `block in run'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:180:in `block in steps_by_score'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:149:in `loop'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:149:in `steps_by_score'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:82:in `run'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:60:in `pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/base.rb:234:in `block in allowed?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/base.rb:234:in `all?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/base.rb:234:in `allowed?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/rule.rb:159:in `pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/step.rb:81:in `pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:92:in `block in run'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:180:in `block in steps_by_score'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:149:in `loop'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:149:in `steps_by_score'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:82:in `run'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/runner.rb:60:in `pass?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/base.rb:234:in `block in allowed?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/base.rb:234:in `all?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/base.rb:234:in `allowed?'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/declarative_policy/base.rb:226:in `can?'
  /home/dgruzd/projects/gdk-ee/gitlab/app/models/ability.rb:72:in `allowed?'
  /home/dgruzd/projects/gdk-ee/gitlab/app/services/search_service.rb:81:in `visible_result?'
  /home/dgruzd/projects/gdk-ee/gitlab/app/services/search_service.rb:85:in `block in redact_unauthorized_results'
  /home/dgruzd/projects/gdk-ee/gitlab/app/services/search_service.rb:85:in `reject'
  /home/dgruzd/projects/gdk-ee/gitlab/app/services/search_service.rb:85:in `redact_unauthorized_results'
  /home/dgruzd/projects/gdk-ee/gitlab/app/services/search_service.rb:65:in `search_objects'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/search_controller.rb:35:in `show'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:546:in `block in allow_gitaly_ref_name_caching'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/gitaly_client.rb:325:in `allow_ref_name_caching'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:545:in `allow_gitaly_ref_name_caching'
  /home/dgruzd/projects/gdk-ee/gitlab/ee/lib/gitlab/ip_address_state.rb:10:in `with'
  /home/dgruzd/projects/gdk-ee/gitlab/ee/app/controllers/ee/application_controller.rb:44:in `set_current_ip_address'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:491:in `set_current_admin'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/session.rb:11:in `with_session'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:482:in `set_session_storage'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/i18n.rb:55:in `with_locale'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/i18n.rb:61:in `with_user_locale'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:476:in `set_locale'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/error_tracking.rb:50:in `with_context'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:541:in `sentry_context'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:469:in `block in set_current_context'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/application_context.rb:52:in `block in use'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/application_context.rb:52:in `use'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/application_context.rb:20:in `with_context'
  /home/dgruzd/projects/gdk-ee/gitlab/app/controllers/application_controller.rb:462:in `set_current_context'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/metrics/elasticsearch_rack_middleware.rb:24:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/metrics/redis_rack_middleware.rb:22:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/rails_queue_duration.rb:29:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/metrics/rack_middleware.rb:17:in `block in call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/metrics/transaction.rb:54:in `run'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/metrics/rack_middleware.rb:17:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/request_profiler/middleware.rb:17:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/query_limiting/middleware.rb:17:in `block in call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/query_limiting/transaction.rb:39:in `run'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/query_limiting/middleware.rb:16:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/ee/lib/gitlab/database/load_balancing/rack_middleware.rb:39:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/ee/lib/gitlab/jira/middleware.rb:19:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/go.rb:20:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/etag_caching/middleware.rb:13:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/multipart.rb:125:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/read_only/controller.rb:51:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/read_only.rb:18:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/same_site_cookies.rb:27:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/basic_health_check.rb:25:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/request_context.rb:23:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/config/initializers/fix_local_cache_middleware.rb:9:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/static.rb:11:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/webpack/dev_server_middleware.rb:27:in `perform_request'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/metrics/requests_rack_middleware.rb:60:in `call'
  /home/dgruzd/projects/gdk-ee/gitlab/lib/gitlab/middleware/release_env.rb:12:in `call'

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Dmitry Gruzd

Merge request reports