Skip to content

Preload group root ancestor for Group Projects API

Describe in detail what your merge request does and why.

Related to #330140 (closed)

Feature flag

While this change should be relatively safe, it would have significant impact if it did cause issues. For that reason I added a feature flag which we can quickly remove if there are no issues in production.

Feature flag rollout issue: #354372 (closed)

Resolves a few related N+1s in the group projects API:

  • Preloads the group relation for projects, after namespace has been preloaded. For all projects except those in a user namespace, group and namespace is identical so it doesn't make sense to request two queries to get the same record.
  • Preloads root_ancestor for namespace and group. Several places in this request path require root_ancestor including the resolution of the default branch name, and some permission policy conditions.

Queries for which N+1's are now eliminated

D, [2022-02-28T14:22:28.375193 #843] DEBUG -- :   CACHE Namespace Load (0.0ms)  SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" WHERE "namespaces"."id" = 132 LIMIT 1
D, [2022-02-28T14:22:28.375832 #843] DEBUG -- :   ↳ app/models/namespaces/traversal/linear.rb:103:in `block in root_ancestor'
D, [2022-02-28T14:22:28.375845 #843] DEBUG -- :   ↳ lib/gitlab/utils/strong_memoize.rb:28:in `strong_memoize'
D, [2022-02-28T14:22:28.375852 #843] DEBUG -- :   ↳ app/models/namespaces/traversal/linear.rb:99:in `root_ancestor'
D, [2022-02-28T14:22:28.375858 #843] DEBUG -- :   ↳ app/models/concerns/has_repository.rb:86:in `default_branch_from_group_preferences'
D, [2022-02-28T14:22:28.375864 #843] DEBUG -- :   ↳ app/models/concerns/has_repository.rb:79:in `default_branch_from_preferences'
D, [2022-02-28T14:22:28.375870 #843] DEBUG -- :   ↳ app/models/concerns/has_repository.rb:75:in `default_branch'
D, [2022-02-28T14:22:28.375875 #843] DEBUG -- :   ↳ app/models/project.rb:2673:in `default_branch_or_main'
D, [2022-02-28T14:22:28.375881 #843] DEBUG -- :   ↳ lib/gitlab/json.rb:110:in `dump'
D, [2022-02-28T14:22:28.375887 #843] DEBUG -- :   ↳ lib/gitlab/json.rb:110:in `adapter_dump'
D, [2022-02-28T14:22:28.375892 #843] DEBUG -- :   ↳ lib/gitlab/json.rb:42:in `dump'
D, [2022-02-28T13:49:45.398313 #89721] DEBUG -- :   CACHE Namespace Load (0.0ms)  SELECT "namespaces"."id", "namespaces"."name", "namespaces"."path", "namespaces"."owner_id", "namespaces"."created_at", "namespaces"."updated_at", "namespaces"."type", "namespaces"."description", "namespaces"."avatar", "namespaces"."membership_lock", "namespaces"."share_with_group_lock", "namespaces"."visibility_level", "namespaces"."request_access_enabled", "namespaces"."ldap_sync_status", "namespaces"."ldap_sync_error", "namespaces"."ldap_sync_last_update_at", "namespaces"."ldap_sync_last_successful_update_at", "namespaces"."ldap_sync_last_sync_at", "namespaces"."description_html", "namespaces"."lfs_enabled", "namespaces"."parent_id", "namespaces"."shared_runners_minutes_limit", "namespaces"."repository_size_limit", "namespaces"."require_two_factor_authentication", "namespaces"."two_factor_grace_period", "namespaces"."cached_markdown_version", "namespaces"."project_creation_level", "namespaces"."runners_token", "namespaces"."file_template_project_id", "namespaces"."saml_discovery_token", "namespaces"."runners_token_encrypted", "namespaces"."custom_project_templates_group_id", "namespaces"."auto_devops_enabled", "namespaces"."extra_shared_runners_minutes_limit", "namespaces"."last_ci_minutes_notification_at", "namespaces"."last_ci_minutes_usage_notification_level", "namespaces"."subgroup_creation_level", "namespaces"."emails_disabled", "namespaces"."max_pages_size", "namespaces"."max_artifacts_size", "namespaces"."mentions_disabled", "namespaces"."default_branch_protection", "namespaces"."unlock_membership_to_ldap", "namespaces"."max_personal_access_token_lifetime", "namespaces"."push_rule_id", "namespaces"."shared_runners_enabled", "namespaces"."allow_descendants_override_disabled_shared_runners", "namespaces"."traversal_ids" FROM "namespaces" WHERE "namespaces"."id" = 132 LIMIT 1
D, [2022-02-28T13:49:45.399008 #89721] DEBUG -- :   ↳ app/models/namespaces/traversal/linear.rb:103:in `block in root_ancestor'
D, [2022-02-28T13:49:45.399019 #89721] DEBUG -- :   ↳ lib/gitlab/utils/strong_memoize.rb:28:in `strong_memoize'
D, [2022-02-28T13:49:45.399025 #89721] DEBUG -- :   ↳ app/models/namespaces/traversal/linear.rb:99:in `root_ancestor'
D, [2022-02-28T13:49:45.399032 #89721] DEBUG -- :   ↳ app/models/project.rb:2634:in `root_namespace'
D, [2022-02-28T13:49:45.288326 #89721] DEBUG -- :   ↳ ee/app/policies/ee/project_policy.rb:130:in `block (2 levels) in <module:ProjectPolicy>'
D, [2022-02-28T13:49:45.288331 #89721] DEBUG -- :   ↳ app/models/ability.rb:79:in `allowed?'
D, [2022-02-28T13:49:45.288337 #89721] DEBUG -- :   ↳ lib/api/entities/project.rb:59:in `block in <class:Project>'
D, [2022-02-28T13:49:45.288342 #89721] DEBUG -- :   ↳ lib/gitlab/json.rb:110:in `dump'
D, [2022-02-28T13:49:45.288356 #89721] DEBUG -- :   ↳ lib/gitlab/json.rb:110:in `adapter_dump'
D, [2022-02-28T13:49:45.288361 #89721] DEBUG -- :   ↳ lib/gitlab/json.rb:42:in `dump'

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Drew Blessing

Merge request reports

Loading