Skip to content

Resolve "Optimize Routable.find_by_full_path by avoiding joins"

Manoj M J requested to merge 292252-optimize-routable-find_by_full_path-by-avoiding-joins into master

What does this MR do and why?

#292252 (closed)

We are building Cells, and tables are being tagged as either a clusterwide table or cell-local table. routes is a clusterwide table and project/namespaces are cell-local tables.

Because these belong to different database schemas, cross-joins between them cannot happen and hence the queries should be rewritten to prevent cross-joins from happening.

This MR attempts to do that, in the Routable.find_by_full_path() method.

Details

This method has different flavours: It can be used as

  • Routable.find_by_full_path(path) or as
  • Project.find_by_full_path(path) or Group.find_by_full_path(path),

or it's chained versions:

  • Project.where(something: something).find_by_full_path(path)/ Group.where(something: something).find_by_full_path(path).

(Note: Routable.find_by_full_path(path) itself cannot be used in chained format, because Routable is not an ActiveRecord model)

Example of one of the changes

Essentially, we are replacing the cross-joins between projects/routes AND namespaces/routes such that they now produce 2 different queries.

Example:

Group.find_by_full_path 'gitlab-org/analytics-section'
Before After
SELECT "namespaces"."id" AS t0_r0, "namespaces"."name" AS t0_r1, "namespaces"."path" AS t0_r2, "namespaces"."owner_id" AS t0_r3, "namespaces"."created_at" AS t0_r4, "namespaces"."updated_at" AS t0_r5, "namespaces"."type" AS t0_r6, "namespaces"."description" AS t0_r7, "namespaces"."avatar" AS t0_r8, "namespaces"."membership_lock" AS t0_r9, "namespaces"."share_with_group_lock" AS t0_r10, "namespaces"."visibility_level" AS t0_r11, "namespaces"."request_access_enabled" AS t0_r12, "namespaces"."ldap_sync_status" AS t0_r13, "namespaces"."ldap_sync_error" AS t0_r14, "namespaces"."ldap_sync_last_update_at" AS t0_r15, "namespaces"."ldap_sync_last_successful_update_at" AS t0_r16, "namespaces"."ldap_sync_last_sync_at" AS t0_r17, "namespaces"."description_html" AS t0_r18, "namespaces"."lfs_enabled" AS t0_r19, "namespaces"."parent_id" AS t0_r20, "namespaces"."shared_runners_minutes_limit" AS t0_r21, "namespaces"."repository_size_limit" AS t0_r22, "namespaces"."require_two_factor_authentication" AS t0_r23, "namespaces"."two_factor_grace_period" AS t0_r24, "namespaces"."cached_markdown_version" AS t0_r25, "namespaces"."project_creation_level" AS t0_r26, "namespaces"."runners_token" AS t0_r27, "namespaces"."file_template_project_id" AS t0_r28, "namespaces"."saml_discovery_token" AS t0_r29, "namespaces"."runners_token_encrypted" AS t0_r30, "namespaces"."custom_project_templates_group_id" AS t0_r31, "namespaces"."auto_devops_enabled" AS t0_r32, "namespaces"."extra_shared_runners_minutes_limit" AS t0_r33, "namespaces"."last_ci_minutes_notification_at" AS t0_r34, "namespaces"."last_ci_minutes_usage_notification_level" AS t0_r35, "namespaces"."subgroup_creation_level" AS t0_r36, "namespaces"."emails_disabled" AS t0_r37, "namespaces"."max_pages_size" AS t0_r38, "namespaces"."max_artifacts_size" AS t0_r39, "namespaces"."mentions_disabled" AS t0_r40, "namespaces"."default_branch_protection" AS t0_r41, "namespaces"."unlock_membership_to_ldap" AS t0_r42, "namespaces"."max_personal_access_token_lifetime" AS t0_r43, "namespaces"."push_rule_id" AS t0_r44, "namespaces"."shared_runners_enabled" AS t0_r45, "namespaces"."allow_descendants_override_disabled_shared_runners" AS t0_r46, "namespaces"."traversal_ids" AS t0_r47, "namespaces"."organization_id" AS t0_r48, "routes"."id" AS t1_r0, "routes"."source_id" AS t1_r1, "routes"."source_type" AS t1_r2, "routes"."path" AS t1_r3, "routes"."created_at" AS t1_r4, "routes"."updated_at" AS t1_r5, "routes"."name" AS t1_r6, "routes"."namespace_id" AS t1_r7 FROM "namespaces" LEFT OUTER JOIN "routes" ON "routes"."source_type" = 'Namespace' AND "routes"."source_id" = "namespaces"."id" WHERE "namespaces"."type" = 'Group' AND "routes"."path" = 'gitlab-org/analytics-section' LIMIT 1 (Turns to 2 different queries) : SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Namespace' AND "routes"."path" = 'gitlab-org/analytics-section' LIMIT 1 , SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 55154808 LIMIT 1

Query plans

Queries before this change

Queries before this change:

We are using project = gitlab-org/gitlab, Project ID = 278964

We are using group = gitlab-org/analytics-section, Group ID = 55154808

  1. Project, without chaining

Project.find_by_full_path 'gitlab-org/gitlab'
Query Query 
SELECT "projects"."id" AS t0_r0, "projects"."name" AS t0_r1, "projects"."path" AS t0_r2, "projects"."description" AS t0_r3, "projects"."created_at" AS t0_r4, "projects"."updated_at" AS t0_r5, "projects"."creator_id" AS t0_r6, "projects"."namespace_id" AS t0_r7, "projects"."last_activity_at" AS t0_r8, "projects"."import_url" AS t0_r9, "projects"."visibility_level" AS t0_r10, "projects"."archived" AS t0_r11, "projects"."avatar" AS t0_r12, "projects"."merge_requests_template" AS t0_r13, "projects"."star_count" AS t0_r14, "projects"."merge_requests_rebase_enabled" AS t0_r15, "projects"."import_type" AS t0_r16, "projects"."import_source" AS t0_r17, "projects"."approvals_before_merge" AS t0_r18, "projects"."reset_approvals_on_push" AS t0_r19, "projects"."merge_requests_ff_only_enabled" AS t0_r20, "projects"."issues_template" AS t0_r21, "projects"."mirror" AS t0_r22, "projects"."mirror_last_update_at" AS t0_r23, "projects"."mirror_last_successful_update_at" AS t0_r24, "projects"."mirror_user_id" AS t0_r25, "projects"."shared_runners_enabled" AS t0_r26, "projects"."runners_token" AS t0_r27, "projects"."build_allow_git_fetch" AS t0_r28, "projects"."build_timeout" AS t0_r29, "projects"."mirror_trigger_builds" AS t0_r30, "projects"."pending_delete" AS t0_r31, "projects"."public_builds" AS t0_r32, "projects"."last_repository_check_failed" AS t0_r33, "projects"."last_repository_check_at" AS t0_r34, "projects"."only_allow_merge_if_pipeline_succeeds" AS t0_r35, "projects"."has_external_issue_tracker" AS t0_r36, "projects"."repository_storage" AS t0_r37, "projects"."repository_read_only" AS t0_r38, "projects"."request_access_enabled" AS t0_r39, "projects"."has_external_wiki" AS t0_r40, "projects"."ci_config_path" AS t0_r41, "projects"."lfs_enabled" AS t0_r42, "projects"."description_html" AS t0_r43, "projects"."only_allow_merge_if_all_discussions_are_resolved" AS t0_r44, "projects"."repository_size_limit" AS t0_r45, "projects"."printing_merge_request_link_enabled" AS t0_r46, "projects"."auto_cancel_pending_pipelines" AS t0_r47, "projects"."service_desk_enabled" AS t0_r48, "projects"."cached_markdown_version" AS t0_r49, "projects"."delete_error" AS t0_r50, "projects"."last_repository_updated_at" AS t0_r51, "projects"."disable_overriding_approvers_per_merge_request" AS t0_r52, "projects"."storage_version" AS t0_r53, "projects"."resolve_outdated_diff_discussions" AS t0_r54, "projects"."remote_mirror_available_overridden" AS t0_r55, "projects"."only_mirror_protected_branches" AS t0_r56, "projects"."pull_mirror_available_overridden" AS t0_r57, "projects"."jobs_cache_index" AS t0_r58, "projects"."external_authorization_classification_label" AS t0_r59, "projects"."mirror_overwrites_diverged_branches" AS t0_r60, "projects"."pages_https_only" AS t0_r61, "projects"."external_webhook_token" AS t0_r62, "projects"."packages_enabled" AS t0_r63, "projects"."merge_requests_author_approval" AS t0_r64, "projects"."pool_repository_id" AS t0_r65, "projects"."runners_token_encrypted" AS t0_r66, "projects"."bfg_object_map" AS t0_r67, "projects"."detected_repository_languages" AS t0_r68, "projects"."merge_requests_disable_committers_approval" AS t0_r69, "projects"."require_password_to_approve" AS t0_r70, "projects"."max_pages_size" AS t0_r71, "projects"."max_artifacts_size" AS t0_r72, "projects"."pull_mirror_branch_prefix" AS t0_r73, "projects"."remove_source_branch_after_merge" AS t0_r74, "projects"."marked_for_deletion_at" AS t0_r75, "projects"."marked_for_deletion_by_user_id" AS t0_r76, "projects"."autoclose_referenced_issues" AS t0_r77, "projects"."suggestion_commit_message" AS t0_r78, "projects"."project_namespace_id" AS t0_r79, "projects"."hidden" AS t0_r80, "projects"."organization_id" AS t0_r81, "routes"."id" AS t1_r0, "routes"."source_id" AS t1_r1, "routes"."source_type" AS t1_r2, "routes"."path" AS t1_r3, "routes"."created_at" AS t1_r4, "routes"."updated_at" AS t1_r5, "routes"."name" AS t1_r6, "routes"."namespace_id" AS t1_r7 FROM "projects" LEFT OUTER JOIN "routes" ON "routes"."source_type" = 'Project' AND "routes"."source_id" = "projects"."id" WHERE "routes"."path" = 'gitlab-org/gitlab' LIMIT 1

Plan: https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/21880/commands/70863

  1. Project, with chaining

Project.where(id: [4,6,8,9, 278964]).find_by_full_path 'gitlab-org/gitlab'
Query Query 
SELECT "projects"."id" AS t0_r0, "projects"."name" AS t0_r1, "projects"."path" AS t0_r2, "projects"."description" AS t0_r3, "projects"."created_at" AS t0_r4, "projects"."updated_at" AS t0_r5, "projects"."creator_id" AS t0_r6, "projects"."namespace_id" AS t0_r7, "projects"."last_activity_at" AS t0_r8, "projects"."import_url" AS t0_r9, "projects"."visibility_level" AS t0_r10, "projects"."archived" AS t0_r11, "projects"."avatar" AS t0_r12, "projects"."merge_requests_template" AS t0_r13, "projects"."star_count" AS t0_r14, "projects"."merge_requests_rebase_enabled" AS t0_r15, "projects"."import_type" AS t0_r16, "projects"."import_source" AS t0_r17, "projects"."approvals_before_merge" AS t0_r18, "projects"."reset_approvals_on_push" AS t0_r19, "projects"."merge_requests_ff_only_enabled" AS t0_r20, "projects"."issues_template" AS t0_r21, "projects"."mirror" AS t0_r22, "projects"."mirror_last_update_at" AS t0_r23, "projects"."mirror_last_successful_update_at" AS t0_r24, "projects"."mirror_user_id" AS t0_r25, "projects"."shared_runners_enabled" AS t0_r26, "projects"."runners_token" AS t0_r27, "projects"."build_allow_git_fetch" AS t0_r28, "projects"."build_timeout" AS t0_r29, "projects"."mirror_trigger_builds" AS t0_r30, "projects"."pending_delete" AS t0_r31, "projects"."public_builds" AS t0_r32, "projects"."last_repository_check_failed" AS t0_r33, "projects"."last_repository_check_at" AS t0_r34, "projects"."only_allow_merge_if_pipeline_succeeds" AS t0_r35, "projects"."has_external_issue_tracker" AS t0_r36, "projects"."repository_storage" AS t0_r37, "projects"."repository_read_only" AS t0_r38, "projects"."request_access_enabled" AS t0_r39, "projects"."has_external_wiki" AS t0_r40, "projects"."ci_config_path" AS t0_r41, "projects"."lfs_enabled" AS t0_r42, "projects"."description_html" AS t0_r43, "projects"."only_allow_merge_if_all_discussions_are_resolved" AS t0_r44, "projects"."repository_size_limit" AS t0_r45, "projects"."printing_merge_request_link_enabled" AS t0_r46, "projects"."auto_cancel_pending_pipelines" AS t0_r47, "projects"."service_desk_enabled" AS t0_r48, "projects"."cached_markdown_version" AS t0_r49, "projects"."delete_error" AS t0_r50, "projects"."last_repository_updated_at" AS t0_r51, "projects"."disable_overriding_approvers_per_merge_request" AS t0_r52, "projects"."storage_version" AS t0_r53, "projects"."resolve_outdated_diff_discussions" AS t0_r54, "projects"."remote_mirror_available_overridden" AS t0_r55, "projects"."only_mirror_protected_branches" AS t0_r56, "projects"."pull_mirror_available_overridden" AS t0_r57, "projects"."jobs_cache_index" AS t0_r58, "projects"."external_authorization_classification_label" AS t0_r59, "projects"."mirror_overwrites_diverged_branches" AS t0_r60, "projects"."pages_https_only" AS t0_r61, "projects"."external_webhook_token" AS t0_r62, "projects"."packages_enabled" AS t0_r63, "projects"."merge_requests_author_approval" AS t0_r64, "projects"."pool_repository_id" AS t0_r65, "projects"."runners_token_encrypted" AS t0_r66, "projects"."bfg_object_map" AS t0_r67, "projects"."detected_repository_languages" AS t0_r68, "projects"."merge_requests_disable_committers_approval" AS t0_r69, "projects"."require_password_to_approve" AS t0_r70, "projects"."max_pages_size" AS t0_r71, "projects"."max_artifacts_size" AS t0_r72, "projects"."pull_mirror_branch_prefix" AS t0_r73, "projects"."remove_source_branch_after_merge" AS t0_r74, "projects"."marked_for_deletion_at" AS t0_r75, "projects"."marked_for_deletion_by_user_id" AS t0_r76, "projects"."autoclose_referenced_issues" AS t0_r77, "projects"."suggestion_commit_message" AS t0_r78, "projects"."project_namespace_id" AS t0_r79, "projects"."hidden" AS t0_r80, "projects"."organization_id" AS t0_r81, "routes"."id" AS t1_r0, "routes"."source_id" AS t1_r1, "routes"."source_type" AS t1_r2, "routes"."path" AS t1_r3, "routes"."created_at" AS t1_r4, "routes"."updated_at" AS t1_r5, "routes"."name" AS t1_r6, "routes"."namespace_id" AS t1_r7 FROM "projects" LEFT OUTER JOIN "routes" ON "routes"."source_type" = 'Project' AND "routes"."source_id" = "projects"."id" WHERE "projects"."id" IN (4, 6, 8, 9, 278964) AND "routes"."path" = 'gitlab-org/gitlab' LIMIT 1

Plan: https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/21880/commands/70865

  1. Group, without chaining

Group.find_by_full_path 'gitlab-org/analytics-section'
Query Query 
SELECT "namespaces"."id" AS t0_r0, "namespaces"."name" AS t0_r1, "namespaces"."path" AS t0_r2, "namespaces"."owner_id" AS t0_r3, "namespaces"."created_at" AS t0_r4, "namespaces"."updated_at" AS t0_r5, "namespaces"."type" AS t0_r6, "namespaces"."description" AS t0_r7, "namespaces"."avatar" AS t0_r8, "namespaces"."membership_lock" AS t0_r9, "namespaces"."share_with_group_lock" AS t0_r10, "namespaces"."visibility_level" AS t0_r11, "namespaces"."request_access_enabled" AS t0_r12, "namespaces"."ldap_sync_status" AS t0_r13, "namespaces"."ldap_sync_error" AS t0_r14, "namespaces"."ldap_sync_last_update_at" AS t0_r15, "namespaces"."ldap_sync_last_successful_update_at" AS t0_r16, "namespaces"."ldap_sync_last_sync_at" AS t0_r17, "namespaces"."description_html" AS t0_r18, "namespaces"."lfs_enabled" AS t0_r19, "namespaces"."parent_id" AS t0_r20, "namespaces"."shared_runners_minutes_limit" AS t0_r21, "namespaces"."repository_size_limit" AS t0_r22, "namespaces"."require_two_factor_authentication" AS t0_r23, "namespaces"."two_factor_grace_period" AS t0_r24, "namespaces"."cached_markdown_version" AS t0_r25, "namespaces"."project_creation_level" AS t0_r26, "namespaces"."runners_token" AS t0_r27, "namespaces"."file_template_project_id" AS t0_r28, "namespaces"."saml_discovery_token" AS t0_r29, "namespaces"."runners_token_encrypted" AS t0_r30, "namespaces"."custom_project_templates_group_id" AS t0_r31, "namespaces"."auto_devops_enabled" AS t0_r32, "namespaces"."extra_shared_runners_minutes_limit" AS t0_r33, "namespaces"."last_ci_minutes_notification_at" AS t0_r34, "namespaces"."last_ci_minutes_usage_notification_level" AS t0_r35, "namespaces"."subgroup_creation_level" AS t0_r36, "namespaces"."emails_disabled" AS t0_r37, "namespaces"."max_pages_size" AS t0_r38, "namespaces"."max_artifacts_size" AS t0_r39, "namespaces"."mentions_disabled" AS t0_r40, "namespaces"."default_branch_protection" AS t0_r41, "namespaces"."unlock_membership_to_ldap" AS t0_r42, "namespaces"."max_personal_access_token_lifetime" AS t0_r43, "namespaces"."push_rule_id" AS t0_r44, "namespaces"."shared_runners_enabled" AS t0_r45, "namespaces"."allow_descendants_override_disabled_shared_runners" AS t0_r46, "namespaces"."traversal_ids" AS t0_r47, "namespaces"."organization_id" AS t0_r48, "routes"."id" AS t1_r0, "routes"."source_id" AS t1_r1, "routes"."source_type" AS t1_r2, "routes"."path" AS t1_r3, "routes"."created_at" AS t1_r4, "routes"."updated_at" AS t1_r5, "routes"."name" AS t1_r6, "routes"."namespace_id" AS t1_r7 FROM "namespaces" LEFT OUTER JOIN "routes" ON "routes"."source_type" = 'Namespace' AND "routes"."source_id" = "namespaces"."id" WHERE "namespaces"."type" = 'Group' AND "routes"."path" = 'gitlab-org/analytics-section' LIMIT 1

Plan: https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/21880/commands/70866

  1. Group, with chaining

Group.where(id: [4,6,8,9,55154808]).find_by_full_path 'gitlab-org/analytics-section'
Query Query 
SELECT "namespaces"."id" AS t0_r0, "namespaces"."name" AS t0_r1, "namespaces"."path" AS t0_r2, "namespaces"."owner_id" AS t0_r3, "namespaces"."created_at" AS t0_r4, "namespaces"."updated_at" AS t0_r5, "namespaces"."type" AS t0_r6, "namespaces"."description" AS t0_r7, "namespaces"."avatar" AS t0_r8, "namespaces"."membership_lock" AS t0_r9, "namespaces"."share_with_group_lock" AS t0_r10, "namespaces"."visibility_level" AS t0_r11, "namespaces"."request_access_enabled" AS t0_r12, "namespaces"."ldap_sync_status" AS t0_r13, "namespaces"."ldap_sync_error" AS t0_r14, "namespaces"."ldap_sync_last_update_at" AS t0_r15, "namespaces"."ldap_sync_last_successful_update_at" AS t0_r16, "namespaces"."ldap_sync_last_sync_at" AS t0_r17, "namespaces"."description_html" AS t0_r18, "namespaces"."lfs_enabled" AS t0_r19, "namespaces"."parent_id" AS t0_r20, "namespaces"."shared_runners_minutes_limit" AS t0_r21, "namespaces"."repository_size_limit" AS t0_r22, "namespaces"."require_two_factor_authentication" AS t0_r23, "namespaces"."two_factor_grace_period" AS t0_r24, "namespaces"."cached_markdown_version" AS t0_r25, "namespaces"."project_creation_level" AS t0_r26, "namespaces"."runners_token" AS t0_r27, "namespaces"."file_template_project_id" AS t0_r28, "namespaces"."saml_discovery_token" AS t0_r29, "namespaces"."runners_token_encrypted" AS t0_r30, "namespaces"."custom_project_templates_group_id" AS t0_r31, "namespaces"."auto_devops_enabled" AS t0_r32, "namespaces"."extra_shared_runners_minutes_limit" AS t0_r33, "namespaces"."last_ci_minutes_notification_at" AS t0_r34, "namespaces"."last_ci_minutes_usage_notification_level" AS t0_r35, "namespaces"."subgroup_creation_level" AS t0_r36, "namespaces"."emails_disabled" AS t0_r37, "namespaces"."max_pages_size" AS t0_r38, "namespaces"."max_artifacts_size" AS t0_r39, "namespaces"."mentions_disabled" AS t0_r40, "namespaces"."default_branch_protection" AS t0_r41, "namespaces"."unlock_membership_to_ldap" AS t0_r42, "namespaces"."max_personal_access_token_lifetime" AS t0_r43, "namespaces"."push_rule_id" AS t0_r44, "namespaces"."shared_runners_enabled" AS t0_r45, "namespaces"."allow_descendants_override_disabled_shared_runners" AS t0_r46, "namespaces"."traversal_ids" AS t0_r47, "namespaces"."organization_id" AS t0_r48, "routes"."id" AS t1_r0, "routes"."source_id" AS t1_r1, "routes"."source_type" AS t1_r2, "routes"."path" AS t1_r3, "routes"."created_at" AS t1_r4, "routes"."updated_at" AS t1_r5, "routes"."name" AS t1_r6, "routes"."namespace_id" AS t1_r7 FROM "namespaces" LEFT OUTER JOIN "routes" ON "routes"."source_type" = 'Namespace' AND "routes"."source_id" = "namespaces"."id" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" IN (4, 6, 8, 9, 55154808) AND "routes"."path" = 'gitlab-org/analytics-section' LIMIT 1

Plan: https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/21880/commands/70867

  1. Routable, for Project

Routable.find_by_full_path 'gitlab-org/gitlab'
Queries Query 
SELECT "routes".* FROM "routes" WHERE "routes"."path" = 'gitlab-org/gitlab' LIMIT 1 
SELECT "projects"."id", "projects"."name", "projects"."path", "projects"."description", "projects"."created_at", "projects"."updated_at", "projects"."creator_id", "projects"."namespace_id", "projects"."last_activity_at", "projects"."import_url", "projects"."visibility_level", "projects"."archived", "projects"."avatar", "projects"."merge_requests_template", "projects"."star_count", "projects"."merge_requests_rebase_enabled", "projects"."import_type", "projects"."import_source", "projects"."approvals_before_merge", "projects"."reset_approvals_on_push", "projects"."merge_requests_ff_only_enabled", "projects"."issues_template", "projects"."mirror", "projects"."mirror_last_update_at", "projects"."mirror_last_successful_update_at", "projects"."mirror_user_id", "projects"."shared_runners_enabled", "projects"."runners_token", "projects"."build_allow_git_fetch", "projects"."build_timeout", "projects"."mirror_trigger_builds", "projects"."pending_delete", "projects"."public_builds", "projects"."last_repository_check_failed", "projects"."last_repository_check_at", "projects"."only_allow_merge_if_pipeline_succeeds", "projects"."has_external_issue_tracker", "projects"."repository_storage", "projects"."repository_read_only", "projects"."request_access_enabled", "projects"."has_external_wiki", "projects"."ci_config_path", "projects"."lfs_enabled", "projects"."description_html", "projects"."only_allow_merge_if_all_discussions_are_resolved", "projects"."repository_size_limit", "projects"."printing_merge_request_link_enabled", "projects"."auto_cancel_pending_pipelines", "projects"."service_desk_enabled", "projects"."cached_markdown_version", "projects"."delete_error", "projects"."last_repository_updated_at", "projects"."disable_overriding_approvers_per_merge_request", "projects"."storage_version", "projects"."resolve_outdated_diff_discussions", "projects"."remote_mirror_available_overridden", "projects"."only_mirror_protected_branches", "projects"."pull_mirror_available_overridden", "projects"."jobs_cache_index", "projects"."external_authorization_classification_label", "projects"."mirror_overwrites_diverged_branches", "projects"."pages_https_only", "projects"."external_webhook_token", "projects"."packages_enabled", "projects"."merge_requests_author_approval", "projects"."pool_repository_id", "projects"."runners_token_encrypted", "projects"."bfg_object_map", "projects"."detected_repository_languages", "projects"."merge_requests_disable_committers_approval", "projects"."require_password_to_approve", "projects"."max_pages_size", "projects"."max_artifacts_size", "projects"."pull_mirror_branch_prefix", "projects"."remove_source_branch_after_merge", "projects"."marked_for_deletion_at", "projects"."marked_for_deletion_by_user_id", "projects"."autoclose_referenced_issues", "projects"."suggestion_commit_message", "projects"."project_namespace_id", "projects"."hidden", "projects"."organization_id" FROM "projects" WHERE "projects"."id" = 278964 LIMIT 1

Plans:

  1. Routable, for Group

Query

Routable.find_by_full_path 'gitlab-org/analytics-section'
Queries Query 
SELECT "routes".* FROM "routes" WHERE "routes"."path" = 'gitlab-org/analytics-section' LIMIT 1
SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."id" = 55154808 LIMIT 1

Plans:

Queries after this change

Queries after this change:

We are using project = gitlab-org/gitlab, Project ID = 278964

We are using group = gitlab-org/analytics-section, Group ID = 55154808

  1. Project, without chaining

Project.find_by_full_path 'gitlab-org/gitlab'
Queries Queries 
SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Project' AND "routes"."path" = 'gitlab-org/gitlab' LIMIT 1
SELECT "projects"."id", "projects"."name", "projects"."path", "projects"."description", "projects"."created_at", "projects"."updated_at", "projects"."creator_id", "projects"."namespace_id", "projects"."last_activity_at", "projects"."import_url", "projects"."visibility_level", "projects"."archived", "projects"."avatar", "projects"."merge_requests_template", "projects"."star_count", "projects"."merge_requests_rebase_enabled", "projects"."import_type", "projects"."import_source", "projects"."approvals_before_merge", "projects"."reset_approvals_on_push", "projects"."merge_requests_ff_only_enabled", "projects"."issues_template", "projects"."mirror", "projects"."mirror_last_update_at", "projects"."mirror_last_successful_update_at", "projects"."mirror_user_id", "projects"."shared_runners_enabled", "projects"."runners_token", "projects"."build_allow_git_fetch", "projects"."build_timeout", "projects"."mirror_trigger_builds", "projects"."pending_delete", "projects"."public_builds", "projects"."last_repository_check_failed", "projects"."last_repository_check_at", "projects"."only_allow_merge_if_pipeline_succeeds", "projects"."has_external_issue_tracker", "projects"."repository_storage", "projects"."repository_read_only", "projects"."request_access_enabled", "projects"."has_external_wiki", "projects"."ci_config_path", "projects"."lfs_enabled", "projects"."description_html", "projects"."only_allow_merge_if_all_discussions_are_resolved", "projects"."repository_size_limit", "projects"."printing_merge_request_link_enabled", "projects"."auto_cancel_pending_pipelines", "projects"."service_desk_enabled", "projects"."cached_markdown_version", "projects"."delete_error", "projects"."last_repository_updated_at", "projects"."disable_overriding_approvers_per_merge_request", "projects"."storage_version", "projects"."resolve_outdated_diff_discussions", "projects"."remote_mirror_available_overridden", "projects"."only_mirror_protected_branches", "projects"."pull_mirror_available_overridden", "projects"."jobs_cache_index", "projects"."external_authorization_classification_label", "projects"."mirror_overwrites_diverged_branches", "projects"."pages_https_only", "projects"."external_webhook_token", "projects"."packages_enabled", "projects"."merge_requests_author_approval", "projects"."pool_repository_id", "projects"."runners_token_encrypted", "projects"."bfg_object_map", "projects"."detected_repository_languages", "projects"."merge_requests_disable_committers_approval", "projects"."require_password_to_approve", "projects"."max_pages_size", "projects"."max_artifacts_size", "projects"."pull_mirror_branch_prefix", "projects"."remove_source_branch_after_merge", "projects"."marked_for_deletion_at", "projects"."marked_for_deletion_by_user_id", "projects"."autoclose_referenced_issues", "projects"."suggestion_commit_message", "projects"."project_namespace_id", "projects"."hidden", "projects"."organization_id" FROM "projects" WHERE "projects"."id" = 278964 LIMIT 1

Plans:

  1. Project, with chaining

Project.where(id: [4,6,8,9, 278964]).find_by_full_path 'gitlab-org/gitlab'
Queries Queries 
SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Project' AND "routes"."path" = 'gitlab-org/gitlab' LIMIT 1
SELECT "projects"."id", "projects"."name", "projects"."path", "projects"."description", "projects"."created_at", "projects"."updated_at", "projects"."creator_id", "projects"."namespace_id", "projects"."last_activity_at", "projects"."import_url", "projects"."visibility_level", "projects"."archived", "projects"."avatar", "projects"."merge_requests_template", "projects"."star_count", "projects"."merge_requests_rebase_enabled", "projects"."import_type", "projects"."import_source", "projects"."approvals_before_merge", "projects"."reset_approvals_on_push", "projects"."merge_requests_ff_only_enabled", "projects"."issues_template", "projects"."mirror", "projects"."mirror_last_update_at", "projects"."mirror_last_successful_update_at", "projects"."mirror_user_id", "projects"."shared_runners_enabled", "projects"."runners_token", "projects"."build_allow_git_fetch", "projects"."build_timeout", "projects"."mirror_trigger_builds", "projects"."pending_delete", "projects"."public_builds", "projects"."last_repository_check_failed", "projects"."last_repository_check_at", "projects"."only_allow_merge_if_pipeline_succeeds", "projects"."has_external_issue_tracker", "projects"."repository_storage", "projects"."repository_read_only", "projects"."request_access_enabled", "projects"."has_external_wiki", "projects"."ci_config_path", "projects"."lfs_enabled", "projects"."description_html", "projects"."only_allow_merge_if_all_discussions_are_resolved", "projects"."repository_size_limit", "projects"."printing_merge_request_link_enabled", "projects"."auto_cancel_pending_pipelines", "projects"."service_desk_enabled", "projects"."cached_markdown_version", "projects"."delete_error", "projects"."last_repository_updated_at", "projects"."disable_overriding_approvers_per_merge_request", "projects"."storage_version", "projects"."resolve_outdated_diff_discussions", "projects"."remote_mirror_available_overridden", "projects"."only_mirror_protected_branches", "projects"."pull_mirror_available_overridden", "projects"."jobs_cache_index", "projects"."external_authorization_classification_label", "projects"."mirror_overwrites_diverged_branches", "projects"."pages_https_only", "projects"."external_webhook_token", "projects"."packages_enabled", "projects"."merge_requests_author_approval", "projects"."pool_repository_id", "projects"."runners_token_encrypted", "projects"."bfg_object_map", "projects"."detected_repository_languages", "projects"."merge_requests_disable_committers_approval", "projects"."require_password_to_approve", "projects"."max_pages_size", "projects"."max_artifacts_size", "projects"."pull_mirror_branch_prefix", "projects"."remove_source_branch_after_merge", "projects"."marked_for_deletion_at", "projects"."marked_for_deletion_by_user_id", "projects"."autoclose_referenced_issues", "projects"."suggestion_commit_message", "projects"."project_namespace_id", "projects"."hidden", "projects"."organization_id" FROM "projects"  WHERE "projects"."id" IN (4, 6, 8, 9, 278964) AND "projects"."id" = 278964 LIMIT 1

Plans:

  1. Group, without chaining

Group.find_by_full_path 'gitlab-org/analytics-section'
Queries Queries 
SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Namespace' AND "routes"."path" = 'gitlab-org/analytics-section' LIMIT 1
SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" = 55154808 LIMIT 1

Plans:

  1. Group, with chaining

Group.where(id: [4,6,8,9,55154808]).find_by_full_path 'gitlab-org/analytics-section'
Queries Queries 
SELECT "routes".* FROM "routes" WHERE "routes"."source_type" = 'Namespace' AND "routes"."path" = 'gitlab-org/analytics-section' LIMIT 1
SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."type" = 'Group' AND "namespaces"."id" IN (4, 6, 8, 9, 77, 55154808) AND "namespaces"."id" = 55154808 LIMIT 1

Plans:

  1. Routable, for Project (this is now same as Project.find_by_full_path(path))

Routable.find_by_full_path 'gitlab-org/gitlab'
Queries Query 
SELECT "routes".* FROM "routes" WHERE "routes"."path" = 'gitlab-org/gitlab' LIMIT 1
SELECT "projects"."id", "projects"."name", "projects"."path", "projects"."description", "projects"."created_at", "projects"."updated_at", "projects"."creator_id", "projects"."namespace_id", "projects"."last_activity_at", "projects"."import_url", "projects"."visibility_level", "projects"."archived", "projects"."avatar", "projects"."merge_requests_template", "projects"."star_count", "projects"."merge_requests_rebase_enabled", "projects"."import_type", "projects"."import_source", "projects"."approvals_before_merge", "projects"."reset_approvals_on_push", "projects"."merge_requests_ff_only_enabled", "projects"."issues_template", "projects"."mirror", "projects"."mirror_last_update_at", "projects"."mirror_last_successful_update_at", "projects"."mirror_user_id", "projects"."shared_runners_enabled", "projects"."runners_token", "projects"."build_allow_git_fetch", "projects"."build_timeout", "projects"."mirror_trigger_builds", "projects"."pending_delete", "projects"."public_builds", "projects"."last_repository_check_failed", "projects"."last_repository_check_at", "projects"."only_allow_merge_if_pipeline_succeeds", "projects"."has_external_issue_tracker", "projects"."repository_storage", "projects"."repository_read_only", "projects"."request_access_enabled", "projects"."has_external_wiki", "projects"."ci_config_path", "projects"."lfs_enabled", "projects"."description_html", "projects"."only_allow_merge_if_all_discussions_are_resolved", "projects"."repository_size_limit", "projects"."printing_merge_request_link_enabled", "projects"."auto_cancel_pending_pipelines", "projects"."service_desk_enabled", "projects"."cached_markdown_version", "projects"."delete_error", "projects"."last_repository_updated_at", "projects"."disable_overriding_approvers_per_merge_request", "projects"."storage_version", "projects"."resolve_outdated_diff_discussions", "projects"."remote_mirror_available_overridden", "projects"."only_mirror_protected_branches", "projects"."pull_mirror_available_overridden", "projects"."jobs_cache_index", "projects"."external_authorization_classification_label", "projects"."mirror_overwrites_diverged_branches", "projects"."pages_https_only", "projects"."external_webhook_token", "projects"."packages_enabled", "projects"."merge_requests_author_approval", "projects"."pool_repository_id", "projects"."runners_token_encrypted", "projects"."bfg_object_map", "projects"."detected_repository_languages", "projects"."merge_requests_disable_committers_approval", "projects"."require_password_to_approve", "projects"."max_pages_size", "projects"."max_artifacts_size", "projects"."pull_mirror_branch_prefix", "projects"."remove_source_branch_after_merge", "projects"."marked_for_deletion_at", "projects"."marked_for_deletion_by_user_id", "projects"."autoclose_referenced_issues", "projects"."suggestion_commit_message", "projects"."project_namespace_id", "projects"."hidden", "projects"."organization_id" FROM "projects" WHERE "projects"."id" = 278964 LIMIT 1

Plans:

  1. Routable, for Group (this is now same as Project.find_by_full_path(path))

Query

Routable.find_by_full_path 'gitlab-org/analytics-section'
Queries Query 
SELECT "routes".* FROM "routes" WHERE "routes"."path" = 'gitlab-org/analytics-section' LIMIT 1
SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."id" = 55154808 LIMIT 1

Plans:

Note:

  • Change has been added behind a feature flag, just to be on the safer side. And hence, no changelog has been added.
  • After the change, the queries generated from Project.find_by_full_path/Group.find_by_full_path are essentially the same as the current queries generated by Routable.find_by_full_path, so we should be safe on the correctness and the query performance part, according to my understanding.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #292252 (closed)

Edited by Manoj M J

Merge request reports