GDK requests are very slow to render views

Description

Within the last week or so, after installing a fresh GDK, requests have been incredibly slow (often > 20 seconds). Looking at the development.log it appears this is because it's having a hard time rendering the views (specifically layout/_head.html.haml):

# Feel free to scroll to the bottom to see the 10+ ms it takes to render the view.
# You'll notice that `layout/_head.html.haml` is the culprit to this large delay

Started GET "/root/codesandbox-example" for 192.168.1.7 at 2019-12-08 20:01:50 -0600
  SQL (0.9ms)  SELECT  "namespaces"."id" AS t0_r0, "namespaces"."name" AS t0_r1, "namespaces"."path" AS t0_r2, "namespaces"."owner_id" AS t0_r3, "namespaces"."created_at" AS t0_r4, "namespaces"."updated_at" AS t0_r5, "namespaces"."type" AS t0_r6, "namespaces"."description" AS t0_r7, "namespaces"."avatar" AS t0_r8, "namespaces"."membership_lock" AS t0_r9, "namespaces"."share_with_group_lock" AS t0_r10, "namespaces"."visibility_level" AS t0_r11, "namespaces"."request_access_enabled" AS t0_r12, "namespaces"."ldap_sync_status" AS t0_r13, "namespaces"."ldap_sync_error" AS t0_r14, "namespaces"."ldap_sync_last_update_at" AS t0_r15, "namespaces"."ldap_sync_last_successful_update_at" AS t0_r16, "namespaces"."ldap_sync_last_sync_at" AS t0_r17, "namespaces"."description_html" AS t0_r18, "namespaces"."lfs_enabled" AS t0_r19, "namespaces"."parent_id" AS t0_r20, "namespaces"."shared_runners_minutes_limit" AS t0_r21, "namespaces"."repository_size_limit" AS t0_r22, "namespaces"."require_two_factor_authentication" AS t0_r23, "namespaces"."two_factor_grace_period" AS t0_r24, "namespaces"."cached_markdown_version" AS t0_r25, "namespaces"."plan_id" AS t0_r26, "namespaces"."project_creation_level" AS t0_r27, "namespaces"."runners_token" AS t0_r28, "namespaces"."trial_ends_on" AS t0_r29, "namespaces"."file_template_project_id" AS t0_r30, "namespaces"."saml_discovery_token" AS t0_r31, "namespaces"."runners_token_encrypted" AS t0_r32, "namespaces"."custom_project_templates_group_id" AS t0_r33, "namespaces"."auto_devops_enabled" AS t0_r34, "namespaces"."extra_shared_runners_minutes_limit" AS t0_r35, "namespaces"."last_ci_minutes_notification_at" AS t0_r36, "namespaces"."last_ci_minutes_usage_notification_level" AS t0_r37, "namespaces"."subgroup_creation_level" AS t0_r38, "namespaces"."emails_disabled" AS t0_r39, "namespaces"."max_pages_size" AS t0_r40, "namespaces"."max_artifacts_size" AS t0_r41, "namespaces"."mentions_disabled" AS t0_r42, "routes"."id" AS t1_r0, "routes"."source_id" AS t1_r1, "routes"."source_type" AS t1_r2, "routes"."path" AS t1_r3, "routes"."created_at" AS t1_r4, "routes"."updated_at" AS t1_r5, "routes"."name" AS t1_r6 FROM "namespaces" LEFT OUTER JOIN "routes" ON "routes"."source_id" = "namespaces"."id" AND "routes"."source_type" = $1 WHERE "namespaces"."type" IN ('Group') AND "routes"."path" = $2 LIMIT $3  [["source_type", "Namespace"], ["path", "root/codesandbox-example"], ["LIMIT", 1]]
  ↳ app/models/concerns/routable.rb:38
  SQL (0.7ms)  SELECT  "namespaces"."id" AS t0_r0, "namespaces"."name" AS t0_r1, "namespaces"."path" AS t0_r2, "namespaces"."owner_id" AS t0_r3, "namespaces"."created_at" AS t0_r4, "namespaces"."updated_at" AS t0_r5, "namespaces"."type" AS t0_r6, "namespaces"."description" AS t0_r7, "namespaces"."avatar" AS t0_r8, "namespaces"."membership_lock" AS t0_r9, "namespaces"."share_with_group_lock" AS t0_r10, "namespaces"."visibility_level" AS t0_r11, "namespaces"."request_access_enabled" AS t0_r12, "namespaces"."ldap_sync_status" AS t0_r13, "namespaces"."ldap_sync_error" AS t0_r14, "namespaces"."ldap_sync_last_update_at" AS t0_r15, "namespaces"."ldap_sync_last_successful_update_at" AS t0_r16, "namespaces"."ldap_sync_last_sync_at" AS t0_r17, "namespaces"."description_html" AS t0_r18, "namespaces"."lfs_enabled" AS t0_r19, "namespaces"."parent_id" AS t0_r20, "namespaces"."shared_runners_minutes_limit" AS t0_r21, "namespaces"."repository_size_limit" AS t0_r22, "namespaces"."require_two_factor_authentication" AS t0_r23, "namespaces"."two_factor_grace_period" AS t0_r24, "namespaces"."cached_markdown_version" AS t0_r25, "namespaces"."plan_id" AS t0_r26, "namespaces"."project_creation_level" AS t0_r27, "namespaces"."runners_token" AS t0_r28, "namespaces"."trial_ends_on" AS t0_r29, "namespaces"."file_template_project_id" AS t0_r30, "namespaces"."saml_discovery_token" AS t0_r31, "namespaces"."runners_token_encrypted" AS t0_r32, "namespaces"."custom_project_templates_group_id" AS t0_r33, "namespaces"."auto_devops_enabled" AS t0_r34, "namespaces"."extra_shared_runners_minutes_limit" AS t0_r35, "namespaces"."last_ci_minutes_notification_at" AS t0_r36, "namespaces"."last_ci_minutes_usage_notification_level" AS t0_r37, "namespaces"."subgroup_creation_level" AS t0_r38, "namespaces"."emails_disabled" AS t0_r39, "namespaces"."max_pages_size" AS t0_r40, "namespaces"."max_artifacts_size" AS t0_r41, "namespaces"."mentions_disabled" AS t0_r42, "routes"."id" AS t1_r0, "routes"."source_id" AS t1_r1, "routes"."source_type" AS t1_r2, "routes"."path" AS t1_r3, "routes"."created_at" AS t1_r4, "routes"."updated_at" AS t1_r5, "routes"."name" AS t1_r6 FROM "namespaces" LEFT OUTER JOIN "routes" ON "routes"."source_id" = "namespaces"."id" AND "routes"."source_type" = $1 WHERE "namespaces"."type" IN ('Group') AND ((LOWER(routes.path) = LOWER('root/codesandbox-example'))) LIMIT $2  [["source_type", "Namespace"], ["LIMIT", 1]]
  ↳ app/models/concerns/routable.rb:38
  Group Load (0.9ms)  SELECT  "namespaces".* FROM "namespaces" INNER JOIN "redirect_routes" ON "redirect_routes"."source_id" = "namespaces"."id" AND "redirect_routes"."source_type" = $1 WHERE "namespaces"."type" IN ('Group') AND (LOWER(redirect_routes.path) = LOWER('root/codesandbox-example')) LIMIT $2  [["source_type", "Namespace"], ["LIMIT", 1]]
  ↳ app/models/concerns/routable.rb:43
  SQL (0.9ms)  SELECT  "projects"."id" AS t0_r0, "projects"."name" AS t0_r1, "projects"."path" AS t0_r2, "projects"."description" AS t0_r3, "projects"."created_at" AS t0_r4, "projects"."updated_at" AS t0_r5, "projects"."creator_id" AS t0_r6, "projects"."namespace_id" AS t0_r7, "projects"."last_activity_at" AS t0_r8, "projects"."import_url" AS t0_r9, "projects"."visibility_level" AS t0_r10, "projects"."archived" AS t0_r11, "projects"."avatar" AS t0_r12, "projects"."merge_requests_template" AS t0_r13, "projects"."star_count" AS t0_r14, "projects"."merge_requests_rebase_enabled" AS t0_r15, "projects"."import_type" AS t0_r16, "projects"."import_source" AS t0_r17, "projects"."approvals_before_merge" AS t0_r18, "projects"."reset_approvals_on_push" AS t0_r19, "projects"."merge_requests_ff_only_enabled" AS t0_r20, "projects"."issues_template" AS t0_r21, "projects"."mirror" AS t0_r22, "projects"."mirror_last_update_at" AS t0_r23, "projects"."mirror_last_successful_update_at" AS t0_r24, "projects"."mirror_user_id" AS t0_r25, "projects"."shared_runners_enabled" AS t0_r26, "projects"."runners_token" AS t0_r27, "projects"."build_coverage_regex" AS t0_r28, "projects"."build_allow_git_fetch" AS t0_r29, "projects"."build_timeout" AS t0_r30, "projects"."mirror_trigger_builds" AS t0_r31, "projects"."pending_delete" AS t0_r32, "projects"."public_builds" AS t0_r33, "projects"."last_repository_check_failed" AS t0_r34, "projects"."last_repository_check_at" AS t0_r35, "projects"."container_registry_enabled" AS t0_r36, "projects"."only_allow_merge_if_pipeline_succeeds" AS t0_r37, "projects"."has_external_issue_tracker" AS t0_r38, "projects"."repository_storage" AS t0_r39, "projects"."repository_read_only" AS t0_r40, "projects"."request_access_enabled" AS t0_r41, "projects"."has_external_wiki" AS t0_r42, "projects"."ci_config_path" AS t0_r43, "projects"."lfs_enabled" AS t0_r44, "projects"."description_html" AS t0_r45, "projects"."only_allow_merge_if_all_discussions_are_resolved" AS t0_r46, "projects"."repository_size_limit" AS t0_r47, "projects"."printing_merge_request_link_enabled" AS t0_r48, "projects"."auto_cancel_pending_pipelines" AS t0_r49, "projects"."service_desk_enabled" AS t0_r50, "projects"."cached_markdown_version" AS t0_r51, "projects"."delete_error" AS t0_r52, "projects"."last_repository_updated_at" AS t0_r53, "projects"."disable_overriding_approvers_per_merge_request" AS t0_r54, "projects"."storage_version" AS t0_r55, "projects"."resolve_outdated_diff_discussions" AS t0_r56, "projects"."remote_mirror_available_overridden" AS t0_r57, "projects"."only_mirror_protected_branches" AS t0_r58, "projects"."pull_mirror_available_overridden" AS t0_r59, "projects"."jobs_cache_index" AS t0_r60, "projects"."external_authorization_classification_label" AS t0_r61, "projects"."mirror_overwrites_diverged_branches" AS t0_r62, "projects"."pages_https_only" AS t0_r63, "projects"."external_webhook_token" AS t0_r64, "projects"."packages_enabled" AS t0_r65, "projects"."merge_requests_author_approval" AS t0_r66, "projects"."pool_repository_id" AS t0_r67, "projects"."runners_token_encrypted" AS t0_r68, "projects"."bfg_object_map" AS t0_r69, "projects"."detected_repository_languages" AS t0_r70, "projects"."merge_requests_disable_committers_approval" AS t0_r71, "projects"."require_password_to_approve" AS t0_r72, "projects"."emails_disabled" AS t0_r73, "projects"."max_pages_size" AS t0_r74, "projects"."max_artifacts_size" AS t0_r75, "projects"."pull_mirror_branch_prefix" AS t0_r76, "projects"."remove_source_branch_after_merge" AS t0_r77, "projects"."marked_for_deletion_at" AS t0_r78, "projects"."marked_for_deletion_by_user_id" AS t0_r79, "routes"."id" AS t1_r0, "routes"."source_id" AS t1_r1, "routes"."source_type" AS t1_r2, "routes"."path" AS t1_r3, "routes"."created_at" AS t1_r4, "routes"."updated_at" AS t1_r5, "routes"."name" AS t1_r6 FROM "projects" LEFT OUTER JOIN "routes" ON "routes"."source_id" = "projects"."id" AND "routes"."source_type" = $1 WHERE "routes"."path" = $2 LIMIT $3  [["source_type", "Project"], ["path", "root/codesandbox-example"], ["LIMIT", 1]]
  ↳ app/models/concerns/routable.rb:38
Processing by ProjectsController#show as HTML
  Parameters: {"namespace_id"=>"root", "id"=>"codesandbox-example"}
  CACHE SQL (0.0ms)  SELECT  "projects"."id" AS t0_r0, "projects"."name" AS t0_r1, "projects"."path" AS t0_r2, "projects"."description" AS t0_r3, "projects"."created_at" AS t0_r4, "projects"."updated_at" AS t0_r5, "projects"."creator_id" AS t0_r6, "projects"."namespace_id" AS t0_r7, "projects"."last_activity_at" AS t0_r8, "projects"."import_url" AS t0_r9, "projects"."visibility_level" AS t0_r10, "projects"."archived" AS t0_r11, "projects"."avatar" AS t0_r12, "projects"."merge_requests_template" AS t0_r13, "projects"."star_count" AS t0_r14, "projects"."merge_requests_rebase_enabled" AS t0_r15, "projects"."import_type" AS t0_r16, "projects"."import_source" AS t0_r17, "projects"."approvals_before_merge" AS t0_r18, "projects"."reset_approvals_on_push" AS t0_r19, "projects"."merge_requests_ff_only_enabled" AS t0_r20, "projects"."issues_template" AS t0_r21, "projects"."mirror" AS t0_r22, "projects"."mirror_last_update_at" AS t0_r23, "projects"."mirror_last_successful_update_at" AS t0_r24, "projects"."mirror_user_id" AS t0_r25, "projects"."shared_runners_enabled" AS t0_r26, "projects"."runners_token" AS t0_r27, "projects"."build_coverage_regex" AS t0_r28, "projects"."build_allow_git_fetch" AS t0_r29, "projects"."build_timeout" AS t0_r30, "projects"."mirror_trigger_builds" AS t0_r31, "projects"."pending_delete" AS t0_r32, "projects"."public_builds" AS t0_r33, "projects"."last_repository_check_failed" AS t0_r34, "projects"."last_repository_check_at" AS t0_r35, "projects"."container_registry_enabled" AS t0_r36, "projects"."only_allow_merge_if_pipeline_succeeds" AS t0_r37, "projects"."has_external_issue_tracker" AS t0_r38, "projects"."repository_storage" AS t0_r39, "projects"."repository_read_only" AS t0_r40, "projects"."request_access_enabled" AS t0_r41, "projects"."has_external_wiki" AS t0_r42, "projects"."ci_config_path" AS t0_r43, "projects"."lfs_enabled" AS t0_r44, "projects"."description_html" AS t0_r45, "projects"."only_allow_merge_if_all_discussions_are_resolved" AS t0_r46, "projects"."repository_size_limit" AS t0_r47, "projects"."printing_merge_request_link_enabled" AS t0_r48, "projects"."auto_cancel_pending_pipelines" AS t0_r49, "projects"."service_desk_enabled" AS t0_r50, "projects"."cached_markdown_version" AS t0_r51, "projects"."delete_error" AS t0_r52, "projects"."last_repository_updated_at" AS t0_r53, "projects"."disable_overriding_approvers_per_merge_request" AS t0_r54, "projects"."storage_version" AS t0_r55, "projects"."resolve_outdated_diff_discussions" AS t0_r56, "projects"."remote_mirror_available_overridden" AS t0_r57, "projects"."only_mirror_protected_branches" AS t0_r58, "projects"."pull_mirror_available_overridden" AS t0_r59, "projects"."jobs_cache_index" AS t0_r60, "projects"."external_authorization_classification_label" AS t0_r61, "projects"."mirror_overwrites_diverged_branches" AS t0_r62, "projects"."pages_https_only" AS t0_r63, "projects"."external_webhook_token" AS t0_r64, "projects"."packages_enabled" AS t0_r65, "projects"."merge_requests_author_approval" AS t0_r66, "projects"."pool_repository_id" AS t0_r67, "projects"."runners_token_encrypted" AS t0_r68, "projects"."bfg_object_map" AS t0_r69, "projects"."detected_repository_languages" AS t0_r70, "projects"."merge_requests_disable_committers_approval" AS t0_r71, "projects"."require_password_to_approve" AS t0_r72, "projects"."emails_disabled" AS t0_r73, "projects"."max_pages_size" AS t0_r74, "projects"."max_artifacts_size" AS t0_r75, "projects"."pull_mirror_branch_prefix" AS t0_r76, "projects"."remove_source_branch_after_merge" AS t0_r77, "projects"."marked_for_deletion_at" AS t0_r78, "projects"."marked_for_deletion_by_user_id" AS t0_r79, "routes"."id" AS t1_r0, "routes"."source_id" AS t1_r1, "routes"."source_type" AS t1_r2, "routes"."path" AS t1_r3, "routes"."created_at" AS t1_r4, "routes"."updated_at" AS t1_r5, "routes"."name" AS t1_r6 FROM "projects" LEFT OUTER JOIN "routes" ON "routes"."source_id" = "projects"."id" AND "routes"."source_type" = $1 WHERE "routes"."path" = $2 LIMIT $3  [["source_type", "Project"], ["path", "root/codesandbox-example"], ["LIMIT", 1]]
  ↳ app/models/concerns/routable.rb:38
  Group Load (0.6ms)  SELECT  "namespaces".* FROM "namespaces" WHERE "namespaces"."type" IN ('Group') AND "namespaces"."id" = $1 AND "namespaces"."type" = $2 LIMIT $3  [["id", 1], ["type", "Group"], ["LIMIT", 1]]
  ↳ ee/app/policies/ee/project_policy.rb:267
  Namespace Load (0.4ms)  SELECT  "namespaces".* FROM "namespaces" WHERE "namespaces"."id" = $1 LIMIT $2  [["id", 1], ["LIMIT", 1]]
  ↳ config/application.rb:288
  Route Load (0.2ms)  SELECT  "routes".* FROM "routes" WHERE "routes"."source_id" = $1 AND "routes"."source_type" = $2 LIMIT $3  [["source_id", 1], ["source_type", "Namespace"], ["LIMIT", 1]]
  ↳ app/models/concerns/routable.rb:77
  ProjectFeature Load (0.4ms)  SELECT  "project_features".* FROM "project_features" WHERE "project_features"."project_id" = $1 LIMIT $2  [["project_id", 20], ["LIMIT", 1]]
  ↳ app/policies/project_policy.rb:516
  ProjectImportState Load (0.5ms)  SELECT  "project_mirror_data".* FROM "project_mirror_data" WHERE "project_mirror_data"."project_id" = $1 LIMIT $2  [["project_id", 20], ["LIMIT", 1]]
  ↳ app/models/project.rb:322
  Rendering projects/show.html.haml within layouts/project
  Rendered projects/_deletion_failed.html.haml (2.7ms)
  Rendered projects/_flash_messages.html.haml (6.2ms)
  Rendered projects/_last_push.html.haml (3.2ms)
  User Load (0.4ms)  SELECT  "users".* FROM "users" WHERE "users"."id" = $1 LIMIT $2  [["id", 1], ["LIMIT", 1]]
  ↳ app/models/project.rb:1285
  Namespace Exists (0.5ms)  SELECT  1 AS one FROM "namespaces" WHERE "namespaces"."id" = $1 AND "namespaces"."emails_disabled" = $2 LIMIT $3  [["id", 1], ["emails_disabled", true], ["LIMIT", 1]]
  ↳ app/models/namespace.rb:185
  ActsAsTaggableOn::Tagging Load (0.4ms)  SELECT "taggings".* FROM "taggings" WHERE "taggings"."taggable_id" = $1 AND "taggings"."taggable_type" = $2  [["taggable_id", 20], ["taggable_type", "Project"]]
  ↳ app/views/projects/_home_panel.html.haml:24
  ActsAsTaggableOn::Tag Load (0.6ms)  SELECT "tags".* FROM "tags" INNER JOIN "taggings" ON "tags"."id" = "taggings"."tag_id" WHERE "taggings"."taggable_id" = $1 AND "taggings"."taggable_type" = $2 AND (taggings.context = 'tags' AND taggings.tagger_id IS NULL) ORDER BY taggings.id  [["taggable_id", 20], ["taggable_type", "Project"]]
  ↳ app/views/projects/_home_panel.html.haml:24
  Rendered projects/buttons/_star.html.haml (6.2ms)
  Rendered projects/buttons/_fork.html.haml (3.8ms)
  Rendered ee/app/views/shared/_mobile_kerberos_clone.html.haml (2.0ms)
  Rendered shared/_mobile_clone_panel.html.haml (9.7ms)
  Rendered ee/app/views/projects/buttons/_kerberos_clone_field.html.haml (3.5ms)
  Rendered projects/buttons/_clone.html.haml (12.7ms)
  ProjectStatistics Load (0.4ms)  SELECT  "project_statistics".* FROM "project_statistics" WHERE "project_statistics"."project_id" = $1 LIMIT $2  [["project_id", 20], ["LIMIT", 1]]
  ↳ app/presenters/project_presenter.rb:159
  Rendered projects/_stat_anchor_list.html.haml (2.3ms)
  ForkNetwork Load (0.5ms)  SELECT  "fork_networks".* FROM "fork_networks" INNER JOIN "fork_network_members" ON "fork_networks"."id" = "fork_network_members"."fork_network_id" WHERE "fork_network_members"."project_id" = $1 LIMIT $2  [["project_id", 20], ["LIMIT", 1]]
  ↳ app/models/project.rb:1369
  Rendered ee/app/views/projects/_home_mirror.html.haml (2.4ms)
  ProjectBadge Load (0.6ms)  SELECT "badges".* FROM "badges" WHERE "badges"."type" IN ('ProjectBadge') AND "badges"."project_id" = $1 ORDER BY "badges"."created_at" ASC  [["project_id", 20]]
  ↳ app/views/projects/_home_panel.html.haml:86
  Rendered projects/_home_panel.html.haml (128.2ms)
  RepositoryLanguage Load (0.5ms)  SELECT "repository_languages".* FROM "repository_languages" WHERE "repository_languages"."project_id" = $1 ORDER BY share DESC  [["project_id", 20]]
  ↳ app/views/projects/show.html.haml:18
  ProgrammingLanguage Load (0.4ms)  SELECT "programming_languages".* FROM "programming_languages" WHERE "programming_languages"."id" IN ($1, $2)  [["id", 2], ["id", 3]]
  ↳ app/views/projects/show.html.haml:18
  License Load (0.3ms)  SELECT  "licenses".* FROM "licenses" ORDER BY "licenses"."id" DESC LIMIT $1  [["LIMIT", 1]]
  ↳ ee/app/models/license.rb:258
  Rendered shared/_ref_switcher.html.haml (6.1ms)
  Rendered ee/app/views/projects/tree/_lock_link.html.haml (2.2ms)
  Rendered projects/_find_file_link.html.haml (2.3ms)
  Ci::Pipeline Load (1.5ms)  SELECT  "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."project_id" = $1 AND ("ci_pipelines"."config_source" IN ($2, $3, $4, $5) OR "ci_pipelines"."config_source" IS NULL) AND "ci_pipelines"."ref" = $6 AND "ci_pipelines"."id" IN (SELECT  "ci_pipelines"."id" FROM "ci_pipelines" WHERE "ci_pipelines"."project_id" = $7 AND ("ci_pipelines"."config_source" IN ($8, $9, $10, $11) OR "ci_pipelines"."config_source" IS NULL) AND "ci_pipelines"."ref" = $12 ORDER BY "ci_pipelines"."id" DESC LIMIT $13) AND ("ci_pipelines"."status" IN ('success')) ORDER BY "ci_pipelines"."id" DESC LIMIT $14  [["project_id", 20], ["config_source", 1], ["config_source", 2], ["config_source", 4], ["config_source", 5], ["ref", "master"], ["project_id", 20], ["config_source", 1], ["config_source", 2], ["config_source", 4], ["config_source", 5], ["ref", "master"], ["LIMIT", 100], ["LIMIT", 1]]
  ↳ app/models/ci/pipeline.rb:254
  Rendered projects/buttons/_download_links.html.haml (5.3ms)
  Rendered projects/buttons/_download.html.haml (18.0ms)
  Rendered projects/tree/_tree_header.html.haml (53.6ms)
  Ci::Pipeline Load (0.8ms)  SELECT "ci_pipelines".* FROM "ci_pipelines" LEFT OUTER JOIN "ci_pipelines" "ci_pipelines_2" ON "ci_pipelines"."sha" = "ci_pipelines_2"."sha" AND "ci_pipelines"."project_id" = "ci_pipelines_2"."project_id" AND "ci_pipelines"."id" < "ci_pipelines_2"."id" AND "ci_pipelines"."ref" = "ci_pipelines_2"."ref" WHERE "ci_pipelines"."project_id" = $1 AND ("ci_pipelines"."config_source" IN ($2, $3, $4, $5) OR "ci_pipelines"."config_source" IS NULL) AND "ci_pipelines"."sha" = $6 AND "ci_pipelines_2"."id" IS NULL AND "ci_pipelines"."ref" = $7  [["project_id", 20], ["config_source", 1], ["config_source", 2], ["config_source", 4], ["config_source", 5], ["sha", "b60227b5816551cfe0f6a0fbe0feb58c584406b5"], ["ref", "master"]]
  ↳ app/models/ci/pipeline.rb:298
  User Load (1.8ms)  SELECT "users".* FROM ((SELECT "users".* FROM "users" WHERE "users"."email" = 'admin@example.com')
UNION
(SELECT "users".* FROM "users" INNER JOIN "emails" ON "emails"."user_id" = "users"."id" WHERE "emails"."email" = 'admin@example.com')) users
  ↳ app/models/commit.rb:252
  Email Load (0.2ms)  SELECT "emails".* FROM "emails" WHERE "emails"."user_id" = $1  [["user_id", 1]]
  ↳ app/models/commit.rb:252
  Rendered ee/app/views/projects/commits/_project_namespace.html.haml (2.1ms)
  Rendered projects/commit/_ajax_signature.html.haml (9.9ms)
  Rendered projects/commits/_commit.html.haml (42.6ms)
  Rendered shared/_commit_well.html.haml (45.4ms)
  ProjectAutoDevops Load (0.3ms)  SELECT  "project_auto_devops".* FROM "project_auto_devops" WHERE "project_auto_devops"."project_id" = $1 LIMIT $2  [["project_id", 20], ["LIMIT", 1]]
  ↳ app/models/project.rb:706
  Rendered projects/_stat_anchor_list.html.haml (0.3ms)
  Rendered collection of projects/tree/_tree_row.html.haml [6 times] (5.6ms)
  Rendered projects/blob/viewers/_loading.html.haml (2.9ms)
  Rendered projects/blob/_viewer.html.haml (7.7ms)
  Rendered projects/tree/_readme.html.haml (14.2ms)
  Rendered projects/tree/_tree_content.html.haml (30.2ms)
  Rendered projects/_files.html.haml (188.4ms)
  Rendered projects/show.html.haml within layouts/project (357.7ms)
  Rendering layouts/application.html.haml
  Appearance Load (0.4ms)  SELECT  "appearances".* FROM "appearances" ORDER BY "appearances"."id" ASC LIMIT $1  [["LIMIT", 1]]
  ↳ app/models/appearance.rb:31
  CACHE Appearance Load (0.0ms)  SELECT  "appearances".* FROM "appearances" ORDER BY "appearances"."id" ASC LIMIT $1  [["LIMIT", 1]]
  ↳ app/models/appearance.rb:31
  CACHE Appearance Load (0.1ms)  SELECT  "appearances".* FROM "appearances" ORDER BY "appearances"."id" ASC LIMIT $1  [["LIMIT", 1]]
  ↳ app/models/appearance.rb:31
  CACHE Appearance Load (0.0ms)  SELECT  "appearances".* FROM "appearances" ORDER BY "appearances"."id" ASC LIMIT $1  [["LIMIT", 1]]
  ↳ app/models/appearance.rb:31
source=rack-timeout id=XnxKDZMxjc1 timeout=60000ms service=1001ms state=active
source=rack-timeout id=XnxKDZMxjc1 timeout=60000ms service=2000ms state=active
source=rack-timeout id=XnxKDZMxjc1 timeout=60000ms service=3094ms state=active
source=rack-timeout id=XnxKDZMxjc1 timeout=60000ms service=4003ms state=active
source=rack-timeout id=XnxKDZMxjc1 timeout=60000ms service=5000ms state=active
source=rack-timeout id=XnxKDZMxjc1 timeout=60000ms service=6003ms state=active
source=rack-timeout id=XnxKDZMxjc1 timeout=60000ms service=7000ms state=active
source=rack-timeout id=XnxKDZMxjc1 timeout=60000ms service=8000ms state=active
source=rack-timeout id=XnxKDZMxjc1 timeout=60000ms service=9004ms state=active
source=rack-timeout id=XnxKDZMxjc1 timeout=60000ms service=10000ms state=active
  Rendered layouts/_snowplow.html.haml (2.9ms)
  Rendered layouts/_head.html.haml (10067.3ms)
  Rendered layouts/_init_client_detection_flags.html.haml (3.0ms)
  Rendered peek/_bar.html.haml (4.4ms)
  Rendered shared/_logo.svg (0.4ms)
  Rendered shared/_logo_type.svg (0.3ms)
  Rendered layouts/nav/_explore.html.haml (3.8ms)
  Rendered layouts/_search.html.haml (10.1ms)
  Rendered ee/app/views/shared/_learn_gitlab_menu_item.html.haml (4.5ms)
  Rendered ee/app/views/shared/_user_dropdown_contributing_link.html.haml (1.7ms)
  Rendered ee/app/views/shared/_user_dropdown_instance_review.html.haml (3.0ms)
  Rendered layouts/header/_help_dropdown.html.haml (18.1ms)
  Rendered layouts/header/_default.html.haml (97.9ms)
  Rendered ee/app/views/layouts/nav/_project_insights_link.html.haml (2.7ms)
  Rendered ee/app/views/projects/sidebar/_repository_locked_files.html.haml (2.8ms)
  Rendered ee/app/views/projects/sidebar/_issues_service_desk.html.haml (2.7ms)
  Rendered ee/app/views/layouts/nav/sidebar/_project_security_link.html.haml (5.7ms)
  Rendered ee/app/views/layouts/nav/sidebar/_project_packages_link.html.haml (5.1ms)
  Rendered shared/_sidebar_toggle_button.html.haml (3.0ms)
  Rendered layouts/nav/sidebar/_project.html.haml (102.6ms)
  Rendered shared/_outdated_browser.html.haml (1.9ms)
  Rendered ee/app/views/layouts/header/_ee_license_banner.html.haml (3.0ms)
  Rendered layouts/_broadcast.html.haml (3.2ms)
  Rendered ee/app/views/layouts/header/_read_only_banner.html.haml (2.2ms)
  Rendered layouts/nav/_classification_level_banner.html.haml (2.0ms)
  Rendered shared/_ping_consent.html.haml (2.5ms)
  Rendered layouts/nav/breadcrumbs/_collapsed_dropdown.html.haml (3.6ms)
  Rendered layouts/nav/_breadcrumbs.html.haml (8.7ms)
  Rendered layouts/_flash.html.haml (2.8ms)
  Rendered layouts/_page.html.haml (140.2ms)
  Rendered ee/app/views/shared/_onboarding_guide.html.haml (2.4ms)
  Rendered layouts/application.html.haml (10357.7ms)
Completed 200 OK in 10791ms (Views: 10682.2ms | ActiveRecord: 43.6ms | Elasticsearch: 0.0ms)

When I experimented removing things in _head.html.haml, I discovered that it was webpack_entrypoint_paths which was the slow call... I was able to speed this up significantly by causing manifest to always be cached, but I'm not sure we want to do this or not 🤔

diff --git a/lib/gitlab/webpack/manifest.rb b/lib/gitlab/webpack/manifest.rb
index 1d2aff5e5b4..0db4b26faf0 100644
--- a/lib/gitlab/webpack/manifest.rb
+++ b/lib/gitlab/webpack/manifest.rb
@@ -9,6 +9,10 @@ module Gitlab
       AssetMissingError = Class.new(StandardError)
 
       class << self
+        def manifest
+          @manifest ||= super
+        end
+
         def entrypoint_paths(source)
           raise ::Webpack::Rails::Manifest::WebpackError, manifest["errors"] unless manifest_bundled?