Viewing issues is incredibly slow. Merge request !3855 finally moves the remaining use of `Issue#referenced_merge_requests` / `Issue#related_branches` to a separate request meaning these methods are no longer used when `IssuesController#show` itself is called. An early investigation shows quite a bit of time is spent in `Issuable#subscribed_without_subscriptions?` which in turn triggers the Markdown pipeline to gather user references. ## TODO * [x] Find out what parts of the Markdown pipeline exactly are used when viewing issues (when applying the changes from !3855) * [x] Find out how these parts perform on GitLab.com This issue is mostly meant as a "journal" of what I'm working on so people can actually see _that_ I'm working on this (instead of there suddenly being a patch after 5 days of silence). I'll be posting my thoughts/findings as comments as I go along. Grafana dashboard for the controller in question (only available to GitLab employees): http://performance.gitlab.net/dashboard/db/rails-controllers?var-action=Projects%3A%3AIssuesController%23show ## Banzai Filters The following filters appear to run on my test issue: * `Banzai::Filter::AbstractReferenceFilter` * `Banzai::Filter::AutolinkFilter` * `Banzai::Filter::EmojiFilter` * `Banzai::Filter::ExternalLinkFilter` * `Banzai::Filter::ImageLinkFilter` * `Banzai::Filter::RedactorFilter` * `Banzai::Filter::ReferenceFilter` * `Banzai::Filter::ReferenceGathererFilter` * `Banzai::Filter::RelativeLinkFilter` * `Banzai::Filter::SyntaxHighlightFilter` * `Banzai::Filter::TableOfContentsFilter` * `Banzai::Filter::UploadLinkFilter` * `Banzai::Filter::UserReferenceFilter` Since gathering references triggers a full render of every object (if not done so and cached already) this is not entirely surprising. This won't be a huge problem when we cache rendered HTML on database level and build these documents incrementally. ## GitLab.com Timings On GitLab.com the timings (in milliseconds) for the various Banzai classes/methods used in this controller are as following (in the last hour): | Method | Mean | 95th percentile | 99th percentile | |:-----------------------------------------------------------------------|:---------|:--------------------|:--------------------| | `Banzai::Filter::ReferenceFilter#ignore_ancestor_query` | 56.56 | 223.53 | 297.87 | | `Banzai::Filter::CommitRangeReferenceFilter.references_in` | 98.65 | 221.78 | 221.78 | | `Banzai::Querying.css` | 42.68 | 197.27 | 303.27 | | `Banzai::Filter::AbstractReferenceFilter#object_sym` | 151.01 | 151.01 | 151.01 | | `Banzai::Filter::AbstractReferenceFilter.object_sym` | 81.0 | 150.99 | 150.99 | | `Banzai::Filter::AbstractReferenceFilter.object_name` | 150.92 | 150.92 | 150.92 | | `Banzai::Filter::SyntaxHighlightFilter#highlight_node` | 37.66 | 142.73 | 142.73 | | `Banzai::Filter::LabelReferenceFilter.object_class` | 139.35 | 139.35 | 139.35 | | `Banzai::Filter::ReferenceFilter#project` | 53.7 | 132.71 | 132.71 | | `Banzai::Filter::UserReferenceFilter#call` | 32.52 | 126.48 | 218.95 | | `Banzai::Filter::UserReferenceFilter.referenced_by` | 49.55 | 123.49 | 139.32 | | `Banzai::Filter::ExternalLinkFilter#call` | 28.85 | 117.69 | 228.62 | | `Banzai::Filter::ReferenceFilter#replace_link_node_with_text` | 39.66 | 113.89 | 115.82 | | `Banzai::Filter::ReferenceGathererFilter#call` | 40.15 | 112.67 | 254.58 | | `Banzai::Filter::CommitReferenceFilter.references_in` | 46.23 | 105.25 | 273.05 | | `Banzai::Filter::CommitReferenceFilter#find_object` | 44.68 | 103.93 | 272.9 | | `Banzai::Filter::RedactorFilter#call` | 34.11 | 102.02 | 376.81 | | `Banzai::Filter::UploadLinkFilter#call` | 46.74 | 101.68 | 491.8 | | `Banzai::Filter::ReferenceFilter.user_can_see_reference?` | 41.5 | 96.2 | 150.61 | | `Banzai::Renderer.render_result` | 33.01 | 93.21 | 227.11 | | `Banzai::Filter::ReferenceFilter#each_node` | 34.15 | 91.03 | 243.18 | | `Banzai::Filter::CommitReferenceFilter.referenced_by` | 38.68 | 90.87 | 195.13 | | `Banzai::Filter::LabelReferenceFilter#references_in` | 40.63 | 90.22 | 158.05 | | `Banzai::Filter::CommitReferenceFilter.find_object` | 37.93 | 89.12 | 174.42 | | `Banzai::Filter::SyntaxHighlightFilter#call` | 38.7 | 88.5 | 247.74 | | `Banzai::Filter::AbstractReferenceFilter#project_from_ref_cache` | 25.89 | 86.68 | 86.68 | | `Banzai::Filter::ReferenceFilter#validate` | 38.7 | 85.98 | 178.85 | | `Banzai::Filter::AbstractReferenceFilter#call` | 33.04 | 85.08 | 243.24 | | `Banzai::Renderer.pre_process` | 29.22 | 83.95 | 214.85 | | `Banzai::Filter::AbstractReferenceFilter#object_link_filter` | 38.52 | 83.79 | 256.19 | | `Banzai::Filter::ExternalIssueReferenceFilter#default_issues_tracker?` | 33.38 | 83.41 | 83.41 | | `Banzai::Filter::AbstractReferenceFilter#find_object_cached` | 36.45 | 83.38 | 256.06 | | `Banzai::Filter::ReferenceFilter#replace_text_when_pattern_matches` | 36.92 | 82.55 | 240.74 | | `Banzai::Filter::AbstractReferenceFilter#references_in` | 37.89 | 82.39 | 256.17 | | `Banzai::Filter::ExternalIssueReferenceFilter#call` | 29.99 | 80.24 | 83.45 | | `Banzai::Renderer.cacheless_render` | 29.57 | 76.52 | 208.2 | | `Banzai::Filter::IssueReferenceFilter.user_can_see_reference?` | 28.2 | 75.1 | 220.79 | | `Banzai::Filter::UserReferenceFilter#user_link_filter` | 23.4 | 71.26 | 79.87 | | `Banzai::Filter::UserReferenceFilter.references_in` | 23.3 | 71.21 | 79.8 | | `Banzai::Renderer.render` | 26.23 | 71.21 | 179.74 | | `Banzai::Filter::ReferenceFilter#yield_valid_link` | 31.73 | 71.08 | 222.42 | | `Banzai::Renderer.post_process` | 27.98 | 70.2 | 235.22 | | `Banzai::Filter::AutolinkFilter#call` | 31.64 | 66.44 | 347.83 | | `Banzai::Filter::LabelReferenceFilter#find_label` | 33.73 | 66.09 | 142.1 | | `Banzai::Filter::AbstractReferenceFilter.references_in` | 31.76 | 65.07 | 114.62 | | `Banzai::Filter::UserReferenceFilter.user_can_see_reference?` | 28.46 | 65.03 | 65.03 | | `Banzai::Filter::CommitRangeReferenceFilter#find_object` | 43.32 | 60.56 | 60.56 | | `Banzai::Filter::CommitRangeReferenceFilter.find_object` | 43.29 | 60.54 | 60.54 | | `Banzai::Filter::IssueReferenceFilter#find_object` | 21.2 | 60.41 | 118.38 | | `Banzai::Renderer.full_cache_key` | 31.17 | 59.81 | 59.81 | | `Banzai::Filter::SnippetReferenceFilter#find_object` | 22.85 | 59.35 | 112.27 | | `Banzai::Filter::TableOfContentsFilter#call` | 28.1 | 57.04 | 216.2 | | `Banzai::Filter::ReferenceFilter#replace_link_node_with_href` | 33.92 | 55.93 | 222.09 | | `Banzai::Filter::RelativeLinkFilter#call` | 23.83 | 53.52 | 145.46 | | `Banzai::Filter::AbstractReferenceFilter#url_for_object_cached` | 23.26 | 51.54 | 140.2 | | `Banzai::Filter::MergeRequestReferenceFilter#find_object` | 33.47 | 51.25 | 83.22 | | `Banzai::Filter::IssueReferenceFilter#url_for_object` | 24.13 | 51.09 | 140.12 | | `Banzai::Filter::MilestoneReferenceFilter#find_object` | 26.73 | 47.55 | 57.84 | | `Banzai::Filter::UserReferenceFilter.user_can_reference?` | 22.69 | 44.66 | 60.97 | | `Banzai::Filter::EmojiFilter#call` | 20.82 | 43.07 | 110.48 | | `Banzai::Filter::EmojiFilter.emoji_pattern` | 20.17 | 42.46 | 109.52 | | `Banzai::Filter::EmojiFilter#emoji_image_filter` | 18.29 | 37.99 | 77.11 | | `Banzai::Filter::AbstractReferenceFilter#object_link_text` | 24.16 | 32.3 | 105.52 | | `Banzai::Filter::AbstractReferenceFilter.referenced_by` | 17.17 | 29.29 | 29.29 | | `Banzai::Filter::YamlFrontMatterFilter#call` | 18.99 | 28.61 | 28.72 | | `Banzai::Filter::AbstractReferenceFilter#object_class` | 21.03 | 28.51 | 139.46 | | `Banzai::Filter::CommitRangeReferenceFilter.object_class` | 19.86 | 28.43 | 28.43 | | `Banzai::Filter::MergeRequestReferenceFilter.object_class` | 20.39 | 24.76 | 24.76 | | `Banzai::Filter::ReferenceFilter#text_node?` | 17.58 | 23.84 | 52.79 | | `Banzai::Filter::LabelReferenceFilter#url_for_object` | 14.46 | 20.67 | 20.67 | | `Banzai::Filter::LabelReferenceFilter#find_object` | 15.07 | 19.32 | 19.32 | | `Banzai::Filter::SnippetReferenceFilter.object_class` | 16.15 | 19.28 | 19.28 | | `Banzai::Filter::SanitizationFilter#whitelist` | 13.62 | 16.83 | 16.83 | | `Banzai::Filter::ReferenceFilter#escape_once` | 12.06 | 12.06 | 12.06 | | `Banzai::Filter::MarkdownFilter#call` | 11.48 | 11.48 | 11.48 | | `Banzai::Filter::MarkdownFilter.renderer` | 11.38 | 11.38 | 11.38 |