Skip to content

Resolve "Merge request dashboard page takes over a minute to load"

Sean McGivern requested to merge 24669-merge-request-dashboard-page-takes-over-a-minute-to-load into master

This does two things:

  1. Don't follow #any? on a relation by loading the relation - just load it once.
  2. When getting the count for the tabs, keep the existing caching. But that only helps on subsequent loads, so on the first load, don't do a query per state, just group and count by state and memoize that hash. This isn't a great test, but on staging it is one slightly slower query compared to four queries:
irb(main):001:0> ActiveRecord::Base.logger = Logger.new(STDOUT)
=> #<Logger:0x007f72883dd748 @progname=nil, @level=0, @default_formatter=#<Logger::Formatter:0x007f72883dd6d0 @datetime_format=nil>, @formatter=nil, @logdev=#<Logger::LogDevice:0x007f72883dd680 @shift_size=nil, @shift_age=nil, @filename=nil, @dev=#<IO:<STDOUT>>, @mon_owner=nil, @mon_count=0, @mon_mutex=#<Thread::Mutex:0x007f72883dd658>>>
irb(main):002:0> MergeRequestsFinder.new(nil, {}).execute.reorder(nil).group(:state).count
D, [2016-11-25T16:27:00.017734 #8150] DEBUG -- :    (1664.7ms)  SELECT COUNT(*) AS count_all, "merge_requests"."state" AS merge_requests_state FROM "merge_requests" INNER JOIN "projects" ON "projects"."id" = "merge_requests"."target_project_id" AND "projects"."pending_delete" = $1 LEFT JOIN project_features ON projects.id = project_features.project_id WHERE "merge_requests"."deleted_at" IS NULL AND "projects"."visibility_level" = $2 AND ("project_features"."merge_requests_access_level" = 20 OR "project_features"."merge_requests_access_level" IS NULL) GROUP BY "merge_requests"."state"  [["pending_delete", "f"], ["visibility_level", 20]]
=> {"merged"=>46277, "closed"=>7640, "opened"=>3431, "locked"=>12, "reopened"=>85}
irb(main):003:0> [:opened, :closed, :merged, :all].map { |state| MergeRequestsFinder.new(nil, state: state).execute.page(1).total_count }
D, [2016-11-25T16:27:08.667762 #8150] DEBUG -- :    (1048.4ms)  SELECT COUNT(*) FROM "merge_requests" INNER JOIN "projects" ON "projects"."id" = "merge_requests"."target_project_id" AND "projects"."pending_delete" = $1 LEFT JOIN project_features ON projects.id = project_features.project_id WHERE "merge_requests"."deleted_at" IS NULL AND ("merge_requests"."state" IN ('opened','reopened')) AND "projects"."visibility_level" = $2 AND ("project_features"."merge_requests_access_level" = 20 OR "project_features"."merge_requests_access_level" IS NULL)  [["pending_delete", "f"], ["visibility_level", 20]]
D, [2016-11-25T16:27:09.709288 #8150] DEBUG -- :    (1036.6ms)  SELECT COUNT(*) FROM "merge_requests" INNER JOIN "projects" ON "projects"."id" = "merge_requests"."target_project_id" AND "projects"."pending_delete" = $1 LEFT JOIN project_features ON projects.id = project_features.project_id WHERE "merge_requests"."deleted_at" IS NULL AND ("merge_requests"."state" IN ('closed')) AND "projects"."visibility_level" = $2 AND ("project_features"."merge_requests_access_level" = 20 OR "project_features"."merge_requests_access_level" IS NULL)  [["pending_delete", "f"], ["visibility_level", 20]]
D, [2016-11-25T16:27:11.291144 #8150] DEBUG -- :    (1576.3ms)  SELECT COUNT(*) FROM "merge_requests" INNER JOIN "projects" ON "projects"."id" = "merge_requests"."target_project_id" AND "projects"."pending_delete" = $1 LEFT JOIN project_features ON projects.id = project_features.project_id WHERE "merge_requests"."deleted_at" IS NULL AND ("merge_requests"."state" IN ('merged')) AND "projects"."visibility_level" = $2 AND ("project_features"."merge_requests_access_level" = 20 OR "project_features"."merge_requests_access_level" IS NULL)  [["pending_delete", "f"], ["visibility_level", 20]]
D, [2016-11-25T16:27:12.870355 #8150] DEBUG -- :    (1574.2ms)  SELECT COUNT(*) FROM "merge_requests" INNER JOIN "projects" ON "projects"."id" = "merge_requests"."target_project_id" AND "projects"."pending_delete" = $1 LEFT JOIN project_features ON projects.id = project_features.project_id WHERE "merge_requests"."deleted_at" IS NULL AND "projects"."visibility_level" = $2 AND ("project_features"."merge_requests_access_level" = 20 OR "project_features"."merge_requests_access_level" IS NULL)  [["pending_delete", "f"], ["visibility_level", 20]]
=> [3516, 7640, 46277, 57445]

This takes out some low-hanging fruit from https://gitlab.com/gitlab-org/gitlab-ce/issues/24669#note_18763469, without necessarily fixing it. My local database isn't in anywhere near a production-like state, so it's hard to check.

Merge request reports