Skip to content

Remove N+1 queries from users autocomplete

Sean McGivern requested to merge fix-routes-n-plus-one-in-user-autocomplete into master

What does this MR do?

Both of these were related to groups:

  1. We need to preload routes (using the with_route scope) if we're going to get the group's path.
  2. We were counting each group's members separately.

They're in the same commit because the spec for N+1 detection wouldn't pass with only one of these fixes.

Before these changes (URL https://gitlab.com/gitlab-org/gitlab-ce/autocomplete_sources/members?type=Issue&type_id=43065, user is me, overall timings are not the point, SQL timings are):

I, [2019-03-22T15:50:32.431297 #5914]  INFO -- : Completed 200 OK in 9643ms (Views: 65.1ms | ActiveRecord: 578.6ms | Elasticsearch: 0.0ms)
I, [2019-03-22T15:50:32.441035 #5914]  INFO -- : Route total (177): 168.5ms
I, [2019-03-22T15:50:32.441279 #5914]  INFO -- : Group total (3): 66.9ms
I, [2019-03-22T15:50:32.441355 #5914]  INFO -- : User total (5): 20.5ms
I, [2019-03-22T15:50:32.441424 #5914]  INFO -- : Upload total (4): 19.2ms
I, [2019-03-22T15:50:32.441499 #5914]  INFO -- : Project total (4): 6.2ms
I, [2019-03-22T15:50:32.441571 #5914]  INFO -- : AwardEmoji total (2): 2.6ms
I, [2019-03-22T15:50:32.441635 #5914]  INFO -- : Note total (1): 1.3ms
I, [2019-03-22T15:50:32.441723 #5914]  INFO -- : Issue total (1): 1.0ms
I, [2019-03-22T15:50:32.441786 #5914]  INFO -- : ProjectFeature total (1): 1.0ms
I, [2019-03-22T15:50:32.441857 #5914]  INFO -- : License total (1): 0.9ms
I, [2019-03-22T15:50:32.441953 #5914]  INFO -- : UserPreference total (1): 0.7ms

This also included N counts which don't show up because they weren't a 'model load' and our profiling logging isn't smart enough for that.

After:

I, [2019-03-22T15:50:50.819841 #5914]  INFO -- : Completed 200 OK in 3794ms (Views: 60.2ms | ActiveRecord: 151.2ms | Elasticsearch: 0.0ms)
I, [2019-03-22T15:50:50.827521 #5914]  INFO -- : Group total (3): 36.9ms
I, [2019-03-22T15:50:50.827736 #5914]  INFO -- : Upload total (4): 17.4ms
I, [2019-03-22T15:50:50.827811 #5914]  INFO -- : User total (5): 17.0ms
I, [2019-03-22T15:50:50.827877 #5914]  INFO -- : UserPreference total (1): 8.4ms
I, [2019-03-22T15:50:50.827958 #5914]  INFO -- : Route total (2): 6.7ms
I, [2019-03-22T15:50:50.828030 #5914]  INFO -- : Project total (4): 5.5ms
I, [2019-03-22T15:50:50.828095 #5914]  INFO -- : AwardEmoji total (2): 2.4ms
I, [2019-03-22T15:50:50.828162 #5914]  INFO -- : Note total (1): 1.2ms
I, [2019-03-22T15:50:50.828255 #5914]  INFO -- : Issue total (1): 0.9ms
I, [2019-03-22T15:50:50.828329 #5914]  INFO -- : ProjectFeature total (1): 0.9ms
I, [2019-03-22T15:50:50.828414 #5914]  INFO -- : License total (1): 0.7ms

What are the relevant issue numbers?

https://gitlab.com/gitlab-org/gitlab-ce/issues/43065 - doesn't close it because there's one other thing I'd like to look at.

Does this MR meet the acceptance criteria?

Merge request reports