Skip to content

Expand compound index on members to include member_role

What does this MR do and why?

Query run by Preloaders::UserMemberRolesInProjectsPreloader is slow. One of the contributing factors is that the database does not pick up the right index. See the investigation for further details: https://gitlab.com/gitlab-org/gitlab/-/issues/386039#note_1234189802.

This MR changes the compound index on members from (user_id, source_id, source_type) to (user_id, source_id, source_type, member_role_id), forcing the database to use the correct index.

Migration output

up

$ rake db:migrate:up:main VERSION=20230111125148                                                                               ─╯
main: == 20230111125148 AddUserIdAndSourceIdAndSourceTypeAndMemberRoleIdIndexToMembers: migrating 
main: -- transaction_open?()
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.0792s
main: -- index_exists?(:members, [:user_id, :source_id, :source_type, :member_role_id], {:name=>"idx_members_on_user_and_source_and_source_type_and_member_role", :algorithm=>:concurrently})
main:    -> 0.0068s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0003s
main: -- add_index(:members, [:user_id, :source_id, :source_type, :member_role_id], {:name=>"idx_members_on_user_and_source_and_source_type_and_member_role", :algorithm=>:concurrently})
main:    -> 0.0024s
main: -- execute("RESET statement_timeout")
main:    -> 0.0002s
main: == 20230111125148 AddUserIdAndSourceIdAndSourceTypeAndMemberRoleIdIndexToMembers: migrated (0.0984s) 

$ rake db:migrate:up:main VERSION=20230111125150                                                                               ─╯
main: == 20230111125150 DropUserIdAndSourceIdAndSourceTypeIndexFromMembers: migrating 
main: -- transaction_open?()
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.0708s
main: -- indexes(:members)
main:    -> 0.0058s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0002s
main: -- remove_index(:members, {:algorithm=>:concurrently, :name=>"index_members_on_user_id_source_id_source_type"})
main:    -> 0.0018s
main: -- execute("RESET statement_timeout")
main:    -> 0.0003s
main: == 20230111125150 DropUserIdAndSourceIdAndSourceTypeIndexFromMembers: migrated (0.0896s) 

down

$ rake db:migrate:down:main VERSION=20230111125150                                                                             ─╯
main: == 20230111125150 DropUserIdAndSourceIdAndSourceTypeIndexFromMembers: reverting 
main: -- transaction_open?()
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.0848s
main: -- index_exists?(:members, [:user_id, :source_id, :source_type], {:name=>"index_members_on_user_id_source_id_source_type", :algorithm=>:concurrently})
main:    -> 0.0069s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0002s
main: -- add_index(:members, [:user_id, :source_id, :source_type], {:name=>"index_members_on_user_id_source_id_source_type", :algorithm=>:concurrently})
main:    -> 0.0021s
main: -- execute("RESET statement_timeout")
main:    -> 0.0003s
main: == 20230111125150 DropUserIdAndSourceIdAndSourceTypeIndexFromMembers: reverted (0.1029s) 

$ rake db:migrate:down:main VERSION=20230111125148                                                                             ─╯
main: == 20230111125148 AddUserIdAndSourceIdAndSourceTypeAndMemberRoleIdIndexToMembers: reverting 
main: -- transaction_open?()
main:    -> 0.0000s
main: -- view_exists?(:postgres_partitions)
main:    -> 0.0742s
main: -- indexes(:members)
main:    -> 0.0074s
main: -- execute("SET statement_timeout TO 0")
main:    -> 0.0003s
main: -- remove_index(:members, {:algorithm=>:concurrently, :name=>"idx_members_on_user_and_source_and_source_type_and_member_role"})
main:    -> 0.0034s
main: -- execute("RESET statement_timeout")
main:    -> 0.0003s
main: == 20230111125148 AddUserIdAndSourceIdAndSourceTypeAndMemberRoleIdIndexToMembers: reverted (0.0956s) 

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Imre Farkas

Merge request reports