Skip to content

Improve the performance of project/users API

Adam Hegyi requested to merge optimize-project-users-api into master

What does this MR do?

This change improves the performance of the api/v4/projects/:id/users endpoint by changing the ORDER BY clause to benefit from an index. The API hits the master DB and it's the top 1 query: https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/snippets/2137909#master-1022020102-16

The current query orders by users.id column, however the main filter is present on the project_authorizations JOIN query. Changing the sort to project_authorizations.user_id column will improve the query performance with the project_id, user_id index.

The API will return records in the same order, so no visible change to the end-user.

Note: the change is behind a feature flag (sort_by_project_users_by_project_authorizations_user_id).

Migration

The index creation takes <10 minutes on pg.ai.

up:

ReplaceProjectAuthorizationsProjectIdIndex: migrating =======
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:project_authorizations, [:project_id, :user_id], {:name=>"index_project_authorizations_on_project_id_user_id", :algorithm=>:concurrently})
   -> 0.0017s
-- add_index(:project_authorizations, [:project_id, :user_id], {:name=>"index_project_authorizations_on_project_id_user_id", :algorithm=>:concurrently})
   -> 0.0036s
-- transaction_open?()
   -> 0.0000s
-- indexes(:project_authorizations)
   -> 0.0015s
-- remove_index(:project_authorizations, {:algorithm=>:concurrently, :name=>"index_project_authorizations_on_project_id"})
   -> 0.0029s
== 20210621155328 ReplaceProjectAuthorizationsProjectIdIndex: migrated (0.0120s)

down:

== 20210621155328 ReplaceProjectAuthorizationsProjectIdIndex: reverting =======
-- transaction_open?()
   -> 0.0000s
-- index_exists?(:project_authorizations, :project_id, {:name=>"index_project_authorizations_on_project_id", :algorithm=>:concurrently})
   -> 0.0026s
-- execute("SET statement_timeout TO 0")
   -> 0.0005s
-- add_index(:project_authorizations, :project_id, {:name=>"index_project_authorizations_on_project_id", :algorithm=>:concurrently})
   -> 0.0144s
-- execute("RESET ALL")
   -> 0.0005s
-- transaction_open?()
   -> 0.0000s
-- indexes(:project_authorizations)
   -> 0.0016s
-- remove_index(:project_authorizations, {:algorithm=>:concurrently, :name=>"index_project_authorizations_on_project_id_user_id"})
   -> 0.0020s
== 20210621155328 ReplaceProjectAuthorizationsProjectIdIndex: reverted (0.0231s)

Query change:

see the query tab for the actual query

Screenshots (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Adam Hegyi

Merge request reports