Skip to content

Remove duplicate where condition for distinct_on_user_with_max_access_level

Alex Pooley requested to merge group-members-list into master

What does this MR do?

The Member scope distinct_on_user_with_max_access_level replaces the FROM members SQL component with an alternative FROM (select * from ...) members SQL component. For example. instead of:

select members.*
from members
where green = true

We get this:

select members.*
from (select *
      from members
      where green = true) members

Unfortunately prior to this MR we were getting duplicate WHERE conditions like so:

select members.*
from (select *
      from members
      where green = true) members
where green = true

In real life usage such as through Groups::GroupController#index we were generating this query:

SELECT 
  "members".* 
FROM 
  (
    SELECT 
      DISTINCT ON (user_id, invite_email) * 
    FROM 
      "members" 
    WHERE 
      "members"."type" = 'GroupMember' 
      AND "members"."source_type" = 'Namespace' 
      AND "members"."requested_at" IS NULL 
      AND "members"."source_id" IN (
        SELECT 
          "namespaces"."id" 
        FROM 
          "namespaces" 
        WHERE 
          "namespaces"."type" = 'Group' 
          AND "namespaces"."id" = 9970
      ) 
      AND (members.access_level > 5) 
    ORDER BY 
      user_id, 
      invite_email, 
      access_level DESC, 
      expires_at DESC, 
      created_at ASC
  ) members 
WHERE 
  "members"."type" = 'GroupMember' 
  AND "members"."source_type" = 'Namespace' 
  AND "members"."requested_at" IS NULL 
  AND "members"."source_id" IN (
    SELECT 
      "namespaces"."id" 
    FROM 
      "namespaces" 
    WHERE 
      "namespaces"."type" = 'Group' 
      AND "namespaces"."id" = 9970
  ) 
  AND (members.access_level > 5)

https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/4623/commands/16387

Time: 1.077 s
  - planning: 3.645 ms
  - execution: 1.074 s
    - I/O read: 1.058 s
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 4126 (~32.20 MiB) from the buffer pool
  - reads: 1138 (~8.90 MiB) from the OS file cache, including disk I/O
  - dirtied: 1 (~8.00 KiB)
  - writes: 0

Instead of this query:

SELECT 
  "members".* 
FROM 
  (
    SELECT 
      DISTINCT ON (user_id, invite_email) * 
    FROM 
      "members" 
    WHERE 
      "members"."type" = 'GroupMember' 
      AND "members"."source_type" = 'Namespace' 
      AND "members"."requested_at" IS NULL 
      AND "members"."source_id" IN (
        SELECT 
          "namespaces"."id" 
        FROM 
          "namespaces" 
        WHERE 
          "namespaces"."type" = 'Group' 
          AND "namespaces"."id" = 9970
      ) 
      AND (members.access_level > 5) 
    ORDER BY 
      user_id, 
      invite_email, 
      access_level DESC, 
      expires_at DESC, 
      created_at ASC
  ) members 
WHERE 
  "members"."type" = 'GroupMember'

https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/4623/commands/16414

Time: 365.252 ms
  - planning: 2.900 ms
  - execution: 362.352 ms
    - I/O read: 356.354 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 111 (~888.00 KiB) from the buffer pool
  - reads: 1138 (~8.90 MiB) from the OS file cache, including disk I/O
  - dirtied: 1 (~8.00 KiB)
  - writes: 0

There is an existing spec for distinct_on_user_with_max_access_level at https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/models/member_spec.rb#L463

The MR was spurred by the problems found at #328464 (comment 603529095)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team

Related to #328464 (closed)

Edited by Alex Pooley

Merge request reports