Add optional owned_only param to namespaces endpoint (!66166) · Merge requests · GitLab.org / GitLab

Corinna Gogolok requested to merge 2955_add_optional_owned_only_param_to_namespaces_endpoint into master Jul 14, 2021

What does this MR do?

Part of https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/2955

The current namespaces endpoint to get all namespaces for the current user, now has a new optional parameter called owned_only to only return owned namespaces. This information is needed in the admin area in CustomersDot.

Query plans

Note: Used user has 61 groups. The previous used user for the old version has 57 groups but the result is worse (cold: 990.271 ms; warm: 3.083 ms).

Formatted Query:

EXPLAIN SELECT "namespaces".*
  FROM (
    (
      SELECT "namespaces".*
        FROM "namespaces"
        INNER JOIN "members" ON "namespaces"."id" = "members"."source_id"
        WHERE "members"."type" = 'GroupMember'
          AND "members"."source_type" = 'Namespace'
          AND "namespaces"."type" = 'Group'
          AND "members"."user_id" = 3927967
          AND "members"."requested_at" IS NULL
          AND (access_level >= 10)
    )
    UNION (
      SELECT "namespaces".* FROM "namespaces" WHERE "namespaces"."id" = 5124429
    )
  ) namespaces

Explain Output (https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/5445/commands/18894):

Unique  (cost=14.23..14.48 rows=2 width=2946) (actual time=379.142..379.267 rows=62 loops=1)
   Buffers: shared hit=212 read=129 dirtied=10
   I/O Timings: read=364.609 write=0.000
   ->  Sort  (cost=14.23..14.23 rows=2 width=2946) (actual time=379.140..379.150 rows=62 loops=1)
         Sort Key: namespaces.id, namespaces.name, namespaces.path, namespaces.owner_id, namespaces.created_at, namespaces.updated_at, namespaces.type, namespaces.description, namespaces.avatar, namespaces.membership_lock, namespaces.share_with_group_lock, namespaces.visibility_level, namespaces.request_access_enabled, namespaces.ldap_sync_status, namespaces.ldap_sync_error, namespaces.ldap_sync_last_update_at, namespaces.ldap_sync_last_successful_update_at, namespaces.ldap_sync_last_sync_at, namespaces.lfs_enabled, namespaces.description_html, namespaces.parent_id, namespaces.shared_runners_minutes_limit, namespaces.repository_size_limit, namespaces.require_two_factor_authentication, namespaces.two_factor_grace_period, namespaces.cached_markdown_version, namespaces.project_creation_level, namespaces.runners_token, namespaces.file_template_project_id, namespaces.saml_discovery_token, namespaces.runners_token_encrypted, namespaces.custom_project_templates_group_id, namespaces.auto_devops_enabled, namespaces.extra_shared_runners_minutes_limit, namespaces.last_ci_minutes_notification_at, namespaces.last_ci_minutes_usage_notification_level, namespaces.subgroup_creation_level, namespaces.emails_disabled, namespaces.max_pages_size, namespaces.max_artifacts_size, namespaces.mentions_disabled, namespaces.default_branch_protection, namespaces.unlock_membership_to_ldap, namespaces.max_personal_access_token_lifetime, namespaces.push_rule_id, namespaces.shared_runners_enabled, namespaces.allow_descendants_override_disabled_shared_runners, namespaces.traversal_ids, namespaces.delayed_project_removal
         Sort Method: quicksort  Memory: 42kB
         Buffers: shared hit=212 read=129 dirtied=10
         I/O Timings: read=364.609 write=0.000
         ->  Append  (cost=0.99..14.22 rows=2 width=2946) (actual time=19.858..378.665 rows=62 loops=1)
               Buffers: shared hit=184 read=129 dirtied=10
               I/O Timings: read=364.609 write=0.000
               ->  Nested Loop  (cost=0.99..10.73 rows=1 width=356) (actual time=19.857..361.272 rows=61 loops=1)
                     Buffers: shared hit=183 read=126 dirtied=10
                     I/O Timings: read=347.325 write=0.000
                     ->  Index Scan using index_members_on_user_id_source_id_source_type on public.members  (cost=0.56..7.28 rows=1 width=4) (actual time=19.546..201.379 rows=61 loops=1)
                           Index Cond: ((members.user_id = 3927967) AND ((members.source_type)::text = 'Namespace'::text))
                           Filter: ((members.requested_at IS NULL) AND (members.access_level >= 10) AND ((members.type)::text = 'GroupMember'::text))
                           Rows Removed by Filter: 0
                           Buffers: shared hit=4 read=60 dirtied=9
                           I/O Timings: read=197.608 write=0.000
                     ->  Index Scan using index_namespaces_on_type_and_id_partial on public.namespaces  (cost=0.43..3.45 rows=1 width=356) (actual time=2.615..2.615 rows=1 loops=61)
                           Index Cond: (((namespaces.type)::text = 'Group'::text) AND (namespaces.id = members.source_id))
                           Buffers: shared hit=179 read=66 dirtied=1
                           I/O Timings: read=149.717 write=0.000
               ->  Index Scan using namespaces_pkey on public.namespaces namespaces_1  (cost=0.43..3.45 rows=1 width=356) (actual time=17.350..17.354 rows=1 loops=1)
                     Index Cond: (namespaces_1.id = 5124429)
                     Buffers: shared hit=1 read=3
                     I/O Timings: read=17.284 write=0.000

Summary:

Cold cache:

Time: 384.871 ms
  - planning: 5.293 ms
  - execution: 379.578 ms
    - I/O read: 364.609 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 212 (~1.70 MiB) from the buffer pool
  - reads: 129 (~1.00 MiB) from the OS file cache, including disk I/O
  - dirtied: 10 (~80.00 KiB)
  - writes: 0

Warm cache:

Time: 2.533 ms
  - planning: 1.487 ms
  - execution: 1.046 ms
    - I/O read: 0.000 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 341 (~2.70 MiB) from the buffer pool
  - reads: 0 from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

How to test

Given the GitLab instance you want to send the request to runs on localhost:3000:

curl --request GET --header "PRIVATE-TOKEN: INSERT_PRIVATE_TOKEN" "http://localhost:3000/api/v4/namespaces?owned_only=true"

Does this MR meet the acceptance criteria?

Conformity

I have included changelog trailers, or none are needed. (Does this MR need a changelog?)
I have added/updated documentation, or it's not needed. (Is documentation required?)
I have properly separated EE content from FOSS, or this MR is FOSS only. (Where should EE code go?)
I have added information for database reviewers in the MR description, or it's not needed. (Does this MR have database related changes?)
I have self-reviewed this MR per code review guidelines.
This MR does not harm performance, or I have asked a reviewer to help assess the performance impact. (Merge request performance guidelines)
I have followed the style guides.
This change is backwards compatible across updates, or this does not apply.

Availability and Testing

I have added/updated tests following the Testing Guide, or it's not needed. (Consider all test levels. See the Test Planning Process.)
I have tested this MR in all supported browsers, or it's not needed.
I have informed the Infrastructure department of a default or new setting change per definition of done, or it's not needed.

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

Label as security and @ mention @gitlab-com/gl-security/appsec
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
Security reports checked/validated by a reviewer from the AppSec team

Edited Jul 28, 2021 by Corinna Gogolok

Add optional owned_only param to namespaces endpoint