Skip to content

Enumerate columns to address db query cache issues on column change

Doug Stull requested to merge fix-members-cached-column-info into master

What does this MR do?

Resolves column cache issue on certain Member queries as mentioned in gitlab-com/gl-infra/production#5171 (comment 628068466)

Queries

Query before(in general)

Query
SELECT "members".*
FROM ((SELECT "members".*
       FROM "members"
                LEFT OUTER JOIN "users" ON "members"."user_id" = "users"."id"
       WHERE "members"."type" = 'GroupMember'
         AND "members"."source_type" = 'Namespace'
         AND "users"."state" = 'active'
         AND "members"."requested_at" IS NULL
         AND "members"."invite_token" IS NULL
         AND (members.access_level > 5)
         AND (members.access_level > 5)
         AND "members"."source_id" = 1)
      UNION
      (WITH "group_group_links_cte" AS MATERIALIZED (SELECT "group_group_links".*
                                                     FROM "group_group_links"
                                                     WHERE "group_group_links"."shared_group_id" = 1)
       SELECT "members"."id",
              LEAST("group_group_links"."group_access", "members"."access_level") AS access_level,
              "members"."source_id",
              "members"."source_type",
              "members"."user_id",
              "members"."notification_level",
              "members"."type",
              "members"."created_at",
              "members"."updated_at",
              "members"."created_by_id",
              "members"."invite_email",
              "members"."invite_token",
              "members"."invite_accepted_at",
              "members"."requested_at",
              "members"."expires_at",
              "members"."ldap",
              "members"."override",
              "members"."invite_email_success"
       FROM "members",
            "group_group_links_cte" AS "group_group_links"
       WHERE "members"."type" = 'GroupMember'
         AND "members"."source_type" = 'Namespace'
         AND "members"."requested_at" IS NULL
         AND "members"."source_id" = "group_group_links"."shared_with_group_id"
         AND "members"."source_type" = 'Namespace'
         AND (members.access_level > 5))) members
         LEFT OUTER JOIN "users" ON "members"."user_id" = "users"."id"
WHERE "members"."type" = 'GroupMember'
  AND "members"."source_type" = 'Namespace'
  AND (("members"."user_id" IS NULL AND "members"."invite_token" IS NOT NULL) OR "users"."state" = 'active')
  AND "members"."requested_at" IS NULL
  AND (members.access_level > 5)
  AND "members"."access_level" = 50

Query after targeted override of build_select (in general)

Query
SELECT "members"."id",
       "members"."access_level",
       "members"."source_id",
       "members"."source_type",
       "members"."user_id",
       "members"."notification_level",
       "members"."type",
       "members"."created_at",
       "members"."updated_at",
       "members"."created_by_id",
       "members"."invite_email",
       "members"."invite_token",
       "members"."invite_accepted_at",
       "members"."requested_at",
       "members"."expires_at",
       "members"."ldap",
       "members"."override"
FROM ((SELECT "members"."id",
              "members"."access_level",
              "members"."source_id",
              "members"."source_type",
              "members"."user_id",
              "members"."notification_level",
              "members"."type",
              "members"."created_at",
              "members"."updated_at",
              "members"."created_by_id",
              "members"."invite_email",
              "members"."invite_token",
              "members"."invite_accepted_at",
              "members"."requested_at",
              "members"."expires_at",
              "members"."ldap",
              "members"."override"
       FROM "members"
                LEFT OUTER JOIN "users" ON "members"."user_id" = "users"."id"
       WHERE "members"."type" = 'GroupMember'
         AND "members"."source_type" = 'Namespace'
         AND "users"."state" = 'active'
         AND "members"."requested_at" IS NULL
         AND "members"."invite_token" IS NULL
         AND (members.access_level > 5)
         AND (members.access_level > 5)
         AND "members"."source_id" = 1)
      UNION
      (WITH "group_group_links_cte" AS MATERIALIZED (SELECT "group_group_links".*
                                                     FROM "group_group_links"
                                                     WHERE "group_group_links"."shared_group_id" = 1)
       SELECT "members"."id",
              LEAST("group_group_links"."group_access", "members"."access_level") AS access_level,
              "members"."source_id",
              "members"."source_type",
              "members"."user_id",
              "members"."notification_level",
              "members"."type",
              "members"."created_at",
              "members"."updated_at",
              "members"."created_by_id",
              "members"."invite_email",
              "members"."invite_token",
              "members"."invite_accepted_at",
              "members"."requested_at",
              "members"."expires_at",
              "members"."ldap",
              "members"."override"
       FROM "members",
            "group_group_links_cte" AS "group_group_links"
       WHERE "members"."type" = 'GroupMember'
         AND "members"."source_type" = 'Namespace'
         AND "members"."requested_at" IS NULL
         AND "members"."source_id" = "group_group_links"."shared_with_group_id"
         AND "members"."source_type" = 'Namespace'
         AND (members.access_level > 5))) members
         LEFT OUTER JOIN "users" ON "members"."user_id" = "users"."id"
WHERE "members"."type" = 'GroupMember'
  AND "members"."source_type" = 'Namespace'
  AND (("members"."user_id" IS NULL AND "members"."invite_token" IS NOT NULL) OR "users"."state" = 'active')
  AND "members"."requested_at" IS NULL
  AND (members.access_level > 5)
  AND "members"."access_level" = 50

Specific Queries(also in resolved comments)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Doug Stull

Merge request reports