Skip to content

Avoid join in PipelinesFinder#by_username

Dylan Griffith requested to merge remove-joins-pipelinesfinder-by-username into master

What does this MR do?

The groupsharding is working on moving all of the ci_* tables to a separate database to reduce load on our main Postgres database &6168 (closed). In order to do this there will be many queries that need to change as certain things will no longer be possible. Primarily joining between these 2 databases is not going to be possible. This MR just removes a single use of join for an infrequently used query where the join is not really improving things by much.

When we move ci_* tables to a separate database this join will not be possible. This was extracted from !62092 (closed) .

Queries

Before

SELECT "ci_pipelines".* FROM "ci_pipelines" INNER JOIN "users" ON "users"."id" = "ci_pipelines"."user_id" WHERE "ci_pipelines"."project_id" = 278964 AND "ci_pipelines"."source" != 12 AND "users"."username" = 'DylanGriffith' ORDER BY "ci_pipelines"."id" DESC LIMIT 15 OFFSET 0

https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/5152/commands/18205

Time: 6.382 ms
  - planning: 1.021 ms
  - execution: 5.361 ms
    - I/O read: 0.000 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 1657 (~12.90 MiB) from the buffer pool
  - reads: 0 from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

After

users

SELECT "users"."id" FROM "users" WHERE (LOWER("users"."username") IN (LOWER('DylanGriffith')))

https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/5152/commands/18206

Time: 4.933 ms
  - planning: 0.385 ms
  - execution: 4.548 ms
    - I/O read: 4.407 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 1 (~8.00 KiB) from the buffer pool
  - reads: 3 (~24.00 KiB) from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

ci_pipelines

SELECT "ci_pipelines".* FROM "ci_pipelines" WHERE "ci_pipelines"."project_id" = 278964 AND "ci_pipelines"."source" != 12 AND "ci_pipelines"."user_id" = 120073 ORDER BY "ci_pipelines"."id" DESC LIMIT 15 OFFSET 0

https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/5152/commands/18208

Time: 5.930 ms
  - planning: 0.502 ms
  - execution: 5.428 ms
    - I/O read: 0.000 ms
    - I/O write: 0.000 ms

Shared buffers:
  - hits: 1653 (~12.90 MiB) from the buffer pool
  - reads: 0 from the OS file cache, including disk I/O
  - dirtied: 0
  - writes: 0

Screenshots or Screencasts (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Dylan Griffith

Merge request reports