Skip to content

Draft: Fix `RunnersFinder#group_runners` cross-joining projects

Adrien Kohlbecker requested to merge ak/belonging-to-group-or-project into master

What does this MR do?

See #336432 (closed)

As part of the initiative to extract the ci_* tables from the main database, we need to rework queries that join between the two. The RunnersFinder#group_runners finder is currently joining ci_runners with projects and groups, through the belonging_to_group_or_project scope on Ci::Runners.

This MR reworks the query to first get the IDs of the runner belonging to a group or project, then returns a relation querying the ci_runner table.

This MR adds 2 new tables namespaces_with_runners/projects_with_runners which will contain entries only when a namespace/runner has any runner associated. These tables are simple helpers to reduce the cardinality of the existing query, since looking at the prod database, the majority of namespaces and projects do not have runners associated and would only inflate the number of IDs plucked from the ci_* tables into the final query.

This MR adds 2 new tables ci_namespace_hierarchies/ci_project_hierarchies which shadow the main database's namespaces/projects tables so that we can quickly answer the question "What is the hierarchy of this namespace/project" in the upcoming ci database without needing to touch the main database. These new tables contain records only for namespaces/projects which have at least one runner registered.

TODO list

  • Migrations
  • Adapt query
  • Sync traversal IDs
    • on ci_project_hierarchies when project runner is created
    • on ci_project_hierarchies when project runner is deleted
    • on ci_namespace_hierarchies when group runner is created
    • on ci_namespace_hierarchies when group runner is deleted
    • on ci_project_hierarchies when project is transferred to another group
    • on ci_project_hierarchies/ci_namespace_hierarchies when group is moved
    • on ci_project_hierarchies when project is deleted
    • on ci_namespace_hierarchies when group is deleted
    • on ci_project_hierarchies when parent group is deleted
  • Finalize migration strategy. Options:
      1. Leave current migration of traversal_ids columns in place, moving it to batches. Only populate table if we see that all traversal_ids fields are populated, which could be seen as a sign that sync_traversal_ids FF is turned on.
      1. Safer: Replace current migration with a background migration using a recursive query (similar to Gitlab::BackgroundMigration::BackfillNamespaceTraversalIdsChildren) to address instances where the sync_traversal_ids FF isn't enabled, and so we can't rely on the traversal IDs field.
  • Address pending MR review comments:
    • Current after_update usage does require 2PC which we will forbid soon, so this needs to be async
    • I wish that we would move traversal_ids into namespaces_hierarchy and projects_hierarchy and store/update this on all databases (treat that as a SharedModel) => so create a general *_hierarchy
    • Update of such table SharedModel.for_each_database {} via sidekiq worker. Likely we need something that would allow to validate the contents of this tables to ensure consistency. The above likely can be defined by SharedModel to perform this periodically.
  • Clean up code/tidy up approach

Tested scenarios on GDK instance

  • Enable/disabled a project runner on a project, ci_project_hierarchies record is added/deleted
  • Delete a group that contains runners, ci_namespace_hierarchies record is deleted
  • Delete parent group of a group that contains runners, ci_namespace_hierarchies record is deleted
  • Delete parent group of a project that contains runners, ci_project_hierarchies record is deleted

Migrations

db:migrate
bin/rails db:migrate RAILS_ENV=development
== 20210920131736 CreateCiNamespaceHierarchies: migrating =====================
-- create_table(:ci_namespace_hierarchies, {:id=>false, :primary_key=>:namespace_xid})
   -> 0.0107s
== 20210920131736 CreateCiNamespaceHierarchies: migrated (0.0108s) ============

== 20210920134647 AddCiNamespaceHierarchies: migrating ========================
-- transaction_open?()
   -> 0.0000s
-- execute("INSERT INTO ci_namespace_hierarchies (namespace_xid, traversal_ids)\nSELECT\n    namespace_id AS namespace_xid,\n    namespaces.traversal_ids AS traversal_ids\nFROM\n    ci_runner_namespaces\n    INNER JOIN namespaces ON ci_runner_namespaces.namespace_id = namespaces.id\nON CONFLICT(namespace_xid) DO NOTHING\n")
   -> 0.0086s
== 20210920134647 AddCiNamespaceHierarchies: migrated (0.0260s) ===============

== 20210921095325 CreateCiProjectHierarchies: migrating =======================
-- create_table(:ci_project_hierarchies, {:id=>false, :primary_key=>:project_xid})
   -> 0.0060s
== 20210921095325 CreateCiProjectHierarchies: migrated (0.0061s) ==============

== 20210921095839 AddCiProjectHierarchies: migrating ==========================
-- transaction_open?()
   -> 0.0000s
-- execute("INSERT INTO ci_project_hierarchies (project_xid, traversal_ids)\nSELECT\n    projects.id AS project_xid,\n    namespaces.traversal_ids AS traversal_ids\nFROM\n    ci_runner_projects\n    INNER JOIN projects ON ci_runner_projects.project_id = projects.id\n    INNER JOIN namespaces ON projects.namespace_id = namespaces.id\nON CONFLICT(project_xid) DO NOTHING\n")
   -> 0.0083s
== 20210921095839 AddCiProjectHierarchies: migrated (0.0121s) =================
db:rollback
bin/rails db:migrate:down VERSION=20210921095839 && bin/rails db:migrate:down VERSION=20210921095325 && bin/rails db:migrate:down VERSION=20210920134647 && bin/rails db:migrate:down VERSION=20210920131736
== 20210921095839 AddCiProjectHierarchies: reverting ==========================
-- transaction_open?()
   -> 0.0000s
-- execute("DELETE FROM ci_project_hierarchies\n")
   -> 0.0009s
== 20210921095839 AddCiProjectHierarchies: reverted (0.0166s) =================

== 20210921095325 CreateCiProjectHierarchies: reverting =======================
-- drop_table(:ci_project_hierarchies, {:id=>false, :primary_key=>:project_xid})
   -> 0.0027s
== 20210921095325 CreateCiProjectHierarchies: reverted (0.0049s) ==============

== 20210920134647 AddCiNamespaceHierarchies: reverting ========================
-- transaction_open?()
   -> 0.0000s
-- execute("DELETE FROM ci_namespace_hierarchies\n")
   -> 0.0010s
== 20210920134647 AddCiNamespaceHierarchies: reverted (0.0165s) ===============

== 20210920131736 CreateCiNamespaceHierarchies: reverting =====================
-- drop_table(:ci_namespace_hierarchies, {:id=>false, :primary_key=>:namespace_xid})
   -> 0.0029s
== 20210920131736 CreateCiNamespaceHierarchies: reverted (0.0045s) ============
New SQL tables after migration
gitlabhq_development=# \d ci_namespace_hierarchies
Did not find any relation named "ci_namespace_hierarchies".
gitlabhq_development=# \d ci_namespace_hierarchies
              Table "public.ci_namespace_hierarchies"
    Column     |   Type    | Collation | Nullable |     Default
---------------+-----------+-----------+----------+-----------------
 namespace_xid | integer   |           | not null |
 traversal_ids | integer[] |           | not null | '{}'::integer[]
Indexes:
    "ci_namespace_hierarchies_pkey" PRIMARY KEY, btree (namespace_xid)
    "index_gin_ci_namespace_hierarchies" gin (traversal_ids)

gitlabhq_development=# \d ci_project_hierarchies
               Table "public.ci_project_hierarchies"
    Column     |   Type    | Collation | Nullable |     Default
---------------+-----------+-----------+----------+-----------------
 project_xid   | integer   |           | not null |
 traversal_ids | integer[] |           | not null | '{}'::integer[]
Indexes:
    "ci_project_hierarchies_pkey" PRIMARY KEY, btree (project_xid)
    "index_gin_ci_project_hierarchies" gin (traversal_ids)

gitlabhq_dblab=# SELECT pg_size_pretty( pg_total_relation_size('ci_namespace_hierarchies') ) AS ci_namespace_hierarchies, pg_size_pretty( pg_total_relation_size('ci_project_hierarchies') ) AS ci_project_hierarchies;
 ci_namespace_hierarchies | ci_project_hierarchies
--------------------------+------------------------
 5808 kB                  | 44 MB
(1 row)

Query plans

NOTE: The queries below were executed against production data using a namespace which is one among the top 5 containing the most sub-groups (~6K), projects (~18K), and runners across groups and projects (~40K).

Existing query
SELECT
    "ci_runners".*
FROM ((
        SELECT
            "ci_runners".*
        FROM
            "ci_runners"
            INNER JOIN "ci_runner_namespaces" ON "ci_runner_namespaces"."runner_id" = "ci_runners"."id"
        WHERE
            "ci_runner_namespaces"."namespace_id" IN (
                SELECT
                    "namespaces"."id"
                FROM
                    "namespaces"
                WHERE
                    "namespaces"."type" = 'Group'
                    AND "namespaces"."id" IN (
                        SELECT
                            "namespaces"."id"
                        FROM
                            "namespaces"
                        WHERE (traversal_ids @> ('{5892345}')))))
        UNION (
            SELECT
                "ci_runners".*
            FROM
                "ci_runners"
                INNER JOIN "ci_runner_projects" ON "ci_runner_projects"."runner_id" = "ci_runners"."id"
            WHERE
                "ci_runner_projects"."project_id" IN (
                    SELECT
                        "projects"."id"
                    FROM
                        "projects"
                    WHERE
                        "projects"."namespace_id" IN (
                            SELECT
                                "namespaces"."id"
                            FROM
                                "namespaces"
                            WHERE (traversal_ids @> ('{5892345}')))))) ci_runners
    ORDER BY
        "ci_runners"."created_at" DESC;

pg.ai link: https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/6512/commands/22434

 Sort  (cost=54988.79..54991.08 rows=915 width=3765) (actual time=18484.706..18491.832 rows=40709 loops=1)
   Sort Key: ci_runners.created_at DESC
   Sort Method: quicksort  Memory: 12671kB
   Buffers: shared hit=310274 read=30540 dirtied=1556
   I/O Timings: read=20477.895 write=0.000
   ->  HashAggregate  (cost=54925.48..54934.63 rows=915 width=3765) (actual time=18412.848..18452.857 rows=40709 loops=1)
         Group Key: ci_runners.id, ci_runners.token, ci_runners.created_at, ci_runners.updated_at, ci_runners.description, ci_runners.contacted_at, ci_runners.active, ci_runners.name, ci_runners.version, ci_runners.revision, ci_runners.platform, ci_runners.architecture, ci_runners.run_untagged, ci_runners.locked, ci_runners.access_level, ci_runners.ip_address, ci_runners.maximum_timeout, ci_runners.runner_type, ci_runners.token_encrypted, ci_runners.public_projects_minutes_cost_factor, ci_runners.private_projects_minutes_cost_factor, ci_runners.config
         Buffers: shared hit=310271 read=30540 dirtied=1556
         I/O Timings: read=20477.895 write=0.000
         ->  Append  (cost=18992.71..54875.16 rows=915 width=3765) (actual time=1062.537..18228.536 rows=40884 loops=1)
               Buffers: shared hit=310271 read=30540 dirtied=1556
               I/O Timings: read=20477.895 write=0.000
               ->  Nested Loop  (cost=18992.71..21233.38 rows=100 width=235) (actual time=1062.535..16636.088 rows=40015 loops=1)
                     Buffers: shared hit=215499 read=25179 dirtied=905
                     I/O Timings: read=17857.586 write=0.000
                     ->  Nested Loop  (cost=18992.28..21165.96 rows=100 width=4) (actual time=1057.373..2535.958 rows=40015 loops=1)
                           Buffers: shared hit=73451 read=7088 dirtied=128
                           I/O Timings: read=4218.004 write=0.000
                           ->  HashAggregate  (cost=18991.86..19006.22 rows=1436 width=8) (actual time=1042.395..1046.932 rows=6202 loops=1)
                                 Group Key: namespaces.id
                                 Buffers: shared hit=18174 read=4764 dirtied=126
                                 I/O Timings: read=2894.024 write=0.000
                                 ->  Gather  (cost=1269.36..18988.27 rows=1436 width=8) (actual time=16.710..1037.017 rows=6202 loops=1)
                                       Workers Planned: 2
                                       Workers Launched: 2
                                       Buffers: shared hit=18174 read=4764 dirtied=126
                                       I/O Timings: read=2894.024 write=0.000
                                       ->  Nested Loop  (cost=269.36..17844.67 rows=598 width=8) (actual time=9.837..1032.057 rows=2067 loops=3)
                                             Buffers: shared hit=18174 read=4764 dirtied=126
                                             I/O Timings: read=2894.024 write=0.000
                                             ->  Parallel Bitmap Heap Scan on public.namespaces namespaces_1  (cost=268.93..12460.42 rows=2765 width=4) (actual time=8.478..589.158 rows=2067 loops=3)
                                                   Buffers: shared hit=349 read=3024 dirtied=126
                                                   I/O Timings: read=1690.962 write=0.000
                                                   ->  Bitmap Index Scan using index_namespaces_on_traversal_ids  (cost=0.00..267.27 rows=6636 width=0) (actual time=12.382..12.384 rows=6206 loops=1)
                                                         Index Cond: (namespaces_1.traversal_ids @> '{5892345}'::integer[])
                                                         Buffers: shared hit=139 read=5
                                                         I/O Timings: read=6.553 write=0.000
                                             ->  Index Only Scan using index_namespaces_on_type_and_id_partial on public.namespaces  (cost=0.43..1.95 rows=1 width=4) (actual time=0.212..0.212 rows=1 loops=6202)
                                                   Index Cond: ((namespaces.type = 'Group'::text) AND (namespaces.id = namespaces_1.id))
                                                   Heap Fetches: 919
                                                   Buffers: shared hit=17825 read=1740
                                                   I/O Timings: read=1203.062 write=0.000
                           ->  Index Scan using index_ci_runner_namespaces_on_namespace_id on public.ci_runner_namespaces  (cost=0.42..1.11 rows=39 width=8) (actual time=0.011..0.237 rows=6 loops=6202)
                                 Index Cond: (ci_runner_namespaces.namespace_id = namespaces.id)
                                 Buffers: shared hit=55277 read=2324 dirtied=2
                                 I/O Timings: read=1323.980 write=0.000
                     ->  Index Scan using ci_runners_pkey on public.ci_runners  (cost=0.43..0.67 rows=1 width=235) (actual time=0.350..0.350 rows=1 loops=40015)
                           Index Cond: (ci_runners.id = ci_runner_namespaces.runner_id)
                           Buffers: shared hit=142048 read=18091 dirtied=777
                           I/O Timings: read=13639.582 write=0.000
               ->  Nested Loop  (cost=23064.06..33628.05 rows=815 width=235) (actual time=676.075..1580.991 rows=869 loops=1)
                     Buffers: shared hit=94772 read=5361 dirtied=651
                     I/O Timings: read=2620.309 write=0.000
                     ->  Nested Loop  (cost=23063.63..33158.93 rows=815 width=4) (actual time=674.599..1220.763 rows=869 loops=1)
                           Buffers: shared hit=92166 read=4490 dirtied=585
                           I/O Timings: read=2272.365 write=0.000
                           ->  HashAggregate  (cost=23063.20..23178.09 rows=11489 width=4) (actual time=672.267..679.288 rows=18536 loops=1)
                                 Group Key: projects.id
                                 Buffers: shared hit=36913 read=3273 dirtied=585
                                 I/O Timings: read=1795.592 write=0.000
                                 ->  Gather  (cost=1269.36..23034.48 rows=11489 width=4) (actual time=9.224..659.783 rows=18536 loops=1)
                                       Workers Planned: 2
                                       Workers Launched: 2
                                       Buffers: shared hit=36913 read=3273 dirtied=585
                                       I/O Timings: read=1795.592 write=0.000
                                       ->  Nested Loop  (cost=269.36..20885.58 rows=4787 width=4) (actual time=4.339..657.509 rows=6179 loops=3)
                                             Buffers: shared hit=36913 read=3273 dirtied=585
                                             I/O Timings: read=1795.592 write=0.000
                                             ->  Parallel Bitmap Heap Scan on public.namespaces namespaces_2  (cost=268.93..12460.42 rows=2765 width=4) (actual time=2.202..9.743 rows=2067 loops=3)
                                                   Buffers: shared hit=3373
                                                   I/O Timings: read=0.000 write=0.000
                                                   ->  Bitmap Index Scan using index_namespaces_on_traversal_ids  (cost=0.00..267.27 rows=6636 width=0) (actual time=5.925..5.925 rows=6206 loops=1)
                                                         Index Cond: (namespaces_2.traversal_ids @> '{5892345}'::integer[])
                                                         Buffers: shared hit=144
                                                         I/O Timings: read=0.000 write=0.000
                                             ->  Index Only Scan using index_projects_on_namespace_id_and_id on public.projects  (cost=0.44..2.85 rows=20 width=8) (actual time=0.228..0.312 rows=3 loops=6202)
                                                   Index Cond: (projects.namespace_id = namespaces_2.id)
                                                   Heap Fetches: 2204
                                                   Buffers: shared hit=33540 read=3273 dirtied=585
                                                   I/O Timings: read=1795.592 write=0.000
                           ->  Index Scan using index_ci_runner_projects_on_project_id on public.ci_runner_projects  (cost=0.43..0.71 rows=16 width=8) (actual time=0.015..0.029 rows=0 loops=18536)
                                 Index Cond: (ci_runner_projects.project_id = projects.id)
                                 Buffers: shared hit=55253 read=1217
                                 I/O Timings: read=476.773 write=0.000
                     ->  Index Scan using ci_runners_pkey on public.ci_runners ci_runners_1  (cost=0.43..0.58 rows=1 width=235) (actual time=0.412..0.412 rows=1 loops=869)
                           Index Cond: (ci_runners_1.id = ci_runner_projects.runner_id)
                           Buffers: shared hit=2606 read=871 dirtied=66
                           I/O Timings: read=347.944 write=0.000
New queries
SELECT
    "namespaces"."id"
FROM
    "namespaces"
WHERE
    "namespaces"."type" = 'Group'
    AND "namespaces"."id" = 5892345
SELECT
    "ci_runners".*
FROM ((
        SELECT
            "ci_runners".*
        FROM
            "ci_runners"
            INNER JOIN "ci_runner_namespaces" ON "ci_runner_namespaces"."runner_id" = "ci_runners"."id"
        WHERE
            "ci_runner_namespaces"."namespace_id" IN (
                SELECT
                    traversal_ids[array_length(traversal_ids, 1)]
                FROM
                    "ci_namespace_hierarchies"
                WHERE (traversal_ids && ARRAY[5892345]::int[])))
    UNION (
        SELECT
            "ci_runners".*
        FROM
            "ci_runners"
            INNER JOIN "ci_runner_projects" ON "ci_runner_projects"."runner_id" = "ci_runners"."id"
        WHERE
            "ci_runner_projects"."project_id" IN (
                SELECT
                    "ci_project_hierarchies"."project_id"
                FROM
                    "ci_project_hierarchies"
                WHERE (traversal_ids && ARRAY[5892345]::int[])))) ci_runners
ORDER BY
    "ci_runners"."created_at" DESC
LIMIT 100;

pg.ai links: https://postgres.ai/console/gitlab/gitlab-production-tunnel-pg12/sessions/6575/commands/23099, https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/6575/commands/23098

 Index Only Scan using index_namespaces_on_type_and_id_partial on public.namespaces  (cost=0.43..3.45 rows=1 width=4) (actual time=0.092..0.092 rows=1 loops=1)
   Index Cond: ((namespaces.type = 'Group'::text) AND (namespaces.id = 5892345))
   Heap Fetches: 0
   Buffers: shared hit=4
   I/O Timings: read=0.000 write=0.000
 Limit  (cost=17548.76..17549.01 rows=100 width=3765) (actual time=407.707..407.747 rows=100 loops=1)
   Buffers: shared hit=207025
   I/O Timings: read=0.000 write=0.000
   ->  Sort  (cost=17548.76..17568.10 rows=7736 width=3765) (actual time=407.704..407.729 rows=100 loops=1)
         Sort Key: ci_runners.created_at DESC
         Sort Method: top-N heapsort  Memory: 87kB
         Buffers: shared hit=207025
         I/O Timings: read=0.000 write=0.000
         ->  HashAggregate  (cost=17098.38..17175.74 rows=7736 width=3765) (actual time=358.817..388.665 rows=41058 loops=1)
               Group Key: ci_runners.id, ci_runners.token, ci_runners.created_at, ci_runners.updated_at, ci_runners.description, ci_runners.contacted_at, ci_runners.active, ci_runners.name, ci_runners.version, ci_runners.revision, ci_runners.platform, ci_runners.architecture, ci_runners.run_untagged, ci_runners.locked, ci_runners.access_level, ci_runners.ip_address, ci_runners.maximum_timeout, ci_runners.runner_type, ci_runners.token_encrypted, ci_runners.public_projects_minutes_cost_factor, ci_runners.private_projects_minutes_cost_factor, ci_runners.config
               Buffers: shared hit=207022
               I/O Timings: read=0.000 write=0.000
               ->  Append  (cost=282.47..16672.90 rows=7736 width=3765) (actual time=0.682..278.809 rows=41235 loops=1)
                     Buffers: shared hit=207022
                     I/O Timings: read=0.000 write=0.000
                     ->  Nested Loop  (cost=282.47..9208.88 rows=5811 width=234) (actual time=0.680..254.562 rows=40237 loops=1)
                           Buffers: shared hit=200676
                           I/O Timings: read=0.000 write=0.000
                           ->  Nested Loop  (cost=282.04..5496.19 rows=5811 width=4) (actual time=0.661..61.402 rows=40237 loops=1)
                                 Buffers: shared hit=39649
                                 I/O Timings: read=0.000 write=0.000
                                 ->  HashAggregate  (cost=281.61..282.73 rows=112 width=26) (actual time=0.615..0.672 rows=112 loops=1)
                                       Group Key: ci_namespace_hierarchies.traversal_ids[array_length(ci_namespace_hierarchies.traversal_ids, 1)]
                                       Buffers: shared hit=95
                                       I/O Timings: read=0.000 write=0.000
                                       ->  Bitmap Heap Scan on public.ci_namespace_hierarchies  (cost=5.37..281.33 rows=112 width=26) (actual time=0.135..0.560 rows=112 loops=1)
                                             Buffers: shared hit=95
                                             I/O Timings: read=0.000 write=0.000
                                             ->  Bitmap Index Scan using index_gin_ci_namespace_hierarchies  (cost=0.00..5.34 rows=112 width=0) (actual time=0.114..0.115 rows=112 loops=1)
                                                   Index Cond: (ci_namespace_hierarchies.traversal_ids && '{5892345}'::integer[])
                                                   Buffers: shared hit=3
                                                   I/O Timings: read=0.000 write=0.000
                                 ->  Index Scan using index_ci_runner_namespaces_on_namespace_id on public.ci_runner_namespaces  (cost=0.43..46.03 rows=52 width=8) (actual time=0.010..0.465 rows=359 loops=112)
                                       Index Cond: (ci_runner_namespaces.namespace_id = (ci_namespace_hierarchies.traversal_ids)[array_length(ci_namespace_hierarchies.traversal_ids, 1)])
                                       Buffers: shared hit=39554
                                       I/O Timings: read=0.000 write=0.000
                           ->  Index Scan using ci_runners_pkey on public.ci_runners  (cost=0.43..0.64 rows=1 width=234) (actual time=0.004..0.004 rows=1 loops=40237)
                                 Index Cond: (ci_runners.id = ci_runner_namespaces.runner_id)
                                 Buffers: shared hit=161027
                                 I/O Timings: read=0.000 write=0.000
                     ->  Nested Loop  (cost=9.62..7347.98 rows=1925 width=234) (actual time=0.279..17.242 rows=998 loops=1)
                           Buffers: shared hit=6346
                           I/O Timings: read=0.000 write=0.000
                           ->  Nested Loop  (cost=9.19..6252.93 rows=1925 width=4) (actual time=0.257..10.777 rows=998 loops=1)
                                 Buffers: shared hit=2353
                                 I/O Timings: read=0.000 write=0.000
                                 ->  Bitmap Heap Scan on public.ci_project_hierarchies  (cost=8.76..827.15 rows=356 width=4) (actual time=0.209..3.554 rows=356 loops=1)
                                       Buffers: shared hit=297
                                       I/O Timings: read=0.000 write=0.000
                                       ->  Bitmap Index Scan using index_gin_ci_project_hierarchies  (cost=0.00..8.67 rows=356 width=0) (actual time=0.124..0.125 rows=356 loops=1)
                                             Index Cond: (ci_project_hierarchies.traversal_ids && '{5892345}'::integer[])
                                             Buffers: shared hit=4
                                             I/O Timings: read=0.000 write=0.000
                                 ->  Index Scan using index_ci_runner_projects_on_project_id on public.ci_runner_projects  (cost=0.43..15.06 rows=18 width=8) (actual time=0.013..0.019 rows=3 loops=356)
                                       Index Cond: (ci_runner_projects.project_id = ci_project_hierarchies.project_id)
                                       Buffers: shared hit=2056
                                       I/O Timings: read=0.000 write=0.000
                           ->  Index Scan using ci_runners_pkey on public.ci_runners ci_runners_1  (cost=0.43..0.57 rows=1 width=234) (actual time=0.006..0.006 rows=1 loops=998)
                                 Index Cond: (ci_runners_1.id = ci_runner_projects.runner_id)
                                 Buffers: shared hit=3993
                                 I/O Timings: read=0.000 write=0.000

Screenshots or Screencasts (strongly suggested)

How to setup and validate locally (strongly suggested)

  1. Enable the traversal IDs FFs (currently enabled in prod):

    Feature.enable(:use_traversal_ids); Feature.enable(:use_traversal_ids_for_ancestors); Feature.enable(:use_traversal_ids_for_root_ancestor)
  2. Run the migrations:

    $ bin/rails db:migrate
  3. Run the finder that is currently leveraging the query:

    group = Group.find(22)
    admin = User.find(1)
    Ci::RunnersFinder.new(current_user: admin, params: { group: group, membership: :descendants }).execute

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Pedro Pombeiro

Merge request reports