Add system_id param to GET runners/:id/jobs endpoint (!140667) · Merge requests · GitLab.org / GitLab

Pedro Pombeiro requested to merge pedropombeiro/432586/1-add-rest-endpoint into master Dec 27, 2023

What does this MR do and why?

This MR adds a system_id param to the GET runners/:id/jobs endpoint, so that a user can ask for the jobs executed by a runner in a specific host machine.

Changelog: added

Part of #432586 (closed)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Run any CI job on your local GDK.

For the next step, we'll need the runner ID and the system ID, which you can find in the GDK console:

Ci::RunnerManagerBuild.last(1).to_h { |b| [b.runner_manager.runner.id, b.runner_manager.system_xid] }

Query for that CI job using the values found in the previous step (9620 and s_c2d22f638c25 are example values here):

curl -X GET http://gdk.test:3000/api/v4/runners/9620/managers/s_c2d22f638c25/jobs -H "private-token: $GITLAB_GDK_TOKEN"

The server should return all the builds executed by the specified runner manager.

Database query plans

app/finders/ci/runner_jobs_finder.rb:19

https://console.postgres.ai/gitlab/gitlab-production-ci/sessions/24995/commands/79414

SELECT "ci_runner_machines".*
FROM "ci_runner_machines"
WHERE "ci_runner_machines"."runner_id" = 13522419
  AND "ci_runner_machines"."system_xid" = '<legacy>'
ORDER BY "ci_runner_machines"."id" ASC
LIMIT 1

[(cost=3.45..3.46 rows=1 width=100) (actual time=3.174..3.176 rows=1 loops=1)
   Buffers: shared hit=6 read=4
   I/O Timings: read=3.081 write=0.000
   ->  Sort  (cost=3.45..3.46 rows=1 width=100) (actual time=3.172..3.173 rows=1 loops=1)
         Sort Key: ci_runner_machines.id
         Sort Method: quicksort  Memory: 25kB
         Buffers: shared hit=6 read=4
         I/O Timings: read=3.081 write=0.000
         ->  Index Scan using index_ci_runner_machines_on_runner_id_and_system_xid on public.ci_runner_machines  (cost=0.42..3.44 rows=1 width=100) (actual time=3.128..3.130 rows=1 loops=1)
               Index Cond: ((ci_runner_machines.runner_id = 13522419) AND (ci_runner_machines.system_xid = '<legacy>'::text))
               Buffers: shared hit=3 read=4
               I/O Timings: read=3.081 write=0.000](Limit)

config/initializers/kaminari_active_record_relation_methods_with_limit.rb:31

https://console.postgres.ai/gitlab/gitlab-production-ci/sessions/24995/commands/79423

SELECT COUNT(*)
FROM (
  SELECT 1 AS one
  FROM "p_ci_builds"
    INNER JOIN "p_ci_runner_machine_builds" ON "p_ci_runner_machine_builds"."build_id" = "p_ci_builds"."id"
  WHERE "p_ci_builds"."type" = 'Ci::Build'
    AND "p_ci_runner_machine_builds"."runner_machine_id" = 323948
  LIMIT 10001) subquery_for_count

 Aggregate  (cost=18359.23..18359.24 rows=1 width=8) (actual time=3526.948..3535.027 rows=1 loops=1)
   Buffers: shared hit=4386 read=2234 dirtied=71
   I/O Timings: read=3606.583 write=0.000
   ->  Limit  (cost=1001.27..18312.37 rows=3749 width=4) (actual time=25.330..3534.362 rows=600 loops=1)
         Buffers: shared hit=4386 read=2234 dirtied=71
         I/O Timings: read=3606.583 write=0.000
         ->  Gather  (cost=1001.27..18312.37 rows=3749 width=4) (actual time=25.328..3533.880 rows=600 loops=1)
               Workers Planned: 2
               Workers Launched: 2
               Buffers: shared hit=4386 read=2234 dirtied=71
               I/O Timings: read=3606.583 write=0.000
               ->  Nested Loop  (cost=1.27..16937.47 rows=1562 width=4) (actual time=22.243..1217.597 rows=200 loops=3)
                     Buffers: shared hit=4386 read=2234 dirtied=71
                     I/O Timings: read=3606.583 write=0.000
                     ->  Parallel Append  (cost=0.56..5472.08 rows=1562 width=8) (actual time=6.131..140.436 rows=200 loops=3)
                           Buffers: shared hit=7 read=610 dirtied=59
                           I/O Timings: read=410.321 write=0.000
                           ->  Parallel Index Scan using ci_runner_machine_builds_100_runner_machine_id_idx1 on gitlab_partitions_dynamic.ci_runner_machine_builds_100 p_ci_runner_machine_builds_1  (cost=0.57..4694.11 rows=1900 width=8) (actual time=5.363..191.820 rows=288 loops=2)
                                 Index Cond: (p_ci_runner_machine_builds_1.runner_machine_id = 323948)
                                 Buffers: shared hit=4 read=582 dirtied=50
                                 I/O Timings: read=376.747 write=0.000
                           ->  Parallel Index Scan using ci_runner_machine_builds_101_runner_machine_id_idx1 on gitlab_partitions_dynamic.ci_runner_machine_builds_101 p_ci_runner_machine_builds_2  (cost=0.56..770.16 rows=305 width=8) (actual time=3.831..18.663 rows=12 loops=2)
                                 Index Cond: (p_ci_runner_machine_builds_2.runner_machine_id = 323948)
                                 Buffers: shared hit=3 read=28 dirtied=9
                                 I/O Timings: read=33.574 write=0.000
                     ->  Append  (cost=0.71..7.32 rows=2 width=8) (actual time=5.338..5.383 rows=1 loops=600)
                           Buffers: shared hit=4379 read=1624 dirtied=12
                           I/O Timings: read=3196.263 write=0.000
                           ->  Index Scan using ci_builds_pkey on public.ci_builds p_ci_builds_1  (cost=0.71..3.73 rows=1 width=8) (actual time=5.221..5.223 rows=1 loops=600)
                                 Index Cond: (p_ci_builds_1.id = p_ci_runner_machine_builds.build_id)
                                 Filter: ((p_ci_builds_1.type)::text = 'Ci::Build'::text)
                                 Rows Removed by Filter: 0
                                 Buffers: shared hit=2019 read=1560 dirtied=11
                                 I/O Timings: read=3110.423 write=0.000
                           ->  Index Scan using ci_builds_101_pkey on gitlab_partitions_dynamic.ci_builds_101 p_ci_builds_2  (cost=0.57..3.58 rows=1 width=8) (actual time=0.156..0.156 rows=0 loops=600)
                                 Index Cond: (p_ci_builds_2.id = p_ci_runner_machine_builds.build_id)
                                 Filter: ((p_ci_builds_2.type)::text = 'Ci::Build'::text)
                                 Rows Removed by Filter: 0
                                 Buffers: shared hit=2360 read=64 dirtied=1
                                 I/O Timings: read=85.840 write=0.000

/lib/api/ci/runners.rb:259

https://console.postgres.ai/gitlab/gitlab-production-ci/sessions/24995/commands/79424

SELECT "p_ci_builds"."status", "p_ci_builds"."finished_at", "p_ci_builds"."created_at", "p_ci_builds"."updated_at",
  "p_ci_builds"."started_at", "p_ci_builds"."runner_id", "p_ci_builds"."coverage", "p_ci_builds"."commit_id",
  "p_ci_builds"."name", "p_ci_builds"."options", "p_ci_builds"."allow_failure", "p_ci_builds"."stage",
  "p_ci_builds"."trigger_request_id", "p_ci_builds"."stage_idx", "p_ci_builds"."tag", "p_ci_builds"."ref",
  "p_ci_builds"."user_id", "p_ci_builds"."type", "p_ci_builds"."target_url", "p_ci_builds"."description",
  "p_ci_builds"."project_id", "p_ci_builds"."erased_by_id", "p_ci_builds"."erased_at", "p_ci_builds"."artifacts_expire_at",
  "p_ci_builds"."environment", "p_ci_builds"."when", "p_ci_builds"."yaml_variables", "p_ci_builds"."queued_at",
  "p_ci_builds"."lock_version", "p_ci_builds"."coverage_regex", "p_ci_builds"."auto_canceled_by_id", "p_ci_builds"."retried",
  "p_ci_builds"."protected", "p_ci_builds"."failure_reason", "p_ci_builds"."scheduled_at", "p_ci_builds"."token_encrypted",
  "p_ci_builds"."upstream_pipeline_id", "p_ci_builds"."resource_group_id", "p_ci_builds"."waiting_for_resource_at",
    "p_ci_builds"."processed",
  "p_ci_builds"."scheduling_type", "p_ci_builds"."id", "p_ci_builds"."stage_id", "p_ci_builds"."partition_id",
  "p_ci_builds"."auto_canceled_by_partition_id"
FROM "p_ci_builds"
  INNER JOIN "p_ci_runner_machine_builds" ON "p_ci_runner_machine_builds"."build_id" = "p_ci_builds"."id"
WHERE "p_ci_builds"."type" = 'Ci::Build'
  AND "p_ci_runner_machine_builds"."runner_machine_id" = 323948
ORDER BY "p_ci_builds"."id" ASC
LIMIT 20 OFFSET 0

 Limit  (cost=6555.67..6704.80 rows=20 width=1236) (actual time=8.087..11.062 rows=20 loops=1)
   Buffers: shared hit=827
   I/O Timings: read=0.000 write=0.000
   ->  Nested Loop  (cost=6555.67..34510.00 rows=3749 width=1236) (actual time=8.085..11.055 rows=20 loops=1)
         Buffers: shared hit=827
         I/O Timings: read=0.000 write=0.000
         ->  Gather Merge  (cost=6554.96..6991.60 rows=3749 width=8) (actual time=8.005..10.659 rows=20 loops=1)
               Workers Planned: 2
               Workers Launched: 2
               Buffers: shared hit=631
               I/O Timings: read=0.000 write=0.000
               ->  Sort  (cost=5554.94..5558.84 rows=1562 width=8) (actual time=1.101..1.104 rows=7 loops=3)
                     Sort Key: p_ci_runner_machine_builds.build_id
                     Sort Method: quicksort  Memory: 53kB
                     Buffers: shared hit=631
                     I/O Timings: read=0.000 write=0.000
                     ->  Parallel Append  (cost=0.56..5472.08 rows=1562 width=8) (actual time=0.019..0.986 rows=200 loops=3)
                           Buffers: shared hit=611
                           I/O Timings: read=0.000 write=0.000
                           ->  Parallel Index Scan using ci_runner_machine_builds_100_runner_machine_id_idx1 on gitlab_partitions_dynamic.ci_runner_machine_builds_100 p_ci_runner_machine_builds_1  (cost=0.57..4694.11 rows=1900 width=8) (actual time=0.021..2.694 rows=577 loops=1)
                                 Index Cond: (p_ci_runner_machine_builds_1.runner_machine_id = 323948)
                                 Buffers: shared hit=581
                                 I/O Timings: read=0.000 write=0.000
                           ->  Parallel Index Scan using ci_runner_machine_builds_101_runner_machine_id_idx1 on gitlab_partitions_dynamic.ci_runner_machine_builds_101 p_ci_runner_machine_builds_2  (cost=0.56..770.16 rows=305 width=8) (actual time=0.052..0.175 rows=23 loops=1)
                                 Index Cond: (p_ci_runner_machine_builds_2.runner_machine_id = 323948)
                                 Buffers: shared hit=30
                                 I/O Timings: read=0.000 write=0.000
         ->  Append  (cost=0.71..7.32 rows=2 width=1236) (actual time=0.014..0.018 rows=1 loops=20)
               Buffers: shared hit=196
               I/O Timings: read=0.000 write=0.000
               ->  Index Scan using ci_builds_pkey on public.ci_builds p_ci_builds_1  (cost=0.71..3.73 rows=1 width=1248) (actual time=0.013..0.013 rows=1 loops=20)
                     Index Cond: (p_ci_builds_1.id = p_ci_runner_machine_builds.build_id)
                     Filter: ((p_ci_builds_1.type)::text = 'Ci::Build'::text)
                     Rows Removed by Filter: 0
                     Buffers: shared hit=120
                     I/O Timings: read=0.000 write=0.000
               ->  Index Scan using ci_builds_101_pkey on gitlab_partitions_dynamic.ci_builds_101 p_ci_builds_2  (cost=0.57..3.58 rows=1 width=539) (actual time=0.004..0.004 rows=0 loops=19)
                     Index Cond: (p_ci_builds_2.id = p_ci_runner_machine_builds.build_id)
                     Filter: ((p_ci_builds_2.type)::text = 'Ci::Build'::text)
                     Rows Removed by Filter: 0
                     Buffers: shared hit=76
                     I/O Timings: read=0.000 write=0.000

Edited Jan 05, 2024 by Pedro Pombeiro

Add system_id param to GET runners/:id/jobs endpoint

What does this MR do and why?

MR acceptance checklist

Screenshots or screen recordings

How to set up and validate locally

Database query plans

Merge request reports