Skip to content

Denormalize shared runners to CI::PendingBuild

Ref: #329762 (closed)

What does this MR do?

This MR denormalizes shared runner data in our new ci_pending_builds table.

It includes the following items:

  • Add new column to instance_runners_enabled: if a shared runner can pick up a build
  • Persist this data when creating a new entry to ci_pending_builds behind a feature flag

Why are we doing this?

This is part of our epic to scale up the Ci::Build table.

By avoiding to query the ci_builds we simplify our queuing system to determine which build will get pick up by shared runners.

Database review

1️⃣ AddInstanceRunnersEnabledToCiPendingBuild

DB migration
$ bin/rails db:migrate
== 20210705144657 AddInstanceRunnersEnabledToCiPendingBuild: migrating ========
-- add_column(:ci_pending_builds, :instance_runners_enabled, :boolean, {:null=>false, :default=>false})
   -> 0.0025s
== 20210705144657 AddInstanceRunnersEnabledToCiPendingBuild: migrated (0.0109s)
$ bin/rails db:rollback
== 20210705144657 AddInstanceRunnersEnabledToCiPendingBuild: reverting ========
-- remove_column(:ci_pending_builds, :instance_runners_enabled)
   -> 0.0015s
== 20210705144657 AddInstanceRunnersEnabledToCiPendingBuild: reverted (0.0078s)

2️⃣ New query

Shared runner enabled
SELECT ci_runners.* FROM ci_runners WHERE ci_runners.runner_type = 1
 Index Scan using index_ci_runners_on_runner_type on public.ci_runners  (cost=0.43..22.96 rows=15 width=234) (actual time=31.542..119.209 rows=15 loops=1)
   Index Cond: (ci_runners.runner_type = 1)
   Buffers: shared hit=4 read=22 dirtied=16
   I/O Timings: read=118.530 write=0.000
Time: 124.367 ms  
  - planning: 5.076 ms  
  - execution: 119.291 ms  
    - I/O read: 118.530 ms  
    - I/O write: 0.000 ms  
  
Shared buffers:  
  - hits: 4 (~32.00 KiB) from the buffer pool  
  - reads: 22 (~176.00 KiB) from the OS file cache, including disk I/O  
  - dirtied: 16 (~128.00 KiB)  
  - writes: 0  

Does this MR meet the acceptance criteria?

Conformity

Edited by Max Orefice

Merge request reports