Investigate using runners with more CPUs for the `gitlab:assets:compile *` jobs
In #34319 (comment 311148991), we reduce the average duration of the gitlab:assets:compile * jobs from 27 to 21 minutes but we'd like to reduce this even more, at least under 15 minutes, and ideally under 10 minutes.
N1 standard machine types (General-purpose machine type family) (current GitLab shared runners)
Machine type
Virtual CPUs
Memory
Price (USD)
Preemptible price (USD)
n1-standard-2
2
7.5GB
$48.5500
$14.60
E2 standard machine types (General-purpose machine type family)
Machine type
Virtual CPUs
Memory
Price (USD)
Preemptible price (USD)
e2-standard-2
2
8GB
$48.92
$14.67
e2-standard-4
4
16GB
$97.83
$29.35
e2-standard-8
8
32GB
$195.68
$58.7
e2-standard-16
16
64GB
$391.35
$117.41
C2 machine types (Compute-optimized machine type family)
Machine type
Virtual CPUs
Memory
Price (USD)
Preemptible price (USD)
c2-standard-4
4
16
$121.9392
$36.8650
c2-standard-8
8
32
$243.8784
$73.8030
c2-standard-16
16
64
$487.7568
$147.5330
Analysis: cost * time gain
Job / Runner
gsrm (n1-standard-2)
e2-standard-2
e2-standard-4
e2-standard-8
e2-standard-16
c2-standard-4
c2-standard-8
c2-standard-16
compile-assets pull-cache
48.5500 * 1 = 48.5500
48.92 * 2.11 = 103.22
97.83 * 1.84 = 180.00
195.68 * 1.72 = 336.56
391.35 * 1.74 = 680.949
121.9392 * 1.10 = 134.13
243.8784 * 1.03 = 251.19
487.7568 * 0.98 = 478.00
gitlab:assets:compile pull-cache
48.5500 * 1 = 48.5500
48.92 * 1.05 = 51.36
97.83 * 0.72 = 70.43
195.68 * 0.62 = 121.32
391.35 * 0.56 = 219.15
121.93 * 0.47 = 57.31
243.87 * 0.40 = 97.55
487.75 * 0.36 = 175.59
Conclusion
Based on the cost * time gain analysis, it looks like the c2-standard-4 machines would be have the best cost * time gain ratio (it costs 18% more than the n1-standard-2 for a 53% faster compilation). I suggest we use this type of machine only for the gitlab:assets:compile pull-cache job since there's almost no gain for the compile-assets pull-cache job.
I think we could make the gitlab:assets:compile pull-cache job run under 15 minutes by using c2-standard-4 machines.