Optimize generate-jsonnet-rules script
Since every rule in rules/ thanos-rules/ and thanos-staging-rules/ are
independent of each other, we can parallelize each rule generation by
forking each of render_multi_jsonnet
call.
From profiling each rule generation in !6593 (comment 1661681851), thanos-rules-jsonnet/service-component-alerts.jsonnet
is the slowest, so we can also parallelize this rule by splitting them by services.
Performance comparison on my machine (M1 Mac):
TLDR ./scripts/generate-jsonnet-rules.sh
took 59s in this MR compared to 150s.
- On master branch (without parallelization):
$ /usr/bin/time ./scripts/generate-jsonnet-rules.sh
151.98 real 230.64 user 4.81 sys
- On this branch with parallelization with 2 processes
$ /usr/bin/time ./scripts/generate-jsonnet-rules.sh
92.17 real 244.56 user 7.26 sys
Below is no longer applicable, keeping as reference:
Optimizing `thanos-rules-jsonnet/service-component-alerts.jsonnet`
``` # running /usr/bin/time ./scripts/generate-jsonnet-rules.sh # 1 process: (without splitting service-component-alerts.jsonnet) 95.09 real 232.87 user 5.38 sys2 processes: (splitting service-component-alerts.jsonnet to 2 files)
59.92 real 234.86 user 5.24 sys
3 processes:
55.53 real 241.67 user 5.87 sys
4 processes:
55.28 real 245.04 user 6.30 sys
The time gain from 2 processes and above is negligible, so this MR splits `service-component-alerts.jsonnet` to `service-component-alerts-1.jsonnet` and `service-component-alerts-2.jsonnet`.~~
</details>
Edited by Gregorius Marco