Cache rendered SEP templates
What does this MR do and why?
This MR introduces caching for rendered scan execution policies CI templates, behind a feature flag.
Background
Scan execution policies allow configuring multiple actions per policy, for example sast, secret_detection, etc. We combine all of a policy's actions into a CI configuration in order to enforce security scan jobs. It is also possible to specify whether a latest template version should be used per action.
For every action, we currently look up and render a CI template to a Ruby Hash. This is inefficient, because calling into Gitlab::Ci::Config is expensive. Instead, we can render templates for each (scan_type, template_type) combination only once, and cache the resulting Hash.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
How to set up and validate locally
See bench.rb.
% env ENABLE_CACHE=true bin/rails runner bench.rb -- benchmark
Warming up --------------------------------------
10 actions 2.000 i/100ms
100 actions 1.000 i/100ms
500 actions 1.000 i/100ms
1500 actions 1.000 i/100ms
Calculating -------------------------------------
10 actions 21.448 (± 4.7%) i/s - 108.000 in 5.050964s
100 actions 10.734 (± 9.3%) i/s - 54.000 in 5.067307s
500 actions 4.163 (± 0.0%) i/s - 21.000 in 5.048169s
1500 actions 1.412 (± 0.0%) i/s - 8.000 in 5.738508s
Comparison:
10 actions: 21.4 i/s
100 actions: 10.7 i/s - 2.00x slower
500 actions: 4.2 i/s - 5.15x slower
1500 actions: 1.4 i/s - 15.19x slower
% env bin/rails runner bench.rb -- benchmark
Warming up --------------------------------------
10 actions 1.000 i/100ms
100 actions 1.000 i/100ms
500 actions 1.000 i/100ms
1500 actions 1.000 i/100ms
Calculating -------------------------------------
10 actions 11.867 (± 8.4%) i/s - 60.000 in 5.071618s
100 actions 1.200 (± 0.0%) i/s - 7.000 in 5.835326s
500 actions 0.216 (± 0.0%) i/s - 2.000 in 9.283873s
1500 actions 0.073 (± 0.0%) i/s - 1.000 in 13.629108s
Comparison:
10 actions: 11.9 i/s
100 actions: 1.2 i/s - 9.89x slower
500 actions: 0.2 i/s - 55.06x slower
1500 actions: 0.1 i/s - 161.74x slower% env ENABLE_CACHE=true bin/rails runner bench.rb -- benchmark_mem
Calculating -------------------------------------
10 actions 29.853M memsize ( 2.615M retained)
277.590k objects ( 21.356k retained)
50.000 strings ( 50.000 retained)
100 actions 29.102M memsize ( 319.803k retained)
274.910k objects ( 3.152k retained)
50.000 strings ( 50.000 retained)
500 actions 71.107M memsize ( 754.616k retained)
601.165k objects ( 9.461k retained)
50.000 strings ( 50.000 retained)
1500 actions 266.858M memsize ( 1.006M retained)
1.588M objects ( 11.186k retained)
50.000 strings ( 50.000 retained)
Comparison:
100 actions: 29101854 allocated
10 actions: 29852640 allocated - 1.03x more
500 actions: 71107106 allocated - 2.44x more
1500 actions: 266858016 allocated - 9.17x more
% bin/rails runner bench.rb -- benchmark_mem
Calculating -------------------------------------
10 actions 36.539M memsize ( 2.225M retained)
309.730k objects ( 16.314k retained)
50.000 strings ( 50.000 retained)
100 actions 241.350M memsize ( 254.757k retained)
2.363M objects ( 2.747k retained)
50.000 strings ( 50.000 retained)
500 actions 1.143B memsize ( 682.314k retained)
10.961M objects ( 7.446k retained)
50.000 strings ( 50.000 retained)
1500 actions 3.507B memsize ( 1.283M retained)
32.910M objects ( 9.972k retained)
50.000 strings ( 50.000 retained)
Comparison:
10 actions: 36538927 allocated
100 actions: 241349722 allocated - 6.61x more
500 actions: 1143480510 allocated - 31.29x more
1500 actions: 3506767086 allocated - 95.97x moreRelated to #436545 (closed)