Prune security_findings partition fanout in descendant-pipeline vulnerability check
What
Follow-up to !252311 (merged) (INC-13566 / corrective action gitlab-com/gl-infra/production-engineering#29649). Adds partition pruning to Ci::Pipeline#has_security_findings_in_self_and_descendants?.
Because this check spans self + descendant pipelines — which can live in different partitions — it can't use the single-partition accessor. Instead it collects the distinct findings_partition_numbers from the pipelines' scans and filters security_findings by that set (partition_number IN (...)), enabling multi-partition pruning. Mirrors the existing pattern in Security::FindingsFinder.
Why pruning by the scans' partition set is correct
Summary: the partition set derived from the scans is guaranteed to cover every partition the findings live in, so the IN (...) filter can never drop a real finding. Details below.
Full correctness reasoning
- Every finding is written with
partition_number = its scan's findings_partition_number(store_findings_service.rb:76), andpartition_numberisattr_readonly— it can never change after insert. - A finding
belongs_to :scanwithoptional: false, so a finding cannot exist without a scan. - Therefore
distinct(findings_partition_number)over the scans is a superset of the partitions the findings occupy: it can only ever over-include an empty partition (harmless), never under-include (which would drop findings). - Empty-set guard: if there are no scans anywhere in the tree, the filter is skipped and the original query runs — safe, and there are no findings to find anyway.
Query plans (postgres.ai, gitlab-production-sec)
Captured against the sec DB (security_findings / security_scans).
Before (no partition_number predicate) — fans out across 66 partitions:
EXPLAIN — Append over 66 partitions, planning 27.55ms / total 31.5ms
Limit (cost=1.14..326.11 rows=1 width=4)
-> Nested Loop (cost=1.14..1626.00 rows=5 width=4)
-> Index Scan using index_security_scans_on_length_of_warnings on security_scans
Index Cond: (pipeline_id = '2796061674'::bigint)
Filter: ((status = 1) AND (project_id = 278964))
-> Append (cost=0.57..1261.09 rows=35472 width=8)
-> Index Only Scan ... security_findings_322 ...
-> Index Only Scan ... security_findings_323 ...
... (66 partitions total, security_findings_322 .. _354+) ...
Settings: work_mem='100MB', random_page_cost='1.5', seq_page_cost='4', effective_cache_size='338688MB', jit='off'
planning: 27.552 ms, execution: 3.983 ms, total: 31.5 msAfter (partition_number IN (350)) — prunes to 1 partition:
EXPLAIN — single-partition scan, planning 4.11ms / total 4.18ms
Limit
-> Nested Loop
-> Index Scan using index_security_scans_on_length_of_warnings on security_scans
Index Cond: (security_scans.pipeline_id = '2796061674'::bigint)
Filter: ((security_scans.status = 1) AND (security_scans.project_id = 278964))
-> Index Scan using security_findings_350_scan_id_deduplicated_idx on security_findings_350
Index Cond: (security_findings.scan_id = security_scans.id)
Filter: (security_findings.partition_number = 350)
planning: 4.110 ms, execution: 0.071 ms, total: 4.18 ms, shared hit=4Net: 66 partitions → 1, planning ~7x lower, total ~31.5ms → ~4.2ms. (A descendant tree spanning multiple partitions prunes to that small set rather than all 66.)
Feature flag
Gated behind its own flag prune_security_findings_partition_descendants (default off), separate from prune_security_findings_partition (!252311 (merged)), so the two are independently mergeable.
Testing
ee/spec/models/ci/pipeline_spec.rb: FF on/off, plus a multi-partition case asserting the emitted SQL carries partition_number IN (...) (on) and not (off). Verified locally green (6 examples, 0 failures).
Closes part of gitlab-com/gl-infra/production-engineering#29649.