Watch more sec tables in SBOM ingestion health check
What does this MR do and why?
Extends the defer_on_database_health_signal table list on Sbom::IngestReportsWorker from one table to three:
- Before:
sbom_occurrences - After:
sbom_occurrences,sbom_occurrence_refs,sbom_occurrences_vulnerabilities
This comes out of the investigation into write-ahead-log (WAL) saturation alerts on the GitLab.com sec PostgreSQL database (patroni-sec), tracked in &23596. Sbom::IngestReportsWorker is the heaviest writer to the sec database of any worker: on 2026-09-15 its p99 was roughly 15,800 writes to the sec database per job.
How deferral works. Gitlab::SidekiqMiddleware::SkipJobs evaluates Gitlab::Database::HealthStatus before perform and reschedules the job if any indicator returns a Stop signal. There are four indicators, but only AutovacuumActiveOnTable reads the table list passed to defer_on_database_health_signal. The other three (WriteAheadLog, PatroniApdex, WalRate) are cluster-wide checks and ignore the list. AutovacuumActiveOnTable uses OR semantics: it stops the job if autovacuum is running on any one of the listed tables. That is why the list is not every table the worker writes to — listing everything would make the worker defer far more often than the evidence justifies.
How the tables were chosen. Two independent signals:
- Index count per table, counted from
db/structure.sql(CREATE INDEXstatements plus primary-key and unique constraints, since each of those is backed by an index too). Every index on a row has to be maintained whenever that row is written, so index count is a direct proxy for how much WAL a table generates per row written. - Observed dead tuples on
patroni-sec, from a Grafana/Mimir query run on 2026-09-14 (36 series, top series peaking around 4 million dead tuples, with a sawtooth pattern where drops mark autovacuum completions). Dead tuples are what triggers autovacuum, so a table high on that panel is one autovacuum actually visits.
Index counts for the 8 ingestion tasks in ee/app/services/sbom/ingestion/ingest_report_slice_service.rb
| table | indexes | columns | included? |
|---|---|---|---|
sbom_occurrences |
25 | 22 | yes, already declared |
sbom_occurrence_refs |
6 | 6 | yes, added by this MR |
sbom_occurrences_vulnerabilities |
6 | 8 | yes, added by this MR |
sbom_sources |
4 | 6 | no |
sbom_components |
3 | 7 | no |
sbom_component_versions |
3 | 7 | no |
sbom_source_packages |
3 | 6 | no |
The four excluded tables are lookup/dedup tables with only 3-4 indexes each. The ingestion tasks only write rows to them that are genuinely new (deduped against existing rows), so their write churn is low.
A caveat on the two added tables. sbom_occurrence_refs and sbom_occurrences_vulnerabilities do not appear in the dead-tuple panel, so their inclusion rests on index count and row fan-out alone: each holds roughly one row per occurrence, so their row counts scale with the same volume that drives sbom_occurrences. A reviewer who prefers the strictly minimal change can drop these two and keep only sbom_occurrences.
Feature flag. This MR reuses the worker's existing flag defer_sbom_ingest_reports_on_database_health (gitlab_com_derisk, default off), introduced in !254955 (merged) with rollout tracked in #628675. No new flag or rollout issue is needed.
Reviewer note: this takes effect on merge. Both gates are open on gprd today. The ops flag autovacuum_indicator_uses_context_connection was set to true on 2026-09-12 (gitlab-com/gl-infra/feature-flag-log#64250). The worker flag defer_sbom_ingest_reports_on_database_health was set to true on 2026-09-14 (gitlab-com/gl-infra/feature-flag-log#64272). AutovacuumActiveOnTable therefore observes the sec database. The two added tables start to defer this worker when this merge request merges. Watch sidekiq_jobs_skipped_total{action="deferred", reason="database_health_check"} after the merge.
References
- Parent epic: &23596
autovacuum_indicator_uses_context_connectionrollout: #628693defer_sbom_ingest_reports_on_database_healthrollout: #628675- MR that introduced the flag: !254955 (merged)
- Related write-reduction work on this worker: #628686
Screenshots or screen recordings
No UI change. Dead tuples on patroni-sec on 2026-09-14, the panel that frames this work. sbom_occurrences is among the top series; the sawtooth drops are autovacuum completions, which is what this worker defers on:
pg_stat_user_tables_n_dead_tup{type="patroni-sec", env="gprd", relname=~"sbom_graph_paths|vulnerability_occurrences|vulnerability_occurrence_identifiers|vulnerabilities|vulnerability_reads|vulnerability_identifiers|security_finding_enrichments|sbom_occurrences|security_scans"}Source: Grafana explore, dead tuples on patroni-sec
Note the relname filter: sbom_occurrence_refs and sbom_occurrences_vulnerabilities were never in this query, which is why the caveat above rests their case on index count and row fan-out rather than on this panel.
How to set up and validate locally
The worker's health-check table list is exposed via described_class.database_health_check_attrs (or Sbom::IngestReportsWorker.database_health_check_attrs in a Rails console). The updated spec asserts all three tables:
bundle exec rspec ee/spec/workers/sbom/ingest_reports_worker_spec.rbMR acceptance checklist
Evaluate this MR against the MR acceptance checklist.
