Update CVE enrichment only if it has changed

Marcos Rocharequested to merge
mc_rocha-optimize-cve-enrichment into master

What does this MR do and why?

To avoid unnecessary writing and to allow querying recently updated CVE enrichments, this MR updates the ingestion process to only update pm_cve_enrichment records if the epss_score or is_known_exploit changed.

Upsert Query

INSERT INTO pm_cve_enrichment (epss_score, created_at, updated_at, cve, is_known_exploit)
VALUES
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21366', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21370', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21373', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21375', FALSE),
    (0.16, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21378', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-65354', FALSE),
    (0.19, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21388', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21391', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21398', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21401', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21404', FALSE),
    (0.01, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21406', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21408', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21410', TRUE),
    (0.94, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21412', TRUE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21416', FALSE),
    (0.01, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21435', FALSE),
    (0.01, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21440', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21428', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-65855', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-7329', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21481', FALSE),
    (0.46, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21514', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21518', FALSE),
    (0.01, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21575', FALSE),
    (0.92, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21534', FALSE),
    (0.35, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-66039', FALSE),
    (0.87, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21644', FALSE),
    (0.03, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21647', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21661', FALSE),
    (0.94, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21683', FALSE),
    (0.03, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21686', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21697', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21722', FALSE),
    (0.73, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21645', FALSE),
    (0.93, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21650', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21747', FALSE),
    (0.03, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21797', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-66329', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-66331', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-66332', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-66333', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-66334', FALSE),
    (0.94, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21893', TRUE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21898', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21910', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21911', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21978', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21979', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21980', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21981', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21982', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21983', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21984', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21985', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21987', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21988', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21993', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21994', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22002', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22004', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22006', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22007', FALSE),
    (0.94, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21887', TRUE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22008', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22009', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21989', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-21990', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22005', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-66491', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8074', FALSE),
    (0.03, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-66644', TRUE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-68337', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22010', FALSE),
    (0.01, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22011', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22012', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22013', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22014', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22015', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22016', FALSE),
    (0.01, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22017', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2024-22018', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-67461', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-67643', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-67742', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-67873', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-67929', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8763', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8853', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8901', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8902', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8904', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8905', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-68961', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-68962', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-69200', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-69821', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-69822', FALSE),
    (0.03, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-6065', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-6082', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-6168', FALSE),
    (0.01, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-6197', FALSE),
    (0.5, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-6205', TRUE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-6217', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-6748', FALSE),
    (0.19, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-6794', FALSE),
    (0.01, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-6795', FALSE),
    (0.16, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-6798', FALSE),
    (0.16, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-6805', FALSE),
    (0.01, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-6807', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-6942', FALSE),
    (0.03, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-7769', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-7850', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-7952', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8018', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8061', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8102', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8191', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8220', FALSE),
    (0.01, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8242', FALSE),
    (0.02, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8296', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8393', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8433', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8434', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8435', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8436', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8437', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8438', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8439', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8440', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8441', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8442', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8443', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8445', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8446', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8447', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8448', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8449', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8450', FALSE),
    (0.0, '2026-02-05 21:53:18.843955', '2026-02-05 21:53:18.843955', 'CVE-2025-8451', FALSE)
ON CONFLICT (cve)
    DO UPDATE SET
        epss_score = excluded.epss_score,
        updated_at = excluded.updated_at,
        is_known_exploit = excluded.is_known_exploit
    WHERE
        pm_cve_enrichment.epss_score IS DISTINCT FROM excluded.epss_score
        OR pm_cve_enrichment.is_known_exploit IS DISTINCT FROM excluded.is_known_exploit

https://postgres.ai/console/gitlab/gitlab-production-main/sessions/48222/commands/145137

References

Related to: #490985 (closed), !220880 (comment 3060296578)

How to set up and validate locally

To simulate an update we can update some existing records.

  1. Using the rails console select 3 PackageMetadata::CveEnrichment records
cve_epss_score, cve_is_known_exploit, unchanged_cve = PackageMetadata::CveEnrichment.last(3)
  1. Update the epss_score for the cve_epss_score 
cve_epss_score.update!(epss_score: 99.9)
  1. Update the cve_is_known_exploit for the cve_is_known_exploit 
cve_is_known_exploit.update!(is_known_exploit: !cve_is_known_exploit.is_known_exploit)
  1. Run the PackageMetadata::CveEnrichmentSyncWorker 
module PackageMetadata; class NullCheckpoint; def update(*args); end; def blank?; true; end; end; end
ENV['PM_SYNC_IN_DEV'] = 'true'
::PackageMetadata::CveEnrichmentSyncWorker.new.perform
  1. Verify that epss_score and updated_at were updated for cve_epss_score 
cve_epss_score.reload
[29] pry(main)> cve_epss_score.reload
  PackageMetadata::CveEnrichment Load (0.8ms)  SELECT "pm_cve_enrichment".* FROM "pm_cve_enrichment" WHERE "pm_cve_enrichment"."id" = 12920091 LIMIT 1             
=> #<PackageMetadata::CveEnrichment:0x000000012f34e100
 id: 12920091,
 epss_score: 0.0,
 created_at: "2026-02-05 17:00:22.328750000 +0000",
 updated_at: "2026-02-05 21:53:18.514310000 +0000",
 cve: "CVE-2025-57793",
 is_known_exploit: false>
  1. Verify that is_known_exploit and updated_at were updated for cve_is_known_exploit 
cve_is_known_exploit.reload
[30] pry(main)> cve_is_known_exploit.reload
  PackageMetadata::CveEnrichment Load (0.8ms)  SELECT "pm_cve_enrichment".* FROM "pm_cve_enrichment" WHERE "pm_cve_enrichment"."id" = 12920092 LIMIT 1            
=> #<PackageMetadata::CveEnrichment:0x000000012f34e240
 id: 12920092,
 epss_score: 0.0,
 created_at: "2026-02-05 17:00:22.328750000 +0000",
 updated_at: "2026-02-05 21:53:18.514310000 +0000",
 cve: "CVE-2025-57794",
 is_known_exploit: false>
  1. Verify that updated_at was not updated for unchanged_cve 
unchanged_cve.reload
[31] pry(main)> unchanged_cve.reload
  PackageMetadata::CveEnrichment Load (0.8ms)  SELECT "pm_cve_enrichment".* FROM "pm_cve_enrichment" WHERE "pm_cve_enrichment"."id" = 12924241 LIMIT 1         
=> #<PackageMetadata::CveEnrichment:0x000000014d165320
 id: 12924241,
 epss_score: 0.0,
 created_at: "2026-02-05 17:00:22.533785000 +0000",
 updated_at: "2026-02-05 20:00:38.266032000 +0000",
 cve: "CVE-2025-58150",
 is_known_exploit: false>

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Marcos Rocha

Merge request reports