Skip to content

Add SettingsBasedUpdateService

Gal Katzrequested to merge
gkatz_setting_based_update_service into master

What does this MR do and why?

This MR introduces setting-based analyzer status tracking alongside the existing pipeline-based statuses.
It uses the aggregated analyzer types (secret_detection and container_scanning) that combine statuses from both pipeline executions and project security settings using existing priority-based OR logic.

The implementation includes a new SettingsBasedUpdateService that can process multiple projects simultaneously to support bulk setting updates like we have in the codebase already. It also extracts shared logic into an AggregatedTypesHandler module to avoid duplication between pipeline-based and setting-based status calculations. This shared logic compares existing statuses and the new status to determine the wanted aggregated status.

Note that the diff service is not yet in use to propagate these new setting-based status changes to ancestor namespaces in the SettingsBasedUpdateService as additional changes are required for that.

This MR is the first in a series of 3 MRs. See the next one and the last one.

Related issue

Namespace level statuses for DB based settings (#548281 - closed) • rossfuhrman, Gal Katz • 18.2

Query plans

upsert_analyzer_statuses

Raw SQL 
INSERT INTO "analyzer_project_statuses" ("project_id", "traversal_ids", "analyzer_type", "status", "last_call", "created_at", "updated_at")
    VALUES (66101996, '{9970,96981785,97058478,98867842,100569068}', 0, 1, '2025-04-06 08:51:29', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP),
    (66101996, '{9970,96981785,97058478,98867842,100569068}', 4, 2, '2025-04-06 08:51:29', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP),
    (66101996, '{9970,96981785,97058478,98867842,100569068}', 5, 2, '2025-04-06 08:51:29', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP),
    (66101996, '{9970,96981785,97058478,98867842,100569068}', 6, 1, '2025-04-06 08:51:29', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP),
    (66101996, '{9970,96981785,97058478,98867842,100569068}', 2, 1, '2025-04-06 08:51:29', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP),
    (66101996, '{9970,96981785,97058478,98867842,100569068}', 1, 1, '2025-04-06 08:51:29', CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)
ON CONFLICT ("project_id", "analyzer_type")
    DO UPDATE SET
        updated_at = (
            CASE WHEN ("analyzer_project_statuses"."traversal_ids" IS NOT DISTINCT FROM excluded."traversal_ids"
                AND "analyzer_project_statuses"."status" IS NOT DISTINCT FROM excluded."status"
                AND "analyzer_project_statuses"."last_call" IS NOT DISTINCT FROM excluded."last_call") THEN
                "analyzer_project_statuses".updated_at
            ELSE
                CURRENT_TIMESTAMP
            END),
        "traversal_ids" = excluded."traversal_ids",
        "status" = excluded."status",
        "last_call" = excluded."last_call"
    RETURNING
        "id"
Query plan

Full details here

 ModifyTable on public.analyzer_project_statuses  (cost=0.00..0.09 rows=6 width=85) (actual time=15.228..19.402 rows=6 loops=1)
   Buffers: shared hit=163 read=38 dirtied=17
   WAL: records=41 fpi=15 bytes=103962
   I/O Timings: read=18.663 write=0.000
   ->  Values Scan on "*VALUES*"  (cost=0.00..0.09 rows=6 width=85) (actual time=0.778..0.807 rows=6 loops=1)
         Buffers: shared hit=20 read=5 dirtied=2
         WAL: records=3 fpi=1 bytes=8363
         I/O Timings: read=0.658 write=0.000
Settings: seq_page_cost = '4', work_mem = '100MB', random_page_cost = '1.5', effective_cache_size = '338688MB', jit = 'off'
Edited by Gal Katz

Merge request reports