Draft: [skip ci] Mark groups for a prevented-stats refresh

What does this MR do and why?

Part 3 of 7 for https://gitlab.com/gitlab-org/gitlab/-/work_items/631890.

This MR adds the triggers that schedule a refresh of a group's precomputed prevented-package summary.

It also adds AffectedGroups.for_projects(project_ids) and .for_namespace(namespace_id), which return the ids of the groups a project or namespace rolls up to, only for top-level groups that have the dependency_firewall_group_prevented_stats flag on.

The stats table's flush worker, FlushPreventedPackagesWriteBufferCronWorker, now schedules RefreshGroupPreventedStatsWorker for the affected groups of the projects it just flushed, via AffectedGroups, after the write commits. This is best effort: errors are tracked and never affect the flush itself, so a scheduling failure cannot break the underlying write path.

A new EventStore subscriber, MarkGroupsForPreventedStatsRefreshWorker, schedules RefreshGroupPreventedStatsWorker for the affected groups (via AffectedGroups) on project transfers and deletions, and on group transfers and deletions. This keeps summaries eligible for recomputation whenever a project moves between groups, or a group's hierarchy otherwise changes, since those events can change which projects roll up into a group's totals.

Stack

Part MR Targets
1 Summary table and flag !261477 (merged) master
2 Paged group refresh service and refresh job !261478 part 1
3 Affected-group lookup and refresh scheduling on flush and transfers !261479 part 2
4 Daily refresh !261485 part 2
5 Group read path !261481 part 2
6 ClickHouse reads !261486 part 5
7 Group page !261483 part 1

Database

AffectedGroups runs three lookups on existing tables: projects joined to namespaces by project id (at most 1,000 ids per call, the flush batch size), namespaces by id and type, and the root groups by id. The raw SQL and Database Lab plans are still to be added here.

How to set up and validate locally

Run:

bin/rspec ee/spec/workers/security/dependency_firewall/flush_prevented_packages_write_buffer_cron_worker_spec.rb ee/spec/workers/security/dependency_firewall/mark_groups_for_prevented_stats_refresh_worker_spec.rb ee/spec/lib/security/dependency_firewall/prevented_stats/affected_groups_spec.rb

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

References

Edited by Arpit Gogia

Merge request reports

Loading
Loading