[Spike] Profile Workload Orchestration
Proposal
We need to establish a clear technical architecture for allowing Profiles to orchestrate scan workloads when a scan trigger is enabled.
One of the most critical conditions we must define is in utilizing a Security Policy Project to enable branch and MR-based pipeline scans, with discussion and validation from groupsecurity policies.
Workload approaches
✅ Git push events
Only Secret Push Protection currently supports git push events. When enabled, we can toggle the corresponding security_project_settings.ssp_enabled project configuration to ensure SPP will run when pushes occur.
🚧 Branch and MR pipeline orchestration
1. Create and maintain per-project Security Policy Project
When a profile has been enabled, we create a SPP and associate to the project via profile. This SPP syncs bidirectionally, allowing any profiles changes through the UI to persist to the SPP and vice-versa (leveraging synchronization through the database read model).
2. Create and maintain per-instance Security Policy Project
Building on the above, instead of maintaining a 1-1 mapping of SPPs per project, we could use an organization-wide SPP.
This should be feasible for default, read-only profiles but lack flexibility once modified. It could prove a good jumping-off point however for beginning with base-enablement projects and "forking" to per-project configurations as needed (i.e. after customization).
3. Running arbitrary workloads without Policies using scheduled jobs
Using a service like RunWorkloadService we can manage scans individually, outside of the Policies framework. This has a number of scaling challenges that would need to be solved from job durability, visibility at scale, failure modes, and thundering heard challenges but is an alternative if we wish to keep the mechanism entirely separate from Category:Security Policy Management
🚧 Scheduled scans
Scheduled scans can rely on either the existing scheduled pipelines feature or scheduled pipeline execution policies.
✅ Event-based triggers
Certain events rely on event-based alerting, such as Continuous Vulnerability Scanning (when advisory sync occurs) or SAST's IDE scanning (on-demand). For these workloads, we simply need to listen to a specific event internally (CVS) or via endpoint (IDE) and ensure the profile is used during scan execution.
