Resync security policies with one configuration event
What does this MR do and why?
A forced security policy resync (the resyncSecurityPolicies mutation, a policy project change, or a CSP change) publishes one PolicyResyncEvent per policy. Each event fans out one SyncProjectPolicyWorker per project, and each in-scope job sweeps the project's open merge requests. A resync of K policies over P projects costs P × K jobs and up to 2 × K merge request sweeps per project. This is the problem from #600291. It replaces !265393 (closed) with a smaller scope, following the review there.
Behind the state_based_security_policy_resync flag (gitlab_com_derisk, off by default), a forced resync works like assignment from #600265 (closed):
- One event per request.
PersistSecurityPoliciesWorkerpublishes onePolicyConfigurationResyncEventfor the configuration.EventPublisherstops publishing aPolicyResyncEventper policy, and checks the flag itself.SyncPolicyConfigurationWorkersubscribes to the new event. It runs the compliance framework sync and the pipeline execution policy metadata sync once for each enabled policy, asSyncPolicyWorkerdid for each resync event. Then it enqueues oneSyncProjectPoliciesWorkerper project.- It also enqueues
DeleteSecurityPolicyWorkerfor the configuration's soft-deleted policies, in case an earlier delete job never ran. - Its dedup key now includes the event type, so an assignment and a resync of the same configuration don't deduplicate into each other.
- One job and one sweep per project.
SyncProjectPoliciesWorkerhands a forced resync to the newResyncProjectPoliciesService. The service runs the same per-policy resync as before (SyncPolicyEventServicewithPolicyResyncEvent) for every undeleted policy of the configuration.- Applicable policies are unlinked and relinked with the merge request sweep deferred. The service then schedules one
SyncMergeRequestsForProjectWorkerand one finding enrichment sync for the project. - Disabled and out-of-scope policies unlink with the sweep deferred as well. Unlinking already deletes their merge request rules and enqueues the bot comment refresh, so a sweep after it has nothing left to do for them.
- The project-wide scheduling helpers move from
LinkPolicyConfigurationServiceinto a smallProjectWideSyncsconcern that both services share. SyncProjectPoliciesWorkerswitches fromuntil_executing, including_scheduledtountil_executed, if_deduplicated: :reschedule_once, the same as the per-policySyncProjectPolicyWorker. Two resyncs of one project, such as a configuration resync and an inherited one, no longer run at the same time: a duplicate enqueued while a job runs is rescheduled once after it.including_scheduledis dropped because aperform_induplicate can't be rescheduled and would be lost; scheduled jobs are deduplicated when they become due instead.
A resync now costs P jobs instead of P × K, and one deferred merge request sweep per project instead of one for each policy. Unlike the per-policy path, which skipped disabled policies, the project job also unlinks them, so stale links are cleaned up. For a disabled pipeline execution schedule policy that enqueues one CancelPolicyPipelinesWorker per project, even where it was never linked. The worker is a no-op in that case, and the inherited resync already does the same.
The project-level resync (inherited relationship) already enqueues SyncProjectPoliciesWorker with the force resync reason, so it uses the new service too when the flag is on.
With the flag off, the previous path runs unchanged. Two changes aren't behind the flag. The SyncPolicyConfigurationWorker dedup key now includes the event type, so an assignment job enqueued before the deploy doesn't deduplicate against one enqueued after it. The SyncProjectPoliciesWorker dedup change above also applies to the assignment path: a duplicate assignment job for a project now waits for the running one and runs after it.
Left out on purpose, compared with !265393 (closed). Policy Store / Security Policies v2 (#549779 (closed)) will replace this path, so this stays small:
- No DB-health deferral on the project policy sync workers.
- No new query on the project fan-out. The service resyncs
undeletedpolicies, the same set the per-policy fan-out uses. The only new query is the per-configurationdeletedlookup inSyncPolicyConfigurationWorker(see Database review). - No
force_resyncfield onPolicyConfigurationAssignedEventand noskip_project_syncmarker onPolicyResyncEvent. - No
merge_request_sync_deferred?tracking.
Rollout: enable the flag only after this change is deployed on every node. An older Sidekiq node doesn't know PolicyConfigurationResyncEvent and raises InvalidEvent until a new node picks the job up. Rollout issue: #636320.
Database review
New scope Security::Policy.deleted (policy_index < 0), used once per resync in SyncPolicyConfigurationWorker:
configuration.security_policies.deleted.pluck_primary_keySELECT "security_policies"."id" FROM "security_policies"
WHERE "security_policies"."security_orchestration_policy_configuration_id" = <configuration_id>
AND (policy_index < 0)Plan on production-size data (gitlab-production-main, Database Lab clone, two configurations tried, same shape):
Index Scan using idx_security_policies_config_id_policy_index on security_policies
Index Cond: ((security_orchestration_policy_configuration_id = <configuration_id>) AND (policy_index < 0))
Buffers: shared hit=6
execution: under 0.1 msBoth predicates are index conditions on the existing composite index idx_security_policies_config_id_policy_index (security_orchestration_policy_configuration_id, policy_index), so no new index is needed. The query runs once per configuration resync and returns the soft-deleted rows of that one configuration, which is normally none.
The configuration-level sync also preloads security_pipeline_execution_policy_config_link for the enabled policies, which is one WHERE security_policy_id IN (...) query on the links table per configuration.
How to set up and validate locally
-
Enable the flag:
Feature.enable(:state_based_security_policy_resync). -
Use a group with a security policy project, a few projects, at least two enabled approval policies, and an open merge request in one project.
-
In the Rails console, change
approvals_requiredon one project approval rule that comes from a policy. -
Run the resync for the group:
mutation { resyncSecurityPolicies(input: { fullPath: "my-top-group" }) { errors } } -
Check that:
- Sidekiq logs show one
Security::SyncPolicyConfigurationWorkerforSecurity::PolicyConfigurationResyncEvent, oneSecurity::SyncProjectPoliciesWorkerper project, and oneSecurity::ScanResultPolicies::SyncMergeRequestsForProjectWorkerper project with open merge requests. - There are no
Security::SyncProjectPolicyWorkerjobs and noSecurity::PolicyResyncEventjobs. - The edited approval rule is re-created with the value from the policy.
- Sidekiq logs show one
-
Disable the flag and repeat. The per-policy fan-out runs.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist.