Reindex vulnerabilities when group tree transfers to new organization
What does this MR do and why?
When a top-level group is transferred to a different organization,
Organizations::Transfer::GroupsService updates organization_id on every
namespace and project in the group tree, then publishes a single
Organizations::GroupTransferredEvent for the root group. However, the
organization_id field on Elasticsearch vulnerability documents was never
refreshed, so transferred vulnerabilities kept appearing on the old
organization's Security Dashboard (which is scoped only by that ES field).
This MR adds an event-driven re-index, following the process of
Vulnerabilities::ProcessTransferEventsWorker:
Vulnerabilities::ProcessOrganizationTransferEventWorkersubscribes toOrganizations::GroupTransferredEvent. Since the event fires only for the root group, it traverses the group tree withGitlab::Database::NamespaceProjectIdsEachBatch, keeps only projects withhas_vulnerabilities, and bulk-enqueues per-project jobs in slices of 1,000.Vulnerabilities::ReindexProjectVulnerabilitiesWorkerre-indexes a single project'svulnerability_readsin batches of 100 viaVulnerabilities::BulkEsOperationService, without touching database rows.
Why a re-index is sufficient
Search::Elastic::References::Vulnerability#as_indexed_jsoncomputesorganization_idfromproject.namespace.organization_idat index time. The event is published after commit, so the new organization ID is already persisted when the worker runs.- ES routing (
es_parent = "group_#{root_ancestor.id}") is unchanged: only root groups can be transferred and the hierarchy stays intact, so documents are updated in place and no stale duplicates are left behind. For the same reason, notraversal_idsupdate is needed. - The
organization_idfield is written viaset_field, gated on theadd_organization_id_to_vulnerabilitymigration, so this is safe on instances where that ES migration has not finished. - No denormalized
organization_idcolumns exist onvulnerability_reads,vulnerability_statistics, orvulnerability_namespace_statistics, so Elasticsearch is the only place this value needs resyncing.
Both workers are idempotent and deduplicated (:until_executing).
Follow-up verification of the end-to-end application logic is tracked in
#605426 (closed).
How to set up and validate locally
- Enable Elasticsearch indexing and index the instance
(
Admin > Settings > Search). - Create two organizations, a group tree (group + subgroup) in the first organization, and a project with vulnerabilities in each group.
- Confirm the vulnerabilities appear on the first organization's Security Dashboard.
- Transfer the top-level group to the second organization.
- After Sidekiq processes the jobs and the ES index refreshes, verify the vulnerabilities now appear on the second organization's Security Dashboard and no longer on the first.
Related to #605425 (closed)