Orgs Feature Parity: Scope stale import queries to organization (Direct Transfer)

Context

Part of &21584 (Organizations Feature Parity: Create – Import). From self-assessment Q1.1.

Problem

Direct Transfer has two distinct classes of unscoped queries that span all organizations:

1. Cron worker queries (cluster-wide)

app/workers/bulk_imports/stale_import_worker.rb is a CronjobQueue worker. Both scopes it iterates are unscoped:

  • BulkImport.stalewhere('updated_at < ?', 24.hours.ago).where(status: [0, 1]) — no org filter
  • BulkImports::Entity.with_trackers.stale — same scope on entities, no org filter

Note

Whether per-org iteration is required for cron workers, or instance-wide is acceptable as cell-local on Cells, is an open question for the Organizations team. Tracked separately in the broader cron-workers scoping discussion. The fix here depends on that decision.

2. Validation/service-path lookups (request-time)

These run at validation/import time and can already be scoped using the entity's or bulk import's organization_id:

  • app/models/bulk_imports/entity.rbvalidate_destination_namespace_ascendency calls Group.find_by_full_path(source_full_path) unscoped. Only runs when source_equals_destination? (same-instance migration). Should use the entity's own organization (already enforced equal to bulk_import.organization via validate_bulk_import_organization_matches).
  • app/services/bulk_imports/create_service.rbGroup.find_by_full_path(namespace)&.organization || fallback_organization is the org resolver itself; globally unscoped lookup can resolve to a group in a different org if the path collides. Should be scoped to Current.organization or fall through to fallback_organization.
  • lib/bulk_imports/groups/transformers/group_attributes_transformer.rbNamespace.find_by_full_path(import_entity.destination_namespace) unscoped. Should scope to import_entity.bulk_import.organization.
  • lib/bulk_imports/projects/transformers/project_attributes_transformer.rb — same pattern.
  • lib/bulk_imports/groups/loaders/group_loader.rb — two unscoped lookups (destination(path) and the present? check) on the destination namespace. Should scope to the import's organization.

Expected

All read/write queries in Direct Transfer scoped to the organization using the sharding key.

For the validation/service-path lookups: use organization.groups.find_by_full_path(...) / organization.namespaces.find_by_full_path(...) (or equivalent scoped relation), deriving the organization from the bulk_import / entity record that already carries it.

For the cron-worker queries: pending the cron-workers scoping decision from the Organizations team, then either keep instance-wide (if cell-local cron is acceptable) or implement per-org iteration.

Open questions

  • Cron-worker scoping pattern: tied to broader question for Organizations team about whether per-org iteration is required for cron cleanup workers, or whether instance-wide (cell-local on Cells) is acceptable. Resolution of that question determines the fix shape for the stale_import_worker.rb portion of this issue.
Edited by 🤖 GitLab Bot 🤖