Track Direct Transfer and Offline Transfer job level events

What does this MR do and why?

Fires start_gitlab_migration, finish_gitlab_migration, fail_gitlab_migration, cancel_gitlab_migration, and timeout_gitlab_migration on BulkImport, covering both Direct Transfer and Offline Transfer.

• start_gitlab_migration carries label (gitlab_migration / offline_transfer) and same_instance.
• finish_gitlab_migration carries label, total_failure_count, duration_seconds, imported_objects_count, and migrated_entities_count, all aggregated across every entity in the request.
• fail_gitlab_migration / cancel_gitlab_migration / timeout_gitlab_migration carry label, total_failure_count, and duration_seconds.

No import_source on these events: a single request can touch several unrelated top-level namespaces on the same source instance, so there's no single path to attach (per the epic's "Re-attempts" note).

duration_seconds is measured from created_at, not from the started transition, so it includes any time the request spent queued before actually starting.

same_instance design decision

BulkImport#source_equals_destination? previously only handled Direct Transfer (comparing configuration.url to the current instance's host). It's extended here to also handle Offline Transfer, by comparing offline_configuration.source_hostname (the hostname the export came from) to Settings.gitlab.host.

As a side effect, this also activates a previously dormant validation: BulkImports::Entity#validate_destination_namespace_ascendency already used source_equals_destination? to block importing a group into a subgroup of itself, but that check was always a no-op for Offline Transfer (the method always returned false). It's now active for Offline Transfer too. Covered by a new spec context in entity_spec.rb.

same_instance is sent as a string ('true'/'false') rather than a boolean, since internal event additional properties only accept String, Integer, or Float.

Database review

This MR adds three read-only aggregate queries on the BulkImport model to build additional properties for job-level internal events. There are no migrations, no schema changes, and no writes to the database.

When these run

All three queries run after the transaction commits (via run_after_commit), once per migration request when it reaches a terminal state, and never in a loop. Finish, fail and timeout transitions happen in Sidekiq workers.

Cancel can also come from POST /bulk_imports/:id/cancel, so on that path the failure count query (Query 1) runs inside the API request.

Query	Events
Failure count	finish_gitlab_migration, fail_gitlab_migration (only when failed from created/started, cancel_gitlab_migration (only when cancelled from created/started), timeout_gitlab_migration
Imported objects sum	finish_gitlab_migration
Finished entities count	finish_gitlab_migration

So at most three queries per terminal transaction run per migration request, and only once in that request's lifetime.

Query 1 — Failure count

Ruby: failures.count (via has_many :failures, through: :entities)

SELECT COUNT(*) FROM "bulk_import_failures" INNER JOIN "bulk_import_entities" ON "bulk_import_failures"."bulk_import_entity_id" = "bulk_import_entities"."id" WHERE "bulk_import_entities"."bulk_import_id" = <bulk_import_id>

This query uses index_bulk_import_entities_on_bulk_import_id_and_status (on bulk_import_id, status) to find the entities for the migration request, and index_bulk_import_failures_on_bulk_import_entity_id (on bulk_import_entity_id) to find their failures.

Query plan: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/58478/commands/163309

Query 2 — Imported objects sum

Ruby: trackers.sum(:imported_objects_count) (via has_many :trackers, through: :entities)

SELECT SUM("bulk_import_trackers"."imported_objects_count") FROM "bulk_import_trackers" INNER JOIN "bulk_import_entities" ON "bulk_import_trackers"."bulk_import_entity_id" = "bulk_import_entities"."id" WHERE "bulk_import_entities"."bulk_import_id" = <bulk_import_id>

This query uses index_bulk_import_entities_on_bulk_import_id_and_status to find the entities, and the unique index bulk_import_trackers_uniq_relation_by_entity (on bulk_import_entity_id, relation) to find their trackers — its leading column is bulk_import_entity_id, so it still supports this lookup.

Query plan: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/58478/commands/163307

Query 3 — Finished entities count

Ruby: entities.with_status(:finished).count

SELECT COUNT(*) FROM "bulk_import_entities" WHERE "bulk_import_entities"."bulk_import_id" = <bulk_import_id> AND "bulk_import_entities"."status" = 2

index_bulk_import_entities_on_bulk_import_id_and_status covers both the bulk_import_id and status conditions in this query directly.

Query plan: https://postgres.ai/console/gitlab/gitlab-production-main/sessions/58478/commands/163308

Notes

All three tables (bulk_import_entities, bulk_import_failures, bulk_import_trackers) live on the main database, so the joins don't cross databases. Rows scanned per migration request are bounded by the number of entities in that request, and the failures/trackers belonging to those entities.

References

• Closes #627871 (closed)
• Part of #617884
• Follows the pattern established in !253895 (merged) and !255888 (merged)

Screenshots or screen recordings

Not applicable — backend-only change (internal event instrumentation), no UI surface.

How to set up and validate locally

1. Enable Snowplow Micro so events are actually collected:

gdk config set snowplow_micro.enabled true
gdk reconfigure
gdk restart

2. Trigger a Direct Transfer migration (New group → Import group → Direct Transfer) and let it run to completion.
3. Optionally trigger an Offline Transfer migration to see the offline_transfer label on the same events.
4. Trigger a cancel (POST /bulk_imports/:id/cancel) and a timeout (via BulkImports::StaleImportWorker) to exercise the remaining two events.
5. Confirm start_gitlab_migration, finish_gitlab_migration, fail_gitlab_migration, cancel_gitlab_migration, and timeout_gitlab_migration reach Snowplow Micro (http://gdk.test:9091/micro/good) with the expected additional_properties.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #627871 (closed)