Direct Transfer - Issue while creating group
For some reason, a migration got stuck as `BulkImports::PipelineWorker` didn't create the group as it was supposed to. `bulk_import_entity_id: 1255928` See the logs below for the sequence of events: [Kibana logs - Internal only](https://log.gprd.gitlab.net/app/r/s/U7Glr)(Valid for 7 days) ![Screenshot_2024-10-10_at_13.38.59](/uploads/5846dd24a1b76b7771b73dad0d3367cf/Screenshot_2024-10-10_at_13.38.59.png) It seems that `BulkImports::PipelineWorker` didn't execute the pipeline because an [exclusive lease](https://gitlab.com/gitlab-org/gitlab/-/blob/f1db82fb4db620ce0cff4f3a9e2ff4277caaca48/app/workers/bulk_imports/pipeline_worker.rb#L55-60) wasn't obtained. However, when that happens, we are supposed to see the log `json.message: "Cannot obtain an exclusive lease. There must be another instance already in execution."`, which isn't present in Kibana. See [these logs as an example](https://log.gprd.gitlab.net/app/r/s/iNtTB). Because the group wasn't created and the `BulkImports::Groups::Pipelines::GroupPipeline` wasn't executed, migration got stuck in stage 0 and eventually timed out. ### Proposed solution At the moment, I'm unsure about the solution. Perhaps we could remove the [try_obtain_lease](https://gitlab.com/gitlab-org/gitlab/-/blob/f1db82fb4db620ce0cff4f3a9e2ff4277caaca48/app/workers/bulk_imports/pipeline_worker.rb#L55), but I'm concerned that it might cause the pipeline to be executed twice, as one of the reasons for adding the lock was to prevent that.
issue