Skip to content

Make BulkImport to handle Net::ReadTimeout

Summary

BulkImport pipelines fail to successfully import a group when the source instance is busy and can't handle the request in a certain amount of time.

This issue was originate after this discussion: #363390 (comment 988768783)

Detail

When the source instance is busy and the response takes more than 30 seconds a Net::ReadTimeout is raised and the pipeline is marked as failed. Because the source instance can become available after a period, the BulkImport should retry the pipeline because the source instance can eventually respond to the request.

Doing this will increase the number of imports that are successfully imported.

On production logs - internal only, it's possible to see that this error happened a few times before

Proposed solution

The BulkImport should retry the pipeline a couple of times when a Net::ReadTimeout is raised before marking the pipeline as failed.

Edited by Rodrigo Tomonari