Limit number of concurrent batch export workers

Problem

Direct Transfer doesn't limit the number of concurrent workers when exporting batches in the source instance, which can cause the source instance to use lots of resources and fail to export the batches.

Depending on the source instance setup, Direct Transfer may fail with the error ActiveRecord::QueryCanceled PG::QueryCanceled: ERROR: canceling statement due to statement timeout or silence fail and get stuck in case Sidekiq starts to consume a lot of memory and jobs getting interrupted due to Sidekiq Memory Killer restarting multiple times Sidekiq.

Proposed solution

Direct Transfer should limit the number of concurrent export jobs to prevent the source instance from getting overloaded.

Rodrigo proposed the following

The limit should be customizable via Application Setting (see comment)

Currently Direct Transfer can limit the number of import batch jobs via the bulk_import_concurrent_pipeline_batch_limit application setting, which can be updated via API.

We should create another application setting to control the number of export batch jobs and make it configurable via API as well. Then, administrators can tune the limit based on their resources.

Documentation

We should document the circumstances Admins should used this settings.