Extend the rebalancer script to allow not waiting for each project to finish migrating
Currently, storage_rebalance.rb waits for each project to finish migrating across shards before moving to the next one. This wouldn't be favorable for a grand-scale migration where we want to move millions of projects en masse. We want to be able to disable such wait, instead, the script should schedule projects continuously and concurrently, respecting API limits while doing so.
Acceptance Criteria
-
storage_rebalance.rb is able to schedule projects for migrations without waiting for each project to finish -
storage_rebalance.rb is doing requests concurrently while respecting API limits
Edited by Ahmad Sherif