[Sorts] Handle groups with a large amount of roots in Sorts
Problem to solve
There are some groups that have a large number of roots, and recently there was one group with so many that it broke Sorts execution job as the amount of data downloaded exceeded the AWS instance capacity. This error was fixed just by handling it and skipping but the need to process these groups to completion still exists.
Proposal
Instead of first downloading all roots and then running Sorts, the job should download only one root and run Sorts on it before deleting it and then proceeding to the next one.
Steps
-
Make sure that the code contributions checklist has been followed.