deploy: upload to mirrors/servers in parallel
Right now, fdroid deploy
always syncs updates to the webservers and mirrors sequentially. If there is an error in one of those, then the ones after it will not be updated. This also slows down the update process, especially when using Tor since bandwidth will be limited. fdroid deploy
should upload to all different servers in parallel, or at the very least still try uploading the rest if one fails.
If there are many mirrors, then the parallel uploads will compete for bandwidth. In the interest of getting updates out as fast as possible, there could be a hybrid process:
- first pass sync without index is uploaded in parallel
- second pass index update is first pushed for the main repo
- second pass index update is pushed in parallel for all mirrors
Once fdroidclient can handle 404 Not Found errors when the mirrors are out of date (fdroidclient#2118 (closed)), this would open up other possibilities, like:
- first pass sync without index for the main repo
- second pass index update is first pushed for the main repo
- first pass sync without index is pushed in parallel for all mirrors
- second pass index update is pushed in parallel for all mirrors
Then when uploading over Tor, it should be possible to use one circuit per parallel upload. Tor circuits are usually slower than the actual direct internet connection. So using parallel circuits would speed up uploading over Tor.