Skip to content

generate binary diffs of APKs for smaller downloads on update

Following up on this topic: https://f-droid.org/forums/topic/why-f-droid-repo-update-is-so-slow/

I have an idea to drastically speed up the download portion of the fdroid update, taking a cue from how Google Chromium push updates. On the server, it uses bsdiff to generate a binary patch for the new update, then Google Chrome will try to download just that binary patch, which is usually a lot smaller than the whole thing. So this can be done by making the server tools generate the binary diffs on each update, which would be named based on the timestamp of the old index.xml.

So for example, the current index.xml is timestamp="1403063785" then there are some changes so there is a new index.xml with timestamp="14030987654". fdroid update will then generate the binary diff by doing: bsdiff index-1403063785.xml index.xml index-1403063785.xml.patch. It can then keep a backlog of these, so like 5 or 10 old versions.

The client will then look at the current timestamp of the index.xml that it has, and generate the file name of the update that it needs. If that update does not exist, it'll just request the full index.xml. And of course, all of these files would be bundled into a signed jar file. The thing is that it'll make the patch files a lot smaller if the diff is run on the XML file rather than the JAR file, since the JAR is compressed. But starting with the JAR might makes sense just because it'll be a lot simpler to implement.

Down the line, this could also be applied to APKs to make download updated APKs a lot faster too. A quick check of using bsdiff on an Orbot update shows a dramatically smaller update file:

$ bsdiff Orbot-v14.0.3.1-release.apk Orbot-v14.0.4-release.apk Orbot-v14.0.4-update.bsdiff
$ ls -l Orbot-v14.0.*
-rw-r--r-- 1 hans hans 5443365 Jun 11 22:08 Orbot-v14.0.3.1-release.apk
-rw-r--r-- 1 hans hans 5443000 Jun 14 06:00 Orbot-v14.0.4-release.apk
-rw-r--r-- 1 hans hans  206089 Jun 18 12:23 Orbot-v14.0.4-update.bsdiff
Edited by Hans-Christoph Steiner
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information