Deliver f-droid.org and f-droid.org/repo via a CDN service
To cure the slowness/frequent timeout issue of F-Droid, we should rather deliver f-droid.org and f-droid.org/repo via a CDN service.
Why
- Slowness is the most common complaint of F-Droid, including long waiting time for a page load, timeouts, and low download speed, which is a result of the small capacity of our caching server regarding the huge traffic we are serving
- About June 6th-7th, 2021, f-droid.org and f-droid.org/repo could hardly be accessed from anywhere.
- Our users are located from all over the world. 2 servers in centralized locations can’t meet their needs.
The present content delivery architecture
An origin server + 2 caching servers.
Daily traffic statistics
bytes/files served on 2021-06-06 per mirror (https://ftp.fau.de/cgi-bin/show-ftp-stats.cgi)
| mirrorname | Bytes total | Bytes IPv4 | Bytes IPv6 | Files total | Files IPv4 | Files IPv6 | ATR IPv4 | ATR IPv6 |
|---|---|---|---|---|---|---|---|---|
| fdroid | 917.20 GB | 673.79 GB | 243.41 GB | 93734 | 68107 | 25627 | 586 KB/s | 876 KB/s |
Proposed solution
Use a CDN service to deliver our static contents.
Requirements for the CDN service
- Respect privacy
- Minimum visitor logs
- Don’t retain the visitor logs for more than 1 month
- Global coverage
CDN comparison
| Name | CloudFlare | Fastly | KeyCDN |
|---|---|---|---|
| Visitor log retain period | Cloudflare may receive certain information regarding use of customers' websites protected by Cloudflare through requests received at our Anycast Network. We receive limited personal data in these requests, such as a website visitor's IP address. We process this type of information on behalf of our customers in our main data center in the United States for a limited period of time. We currently store detailed end user request traffic logs of the data we process on behalf of our Customers (“End User Log Data”) for up to 7 days and may store such data longer if requested by the Customer and where technically possible. We store a small sample -- approximately 1% -- of all End User traffic logs for up to 12 months. We may store firewall event logs for up to 12 months. When a Customer terminates their account, End User Log Data is deleted in accordance with these retention periods unless other treatment is required to comply with our legal obligations. (Responded from the Cloudflare Privacy Team) | Not retained | |
| Global coverage | Yes | Yes | Yes |
| GDPR compliance | Yes | Yes | Yes |
| Free for open source projects | Yes (Business Plan) | Yes | Yes |
| Test | https://fdroidgoes.live (updated but will not always hit) https://pages.fdroidgoes.live/ (always hit but not updated) | http://fastly.fdroidgoes.live/ | https://fdroid-1a5d1.kxcdn.com/ |
What are affected
- f-droid.org
- The main repository mirror at f-droid.org/repo
- fdroidclient which relies on the database and apks from f-droid.org/repo
What are not affected
- Mirrors
- Third-party repos
Important websites use Cloudflare by default
- https://nodejs.org/
- https://git-scm.com/
- https://www.kali.org/
- https://yarnpkg.com/en/
- https://app.element.io/
- https://gitlab.com
f-droid.org and CDNs censorship in China
Progress
Open source grants application submitted
-
CloudFlare: https://www.cloudflare.com/galileo/ -
Fastly: https://www.fastly.com/open-source
Open source grants application responded
-
CloudFlare: https://www.cloudflare.com/galileo/ -
Fastly: https://www.fastly.com/open-source
Platform decision
-
CloudFlare: https://www.cloudflare.com/galileo/ -
Fastly: https://www.fastly.com/open-source