Implementation proposal for background downloading

Overview

General Process

• User clicks "Install" from AppDetails (note: also can install from "Swap" UI)
• Background download is asked to start
• When complete, user needs some way to initiate an install action for the downloaded app.

What is displayed when the download is queued but not yet begun? How does the user initiate an install of the app ones it is downloaded:

• If the download completes while they are still using F-Droid?
• If the download completes after they close F-Droid?

Scope

Things to run in background service:

• Downloading of Apks in preparation for installing

Things not to run in background service:

• Downloading icons for list of apps

Also, this spec assumes that there is not a privileged installer available. This will come later.

Related Issues

• #592 (closed) is the proposal for the UI portion of this work.
• #103 (closed) is the original issue for background downloading.

Implementation Details

In #103 (closed), @eighthave suggested using an IntentService to form a queue that will process downloads one at a time. This sounds like the right tool for the job, but it will require some minor additions from the API exposed by the SDK.

Notification

As discussed in #592 (closed), the notification is a requirement of using an IntentService. Thus, the service itself will be directly responsible for initializing and managing the state of the notification.

App Details screen

Currently, with the single download setup of F-Droid, the AppDetails Activity is responsible for:

• Initializing and running the download
• Keeping track of progress events emitted from the download
• Updating the UI in response to these events

Once the background downloading is implemented, the AppDetails will no longer be responsible initializing the download (only for sending an initial communication to the service to kickstart it). In addition, keeping track of the download progress via events is necessary but not sufficient. It is necessary because users would likely still like to see progress of an app downloading when viewing details of that app (even if the same info can be obtained from the notification). It is not sufficient, because they may navigate away from an app, do some stuff (maybe view some other apps that are downloading, or some which are not downloading), then navigate back again. In this case, they should not have to wait until the IntentService decides to grace them with a notification about the next stage of the progress. Rather, they should instead be able to ask the question of some class: "What is the current status (if any) of the download for app Blah?".

Keeping track of queue

There is no way to ask an IntentService: "What Intents are in your queue ready to run next. This means that the Notification will not be able to say anything like "Downloading App #1 (closed)... 3 other apps waiting to download".

It seems like the common solution to this on the interwebs is to override onStartCommand and keep a Queue of Intent objects that have been sent to the IntentService. This will also allow for ignoring duplicate Intents so that two requests to install a single app do not result in "2 other apps waiting to download".

Remembering the Queue

As discussed below, the queue needs to live somewhere that can be queried by F-Droid. This way, the AppDetails screen can show a message such as "Waiting to download..." and disable the "Install" button when the app is in the queue. This could be permanent or not-so-permanent.

Permanent queue

If each time an app is queued up for download, the service was to save some state to the database, this could be remembered between different invocations of F-Droid (i.e. after being forceably closed).

Pros

• If the user asks to download an app, it will be much more likely to actually get downloaded.

Cons

• Need to resend queued intents to the download service when restarting F-Droid, which would add complexity.
• May not be what the user expects. If they forceably close F-Droid and its associated download service - should that not also cancel the downloads?

Transient queue

If the queue was instead maintained in memory by the Android process, then when it is terminated, the queue would be cleared.

Pros & Cons

Essentially the opposite of the Permanent queue listed above.

Security Considerations

The Cure53 security audit threw up an issue to do with permissioning of .apk files while they are stored on disk. The problem was essentially that by downloading them to the SD card, so that the package manager had permission to read them, we left open the door for any app with "Write External Storage" permissions to do a switcheroo between us verifying the hash of the downloaded file and us passing the path to the file to package manager. The solution was to download apps to internal protected partition, but then provide read only access to the package manager.

Any changes to the caching code should keep this in mind and not break it.

Tasks

In the interests of preventing monolithic MRs as I have unfortunately done in the past, this section discusses smaller tasks to be implemented and merged one at a time. I've done my best to make it exhaustive, but it will likely require modification as we go. Once it has been discussed here, I'll create issues on GitLab for each task.

Ensure Installs Use Cached .apk File if Available

I've investigated this, works as expected if the "Cache packages" preference is enabled. If that is enabled, then downloaded .apks are copied to the "cache/apks/" dir. If not, they are left in the "cache/temp/" dir. All we will need to do is make sure that the IntentService knows which of these two locations to get the downloaded apk from.

In the following workflow:

• User presses "Install" from AppDetails screen.
• apk file downloads to cache.
• Once downloaded, user cancels the package manager installation.
• User presses "install" again.

Need to make sure this doesn't download the apk file again, but rather uses the file from the cache. Proper function of this "download to cache, then install at a later point" is crucial to getting background downloading working. It is also a nice to have fix on its own.

Perform Single App Download via IntentService

Before proceeding with queuing of multiple downloads, F-Droid should be able to continue to download a single apk via an IntentService. Currently this is done in an AsyncTask or something which is tied to the lifecycle of the AppDetails activity.

• Add intent service.
• Make "Install" button trigger intent, move all subsequent downloading of apk code to intent service.
• Intent service sends broadcasts.
• App details listens to those events.

This will be equally as dumb as the current setup to begin with. By this, I mean that navigating away from AppDetails should cause the download to cease. That way, we don't need to worry so much when navigating to the AppDetails of a different app, but receiving broadcasts about downloads of the previous app we were looking at. In addition, when the IntentService fires a "Download Complete" broadcast, if we are not in the relevant AppDetails screen, then we will ignore it. Future tasks will handle this with more finesse.

Not sure if this can be broken down into smaller tasks or not.

Allow Cancelling of Downloads

The previous task may not include the ability to cancel downloads, as it clutter the MR a bit. If it doesn't, then the next task should be to hook the "Cancel" button in AppDetails up to the IntentService somehow. By definition, IntentServices run to completion, however in reality, we can likely interrupt them like we do for any other Thread. From here on, it should be similar to how the current ApkDownloader deals with cancel requests.

Allow User to Navigate to Other Apps While Downloading

Once the download of a single app is done via IntentService, then this should be done. The following process should work as expected:

• User navigates to AppDetails for "App 1".
• Touches "Install", which kicks off download via service.
• While still in the same AppDetails, download progress is shown.
• User presses "Back" and goes to "App 2"
• Download of "App 1" is still happening in the background, and still emitting broadcast events.
• AppDetails for "App 2" should not show feedback about the progress of "App 1"s download.
• At this stage, "App 2" should probably grey out the "Install" button until the download is complete. This will only be temporary until a proper queue of apks to download is implemented in a following task.
• Navigate back to AppDetails for "App 1" while the download is still happening.
• AppDetails should immediately show feedback about download (rather than waiting for next broadcast event)

Given the task above which says that "Download Complete" broadcasts are ignored if viewing the wrong AppDetails. That behaviour is still the expected behaviour at the end of this task. It will be addressed in a future task.

Show Download Feedback of Single App Download in Notification

Once the AppDetails view is able to do its downloading via an IntentService instead of some local background task, we should then work on getting the Notification to display feedback correctly. This will prep us for showing feedback about multiple downloads sitting in the queue at a later stage.

Download Complete Causes Notification

Once a download has completed, the service first tries to notify the AppDetails activity. If that receives the broadcast and is displaying details for the correct app, then it will initiate the package manager install dialog. However, in the likely event that it is no longer being shown, then a new Notification should be created. When touched, it will launch the package manager to install the downloaded .apk.

• Should the notification be sticky?
• Should it have an "Install" button, perhaps with an "Ignore"/"Cancel" button?

Subsequent Download Requests Form Queue

The IntentService does most of this, however our subclass of the IntentService will need to:

• Keep track of incoming Intents in order to display a queue to users.
• Prevent queuing the same intent twice (as identified by the .apk hash, because two versions can be from different repos)
• Cancelling of pending Intents (probably in future task)
• Display of pending Intents in Notification (probably in future task).