Apks with non-free components, AntiFeatures etc

This is related to / a continuation of #2070 (closed).

There are "several" APKs which somehow slipped through our scanners with their non-free components. Last night I did run my library-scanner against the TtT mirror (which was last updated 2020-01-23, so apks newer than that were not scanned).

Summary

  • Scanning time: exactly 10h (to the minute)
  • 709 APKs (out of 8310 scanned = 8.5% of all APKs had hits) of 183 packages (apps)
  • 42 libs were detected
  • covering antifeatures Tracking, NonFreeNet, NonFreeDep, Ads, NonFreeAdd – where NonFreeDep stands for the library itself being non-free

Libraries

To narrow things down, we might need to "white-list" some libraries – for example those marked Tracking but might be configured opt-in (some of them permit this: I just learned MS AppCenter belongs to that group). Further, if such a library only demands an AntiFeature (e.g. NonFreeAdd for the Tasker/Locale addon), this just needs cross-checking against the index whether that AntiFeature is set; I didn't yet perform this step as it's not yet 100% clear which libraries we want to consider. So here the list of libraries that were found, together with their "assigned AntiFeatures":

  • Adjust Android SDK (/com/adjust/sdk): Tracking
  • Alipay ZeroSdk (/moe/feng/alipay/zerosdk): NonFreeNet
  • Android In-App Billing v3 Library (/com/anjlab/android/iab/v3): NonFreeNet
  • android-issue-reporter (/com/heinrichreimersoftware/androidissuereporter): NonFreeNet
  • Android Market (/com/google/android/finsky): NonFreeNet
  • AppMetrica (/com/yandex/metrica): Tracking
  • AppsFlyer (/com/appsflyer): Tracking
  • BillingClient (/com/android/billingclient): NonFreeNet
  • Bugly Android SDK (/com/tencent/bugly): Tracking
  • BugSense (/com/bugsense): NonFreeNet
  • Bugsnag exception reporter for Android (/com/bugsnag): Tracking
  • Countly (/ly/count/android): Tracking
  • Crashlytics (/com/crashlytics): Tracking
  • Dropbox Core SDK for Java (/com/dropbox/core): NonFreeNet
  • Fabric (/io/fabric/sdk/android): Tracking
  • Facebook Analytics (/com/facebook/appevents): Tracking
  • Facebook SDK for Android (/com/facebook): Tracking
  • Firebase (/com/google/firebase): NonFreeNet
  • Firebase JobDispatcher (/com/firebase/jobdispatcher): NonFreeDep
  • Google Ads (/com/google/ads): Ads
  • Google Analytics (/com/google/android/gms/analytics): Tracking
  • Google API Client Libraries (/com/google/api/client): NonFreeDep
  • Google Cast (/com/google/android/gms/cast): NonFreeDep
  • Google GCM (/com/google/android/gcm): NonFreeNet
  • Google Mobile Services (/com/google/android/gms): NonFreeDep
  • Google Play (/com/android/vending): NonFreeNet
  • HockeyApp (/net/hockeyapp): Tracking
  • last-fm-library (/com/antoniotari/android/lastfm): NonFreeNet
  • Locale (/com/twofortyfouram/locale): NonFreeAdd
  • Locus API (/locus/api): NonFreeAdd
  • Mapbox SDK Telemetry component (/com/mapbox/android/telemetry): Tracking
  • Matomo (formerly Piwik) (/org/matomo): Tracking
  • Microsoft Appcenter Crashes (/com/microsoft/appcenter/crashes): Tracking
  • Microsoft MapPoint (/microsoft/mappoint): NonFreeNet
  • Piwik (/org/piwik): Tracking
  • Play Install Referrer Library (/com/android/installreferrer): NonFreeNet
  • play-store-api (/com/github/yeriomin/playstoreapi): NonFreeNet
  • PushWoosh (/com/pushwoosh): Tracking
  • Sentry SDK for Java (/io/sentry): Tracking
  • telemetry-android (/org/mozilla/telemetry): Tracking
  • Twitter Kit for Android (/com/twitter/sdk/android): NonFreeNet
  • YouTube Data API (/com/google/api/services/youtube): NonFreeNet

APKs

Above summary already indicated that list would get a bit longer, so I will not "inline" it. Instead I attach the resulting JSON for anyone interested. It's 150k or, gzipped, 7.5k: result.json.gz. Some examples of content to show the structure:

    "me.zeeroooo.materialfb_50.apk": [
        {
            "name": "Facebook SDK for Android",
            "id": "/com/facebook",
            "anti": "Tracking"
        }
    ],
    "me.zeeroooo.materialfb_54.apk": [
        {
            "name": "Firebase JobDispatcher",
            "id": "/com/firebase/jobdispatcher",
            "anti": "NonFreeDep"
        }
    ],
    "org.wikipedia_279.apk": [
        {
            "name": "Mapbox SDK Telemetry component",
            "id": "/com/mapbox/android/telemetry",
            "anti": "Tracking"
        },
        {
            "name": "HockeyApp",
            "id": "/net/hockeyapp",
            "anti": "Tracking"
        }
    ],

Suggested approach

I'd suggest to first concentrate on "clear candidates". I guess noone will argue that e.g. Firebase and Crashlytics are non-free libraries, and thus all APKs containing those should be removed (and their corresponding builds disabled). I can easily generate a JSON stripped down to selected libraries to work on. I can also "disable builds", as metadata are easily accessible (I'd suggest a disable: contains XXX) – but I don't know whether that would automatically remove the corresponding APK files, or if that has to be done separately.

So let's split this into "first-things-first" sections:

  • Crashlytics (/com/crashlytics): Tracking
  • Fabric (/io/fabric/sdk/android): Tracking
  • Firebase (/com/google/firebase): NonFreeNet
  • Firebase JobDispatcher (/com/firebase/jobdispatcher): NonFreeDep
  • Google Analytics (/com/google/android/gms/analytics): Tracking
  • Google Cast (/com/google/android/gms/cast): NonFreeDep
  • Google GCM (/com/google/android/gcm): NonFreeNet
  • Google Mobile Services (/com/google/android/gms): NonFreeDep

are IMHO libraries noone will argue off the black-list: they are clearly non-free components which should not be permitted at F-Droid. I'd suggest starting with those, disabling all related builds (and deleting their APKs). While that is in progress, we can discuss the remaining list and how to deal with it. For example:

  • take Bugsnag off the list as it can be configured opt-in (see e.g. FairEmail)
  • similarly: Bugly, Bugsense, Matomo, Piwik, Microsoft Appcenter Crashes, Sentry?

Next round:

  • Play Install Referrer Library seems to always automatically "ping" Google on the very first app start (or right at install), as I noticed in some of my MR reviews. So if it's not considered non-free code itself, it at least warrants Tracking, right?
  • which other libraries should just be cross-checked against our index whether the corresponding AntiFeature is set (and if not, have it set)?

Last round: What about the remaining libs?

Edit: JSON grouped by packageName: result2.json.gz

Edited by Izzy