Language filter list suggestions

Background / User story

As an ABP user, who frequents websites in different languages, I expect the ABP to automatically block ads on any website, irrespective of the language so I get the optimal experience with minimum manual configuration

Approximately 5% of our users don't have the language filter they need because their computer has one language installed although they visit other languages. Therefor we would like to detect and install relevant languages

- User should be able to install/remove lang filters even if SLFS is enabled - If SLFS disabled - stop adding new lang but don’t remove already installed - Mark with tooltip lang that were installed by SLFS - For every new lang installed by SLFS - inform user with icon notification - Enable by default SLFS - When SLFS is enabled - don’t remove existing lang, just add new - If lang list already installed - don’t modify manual user settings - Add language only after user has visited 3 times - add on the 3rd time

Our main KPI for success will be Amount of reports for Polish websites (Nicole Jones) (Outdated heavily, follow the specs instead)

What to change

• Design: https://eyeogmbh.invisionapp.com/console/share/AYVJBL68QWU/400579447
• Research: TBD
• Legal: https://jira.eyeo.com/browse/ABPUI-39
• Spec: https://gitlab.com/eyeo/specs/spec/-/merge_requests/245
• Development: Fix broken language_subscription documentation link.

Hints for testers

In order to test the functionality, you can use the following websites:

1. Recommended language (German) implicitly and explicitly detected:
   • https://adblockplus.org/de/
   • https://www.zeit.de/
   • https://orf.at/
2. Recommended default language (English) implicitly and explicitly detected:
   • https://adblockplus.org/en/
   • https://www.washingtonpost.com/
   • https://www.nytimes.com/
3. Not recommended language (Japanese) explicitly detected and no language implicitly detected:
   • https://www.google.com/search?q=site:ru&hl=ja
   • https://www.google.de/search?q=site:ru&hl=ja
   • https://www.google.fr/search?q=site:ru&hl=ja

You can check the implicitly and explicitly detected languages by logging the variables explicit and implicit by setting a log breakpoint in this line.

This change also includes a fix for #944 (closed).

Hints for translators

• Added strings:
  • notification_recommendLanguage_message
  • options_language_recommend
  • options_language_recommend_tooltip
• Updated strings:
  • options_language_add
  • options_language_description
  • options_language_header
  • options_more_filters_header

Further information

• Spec: eyeo/specs/spec!245 (Old version: Working specs (acknowledgments to Thomas!))

• Website language detection: Proof-of-concept extension
  • Results on reliability of different detection methods: spreadsheet
  • See comment in GitLab for recommendation

• Performance
  • Performance impact of en-/disabling ABP: Performance report on Automatic Filters VS spiegel.de
  • Per-pageview impact of language detection and filter list processing delay: https://gitlab.com/eyeo/specs/spec/merge_requests/177#note_119177056
• Language List Suggestion Requirements Doc - Created by Nicole Jones
• Usability Study - Qual - Final Results
• Design Variants Survey - Quant - Final Results
• Filter list download stats for automation tests, hub ticket

Checklist

For the above to be a success, the following items need to happen

• Flow for the notification system is defined
• Copy for the notification is defined (@lisabielik)
• Data of current issue reports have been collected and analyzed (@data-raccoon) - http://hub.eyeo.com/issues/14043 (OUT OF SCOPE)
• Entire flow has been user tested and verified/ modified accordingly (@achou)
• Ensure our infrastructure will be able to handle automatically installing 000`s of filter lists, by having this feature as opt-in by default.
• Ensure that we will be able to identify filter lists that have been automatically added by the feature vs. added by the user. (OUT OF SCOPE, we'll use telemetry when it's implemented)
• Ensure that our notification system works on all platforms (the logic right now is handled by WebExt) (OUT OF SCOPE; there was a problem with older Firefox versions not supporting the language detection feature we were using, but since the feature took so long, that problem resolved itself)
• Ensure that depending on the outcome of hub#14514 the feature is not rolled out to German/French users
• T&Cs and Privacy Policy has been updated to account for the new feature (@Zarillo) - http://hub.eyeo.com/issues/13531 (OUT OF SCOPE)
• The filter list activation system is compliant with GDPR (@Zarillo) (OUT OF SCOPE, information is not collected that could be considered not compliant and is not already covered in our Privacy Policy)
• The feature is translated/ localized (11 core languages by the agency, remainder [30+] by Crowdin) (@Shikitita, @amazurek)
• The feature is implemented according to the specified design
• Feature behaves as outlined in the specs
• Add "detectedLanguage" data point to issue reporter https://gitlab.com/eyeo/adblockplus/abpui/adblockplusui/issues/168 (OUT OF SCOPE, will be treated separately)
• Defined roll out plan of the feature (OUT OF SCOPE; initially to be rolled out to Polish users but this feature has been in the works for 3 years)