Reconsider the internals of the filter engine

Reconsider the internals of the filter engine

Background / User story

There are a bunch of issue with the filterStorage and filterEnginer we currently have:

  • Issue #381 (moved) show it is harder to implement as we don't properly handle having duplicate filters
  • Downloadable filter list are immutable
  • The implementation of issue #407 (closed) and #419 (closed) makes for an awkward API, including limitations like not being able to add metadata to filters that don't have metadata. (see webext-sdk!291)

Manifest v3

Manifest V3 brings a set of extra requirements:

  • Split between URL blocking filters (DNR) & content + popup filters.
  • DNR static filters are immutable, updates are provided separately.
  • Dynamic DNR rules are in one single ruleset on the browser side, so we need a mechanism to associate them.

What to change

Proposal

Subscriptions in MV3 are no longer a single data set, because URL filter are now DNR.

  • DownloadableSubscription
    • They are associated to a ruleset ID - the core doesn't know DNR but the ID is a string the SDK can use. (MV3)
    • They have a list of dynamic DNR rules. We need to store these to be able to enable / disable (MV3)
    • Their content is immutable, unless it's the synchronization process (all)
  • Issue #449 (closed) - CustomFiltersSubscription replace the SpecialSubscription. (all)
    • They can have metadata. (all)
    • They might have a list of dynamic DNR rules (MV3) for custom URL filters. The whole rules must be stored.
  • The core will need to have the index of subscriptions to be able to locate them by url or mv2-url.
    • Subscription.fromURL() should sort this out.

In Manifest V3 we don't have blocking URL filters.

  • filterEngine will be instantiated in a mv3 mode. This is necessary has the core doesn't have access to the webextension API.
    • Issue #450 - The CombinedMatcher should only contain CSP and POPUP filters in the blocking section.

We need a custom list of recommended list.

Operations (example)

  • enable/disable subscription in EWE:
    • get subscription from filter engine (using the ID)
    • enable/disable subscription
    • get ruleset ID.
      • enable/disable ruleset (webext)
    • get dynamic rules
      • enable/disable the dynamic rules. This is done by adding or removing the rules (webext).

Question

  • how do we handle duplicate filters ?
  • how to we handle immutability ?

Hints for testing

TBD

Edited by Hubert Figuière