Dynamic updates to all filters
Background
A problem that we have now is that we're no longer able to update filter lists in between releases due to the imposition by Mv3 that DNR rules are statically declared at build time. We are planning to implement a diffing mechanism that will work for any type of filter list but it is a complex one and still depending on some modifications to the DNR API that will be released by google in 2023.
Needs:
- Increase the frequency of filter updates to support all available subscription update frequency.
This issue is done when:
-
The SDK provides the users the ability to have all filters updated and usable in between extension updates and without extension releases (manual filter updates), as long as the filter limits imposed by Google are not surpassed.
-
Errors are handled in a manner where the extension does not stay on a broken state once filter limits are met and the filter lists are reversed to the latest working version. We should anticipate errors. Also the SDK provides a message to the extension regarding the reached limit (including category details if applicable).
-
Filters can be removed from a static ruleset. When a filter needs to be removed, we would need to determine which related rules are in the relevant ruleset. In order to remove rules from a static ruleset we would need the correct id. Currently we don't have a way of determining this.
Assumptions
- Updates are atomic. We won't apply updates partially. When a filter in a diff update is invalid, we would not apply the entire update. For now we do not apply any ideas around "partial updates". This is in line with Chrome's
updateDynamicRules
API function. - We shouldn't get invalid filters as the filter delivery system should have already cleared those.
- The updating mechanism should handle errors in a similar way to how we currently handle MV2 filter lists.
- This functionality is only possible with Chrome 111+.
- When a subscription has a
diffUrl
property it will be considered for the diffing mechanism. - We will consider the
expires
property on a subscription to know when to do the diffing check. (This is already handled by the current synchronizer in core)
Questions
- Are some lists more relevant (higher priority) for users than others?
- How do we handle the idea of this functionality only being possible from Chrome 111+? What happens to users using older extensions? Do we set
minversion
in the manifest for MV3 extensions?
Dependencies
-
Static rules can be disabled through a Chrome API: https://chromium-review.googlesource.com/c/chromium/src/+/4005612
Tasks
-
#503 (closed) Create UpdatableSubscription
to MV3 updates (Have the core classes needed to describe this new separate subscription type.DownloadableSubscription
subscriptions feel out of place in this context. This new class is focused on updating differences not downloading the full lists.) -
#504 (closed) Download the updates UpdatableSubscription
(Implement the downloading mechanism. This would look at thediffUrl
of the relevant subscription and download its content. ) -
#505 (closed) Apply the updates to the DNR (This is not only the most complicated feature but the most important one. We should verify our thinking here first. This is where we'll most likely have interesting corner cases.) -
#528 (closed) Map filter to DNR rules.
-
-
#506 (closed) Remove the CV list exception in the DNR conversion -
#507 (closed) Migrate CV list with filters updates -
#516 (closed) Notify update status to the extension
Previous related issues:
https://gitlab.com/eyeo/adblockplus/abc/webext-sdk/-/issues/128, https://gitlab.com/eyeo/adblockplus/abc/webext-sdk/-/issues/129, https://gitlab.com/eyeo/adblockplus/abc/webext-sdk/-/issues/130