Integrate testpages end-to-end tests in our CI
Background / User story
In #1000 (closed), we're going to remove the "webext" child pipeline, which includes the end-to-end tests that are ensuring that the ad blocking functionality works across supported platforms. While the integration of the SDK (see #972 (closed)) will make those tests redundant, we're still relying on them, for the time being. The initial idea was to integrate them with our own end-to-end tests,
An easier solution, that would also require less maintenance on our end, would be to reuse the testpages project: https://gitlab.com/eyeo/adblockplus/abc/testpages.adblockplus.org. Testpages, as well as the tests are maintained in there, so we could add the repository as a submodule (or just clone it when the tests are run) and run the tests in our CI.
What to change
- Design: N/A
- Research: N/A
- Spec: N/A
- Legal: N/A
- Development:
❌ Modifications needed for integrating the tests:
Make the following modifications to the "webext" child pipeline:
- Remove the following CI jobs:
- prepare-dependencies
- build:chrome
- build:firefox
- Move testpages-based end-to-end tests from adblockpluschrome/test/ to test/end-to-end/.
Note that those tests are based Selenium so, in order to integrate them with existing end-to-end tests, they'll need to be converted to WebdriverIO.
✔ Modifications needed for running the testpages in our CI:
- add testpages project as a submodule
- add a new stage in which tests in the testpages project should be run against our CI artifacts
- remove the
webext
child pipeline - remove all webext tests under
adblockpluschrome
- Remove the following, redundant adblockpluschrome npm dependencies:
- chromedriver
- dmg
- extract-zip
- geckodriver
- got
- jimp
- msedgedriver
- ncp
- selenium-webdriver
- semver
Hints for testers
Testpages autotests were integrated in our CI:
- tests should be automatically triggered on the
release
andmaster
branch - tests needs to be manually triggered on all other branches
- tests should be passing
- note: the Snippets tests are skipped for now, see comment: https://gitlab.com/adblockinc/ext/adblockplus/adblockplusui/-/issues/1002#note_1016556438
Hints for translators
N/A