Skip to content

Make a Selenium test version

Selenium is heavier and slower but as a first version it may be enough. The interesting thing with Selenium is that we don't have to minimize requests to the simplest one (it's a long process and it is especially annoying in case of very secure/complex websites or in the case of the presence of magic random values used), just filling textfields and clicking on buttons. However to keep original plan of making a library to be compatible with password managers for instance, testing Selenium headless may be interesting. In addition that in case of captchas, we could just request the user interaction using a notification. Note that could also add the ability to pay someone else to proceed to the captcha. Could maybe exploit the fact that on smartphones, most of the time it's not necessary to have to fill a captcha. Is there any library in Rust to use Selenium?

As the most time spent is in reverse-engineering the websites (can not limit to the websites, for applications-only for instance), so using Selenium can make the project raise very quickly in my opinion.

Could also look for alternatives to Selenium but as far as I know it is open-source (when not used with the well-known Google Chrome setup) and it works very well.

Rust libraries for working with Selenium:

As Selenium is slow, could even use Python, but the main problem with that is that it's hard to make Python work in a standalone manner on non developers computers. However as the main point of this project is the dataset, having a Python interpreter wouldn't be much a problem as we could code one in another programming language quite easily and quickly.

Note that just using Selenium makes the installation on non developers computers harder.

Can proceed using the list of most visited websites, as it's no more very particularly difficult to authenticate to these websites. Similarweb may have a more exhaustive list, as its Wikipedia data source.

Doesn't seem able to find a list of services with most users. Anyway it would help but not so much.

Edited by Benjamin Loison