Skip to content

Add alternative test method to test pickled results.

Problem

Scrapy is capable of exporting crawl results to pickled format which means the testing could be done on pickled results rather than starting up scrapy crawling and crawling cached/live data.
Right now it's not possible to test only the results only as item types are not preserved in json,csv exports.

Advantages

Testing only the results would mean very fast testing because scrapy crawler doesn't need to start

Disadavtages

Doesn't test scrapy crawl itself.