webcrawler

A multi-threaded webcrawler written in C with libcURL and Lexbor.

A web crawler to collect data from a public site using python with scrappy.