Sitemap Crawler

Project information

From a base URL, crawl a websites URLs (via href link traversal) and generate a Sitemap. Supports checking and cleaning up sitemap, and checking paths on a different base/host.

sitemap crawler checker

+ 1 more

15 Commits
1 Branch
0 Tags

Created on

November 11, 2018

Copy SSH clone URLgit@gitlab.com:Kissaki/sitemap-crawler.git
Copy HTTPS clone URLhttps://gitlab.com/Kissaki/sitemap-crawler.git