S
Sitemap Crawler
From a base URL, crawl a websites URLs (via href link traversal) and generate a Sitemap. Supports checking and cleaning up sitemap, and checking paths on a different base/host.
-
Jan Klass authored27a86538
| Name |
Last commit
|
Last update |
|---|---|---|
| Project.toml | ||
| check.jl | ||
| crawl-sitemap.jl | ||
| crawl.jl | ||
| crawldata.jl | ||
| filenames.jl | ||
| test_sitemap.jl |