xcrawl3r

Crawls a website from a start URL, parsing links, scripts, sitemaps and robots.txt, and lists the URLs it finds

Community-packaged from github.com/hueristiq/xcrawl3r: the registry's community pins the upstream release; the tool is its authors' work. Maintain that project? Claim this package.