wget Builder
Set the options for a wget download or site mirror, then get the exact command.
This is generated in browser and is not sent to pah.moi servers.
About this tool2 paragraphs
The options are how deep to recurse, what to bring along (--page-requisites, --convert-links, --adjust-extension), how politely to ask (--wait, --random-wait, --limit-rate), and, under Expert, the scope, reliability, identity and output flags.
Beside the command the builder says what recursion the flags actually ask for, warns about the options that carry a cost (robots.txt ignored, certificate checks skipped), and shows how the crawl looks from the target server's logs: the request rate, the user agent it sends, whether robots.txt was honoured, and the size of the footprint.
Use it locally
This tool has a native command line twin. Build
wget-cli from the site's source with:
cargo build --release --bin wget-cliSource and licence terms