896 stars · MIT · v2.5.1 (2026-06-27)
One command, one HTML report: broken links, redirects, titles, descriptions, image weight, response times.
▶Repo detailsthe review · specs · pros & cons · install
What it is
A command-line site crawler and auditor. It walks every page of a site and reports back on broken links, redirects, response times, page sizes, headings, titles and meta descriptions, images, accessibility basics and security headers — as a single self-contained HTML report, or as Markdown, JSON or a spreadsheet. It also generates sitemaps and can export a working offline copy of a site.
Why it matters
Edition 2 gave you Unlighthouse, which is excellent at one question — how does each page score in Lighthouse — and silent about the boring failures that lose you more traffic than a performance score does. Dead links are the clearest case: you will rename a docs page, you will change a video URL, and six months later Google is still indexing a page that links into nothing. This finds those in one pass, along with the missing meta descriptions and the untitled pages that stop the Starlight docs from ranking properly. Two details make it fit you specifically: it is one binary with no Node project and no install ceremony, and its report is a file you can hand to Claude Code and say "work through this." At 764 stars it is genuinely under the radar, and unlike the last two SEO auditors this radar considered, its releases are dated and recent.
- MIT, self-contained binaries for macOS, Linux and Windows — nothing to host and nothing to maintain
- Covers the unglamorous checks in one pass: broken links, redirect chains, titles, descriptions, image weight, response times
- The offline-export and sitemap features are genuinely useful side effects — an archived copy of the site before a redesign costs one flag
- 764 stars and essentially one maintainer. For a tool you run occasionally on your Mac that risk is small, but do not build a pipeline that depends on it
- It reports; it fixes nothing. The report is only worth the morning you spend acting on it
- Crawling a JavaScript-rendered React app needs the browser-rendering mode added in v2.5.0, which is slower and heavier — the static Astro side is where this shines
# download the binary for your platform from the releases page, then: ./crawler --url=https://yourdomain.com \ --output-html-report=report.html \ --max-visited-urls=500