81.9k stars · Apache-2.0 · v0.9.3 (2026-08-31) · Track this in Scout
LLM-friendly crawler — renders JS pages and returns clean markdown. Competitor research plus a 'paste a URL' front door for Grasppy.
▶Repo detailsthe review · specs · pros & cons · install
What it is
A Python library and a command-line tool. It drives a real browser, strips a page down to its content, and returns Markdown — the plain text format this report itself is written in. It can also take a description of the fields you want and return structured data instead, and it caches what it has fetched so a second run is fast.What it is good for. Anyone building something that reads the web and then asks a model about it. The problem it removes is the cost and the noise of sending a whole raw page to a model. A short line about Grasppy: it maps long text into subtopics, and this is a clean way to get long text out of a set of pages in the first place.
- The output is already clean, so the next step costs less and gives better answers.
- It runs a real browser, so pages that build themselves with JavaScript still work.
- Apache-2.0 licence, a very large and active project at 80,900 stars, and there is a Docker image so you do not have to install a browser by hand.
- It is still before version 1.0, at v0.9.3, and things move. Pin the version.
- Asking it to extract fields with a model costs money on every page. The plain text path costs nothing.
- Browsers are heavy. Give it a gigabyte or more, and do not run twenty at once on a small server.
firecrawl/firecrawlPublished in Edition 9, it does the same job as a hosted service with a generous library, and its main code is AGPL-3.0 rather than Apache-2.0.
Track this in Scout- ScrapeGraphAI/Scrapegraph-ai
It builds the whole extraction as a graph of model calls, which is more automatic and costs a model call at every step.
Track this in Scout 
microsoft/markitdownPublished in Edition 14, it converts files on your disk into Markdown rather than fetching anything.
Track this in Scout
python3 -m venv venv
source venv/bin/activate
pip install -U crawl4ai
crawl4ai-setup # installs the browser it needs
# Check it works:
crwl https://grasppy.com/radar/ -o markdown
# Or from Python:
# import asyncio
# from crawl4ai import AsyncWebCrawler
# async def main():
# async with AsyncWebCrawler() as c:
# r = await c.arun("https://example.com")
# print(r.markdown)
# asyncio.run(main())