Automation and scraping · Edition No. 6 · 31 Aug 2026

adbar/trafilatura

Extracts the actual article from any web page as clean Markdown, with no browser and negligible memory.

← Automation and scrapingRead the whole edition →

6.8k stars · Apache-2.0 · v2.2.0 (2026-07-31)

Extracts the actual article from any web page as clean Markdown, with no browser and negligible memory.

Repo detailsthe review · specs · pros & cons · install

What it is

A Python library and command-line tool that fetches a page and extracts the main body text plus its metadata — title, author, date, tags — discarding navigation, adverts, footers and comment sections. It outputs text, Markdown, JSON, CSV or XML, and it can walk a sitemap or a feed to do this across a whole site.

Why it matters

Two jobs, and they are the two halves of your business. For marketing: point it at your competitors' sitemap.xml and you have every article they have written as clean Markdown in one folder, which is how "write something about AI trends" becomes "here are the eleven questions our competitors answer and the four nobody has answered properly." That corpus is exactly what 06 needs. For the product: Grasppy's own pitch says chat, discussion or document, and "paste a link" is a lower-friction front door than "upload a file" — Docling handles the file, this handles the link. Crucially it is roughly two orders of magnitude lighter than crawl4ai, because it never launches a browser, which is what makes it safe to run on a VPS that is already carrying two products.

Stars6.8k
LicenceApache-2.0
Latestv2.2.0 (2026-07-31)checked 7 Sep 2026
Written inPython
Setup effort
1 / 5
Payoff
4 / 5
Good
  • Apache-2.0 since v1.8.0, so it is safe inside a commercial Grasppy — worth knowing, because anything older, and some forks, are GPLv3+
  • No browser, no Playwright, no Chromium: a plain HTTP fetch and a parser, so memory use is negligible
  • Consistently at or near the top of published extraction benchmarks, which is not a claim most scrapers can make
Watch for
  • It does not run JavaScript. A page that renders its article client-side comes back empty, and that is the moment to reach for crawl4ai instead
  • Extraction is heuristic, so an unusually built page can quietly lose a section without erroring
  • The command-line tool has a great many flags and the defaults are not always what you want for clean Markdown — budget one session of reading
Install
source venv/bin/activate
pip install trafilatura
# one page
trafilatura --markdown -u "https://example.com/some-article"
# a competitor's whole blog, as Markdown files
trafilatura --sitemap "https://competitor.com/sitemap.xml" \
  --markdown -o ./corpus

Get the next edition in your inbox

A dozen repositories, opened and checked. The licence read, the last release dated, and the ones that did not make it named with the reason. It is the half most lists leave out.

No tracking pixels. One click to leave. The archive stays free either way.

We use your address to send the edition and nothing else. Confirm by email, leave in one click. How we handle it.