4.7k stars · MIT · v1.6.8 (2026-06-01) · Track this in Scout
Focused text-chunking library — semantic and recursive splitters that respect message boundaries, so subtopics come out clean.
▶Repo detailsthe review · specs · pros & cons
What it is
A small Python library devoted to a single step — splitting documents into chunks — offering a dozen strategies from plain token windows up to semantic and recursive splitting that respect meaning and structure. It is built to be fast and to carry almost no dependencies.
What it is good for. In a retrieval or clustering pipeline the quality ceiling is set before any clustering happens, at the moment you decide where one piece of the text ends and the next begins. Split a chat mid-argument and everything downstream inherits the mistake — two half-ideas instead of one clear unit — and no amount of clever embedding repairs it afterwards. Chonkie's semantic and recursive chunkers respect message boundaries and topic shifts, which is exactly the shape of chat and transcript input. It is also the natural partner to model2vec from Edition 1: both are about doing the expensive step cheaply, on CPU, on hardware that is already paid for.
- MIT, pure Python, tiny install — drops into a FastAPI venv without a dependency fight
- A dozen strategies behind one API, so you can A/B your chunking without rewriting the pipeline
- Deliberately lightweight, which matters on a VPS that is already running other services
- It solves one narrow step. Useful, not transformative on its own
- The semantic chunkers need an embedding model, so they are not free in CPU terms
- The company behind it sells a hosted product — watch for features drifting to the paid side over time
