1.8k stars · Apache-2.0 · v2.2.0 (2026-03-09) · Track this in Scout
Knowing which language a paragraph is in — before you spend a model call finding out.
▶Repo detailsthe review · specs · pros & cons
What it is
A language-detection library for Python covering 75 languages, built specifically to stay accurate on short and mixed-language text, which is where most detectors collapse. No neural network and no API call — it ships statistical models and runs entirely offline, returning a confidence value rather than a bare guess.
What it is good for. A chat log is a common import, and a genuinely bilingual chat log switches language mid-thread — often mid-paragraph. Almost everything downstream depends on knowing which: which stopword list, which cluster label language, which Presidio (Ed.7 #10) recogniser set, and whether a chunk deserves a cheap model or an expensive one. Deciding that per chunk with an LLM is a recurring bill; deciding it here takes microseconds and costs nothing. It also unlocks a small honest feature you can demo: telling the user "this conversation is 62% Russian" before they have read a word of it.
- Apache-2.0 and fully offline — nothing to call, nothing to pay for, nothing that can rate-limit you mid-import.
- Unusually strong on the short fragments that chunking produces, which is exactly the failure case for simpler detectors.
- Returns confidence values you can threshold, so ambiguous chunks can be routed differently instead of guessed at.
- Loading all 75 languages costs real memory. Restrict it to English and Russian at construction time and it becomes small — but you must remember to.
- Requires Python 3.12 or newer; check the interpreter on the target machine before planning around it.
- Single-author project. Stable and well-tested, but a bus factor of one is still a bus factor.
The detected language becomes a column beside each chunk in Postgres, and every later stage reads it instead of re-guessing.
The detected language becomes a column beside each chunk in Postgres, and every later stage reads it instead of re-guessing.
