AI and models · Edition No. 10 · 4 Sep 2026

pemistahl/lingua-py

A Python library that detects which of seventy-five languages a piece of text is written in.

← AI and modelsRead the whole edition →
08

pemistahl/lingua-py

💎 hidden gem
Lingua: which language is this text. Detects languages offline, with confidence.

1.8k stars · Apache-2.0 · v2.2.0 (2026-03-09) · Track this in Rising ↗

Knowing which language a paragraph is in — before you spend a model call finding out.

Verified27 Oct 2026

Radar check

What we found when we opened it. Every bar is one fact, with its date.

Alive50

Last code pushed 20 Jul 2026, 3 months before this check.

Releases50

Latest release v2.2.0 on 9 Mar 2026.

Maturity90

Public since July 2021, a little over 5 years.

Adoption65

1,811 stars and 62 forks.

Activity0

0 commits and 2 new issues in the last 90 days.

Community40

8 contributors. No pull requests merged in 90 days; changes go in directly. A small team carries it.

Security40

No security policy file says how to report a problem. No published security advisories.

Licence Apache-2.0 | none listed: permissive: use it for anything, keep the noticeSetup effort 1 of 5Payoff 3 of 5
How the bars are scored
  • Alive. Days since the last code push: a week or less 100, a month 90, three months 70, six months 50, a year 30, eighteen months 15, older 0.
  • Releases. Age of the latest GitHub release, on a similar scale; no bar when the project publishes no releases.
  • Maturity. Age of the project: eight years 100, five 90, three 70, two 55, one 40, six months 25, younger 10.
  • Adoption. Stars on a log scale: 100,000 stars is 100, 10,000 is 80, 1,000 is 60, 100 is 40.
  • Activity. Commits in the last 90 days: 300 or more 100, 100 85, 30 65, 10 45, any 25.
  • Community. People who have contributed: 100 or more 100, 30 80, 10 60, 5 40, 2 25, one 10.
  • Security. A published security policy 80, none 40; published advisories are listed, not scored.

Checked by Repo Radar on 27 Oct 2026 from the repository on GitHub. Covered in Edition 10.

▶Repo detailsthe review · specs · pros & cons · install

What it does

A detector is created first, optionally restricted to the languages that matter, and then asked about a piece of text. It returns the most likely language, and a separate call returns a confidence value between zero and one for every candidate, with the values adding up to one. It was built for short and mixed-language text, which is where simpler detectors fail. The models are stored as finite-state transducers, a compact way of holding a large word list, which is why the memory cost is a few dozen megabytes rather than hundreds. Everything runs offline once installed. There is also a low-accuracy mode that loads only a small subset of each model; the README states that it reduces accuracy for texts under 120 characters. What it does not do is translate anything, and it needs a recent interpreter: Python 3.12 or later.

Why it matters

Who it suits. Anyone importing text written by many people, where the language is not recorded and almost everything downstream depends on knowing it — which word list to use, which rules to apply, which model to send a piece to. It also suits anyone who wants to show a reader a fact about their own material, such as how much of a document is in each language. Skip it if every piece of text already carries a language tag you trust.

Verdict. Worth twenty minutes, and it replaces a recurring cost with a free one: deciding the language with a model call is a bill that never stops, while deciding it here takes microseconds. Two things to remember. Loading all seventy-five languages costs real memory, so restrict the list when the detector is built. And this is a single-author project, stable and well tested, but with one person behind it. Version 2.2.0 was published on 9 March 2026 and code landed on 20 July 2026, so it is maintained and unhurried. The same author's Rust version, pemistahl/lingua-rs, is at 1,144 stars with code on 5 October 2026.

Stars1.8k
LicenceApache-2.0
Latestv2.2.0 (2026-03-09)checked 7 Sep 2026
Written inPython
NeedsPython 3
RunsLocally, on your own machine
Summary (Key Points)from the README
Key features
  • Language coverage: Detects 75 languages, with the README stressing quality over quantity.
  • Short text support: Uses n-grams of sizes 1 to 5 so it can handle single words and short phrases, not just long text.
  • Hybrid detection: A rule-based engine filters languages by alphabet and unique characters, then a statistical Naive Bayes n-gram model decides the rest.
  • Small memory footprint: Models are stored as finite-state transducers, so the library uses only a few dozen megabytes at runtime.
  • Offline use: No external API or service connection is needed once the library is downloaded.
  • Confidence values: Returns per-language probabilities between 0.0 and 1.0 that sum to 1.0, plus a single-language confidence method.
  • Mixed-language detection: Can return contiguous single-language sections with start and end indices, marked as experimental.
  • Thread safety and parallel methods: A single detector can be used across threads, and multi-threaded variants accept lists of texts.
  • Eager or lazy loading: Lazy loading is the default, and eager loading preloads all models to avoid latency in web services.
  • Low accuracy mode: Loads only a subset of models for faster runs on long texts, at the cost of accuracy on texts under 120 characters.
Ways to install
  • PyPI package: Install with pip from the Python Package Index. Needs Python 3.12 or newer and a wheel for your platform.
  • Pure Python branch: A separate branch in the repository for environments that do not support native Python extensions. Needs no compiled extension; the README gives no install effort for it. It is kept up to date only within the 1.* releases.
Important
  • Python version: Requires Python 3.12 or newer.
  • Pure Python limits: The pure Python implementation is slower and only receives updates in the 1.* line.
  • Short text results: Minimum relative distance can cause None to be returned for short phrases, and low accuracy mode drops accuracy for texts under 120 characters.
  • Experimental mixed-language mode: Results depend heavily on the input and are less accurate with short phrases.
  • Enum differences: Because of PyO3, enums do not behave like native Python enums; they cannot be subscripted and do not support metaclasses.
  • Benchmark source: Accuracy and speed figures come from the author's own tests on bundled Wortschatz-based data and an iMac, so results on other data may differ.
  • Language set: Restricting the language set is recommended, since more languages reduce accuracy.

Written by an AI model from the project’s README; not part of the review, and it can be wrong.

Setup effort
1 / 5
Payoff
3 / 5
Good
  • Apache-2.0 and fully offline, so nothing can rate-limit it or charge for it.
  • Unusually accurate on short fragments, which is the normal failure case elsewhere.
  • Returns confidence values you can threshold, so an ambiguous piece can be routed differently.
Watch for
  • Loading all seventy-five languages costs real memory unless the list is restricted.
  • Requires Python 3.12 or later, which rules out older installations.
  • One author, so stability today is not the same as cover tomorrow.
Replaces

The detected language becomes a column beside each chunk in Postgres, and every later stage reads it instead of re-guessing.

Works well with

The detected language becomes a column beside each chunk in Postgres, and every later stage reads it instead of re-guessing.

Install
python3 -m venv venv
source venv/bin/activate
pip install lingua-language-detector
Screenshots
pemistahl/lingua-py: GitHub preview cardpemistahl/lingua-py: Screenshot 1pemistahl/lingua-py: Screenshot 2pemistahl/lingua-py: Screenshot 3pemistahl/lingua-py: Screenshot 4

On Stack Radar: AI & Automation › AI Models & Inference › Language Models.

Get the next edition in your inbox

A dozen repositories, opened and checked. The licence read, the last release dated, and the ones that did not make it named with the reason. It is the half most lists leave out.

No tracking pixels. One click to leave. The archive stays free either way.

We use your address to send the edition and nothing else. Confirm by email, leave in one click. How we handle it.