Data and databases · Independent · 26 Oct 2026

dedupeio/dedupe

:id: A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.

← Data and databases
Verified26 Oct 2026

Radar check

What we found when we opened it. Every bar is one fact, with its date.

Alive15

Last code pushed 29 Jul 2025, 15 months before this check.

Releases—

No releases published on GitHub; the code date above is the one to trust.

Maturity100

Public since April 2012, a little over 14 years.

Adoption73

4,523 stars and 578 forks.

Activity0

0 commits and 1 new issues in the last 90 days.

Community80

72 contributors. No pull requests merged in 90 days; changes go in directly.

Security40

No security policy file says how to report a problem. 1 published security advisory.

Licence MIT: permissive: use it for anything, keep the noticeSetup effort 3 of 5Payoff 4 of 5
How the bars are scored
  • Alive. Days since the last code push: a week or less 100, a month 90, three months 70, six months 50, a year 30, eighteen months 15, older 0.
  • Releases. Age of the latest GitHub release, on a similar scale; no bar when the project publishes no releases.
  • Maturity. Age of the project: eight years 100, five 90, three 70, two 55, one 40, six months 25, younger 10.
  • Adoption. Stars on a log scale: 100,000 stars is 100, 10,000 is 80, 1,000 is 60, 100 is 40.
  • Activity. Commits in the last 90 days: 300 or more 100, 100 85, 30 65, 10 45, any 25.
  • Community. People who have contributed: 100 or more 100, 30 80, 10 60, 5 40, 2 25, one 10.
  • Security. A published security policy 80, none 40; published advisories are listed, not scored.

Checked by Repo Radar on 26 Oct 2026 from the repository on GitHub.

In simple English

Real-world data is often messy and full of duplicates. Names and addresses get spelled differently in different spreadsheets. This Python library (a collection of ready-to-use programming code) uses machine learning (a way for computers to learn patterns from examples) to match those records up. You give it a few examples of matches by hand. Then it finds the rest of the duplicates across your large datasets automatically.

What it is

Dedupe is an open-source Python library for record deduplication and entity resolution. It reads human training choices to build custom matching rules for your specific data.

What it is good for

Solo builders who need to clean up dirty customer lists, merge two databases without shared keys, or spot identical entries hiding under slight spelling mistakes.

Stars4.5k
LicenceMIT
LatestNo tagged releasechecked 26 Oct 2026
Last code change2025-07-29
Written inPython
Setup effort
3 / 5
Payoff
4 / 5
Good
  • Learns from your own training data to get more accurate results over time.
  • Handles very large datasets quickly using active learning algorithms.
  • Free to use under the permissive MIT licence.
Watch for
  • Requires writing Python code and training the model by hand.
  • Needs enough computer RAM (temporary memory) to hold your dataset while processing.
  • Maintenance falls entirely on you since this is a community code library.
Replaces

manual spreadsheet cleanup and custom regex scripts

Works well with

pandas for data manipulation in Python workflows

Similar repositories
  • cleanlab/cleanlab12k stars

    Finds label errors and data issues in machine learning datasets using statistical methods.

Also reviewed on Repo Radar
  • tobymao/sqlglot9.7k stars

    Reads, rewrites and translates SQL between more than twenty database dialects, without a database.

  • patroni/patroni8.8k stars

    Watches a group of PostgreSQL copies and promotes a new leader when the leader fails.

  • pgEdge/spock758 stars

    An add-on that lets several PostgreSQL databases all accept changes and keep each other in step.

Install
python3 -m venv venv
source venv/bin/activate
pip install dedupe

Independent: reviewed for Repo Radar outside the daily editions. Every independent repository

On Stack Radar: Data & Research › Data Enrichment & Collection › Data Cleaning & Deduplication.

Get the next edition in your inbox

A dozen repositories, opened and checked. The licence read, the last release dated, and the ones that did not make it named with the reason. It is the half most lists leave out.

No tracking pixels. One click to leave. The archive stays free either way.

We use your address to send the edition and nothing else. Confirm by email, leave in one click. How we handle it.