In simple English
Moving data between databases is often slow and hard to set up. PeerDB lets you stream data from PostgreSQL to ClickHouse in real time. You control the syncs using standard SQL commands instead of complex configuration files. It runs locally with Docker and handles large tables quickly. You need some server management experience to run and monitor the containers.
What it is
PeerDB (a data replication tool built in Go) streams database tables continuously using change data capture. It provides a Postgres-compatible SQL interface to manage sync jobs and transformations.
What it is good for
Solo builders who need to move large volumes of transactional data from PostgreSQL into ClickHouse for fast analytics without writing custom pipelines.
- Streams data in real time with low latency.
- Uses standard SQL commands for setting up data movement.
- Handles large tables and complex data types efficiently.
- Requires Docker and Docker Compose installed on your machine.
- Consumes significant memory when running multiple database containers.
- Licensed under the AGPL, which requires releasing source code if you offer it as a hosted service.
custom Python data sync scripts
ClickHouse
- airbytehq/airbyte22k stars
A massive data integration platform with hundreds of connectors, but much heavier to run.
- meltano/meltano2.7k stars
A command-line tool for managing data pipelines using Singer taps and targets.
- meilisearch/meilisearch59k stars
A search engine you run yourself, built for fast answers and forgiving spelling.
- beekeeper-studio/beekeeper-studio24k stars
A desktop window onto a database, so you can read and edit rows without writing commands.
- sqliteai/sqlite-vector1.1k stars
An add-on that gives the single-file SQLite database meaning-based search, with its own accuracy figures printed in the README.
git clone https://github.com/PeerDB-io/peerdb.git cd peerdb bash ./run-peerdb.sh