Data and databases · Edition No. 8 · 2 Sep 2026

simonw/sqlite-utils

Pours JSON and CSV into a real SQLite database with no schema design, then indexes it and adds full-text search.

← Data and databasesRead the whole edition →
05

simonw/sqlite-utils

💎 hidden gem

2.2k stars · Apache-2.0 · 4.2.1 (2026-08-13) · Track this in Scout

Pours JSON and CSV into a real SQLite database with no schema design, then indexes it and adds full-text search.

▶Repo detailsthe review · specs · pros & cons

What it is

A command-line tool and Python library that turns JSON and CSV into a real SQLite database without you designing a schema, then lets you query it, index it and add full-text search. It is the plumbing between "I have a folder of files" and "I have a question".

What it is good for. Right now the output of yt-dlp, youtube-transcript-api, youtube-comment-downloader and PRAW is four incompatible piles of JSON, and the questions worth asking cut across all of them: which complaint appears under five different competitors' videos, which phrases show up in both Reddit threads and YouTube comments, what titles the fastest-growing channel in a niche used this quarter. One command per file and all of that is one research.db you can query in SQL, search full-text, and — the part that matters most — hand to Claude Code as a database it can query, instead of forty attachments it has to read. It is ten minutes of work that upgrades three previous editions.

Stars2.2k
LicenceApache-2.0
Latest4.2.1 (2026-08-13)checked 7 Sep 2026
Written inPython
NeedsPostgreSQL and MySQL
RunsLocally, on your own machine
Setup effort
1 / 5
Payoff
4 / 5
Good
  • No schema design. It reads the JSON and creates the columns, which is exactly right for messy scraped data.
  • Built-in full-text search (enable-fts) means "every comment containing 'confusing'" is one query.
  • SQLite is one file: back it up with entry 11, copy it to another machine, open it anywhere. There is no server to run and nothing to break.
Watch for
  • You will be writing SQL, or asking Claude Code to. Not hard, but it is a real skill boundary and the tool does nothing until you cross it.
  • Verification trap: the GitHub releases page stops at 3.39 (November 2025) and looks stale. PyPI shows 4.2.1 on 13 August 2026. Always check PyPI for this project — and note that 3.x → 4.x is a major version, so read the changelog before upgrading an existing database.
  • SQLite is not the right home for a production application; where PostgreSQL is already running, keep this on the research side of the fence.
Replaces

Its sibling Datasette will serve the same file as a browsable web UI if you ever want to click instead of type, and entry 06 will chart it.

Works well with

Standalone on your Mac, alongside the scrapers, which for yt-dlp and the comment downloader is where they have to run anyway.

Screenshots
simonw/sqlite-utils: GitHub preview card

Get the next edition in your inbox

A dozen repositories, opened and checked. The licence read, the last release dated, and the ones that did not make it named with the reason. It is the half most lists leave out.

No tracking pixels. One click to leave. The archive stays free either way.

We use your address to send the edition and nothing else. Confirm by email, leave in one click. How we handle it.