Data and databases · Edition No. 8 · 2 Sep 2026

simonw/sqlite-utils

Pours JSON and CSV into a real SQLite database with no schema design, then indexes it and adds full-text search.

← Data and databasesRead the whole edition →
05

simonw/sqlite-utils

💎 hidden gem

2.2k stars · Apache-2.0 · 4.2.1 (2026-08-13)

Pours JSON and CSV into a real SQLite database with no schema design, then indexes it and adds full-text search.

Repo detailsthe review · specs · pros & cons · install

What it is

A command-line tool and Python library that turns JSON and CSV into a real SQLite database without you designing a schema, then lets you query it, index it and add full-text search. It is the plumbing between "I have a folder of files" and "I have a question".

Why it matters

Right now the output of yt-dlp, youtube-transcript-api, youtube-comment-downloader and PRAW is four incompatible piles of JSON, and the questions you actually want to ask cut across all of them: which complaint appears under five different competitors' videos, which phrases show up in both Reddit threads and YouTube comments, what titles the fastest-growing channel in your niche used this quarter. One command per file and all of that is one research.db you can query in SQL, search full-text, and — the part that matters most for you — hand to Claude Code as a database it can query, instead of forty attachments it has to read. It is ten minutes of work that upgrades three previous editions.

Stars2.2k
LicenceApache-2.0
Latest4.2.1 (2026-08-13)checked 7 Sep 2026
Written inPython
NeedsPostgreSQL and MySQL
RunsLocally, on your own machine
Setup effort
1 / 5
Payoff
4 / 5
Good
  • No schema design. It reads the JSON and creates the columns, which is exactly right for messy scraped data.
  • Built-in full-text search (enable-fts) means "every comment containing 'confusing'" is one query.
  • SQLite is one file: back it up with entry 11, copy it to your Mac, open it anywhere. There is no server to run and nothing to break.
Watch for
  • You will be writing SQL, or asking Claude Code to. Not hard, but it is a real skill boundary and the tool does nothing for you until you cross it.
  • Verification trap: the GitHub releases page stops at 3.39 (November 2025) and looks stale. PyPI shows 4.2.1 on 13 August 2026. Always check PyPI for this project — and note that 3.x → 4.x is a major version, so read the changelog before upgrading an existing database.
  • SQLite is not the right home for the products themselves; you already run PostgreSQL. Keep this on the research side of the fence.
Replaces

Its sibling Datasette will serve the same file as a browsable web UI if you ever want to click instead of type, and entry 06 will chart it.

Works well with

Standalone on your Mac, alongside the scrapers, which for yt-dlp and the comment downloader is where they have to run anyway.

How to get it

Standalone on your Mac, alongside the scrapers, which for yt-dlp and the comment downloader is where they have to run anyway.

Get the next edition in your inbox

A dozen repositories, opened and checked. The licence read, the last release dated, and the ones that did not make it named with the reason. It is the half most lists leave out.

No tracking pixels. One click to leave. The archive stays free either way.

We use your address to send the edition and nothing else. Confirm by email, leave in one click. How we handle it.