2.2k stars · MIT · v0.9.0 (2026-08-12)
Distills sentence-transformers into static embeddings ~15x smaller and hundreds of times faster on plain CPU.
▶Repo detailsthe review · specs · pros & cons · install
What it is
It distills a sentence-transformer into a static embedding model roughly fifteen times smaller and hundreds of times faster, so producing an embedding becomes a table lookup instead of running a neural network. It needs tens of megabytes of RAM rather than gigabytes, and no GPU.
Why it matters
Every ambitious thing you might want Grasppy to do — clustering, semantic search, topic modelling — bottlenecks on the same step: embedding the text. On a Hetzner CPU box that step is what makes the whole idea feel too expensive. This removes it. You can embed a full transcript in seconds and keep clustering affordable, which quietly makes several features on your roadmap possible instead of theoretical. My favourite find of the day.
- No GPU, no torch at inference — runs comfortably on your existing VPS
- Drop-in replacement for sentence-transformers in BERTopic, Turftopic and vector search
- MIT, actively released, pretrained models ready on Hugging Face
- Static embeddings lose some accuracy, especially on nuance and negation
- Weaker on very short strings and multilingual edge cases — worth testing on Russian text before committing
- Small team behind it; quality depends on which distilled model you pick
You can embed a full transcript in seconds and keep clustering affordable, which quietly makes several features on your roadmap possible instead of theoretical.
Use it as the embedding function inside your FastAPI ingest worker and write the vectors to a pgvector column.
