2.2k stars · MIT · v0.9.0 (2026-08-12) · Track this in Scout
Distills sentence-transformers into static embeddings ~15x smaller and hundreds of times faster on plain CPU.
▶Repo detailsthe review · specs · pros & cons
What it is
It distills a sentence-transformer into a static embedding model roughly fifteen times smaller and hundreds of times faster, so producing an embedding becomes a table lookup instead of running a neural network. It needs tens of megabytes of RAM rather than gigabytes, and no GPU.
What it is good for. Clustering, semantic search and topic modelling all bottleneck on the same step: embedding the text. On a plain CPU server that step is what makes the whole idea feel too expensive. This removes it. You can embed a full transcript in seconds and keep clustering affordable, which quietly moves several features from theoretical to possible. My favourite find of the day.
- No GPU, no torch at inference — runs comfortably on a modest CPU server
- Drop-in replacement for sentence-transformers in BERTopic, Turftopic and vector search
- MIT, actively released, pretrained models ready on Hugging Face
- Static embeddings lose some accuracy, especially on nuance and negation
- Weaker on very short strings and multilingual edge cases — worth testing on Russian text before committing
- Small team behind it; quality depends on which distilled model you pick
You can embed a full transcript in seconds and keep clustering affordable, which quietly makes several features on your roadmap possible instead of theoretical.
