34.3k stars · MIT, except the ee/ directories which are commercially licensed · v4.30.0 (2026-09-04) · Track this in Scout
LiteLLM meters what the AI costs. This one tells you whether it was any good.
▶Repo detailsthe review · specs · pros & cons
What it is
An LLM engineering platform. It traces every model call inside a request, stores prompts as versioned artefacts outside your code, runs evaluations over saved datasets, and gives you a UI to compare outputs across prompt and model versions side by side.
What it is good for. When all of a product's value sits in the quality of one model output, the usual way to judge a prompt change is to open one conversation and have a feeling. Langfuse converts that into twenty real conversations saved as a dataset, a score per run, and a visible diff between prompt v3 and prompt v4 — which is what makes iteration compound rather than wander. It is also the direct answer to the decision this ledger has now deferred for seven editions: BERTopic vs turftopic vs LightRAG vs GraphRAG. You cannot pick a winner without a scoreboard. This is the scoreboard.
- MIT for everything you would actually use, with a first-class LiteLLM (Ed.5 #9) integration — cost and quality land in the same trace.
- Prompt management moves prompts out of the codebase, so tuning wording stops requiring a deploy.
- A free hosted tier means you can evaluate the whole idea before deciding whether it deserves space on a server you maintain.
- Self-hosting v3 and later wants Postgres, ClickHouse, Redis and S3-compatible storage — as heavy as OpenReplay, and you should not stand both up new in the same month.
- The
ee/directories are commercially licensed; read them before assuming the whole repository is MIT. - It ships close to daily. That is momentum, but self-hosters inherit the churn — pin a version.
