58.2k stars · MIT, with a separate licence over the enterprise/ directory · v1.100.0 (2026-09-06)
One interface over 100+ model providers, with per-key budgets, spend tracking, caching and fallbacks — the meter on the AI bill.
▶Repo detailsthe review · specs · pros & cons · install
What it is
A gateway that puts a hundred-plus model providers behind one OpenAI-shaped interface, as either a Python library or a proxy server you run. The proxy adds the parts that matter operationally: virtual keys, per-key budgets, spend tracking, caching, retries and automatic fallback to a second provider.
Why it matters
Grasppy's unit economics are the thing that decides whether it can have a free tier, and right now nothing in your stack measures them. GraphRAG costs dollars an import and BERTopic costs cents — Edition 3 told you that, but you cannot confirm it, price a plan, or notice the afternoon a bug puts your ingest worker into a retry loop. A gateway with a budget per feature turns that into a dashboard and a hard cap. The caching alone tends to pay for the setup, because during development you re-run the same conversation through the same prompt twenty times, and nineteen of those are currently full price. Wire it in before you ship an AI feature to strangers rather than after.
- Per-feature virtual keys with budgets and alerts — you can cap a runaway loop instead of discovering it on a statement
- One interface across providers, so switching a step from an expensive model to a cheap one is a config change, not a rewrite
- Response caching and automatic fallback, which cut both your development spend and your outage risk
- Two very different products under one name. The Python library is a small dependency; the proxy wants its own PostgreSQL and Redis, so it is a real install
- MIT except for an
enterprise/directory under a separate licence — check any feature you come to depend on - Enormous surface area and a very fast release cadence. Pin the version and use the tenth of it you need
source venv/bin/activate pip install litellm