58.2k stars · MIT, with a separate licence over the enterprise/ directory · v1.100.0 (2026-09-06) · Track this in Scout
One interface over 100+ model providers, with per-key budgets, spend tracking, caching and fallbacks — the meter on the AI bill.
▶Repo detailsthe review · specs · pros & cons · install
What it is
A gateway that puts a hundred-plus model providers behind one OpenAI-shaped interface, as either a Python library or a proxy server you run. The proxy adds the parts that matter operationally: virtual keys, per-key budgets, spend tracking, caching, retries and automatic fallback to a second provider.
What it is good for. For a product built on paid models, unit economics are the thing that decides whether it can have a free tier, and most stacks measure them nowhere. GraphRAG costs dollars an import and BERTopic costs cents — Edition 3 covered that, but without a meter you cannot confirm it, price a plan, or notice the afternoon a bug puts an ingest worker into a retry loop. A gateway with a budget per feature turns that into a dashboard and a hard cap. The caching alone tends to pay for the setup, because during development the same conversation goes through the same prompt twenty times, and nineteen of those are currently full price. Wire it in before shipping an AI feature to strangers rather than after.
- Per-feature virtual keys with budgets and alerts — you can cap a runaway loop instead of discovering it on a statement
- One interface across providers, so switching a step from an expensive model to a cheap one is a config change, not a rewrite
- Response caching and automatic fallback, which cut both your development spend and your outage risk
- Two very different products under one name. The Python library is a small dependency; the proxy wants its own PostgreSQL and Redis, so it is a real install
- MIT except for an
enterprise/directory under a separate licence — check any feature you come to depend on - Enormous surface area and a very fast release cadence. Pin the version and use the tenth of it you need
pip install litellm
