15.7k stars · Apache-2.0 · v0.4.3 (2026-01-13) — GitHub and PyPI agree · Track this in Scout
Scores how good a retrieval-based AI answer is, so a change can be shown to have helped or hurt.
▶Repo detailsthe review · specs · pros & cons · install
What it is
Ragas is an evaluation framework for AI applications that look things up before answering. It measures things like whether the answer is supported by the source text, and whether the right parts of the source were found.
What it is good for. There are several ways to build the retrieval step behind an answer. LightRAG (Ed. 6 #8), pgvector (Ed. 2 #11), BERTopic and turftopic all change the result. Without a score, choosing between them is guessing, and you keep the one you happened to build last. Two earlier entries sit either side of this. Instructor (Ed. 5 #11) makes the output the right shape. Langfuse (Ed. 10 #7) watches what happened after release. This is the missing middle: a number before release that you can compare.
- It turns "does this feel better" into a number you can put in a table.
- It works with tools already in common use, including LangChain and plain Python.
- Apache-2.0 licence, so there is nothing to worry about in a paid product.
- Every run costs money, because a language model does the scoring. Keep the test set small, twenty to fifty examples, not thousands.
- You have to write that test set yourself. That is the real work, and it is most of the day.
- Scores from a language model wobble between runs. Compare one run against another. Never treat a single number as the truth.
source venv/bin/activate pip install ragas
