15.7k stars · Apache-2.0 · v0.4.3 (2026-01-13) — GitHub and PyPI agree
Scores how good a retrieval-based AI answer is, so a change to Grasppy can be shown to have helped or hurt.
▶Repo detailsthe review · specs · pros & cons · install
What it is
Ragas is an evaluation framework for AI applications that look things up before answering. It measures things like whether the answer is supported by the source text, and whether the right parts of the source were found.
Why it matters
Grasppy's entire pitch is the map. You have several ways to change how it is built. LightRAG (Ed. 6 #8), pgvector (Ed. 2 #11), BERTopic and turftopic all change the result. Without a score, choosing between them is guessing, and you will keep the one you happened to build last. Two earlier entries sit either side of this. Instructor (Ed. 5 #11) makes the output the right shape. Langfuse (Ed. 10 #7) watches what happened after release. This is the missing middle: a number before release that you can compare.
- It turns "does this feel better" into a number you can put in a table.
- It works with what you already use, including LangChain and plain Python.
- Apache-2.0 licence, so there is nothing to worry about in a paid product.
- Every run costs money, because a language model does the scoring. Keep the test set small, twenty to fifty examples, not thousands.
- You have to write that test set yourself. That is the real work, and it is most of the day.
- Scores from a language model wobble between runs. Compare one run against another. Never treat a single number as the truth.
source venv/bin/activate pip install ragas