15.7k stars · Apache-2.0 · v0.4.3 (2026-01-13) — GitHub and PyPI agree · Track this in Rising ↗
Scores how good a retrieval-based AI answer is, so a change can be shown to have helped or hurt.
Radar check
What we found when we opened it. Every bar is one fact, with its date.
Last code pushed 24 Feb 2026, 8 months before this check.
Latest release v0.4.3 on 13 Jan 2026.
Public since May 2023, a little over 3 years.
16,165 stars and 1,780 forks.
0 commits and 38 new issues in the last 90 days.
246 contributors. No pull requests merged in 90 days; changes go in directly.
A security policy says how to report a problem. No published security advisories.
How the bars are scored
- Alive. Days since the last code push: a week or less 100, a month 90, three months 70, six months 50, a year 30, eighteen months 15, older 0.
- Releases. Age of the latest GitHub release, on a similar scale; no bar when the project publishes no releases.
- Maturity. Age of the project: eight years 100, five 90, three 70, two 55, one 40, six months 25, younger 10.
- Adoption. Stars on a log scale: 100,000 stars is 100, 10,000 is 80, 1,000 is 60, 100 is 40.
- Activity. Commits in the last 90 days: 300 or more 100, 100 85, 30 65, 10 45, any 25.
- Community. People who have contributed: 100 or more 100, 30 80, 10 60, 5 40, 2 25, one 10.
- Security. A published security policy 80, none 40; published advisories are listed, not scored.
Checked by Repo Radar on 27 Oct 2026 from the repository on GitHub. Covered in Edition 14.
▶Repo detailsthe review · specs · pros & cons · install
What it does
Ragas is an evaluation library for applications that look up text before answering. It scores things such as whether an answer is supported by the source text, and whether the right parts of the source were found. A test set of questions, source passages and expected answers is supplied, and scores come back for each example and as an average. A language model does most of the judging, which is why each run costs money. It works with plain Python and with the common frameworks for building these applications. Version 0.4.3 was published on 13 January 2026, code last landed on 24 February 2026, and the repository now sits in the vibrantlabsai organisation, with the old address redirecting to it. It does not say how to fix a low score, and it does not watch live traffic.
Why it matters
Who it suits. It suits a team changing how text is retrieved and wanting to know whether the change helped. It suits a small, real test set of twenty to fifty examples drawn from actual use. Skip it before there is a working application to measure, and skip it when the real question is speed and cost rather than answer quality.
Verdict. Worth a day once the answers matter, and the day goes almost entirely on writing the test set, which is the part worth having anyway. Treat the numbers as a comparison between runs rather than as truth, because a language model's scores wobble. The newest code is from February 2026, eight months ago, which is older than most projects here, so check the repository before depending on it. confident-ai/deepeval is the closest alternative, and langfuse/langfuse covers the live traffic this does not.
Summary (Key Points)from the README
Key features
- Objective metrics: Scores LLM app outputs with both LLM-based and traditional metrics, such as the pre-built Aspect Critique metric that uses DiscreteMetric.
- Test data generation: Creates test datasets covering a wide range of scenarios, and the README also mentions production-aligned test set generation.
- Framework integrations: Works with popular LLM frameworks such as LangChain and with major observability tools.
- Feedback loops: Uses production data to keep improving an LLM application.
- Project templates: The `ragas quickstart` command clones a complete example project; the rag_eval template evaluates RAG systems.
- Planned templates: agent_evals, benchmark_llm, prompt_evals and workflow_eval are listed as coming soon.
- Anonymized usage analytics: Collects minimal anonymized usage data, with open-source collection code and publicly available aggregated data.
Ways to install
| Platform or method | What it needs | Effort |
|---|---|---|
| PyPI package | Not stated in the README | Not stated in the README |
| From source (cloned repository) | Not stated in the README | Not stated in the README |
Important
- API key: The quickstart evaluation example requires the OPENAI_API_KEY environment variable to be set.
- Analytics on by default: Usage data is collected unless you set RAGAS_DO_NOT_TRACK to true.
- Partial templates: Only the rag_eval quickstart template is available; the other four are marked as coming soon.
- Missing install details: The README's install commands and supported platforms are not reproduced in the text, so check the documentation before installing.
Written by an AI model from the project’s README; not part of the review, and it can be wrong.
- It turns "this feels better" into a number that can be written in a table.
- It works with plain Python, so it drops into scripts and tests that already exist.
- Apache-2.0 licence, so there is nothing to worry about in a paid product.
- Every run costs money, because a language model does the scoring. Keep the test set to twenty to fifty examples.
- The test set has to be written by hand. That is the real work and it fills most of the first day.
- Scores from a language model wobble between runs, and the newest code is dated February 2026. Compare one run with another, and never treat a single number as the truth.
source venv/bin/activate pip install ragas
