⚡Stack Radar

AI & Automation · AI Development & Evaluation

AI Observability

9 paid against 11 free you can run yourself.

Paid against free › AI & Automation › AI Development & Evaluation › AI Observability · AI and models on Repo Radar

AI observability tools record what LLM apps and agents do (every model call, tool call and step), score the results and show cost, so teams can debug and improve them in production. The paid options include framework-specific platforms (LangSmith), open-core services with hosted and self-hosted versions (Langfuse, Arize, Opik, Laminar), evaluation-first tools (Braintrust), proxy-based logging (Helicone), OpenTelemetry-based observability (Pydantic Logfire) and enterprise platforms (HoneyHive). On the open-source side Langfuse, Phoenix, Opik, MLflow, Laminar, OpenLIT and Latitude can be self-hosted, and OpenLLMetry provides standard instrumentation for any backend. Marks come from vendor pricing pages and project READMEs as of 19 October 2026; nothing was tested hands-on.

Paid 9 reviewed

Prices and free tiers as checked, dated on each review.

LLangSmith

Platform for agent engineering: observe, evaluate, and deploy AI agents.

free tierLLangfuse

Open-source AI engineering platform for tracing, evaluating, and improving LLM applications.

$29free tierAArize AI

Arize AI is an agent observability and evaluation platform for AI teams.

$50free tierBBraintrust

Active observability platform for instrumenting, understanding, and improving AI agents.

$249free tierHHelicone

AI gateway and LLM observability platform for routing, debugging, and analyzing AI applications.

$79free tierHHoneyHive

Agent observability and evaluation platform for enterprises to trace, evaluate, and improve production AI…

free tierLLaminar

Open-source observability platform for AI agents with tracing, evals, and alerts.

$30free tierOOpik

Open-source LLM observability and evaluation platform by Comet.

$19free tierPPydantic Logfire

AI observability platform for tracing, monitoring, and optimizing AI agents and applications.

$49free tier

Free 11 direct · 6 related

Free to use. You run it on your own server.

Direct alternatives

They do this job.

LAlangfuse/langfuse

A platform that traces language model calls, versions the prompts, and scores the output.

★ 35.7kFreeMLmlflow/mlflow

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

★ 28.4kFreeOPcomet-ml/opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

★ 22.7kFreeOPtraceloop/openllmetry

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

★ 7.5kFreeHEHelicone/helicone

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

★ 6.2kFreeEMapple/embedding-atlas

Apple's interactive embedding map with automatic cluster labelling — worth studying as a design reference for an overview screen rather than adding as a dependency.

★ 5kFreeLAlatitude-dev/latitude-llm

Open-source observability for AI agents. Find where your agents fail, dispatch your coding agent to fix it, and verify the fix against real traces.

★ 4.7kFreeLOpydantic/logfire

AI observability platform for production LLM and agent systems.

★ 4.5kFreeLMlmnr-ai/lmnr

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

★ 3.4kFreeOPopenlit/openlit

OpenLIT is the open-source agent harness engineering platform: trace, evaluate, guard, and improve everything around the model in your AI agents, on OTEL.

★ 2.9kFreeDATutteInstitute/datamapplot

A Python library that draws readable, labelled maps of text collections.

★ 1kFree

Related alternatives

A second choice: they cover part of this job or are a building block for it.

LIBerriAI/litellm

A gateway that puts a hundred or more model providers behind one interface, with spend tracking and limits.

★ 61kFreeUMumami-software/umami

Self-installed website analytics with no cookies, running on an existing PostgreSQL database.

★ 39.3kFreePRpromptfoo/promptfoo

A test runner for prompts: you describe a good answer once, and it grades every model against it.

★ 26.1kFreeDEconfident-ai/deepeval

Tests for language-model output, written and run like ordinary unit tests.

★ 18.8kFreeLMEleutherAI/lm-evaluation-harness

Runs standard question sets against a language model and reports the scores.

★ 14.3kFreePEpezzolabs/pezzo

🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.

★ 3.3kFree

Feature by feature

Every paid tool and free repository we compared, marked against the same features.

✓has itPartlysome plans or extra work—confirmed not available?not confirmed✓the vendor's claim onlypricingClick a mark for its note and source

Paid · 9Free · 10
Featurelangfuse/langfuseArize-ai/phoenixcomet-ml/opikHelicone/heliconemlflow/mlflowtraceloop/openllmetrylmnr-ai/lmnropenlit/openlitpydantic/logfirelatitude-dev/latitude-llm
Starts at$39 per seat per month (Plus)$29 per month (100k units)$50 per month (AX Pro)$249 per month (Pro)$79 per month (Pro)Enterprise by quote$30 per month plus $2 per GB$19 per month (Pro Cloud)$49 per month (Team)MITElasticApache-2.0Apache-2.0Apache-2.0Apache-2.0Apache-2.0Apache-2.0MITMIT
Observe
Tracing of LLM calls and agentsRecords each model call, tool call and agent step as a trace.
Evaluate
EvaluationsScores outputs with LLM-as-judge or code evaluators.
Improve
Prompt management or playgroundVersion prompts or try them in a playground.
Evaluate
Datasets and experimentsTest sets and side-by-side experiments.
Integration
OpenTelemetry supportAccepts or emits standard OpenTelemetry traces.
Observe
Cost and token trackingShows tokens and spend per call or user.
Integration
Gateway or proxy loggingCaptures traffic by routing model calls through a proxy.
Observe
Sessions and user trackingGroups traces by conversation or end user.
Alerts and monitoringDashboards and alerts on quality, cost or errors.
Deployment
Self-hostingRun it on your own infrastructure.
Pricing
Free tierA free plan with real usage included.
Usage-based pricingPay for traces, spans, events or data volume beyond the plan.
Teams
SSO and RBACSingle sign-on or role-based access.
Data retention controlsChoose how long trace data is kept.
Licence
Open sourceSource published under an open or source-available licence.
Pricing
No per-seat chargePaid plans include unlimited team members.

Excluded candidates

Looked at, and left out of the comparison, with the reason.

Paid apps

  • Weights & Biases WeaveWeights & Biases' pricing address now redirects to CoreWeave Forge pricing, so Weave has no standalone observability plan to compare.Source
  • GalileoGalileo's pricing address now redirects to Splunk's product pricing page, so it has no standalone plan to compare.Source

Repositories

  • apple/embedding-atlasEmbedding Atlas gives interactive visualisations of large embedding collections. It is an embedding exploration tool, not observability for running AI apps.Source
  • TutteInstitute/datamapplotDataMapPlot creates plots of data maps from embeddings. It is a visualisation library rather than an observability tool.Source
  • Agenta-AI/agentaAgenta now describes itself as a workspace for building agents and automations, so it fits agent builders better than observability.Source
  • promptfoo/promptfoopromptfoo tests prompts, agents and RAG apps and runs red-teaming scans. It is a testing and security tool rather than production observability.Source
  • confident-ai/deepevalDeepEval is an LLM evaluation framework for tests. It covers evaluation only, better compared with evaluation tools.Source
  • evidentlyai/evidentlyEvidently is an ML and LLM evaluation and monitoring framework. It overlaps Opik and Phoenix but is more focused on classic ML data drift.Source
  • wandb/weaveWeave is a toolkit from Weights & Biases for developing AI apps with tracing. It overlaps Opik and Langfuse with a much smaller open-source community.Source
  • coze-dev/coze-loopCoze Loop is an agent optimisation platform from ByteDance's Coze. It overlaps Langfuse and Opik and is tied to the Coze ecosystem.Source
  • VoltAgent/voltagentVoltAgent is a TypeScript agent framework with observability built in. It is an agent framework first, better compared with agent frameworks.Source
  • FailproofAI/failproofaiFailproof AI captures and enforces behaviour of AI agent harnesses. It is closer to agent guardrails than to general LLM observability.Source
  • maximhq/bifrostBifrost is a high-performance AI gateway. It logs traffic as a side effect of routing, which fits LLM gateways better than observability.Source

Gaps

What the research could not settle from the vendors' own pages.

What vendors don't publish

  • Observability vendors bill in different units (traces, units, spans, events, GB of data, records, requests), so the cost of observing the same app cannot be compared without each calculator.LangSmith, Langfuse, Arize AI, Braintrust, Helicone, Laminar, Pydantic Logfire · checked 2026-10-19 · source 1 · source 2 · source 3 · source 4
  • HoneyHive publishes only its free tier; every paid option is by quote, so teams cannot estimate cost before talking to sales.HoneyHive · checked 2026-10-19 · source 1

Paid against free, in your inbox

One job at a time: the paid tools worth paying for, the free ones you can run yourself, and who should pick which.

No tracking pixels. One click to leave. We use your address to send Stack Radar and nothing else. Privacy.