Hayz Labs · https://hayzlabs.com/reviews/top/evals-observability/
reviews · top 20 list · checked 18 Oct
Top tools for testing and tracing AI apps
Tools that show you what your AI app did, score its answers, and let you test prompts before you ship.
Nothing on this list is scored yet: no outside source we can turn into a number covers them. The order below follows the public measure named on each row.
By share of developers using it in the Stack Overflow 2026 survey; the two tools not in the survey follow by PyPI downloads. Contains information from the Stack Overflow Developer Survey 2026, which is made available under the Open Database License (ODbL). Open the Table to see the number that placed each one and how it was scored.
- 1LangSmithLangChain
Hosted tracing and evals from the LangChain team; the easy fit if your app is built on LangChain or LangGraph.
Best for Apps built on LangChain or LangGraph, though it works with any frameworkYou give up The free plan is 1 seat and 5k traces a month; Plus costs $39 per seat per month
Not scored yetFree - 2MLflowDatabricks (MLflow project)
Free classic ML tracker that now also traces and scores AI apps; a natural pick if you already use MLflow.
Best for Teams wanting one free tool for model training records and AI app tracesYou give up Not scored yet: recent write-ups found are based on docs, not hands-on tests
Not scored yetFree - 3LangfuseLangfuse (ClickHouse)
Open-source tracing, prompts and evals to self-host; ClickHouse bought it in Jan 2026, says core stays MIT.
Best for Teams that want to self-host tracing and prompt management for freeYou give up Some enterprise features are licensed separately; now owned by ClickHouse
Not scored yetFree - 4Datadog LLM ObservabilityDatadog
Shows AI app traces next to the rest of your Datadog monitoring; best if your team already uses Datadog.
Best for Teams that want AI traces next to their existing app monitoringYou give up Pro costs $160 a month for up to 100k traced steps; the free plan stops at 40k
Not scored yetFree - 5W&B WeaveCoreWeave (Weights & Biases)
Tracing and evals from Weights & Biases, now part of CoreWeave; best for teams already on Weights & Biases.
Best for Model teams that want AI app traces in the same place as training runsYou give up Free plan allows 1 GB a month; pricing has moved under CoreWeave Forge; not scored yet
Not scored yetFree - 6Arize PhoenixArize AI
Free tracing and evals app built on OpenTelemetry; Elastic licence, so source-available, not OSI open source.
Best for Teams that want free self-hosted tracing built on an open tracing standardYou give up Source-available, not true open source; not scored yet: no hands-on outside test
Not scored yetFree - 7BraintrustBraintrust
Hosted platform built around evals, free tier with unlimited users; good when testing before release is key.
Best for Teams that test AI changes before release and want everyone on one free planYou give up Big jump from free to Pro at $249 a month; free plan caps at 1 GB and 10k scores
Not scored yetFree - 8DeepEvalConfident AI
Free Python library that checks AI answers like unit tests, with ready-made scores for RAG apps and agents.
Best for Python developers adding pass or fail checks on AI answers to their testsYou give up A library, not a hosted service; not scored yet: recent write-ups are docs-based
Not scored yetFree - 9promptfooPromptfoo (OpenAI deal announced)
Free command-line tool to test prompts and red-team AI apps before release; OpenAI agreed to buy it in March.
Best for Checking prompts and agents for security problems on every code changeYou give up Command line only; its owner is changing as OpenAI agreed to buy it; not scored yet
Not scored yetFree - 10RagasRagas team (vibrantlabsai)
Free scores for RAG apps: did it find the right text and stick to it. No new release since January 2026.
Best for Scoring document search apps on whether answers stick to their sourcesYou give up Aimed at document search apps only; no new release since January 2026; not scored yet
Not scored yetFree - 11OpikComet
Comet's tracing and evals platform; all of it is Apache-2.0 and free to self-host, with a free cloud tier.
Best for Teams that want a fully open, free self-hosted option with a free cloud fallbackYou give up Used by 3.7% in the 2026 Stack Overflow survey, fewer than most here; not scored yet
Not scored yetFree - 12LaminarLaminar (lmnr-ai)
Open-source tracing built for AI agents, including browser agents; a small project that releases often.
Best for Teams tracing AI agents, including agents that use browsersYou give up Small project with 3.2k stars; its download count may include installs by other tools
Not scored yetFree - 13Inspect AIUK AI Security Institute
UK AI Security Institute's free framework for careful model and agent evals; suits researchers over app teams.
Best for Researchers who want repeatable, detailed tests of models and agentsYou give up Built for research more than app monitoring; not in the 2026 survey; not scored yet
Not scored yetFree
| # | Name | Score | What placed it | Price | Best for | You give up |
|---|---|---|---|---|---|---|
| 1 | LangSmithLangChain · Closed | Not scored | Used by about 17% of developers asked (Stack Overflow survey 2026)NotesTheir figure: SO 2026 AI observability/evals 17.3% (393 of 2,277). not scored: no hands-on independent test since 2026-07-20; recent write-ups (Majestic Labs 2026-08-19, MarkTechPost 2026-08-09) are docs-based Report a mistake | Developer free (1 seat, 5k traces/mo); Plus $39/seat/mo | Apps built on LangChain or LangGraph, though it works with any framework | The free plan is 1 seat and 5k traces a month; Plus costs $39 per seat per month |
| 2 | MLflowDatabricks (MLflow project) · Open source | Not scored | Used by about 15% of developers asked (Stack Overflow survey 2026)NotesTheir figure: SO 2026 AI observability/evals 15.2% (347 of 2,277). not scored: no hands-on independent test since 2026-07-20; recent write-ups (Majestic Labs 2026-08-19, MarkTechPost 2026-08-09) are docs-based Report a mistake | Free (open source) | Teams wanting one free tool for model training records and AI app traces | Not scored yet: recent write-ups found are based on docs, not hands-on tests |
| 3 | LangfuseLangfuse (ClickHouse) · Open source | Not scored | Used by about 15% of developers asked (Stack Overflow survey 2026)NotesTheir figure: SO 2026 AI observability/evals 15.2% (345 of 2,277). not scored: no hands-on independent test since 2026-07-20; recent write-ups (Majestic Labs 2026-08-19, MarkTechPost 2026-08-09) are docs-based; AIMultiple overhead test dated 2026-06-03, too old Report a mistake | Open source (MIT core); Cloud Hobby free (50k units/mo); Core $29/mo | Teams that want to self-host tracing and prompt management for free | Some enterprise features are licensed separately; now owned by ClickHouse |
| 4 | Datadog LLM ObservabilityDatadog · Closed | Not scored | Used by about 12% of developers asked (Stack Overflow survey 2026)NotesTheir figure: SO 2026 'Datadog LLM' 11.9% (271 of 2,277). not scored: no hands-on independent test since 2026-07-20; recent write-ups (Majestic Labs 2026-08-19, MarkTechPost 2026-08-09) are docs-based Report a mistake | Free (up to 40k LLM spans/mo); Pro $160/mo (up to 100k spans) | Teams that want AI traces next to their existing app monitoring | Pro costs $160 a month for up to 100k traced steps; the free plan stops at 40k |
| 5 | W&B WeaveCoreWeave (Weights & Biases) · Closed | Not scored | Used by about 11% of developers asked (Stack Overflow survey 2026)NotesTheir figure: SO 2026 'Weights & Biases' 10.7% (244 of 2,277); counts the whole W&B brand, not only Weave. not scored: no hands-on independent test since 2026-07-20; recent write-ups (Majestic Labs 2026-08-19, MarkTechPost 2026-08-09) are docs-based Report a mistake | Free (1 GB/mo ingestion); Pro from $60/mo | Model teams that want AI app traces in the same place as training runs | Free plan allows 1 GB a month; pricing has moved under CoreWeave Forge; not scored yet |
| 6 | Arize PhoenixArize AI · Open source | Not scored | Used by about 6% of developers asked (Stack Overflow survey 2026)NotesTheir figure: SO 2026 'Arize AI' 5.8% (133 of 2,277); covers Phoenix and paid Arize AX. not scored: no hands-on independent test since 2026-07-20; recent write-ups (Majestic Labs 2026-08-19, MarkTechPost 2026-08-09) are docs-based; licence Elastic-2.0, not OSI Report a mistake | Free to self-host (Elastic-2.0 licence) | Teams that want free self-hosted tracing built on an open tracing standard | Source-available, not true open source; not scored yet: no hands-on outside test |
| 7 | BraintrustBraintrust · Closed | Not scored | Used by about 6% of developers asked (Stack Overflow survey 2026)NotesTheir figure: SO 2026 AI observability/evals 5.8% (132 of 2,277). not scored: no hands-on independent test since 2026-07-20; recent write-ups (Majestic Labs 2026-08-19, MarkTechPost 2026-08-09) are docs-based Report a mistake | Starter free (1 GB data, 10k scores); Pro $249/mo | Teams that test AI changes before release and want everyone on one free plan | Big jump from free to Pro at $249 a month; free plan caps at 1 GB and 10k scores |
| 8 | DeepEvalConfident AI · Open source | Not scored | Used by about 6% of developers asked (Stack Overflow survey 2026)NotesTheir figure: SO 2026 AI observability/evals 5.8% (131 of 2,277). not scored: no hands-on independent test since 2026-07-20; recent write-ups (Majestic Labs 2026-08-19, MarkTechPost 2026-08-09) are docs-based Report a mistake | Free (open source) | Python developers adding pass or fail checks on AI answers to their tests | A library, not a hosted service; not scored yet: recent write-ups are docs-based |
| 9 | promptfooPromptfoo (OpenAI deal announced) · Open source | Not scored | Used by about 5% of developers asked (Stack Overflow survey 2026)NotesTheir figure: SO 2026 AI observability/evals 5.3% (121 of 2,277). not scored: no hands-on independent test since 2026-07-20; recent write-ups (Majestic Labs 2026-08-19, MarkTechPost 2026-08-09) are docs-based Report a mistake | Free (open source) | Checking prompts and agents for security problems on every code change | Command line only; its owner is changing as OpenAI agreed to buy it; not scored yet |
| 10 | RagasRagas team (vibrantlabsai) · Open source | Not scored | Used by about 5% of developers asked (Stack Overflow survey 2026)NotesTheir figure: SO 2026 AI observability/evals 4.9% (111 of 2,277). not scored: no hands-on independent test since 2026-07-20; recent write-ups (Majestic Labs 2026-08-19, MarkTechPost 2026-08-09) are docs-based; no PyPI release since 0.4.3 on 2026-01-13 Report a mistake | Free (open source) | Scoring document search apps on whether answers stick to their sources | Aimed at document search apps only; no new release since January 2026; not scored yet |
| 11 | OpikComet · Open source | Not scored | Used by about 4% of developers asked (Stack Overflow survey 2026)NotesTheir figure: SO 2026 AI observability/evals 3.7% (84 of 2,277). not scored: no hands-on independent test since 2026-07-20; recent write-ups (Majestic Labs 2026-08-19, MarkTechPost 2026-08-09) are docs-based Report a mistake | Free to self-host (Apache-2.0); free Comet cloud account | Teams that want a fully open, free self-hosted option with a free cloud fallback | Used by 3.7% in the 2026 Stack Overflow survey, fewer than most here; not scored yet |
| 12 | LaminarLaminar (lmnr-ai) · Open source | Not scored | About 5.5 million downloads a month from PyPI, Python's package storeNotesTheir figure: PyPI lmnr 5.52M downloads in 30 days to 2026-10-01 (not in SO survey; high next to 3.2k stars, may count installs pulled in by other tools). not scored: no hands-on independent test since 2026-07-20; recent write-ups (Majestic Labs 2026-08-19, MarkTechPost 2026-08-09) are docs-based Report a mistake | Code Apache-2.0; Cloud free (1 GB); Starter $30/mo | Teams tracing AI agents, including agents that use browsers | Small project with 3.2k stars; its download count may include installs by other tools |
| 13 | Inspect AIUK AI Security Institute · Open source | Not scored | About 2.8 million downloads a month from PyPI, Python's package storeNotesTheir figure: PyPI inspect-ai 2.80M downloads in 30 days to 2026-10-01 (not in SO survey). not scored: no hands-on independent test since 2026-07-20; recent write-ups (Majestic Labs 2026-08-19, MarkTechPost 2026-08-09) are docs-based Report a mistake | Free (open source) | Researchers who want repeatable, detailed tests of models and agents | Built for research more than app monitoring; not in the 2026 survey; not scored yet |
Nothing matches all of that. , or try everything.
How we ranked this
Order. By share of developers using it in the Stack Overflow 2026 survey; the two tools not in the survey follow by PyPI downloads. Contains information from the Stack Overflow Developer Survey 2026, which is made available under the Open Database License (ODbL).
How we score this list. Our checklist. Recent write-ups compare documentation, not measured runs, so every row is Not scored. The full rules are on How we score. Open "How it was scored" on any row to see the working.
What we did not do. We did not run any of these ourselves. Every number comes from the public source linked on its row, checked on 18 Oct 2026.
Spot a mistake? A wrong rank, price or link: tell us and we will check it against the source. Corrections are dated in News.
Reviews are drafted with help from AI models, including Anthropic's Claude. Model scores come only from outside leaderboards and independent evaluations, by a written rule.