Report · agent-observability · updated 2026-09-24
Agent observability and guardrails
Platforms that record agent traces, run evaluations, or enforce runtime guardrails. Included only with public, dated sources. Helicone is included because its acquisition and maintenance-mode statement are themselves evidence of a shipped platform — not a prediction of future investment.
Placements
10 of 10 entities have a Jev placement. Axes: Proven Execution × Validated Direction.
| Entity | Status | Proven Execution | Validated Direction | uX | uY | Region | P(region) |
|---|---|---|---|---|---|---|---|
| LangSmith | scored | 0.542 | 0.580 | 0.347 | 0.324 | OPERATIONAL | OPERATIONAL:0.78 ANCHORED:0.14 FORMING:0.02 DIRECTED:0.06 |
| Langfuse | scored | 0.530 | 0.537 | 0.326 | 0.298 | OPERATIONAL | DIRECTED:0.03 FORMING:0.02 ANCHORED:0.12 OPERATIONAL:0.83 |
| Arize Phoenix | scored | 0.378 | 0.430 | 0.405 | 0.356 | OPERATIONAL | DIRECTED:0.09 ANCHORED:0.01 OPERATIONAL:0.67 FORMING:0.23 |
| Braintrust | scored | 0.635 | 0.445 | 0.337 | 0.349 | OPERATIONAL | DIRECTED:0.29 FORMING:0.02 ANCHORED:0.32 OPERATIONAL:0.37 |
| Helicone | scored | 0.715 | 0.287 | 0.353 | 0.368 | OPERATIONAL | ANCHORED:0.16 OPERATIONAL:0.78 FORMING:0.04 DIRECTED:0.02 |
| Galileo | scored | 0.368 | 0.292 | 0.375 | 0.380 | OPERATIONAL | DIRECTED:0.27 ANCHORED:0.04 FORMING:0.20 OPERATIONAL:0.49 |
| Patronus AI | scored | 0.302 | 0.268 | 0.351 | 0.358 | FORMING | FORMING:0.45 OPERATIONAL:0.18 ANCHORED:0.01 DIRECTED:0.36 |
| AgentOps | scored | 0.185 | 0.177 | 0.327 | 0.327 | FORMING | DIRECTED:0.04 ANCHORED:0.00 FORMING:0.64 OPERATIONAL:0.32 |
| Lakera | scored | 0.190 | 0.190 | 0.325 | 0.326 | FORMING | FORMING:0.75 DIRECTED:0.11 OPERATIONAL:0.14 ANCHORED:0.00 |
| Guardrails AI | scored | 0.227 | 0.180 | 0.343 | 0.325 | FORMING | DIRECTED:0.08 FORMING:0.71 ANCHORED:0.00 OPERATIONAL:0.21 |
Entities
- LangSmith
LangChain’s tracing, evaluation, and agent-operations platform.
3 evidence · scored · OPERATIONAL
- Langfuse
Open-source LLM and agent observability with cloud and self-host options.
3 evidence · scored · OPERATIONAL
- Arize Phoenix
Open-source AI observability with Phoenix Intelligence (PXI) agent.
2 evidence · scored · OPERATIONAL
- Braintrust
Evaluation-first observability and CI quality gates for production AI.
2 evidence · scored · OPERATIONAL
- Helicone
Open-source LLM gateway and observability platform; acquired by Mintlify.
2 evidence · scored · OPERATIONAL
- Galileo
Evaluation, observability, and runtime protection for GenAI applications.
1 evidence · scored · OPERATIONAL
- Patronus AI
Evaluator models and domain-specific testing for LLM applications.
1 evidence · scored · FORMING
- AgentOps
Session-level observability aimed at agent runs.
1 evidence · scored · FORMING
- Lakera
API-first AI security and prompt-injection guardrails.
1 evidence · scored · FORMING
- Guardrails AI
Open-source and commercial validators for LLM outputs.
1 evidence · scored · FORMING
Payment does not affect scores. Submit evidence or read the rubric.