LangSmith

Quadrant distribution

Jev `choice` — not a hard 0.5 cut. Confidence 0.70.

• Operational78.0%
• Anchored14.0%
• Forming2.0%
• Directed6.0%

Evidence ledger

Recency half-life 180 days. Weight = 2^(-age/180).

1. announcement · published 2026-05-13 · w=0.595

   SmithDB, the data layer for agent observability

   LangChain launched SmithDB, a Rust/DataFusion store backing LangSmith, stating 100% of US Cloud ingestion and tracing UI query traffic run on it and claiming up to 12× faster core experiences.

2. announcement · published 2026-05-14 · w=0.597

   Everything we shipped at Interrupt

   At Interrupt 2026 LangChain announced LangSmith Engine (autonomous issue clustering and fix proposals, public beta), Managed Deep Agents, LangSmith LLM Gateway for spend limits and sensitive-data detection, and related agent-lifecycle products.

3. release · published 2026-07-21 · w=0.776

   Trace voice agents in LangSmith

   LangSmith added Python tracing integrations for Pipecat, LiveKit, OpenAI Realtime, and Gemini Live, capturing conversation audio, STT/TTS, interruptions, and tool calls.

Reproducibility

resolved model: jev-1.13.0

requested model: jev-latest

request id: req_01a0d5e179c979c6b88de4161c850521

timestamp: 2026-09-25T00:05:17.859Z

request hash: a6d9109974c5326bb04794df7cc9717493cc7b34aaa0fba5c03670e8a6103271

{
  "model": "jev-latest",
  "state": {
    "goal": "Score this entity on the Jev Quadrant rubric using only the supplied evidence.",
    "framework": "jev-quadrant-v1",
    "topic": {
      "id": "agent-observability",
      "title": "Agent observability and guardrails",
      "x_axis": "Proven Execution",
      "y_axis": "Validated Direction"
    },
    "entity": {
      "id": "langsmith",
      "name": "LangSmith",
      "website": "https://www.langchain.com/langsmith",
      "description": "LangChain’s tracing, evaluation, and agent-operations platform."
    },
    "evidence": [
      {
        "id": "smithdb",
        "kind": "announcement",
        "url": "https://www.langchain.com/blog/introducing-smithdb",
        "date": "2026-05-13",
        "date_kind": "published",
        "recency_weight": 0.5946,
        "summary": "LangChain launched SmithDB, a Rust/DataFusion store backing LangSmith, stating 100% of US Cloud ingestion and tracing UI query traffic run on it and claiming up to 12× faster core experiences.",
        "title": "SmithDB, the data layer for agent observability"
      },
      {
        "id": "interrupt-2026",
        "kind": "announcement",
        "url": "https://www.langchain.com/blog/interrupt-2026-overview",
        "date": "2026-05-14",
        "date_kind": "published",
        "recency_weight": 0.5969,
        "summary": "At Interrupt 2026 LangChain announced LangSmith Engine (autonomous issue clustering and fix proposals, public beta), Managed Deep Agents, LangSmith LLM Gateway for spend limits and sensitive-data detection, and related agent-lifecycle products.",
        "title": "Everything we shipped at Interrupt"
      },
      {
        "id": "voice-tracing",
        "kind": "release",
        "url": "https://www.langchain.com/blog/trace-voice-agents-in-langsmith",
        "date": "2026-07-21",
        "date_kind": "published",
        "recency_weight": 0.7756,
        "summary": "LangSmith added Python tracing integrations for Pipecat, LiveKit, OpenAI Realtime, and Gemini Live, capturing conversation audio, STT/TTS, interruptions, and tool calls.",
        "title": "Trace voice agents in LangSmith"
      }
    ],
    "claims": [],
    "independent_checks": [],
    "recency": {
      "half_life_days": 180,
      "as_of": "2026-09-25",
      "mass": 1.9671
    }
  },
  "questions": {
    "exec_shipped": {
      "type": "noul",
      "instructions": "Use only the supplied state. Do not use prior knowledge that is not represented as evidence items. If evidence is thin, return a probability near 0.5 with the understanding that uncertainty is handled separately. Has this entity shipped a generally available product used by third parties?",
      "criteria": {
        "true": "Independent public evidence shows a generally available product used by third parties.",
        "false": "The product is announced, preview-only, or evidenced only by unsourced vendor marketing with no third-party use."
      }
    },
    "exec_adoption": {
      "type": "noul",
      "instructions": "Use only the supplied state. Do not use prior knowledge that is not represented as evidence items. If evidence is thin, return a probability near 0.5 with the understanding that uncertainty is handled separately. Is there credible adoption evidence?",
      "criteria": {
        "true": "Credible adoption evidence exists: named customers, published user or download counts, case studies, or equivalent.",
        "false": "Adoption claims are absent, purely anecdotal, or unsourced."
      }
    },
    "exec_reliability": {
      "type": "noul",
      "instructions": "Use only the supplied state. Do not use prior knowledge that is not represented as evidence items. If evidence is thin, return a probability near 0.5 with the understanding that uncertainty is handled separately. Is there evidence of reliability, quality, security/compliance, or operational maturity?",
      "criteria": {
        "true": "Evidence of reliability, quality, security/compliance, or operational maturity (uptime, incidents handled, certifications, long-running open-source project).",
        "false": "No reliability or quality evidence is present."
      }
    },
    "exec_cadence": {
      "type": "noul",
      "instructions": "Use only the supplied state. Do not use prior knowledge that is not represented as evidence items. If evidence is thin, return a probability near 0.5 with the understanding that uncertainty is handled separately. Is there evidence of continued delivery after launch?",
      "criteria": {
        "true": "Evidence of continued delivery after launch (dated releases, changelog, subsequent features).",
        "false": "Delivery appears one-shot or stalled."
      }
    },
    "dir_strategy": {
      "type": "noul",
      "instructions": "Use only the supplied state. Do not use prior knowledge that is not represented as evidence items. If evidence is thin, return a probability near 0.5 with the understanding that uncertainty is handled separately. Is there a specific, consistent, dated public strategy or roadmap?",
      "criteria": {
        "true": "A public strategy or roadmap is specific, consistent, and dated.",
        "false": "Direction is vague, contradictory, or only implied."
      }
    },
    "dir_backed": {
      "type": "noul",
      "instructions": "Use only the supplied state. Do not use prior knowledge that is not represented as evidence items. If evidence is thin, return a probability near 0.5 with the understanding that uncertainty is handled separately. Are strategy or roadmap claims backed by shipped artifacts in the evidence pack?",
      "criteria": {
        "true": "Roadmap or strategy claims are backed by shipped artifacts in the evidence pack.",
        "false": "Direction is promised without corresponding delivery evidence."
      }
    },
    "dir_distinct": {
      "type": "noul",
      "instructions": "Use only the supplied state. Do not use prior knowledge that is not represented as evidence items. If evidence is thin, return a probability near 0.5 with the understanding that uncertainty is handled separately. Is the evidenced direction distinct from generic category copy?",
      "criteria": {
        "true": "The evidenced direction is distinct from generic category copy.",
        "false": "Positioning is interchangeable with peers."
      }
    },
    "dir_invested": {
      "type": "noul",
      "instructions": "Use only the supplied state. Do not use prior knowledge that is not represented as evidence items. If evidence is thin, return a probability near 0.5 with the understanding that uncertainty is handled separately. Is there evidence of sustained investment in this product?",
      "criteria": {
        "true": "Evidence of sustained investment (funding used for the product, research, open-source velocity, or platform expansion).",
        "false": "No evidence of continued investment."
      }
    },
    "quadrant": {
      "type": "choice",
      "instructions": "Use only the supplied state. Do not use prior knowledge that is not represented as evidence items. If evidence is thin, return a probability near 0.5 with the understanding that uncertainty is handled separately. Assign the entity to one Jev Quadrant cell. Do not use a hard 0.5 cut. Weigh the full evidence pack.",
      "criteria": {
        "ANCHORED": "Both delivery (proven execution) and evidenced direction (validated direction) are strong.",
        "DIRECTED": "Validated direction is better evidenced than proven execution.",
        "OPERATIONAL": "Proven execution is better evidenced than validated direction.",
        "FORMING": "Both axes are thin, early, or contradicted."
      }
    }
  }
}

Challenges

Anyone can submit a source. Payment never affects review.

No pending or accepted challenges yet.