LangSmith integrates TypeSafe Jev for faster agent evals

LangChain has added TypeSafe AI's Jev model as an evaluation judge within its LangSmith platform. Unlike traditional large language models that generate free-text reasoning, Jev is a System One model designed to make rapid, structured decisions. It evaluates states, such as agent traces or messages, and returns typed answers and probabilities instead of text blocks. This allows developers to track structured feedback on open-ended agent behaviors directly within LangSmith.

In comparative testing against frontier models, Jev demonstrated significant advantages in speed, cost, and consistency. Evaluated alongside GPT-5.6 Luna, GPT-5.6 Terra, and Claude Sonnet 4.6, Jev matched a human reviewer on every decision while exhibiting 92-913x lower variance than the LLM judges. Jev averaged just 0.44 seconds per call, compared to the 2.16 to 2.83 seconds required by the LLMs. Financially, Jev cost $0.00035 per call, bringing the total cost for the test suite to $0.34. In contrast, the same evaluations cost $0.39 with GPT-5.6 Luna, $2.90 with GPT-5.6 Terra, and $28.17 with Claude Sonnet 4.6. TypeSafe AI claims Jev can be up to 450x cheaper and 200x faster than comparable LLMs on classification tasks.

For practitioners, this integration addresses the high latency and prohibitive costs of running LLM-as-a-judge evaluations on live production traffic. Jev can evaluate multiple questions in parallel, meaning checking for PII leakage, user intent, and user frustration simultaneously adds minimal extra cost or time. To use the new capability, developers can add their TypeSafe API key under LangSmith's provider secrets, create a new evaluator, and select the jev-latest model. Instead of writing complex prompts, users define the state and configure typed questions, such as yes/no queries, multiple-choice options, or numerical scales, to receive clean, automated feedback.