@LangChain: Point a coding agent at the LangSmith CLI and it can build you an online LLM-as-a-judge: define the rubric, wire it to …
Summary
A tutorial on using a coding agent with LangSmith CLI to set up an online LLM-as-a-judge for evaluating LLM traces, all configurable from the terminal.
View Cached Full Text
Cached at: 09/11/26, 12:25 AM
Point a coding agent at the LangSmith CLI and it can build you an online LLM-as-a-judge: define the rubric, wire it to every incoming trace, adjust the sampling rate, all from your terminal.
Here’s a 5 minute tutorial. https://t.co/m8IBLK4pH2
Similar Articles
@LangChain: You can use the LangSmith CLI to pull your flagged traces, classify each one by issue type, and build an eval dataset w…
This article explains how to use the LangSmith CLI to manage traces and build evaluation datasets through a quick tutorial.
@akshay_pachaar: If you use LLM-as-judge, this one is for you. (bookmark it) Most teams validate their agent's outputs by calling a fron…
Details an approach to train a small LLM judge for evaluating agent outputs, replacing costly frontier models, with a Claude Code plugin for deployment.
@LangChain: Great thread from @AdamRLucek on LangSmith Engine!
Adam Łucek discusses LangSmith Engine, an agent for automating agent development using trace data.
@LangChain: Introducing LangSmith LLM Gateway: The runtime governance layer for your agents. Enforce cost limits Detect PII Act on …
LangChain announces LangSmith LLM Gateway, a runtime governance layer for AI agents that enforces cost limits, detects PII, and enables policy enforcement directly within LangSmith, now in private beta.
@LangChain: Now you can learn LangChain using your favorite coding agent with LangChain Academy Tutors. Tutors use skills to teach …
LangChain Academy introduces Tutors that use coding agents to teach AI concepts through interactive lessons, quizzes, and feedback, with featured courses on Deep Agents and LangSmith.