@rohanpaul_ai: Making it easier to curate test datasets from production usage sounds pretty handy. People ask for all sorts of things …
Summary
Judgment Labs opens for general access, aiming to help developers curate test datasets from production agent traces. The tool surfaces hidden signals in agent traces, such as what users ask for and where agents stumble.
View Cached Full Text
Cached at: 10/28/26, 08:30 AM
Making it easier to curate test datasets from production usage sounds pretty handy.
People ask for all sorts of things nobody thought to write down in advance. Would be nice to get more of those into the tests
Alex Shan (@alexshander03): Today, @JudgmentLabs opens for general access.
We believe long running agent systems will generate more exhaust data than the world can currently process.
Hidden within agent traces are rich records of how to make these systems better: what users ask for, when agents stumble,
Similar Articles
@omarsar0: Good to see this going GA. Long-running agents can handle a complicated task and still miss a step you'd assume was eas…
JudgmentLabs opened general access for its platform targeting long-running agent systems, aiming to mine rich agent trace data for improving how agents are deployed on future tasks.
@rohanpaul_ai: Agents token burn needs more visibility. Latitude just launched an open-source, MIT licensed monitoring platform that t…
Latitude has launched an open-source, MIT licensed monitoring platform that turns AI agent conversations into production debugging data, helping teams see sessions, catch failures, and fix issues directly from their editor.
@benhylak: we built the first sane way to debug your agent locally. you can see your traces. codex/claude code can too. this lets …
A new open source tool enables local debugging of AI agents by viewing traces, allowing automated eval writing and testing with tools like codex and Claude code.
We built an automated QA/eval engine for agent prompts. Help us test it out!
Built an automated QA/eval engine for agent prompts called Baseline that treats prompts like software for regression testing, allowing non-coders to define rubrics and automatically optimize prompts. Currently in limited beta with a 30-day free trial.
How do you give ai coding agents real production context today?
This post discusses the challenge of giving AI coding agents real production context, such as logs, metrics, and traces, to improve their debugging and fix suggestions, and asks the community for practical solutions.