@omarsar0: Your agent traces are your best eval data. The most important failures only show up in production, and they rarely make…
Summary
DigitalOcean 推出 Signals（Agent Learning Services 与 Agent Data Services），可从生产环境的 agent 轨迹中自动标记失败会话并一键转化为评测集，在真实运行环境中回放验证修复效果。
View Cached Full Text
Cached at: 10/12/26, 07:30 PM
Your agent traces are your best eval data.
The most important failures only show up in production, and they rarely make it into your test set.
DigitalOcean’s Signals flags sessions where users got frustrated or tools failed and turns them into evals with one click. Simulations then replay them against your fix on your agent’s real harness before it ships.
If you’re running agents in production, you need this loop.
https://x.com/digitalocean/status/2109658818196242684/video/1…
DigitalOcean (@digitalocean): Catch it at the first user, not the four-hundredth.
A new user’s bank link fails, the agent says “try again later,” and nothing shows up in the logs. Watch a team find the bug and fix it before anyone else hits it, with Agent Learning Services and Agent Data Services, launching
Similar Articles
@rohanpaul_ai: Agent failures often pass every health check. A tool call hits the wrong database region, returns 200, and the user get…
DigitalOcean 推出 Agent Learning Services 与 Agent Data Services，用低成本规则监控线上 AI agent 会话、发现静默失败，并将真实故障自动重放为测试用例，在发布前验证修复效果。
"Your agent's failures are your regression suite" is great advice. Building eval datasets by hand isn't. Here's what I do instead.
作者分享了一套基于真实会话失败用例构建 AI agent 回归测试集的流程：先手动标注 traces，再按失败类型分组、写廉价检查项，并用 Kitaru 的 MCP server 与重放机制加速评测集的维护。
@omarsar0: // Agents Do Not Fail Alone // Very nice open-source eval tool to check agent reliability. Lots of cool ideas in there.…
This paper introduces ProofAgent-Harness, an open-source evaluation tool that measures context engineering quality for AI agents across seven criteria, validating it as a leading indicator of agent reliability.
@Xudong07452910: This LangChain article on Agent Eval is very practical, especially for those who have already started integrating Agent…
LangChain 发布关于 Agent 评估（Agent Evals）的实用指南，建议从真实 trace 中选取 10–20 个失败案例构建小型评估集，结合代码检查与 LLM Judge 逐步完善评估流程。
when your agent eval catches a failure what do u actually do next?
The author discusses common challenges in debugging AI agent evaluation failures, seeking insights on efficient investigation methods and the reliability of comparison runs versus other evidence sources.