Tag
作者分享了一次因重试导致重复扣费的线上事故，并介绍其正在构建的 FetchSandbox 工具——通过确定性故障注入检查工作流重试时的幂等性是否真正生效。
The tweet introduces State Machines as a tool to solve the challenge of testing AI agents by providing stateful, repeatable enterprise environments for API mocking.
This paper introduces Agents' Last Exam, a benchmark that tests AI agents on real expert work across 55 digital work areas. Current best agents fail most tasks, averaging only 2.6% pass rate on the hardest tier, revealing a large gap between benchmark scores and real-world automation readiness.
A developer shares a personal open-source benchmark runner for testing OpenClaw agents on real, messy workflows. The tool allows users to define private evaluation cases, run agents in their actual workspace, and generate reports, aiming to provide more relevant signals than public benchmarks.
The author questions whether current LLM evaluation tools are too focused on isolated prompts rather than full workflows and agent interactions, noting that step-by-step accuracy can mask overall behavioral drift in production.