Tag
A practitioner shares their deterministic evaluation harness for QA/coding agents — fixed corpora, injected bugs, hidden tests, and pass^k/token metrics — and asks the community how others balance smoke tests vs full benchmarks, whether LLM-as-a-judge adds value, and whether tools like Promptfoo or DeepEval reduce maintenance burden.
Hugging Face introduces transformers-mic, a Model Integration Contract (MIC) standard that packages tiny published fixtures, runnable usage paths, and human-reviewed baselines to catch model regressions on cheap CPU runners before they reach the Transformers main branch.
A practitioner reflects on the hardest AI agent bugs: failures that produce confident, well-formatted but subtly wrong output, and shares mitigation tactics like output assertion checks, independent math verification, and source citation.
介绍了「Kluever Maneuver」这一 bug 修复实践：先提交一个能通过的失败行为测试来复现 bug，再在第二个提交中修复代码并同时修正该测试，从而证明修复真正有效且每个提交都能通过 CI。
Jepsen 的这份分析报告测试了面向 AI 工作负载的多模型数据库 ParticleDB（1.1.4 至 1.1.18.11.rc-22），发现其宣称的"完全 ACID"并不成立，共报告 96 个问题，包括崩溃、死锁、数据丢失及各类一致性异常，最新的 1.1.19-rc.13 声称已修复全部问题。
SVPino argues that deterministic tests should replace written instructions when working with AI agents: anything checkable automatically—formatting, types, API contracts, security—should be enforced by tests, linters, and static analysis rather than by prompting the agent with rules.
Capy now offers cloud-based mobile devices for building and testing iOS and Android apps on cloud phones, powered by Revyl.
WebYes launched a free PDF Accessibility Checker that detects accessibility issues in PDF files and provides guidance on how to fix them.
A solo practitioner describes a service that reproduces and minimizes AI agent failures into tiny reproducible test cases, then asks the community where to find test customers after getting zero replies from outreach.
YouTube appears to be testing moving the dislike button into the three-dots submenu, based on screenshots shared by users on X and Reddit. This follows earlier changes, including hiding public dislike counts in 2021 and removing the button from YouTube Shorts.
A recap of BazelCon 26 highlighting hallway-track discussions on the value of tests, criticizing frontier coding agents for over-generating unit tests while weighing the trade-offs between unit and e2e testing based on business risk tolerance.
Claude Code 的开发流程表明，为 AI 代理提供自动化验证机制（如 E2E 测试、属性测试和多代理对抗审查）可让 Claude 自行编写 90% 以上的代码，同时在 PR 合并量翻倍的情况下将回滚率降低 40% 以上。
作者分享了为 ActivityPub 机器人框架（ActivityBot，NLnet 资助项目）编写单元测试时遇到的四大难题：用户名与域名校验、消息验证、HTTP 签名的生成与重放，以及与外部服务器的真实互操作测试。
A Lobsters discussion asking whether teams allow LLMs to write both production code and tests, with the author considering a policy restricting AI generation to only one of the two to avoid tautological, over-engineered results.
A practitioner asks the community how they test tool-calling AI agents before and after deployment, comparing scenario tests, LLM-as-judge, tool-call trajectory checks, and regression suites while raising doubts about coverage, judge reliability, and nondeterminism.
A practitioner reports that two AI voice agents passed demos but failed on real financial services calls — one hallucinated a fee, another skipped a required disclosure when interrupted. The agent that held up used locked, mandatory workflow steps; the post asks how others test for such compliance failures before launch.
作者用「布料、支柱与地钉」比喻 agentic 代码：AI 生成的代码柔软易变形，需要架构约束（支柱）才能成型，测试只是地钉。文中还强调数据模型是最承重的支柱，并向读者征询看法。
A blog post exploring the implications of releasing AI-generated software, prompted by Robert C. Martin's claim that he no longer reads agent-written code and instead relies on tests and constraints, discussing epistemology, verification, and liability when LLMs produce code.
作者发布了 Covenant——一个 Elixir 开源测试库（Hex 上的 covenant 0.1.0，MIT 协议），它将 Markdown/YAML 编写的 Contract 与自动化测试绑定，支持决策记录、按 obligation 计分的变异测试以及面向 AI agent 的组件重建包。
DigitalOcean 推出 Agent Learning Services 与 Agent Data Services，用低成本规则监控线上 AI agent 会话、发现静默失败，并将真实故障自动重放为测试用例，在发布前验证修复效果。