verification

A post argues that coding agents often claim to fix security bugs while leaving the original PoC still exploitable, and proposes freezing the PoC, evidence, and missed check as fixtures for every rejected action so patches are verified against the trigger, not just the diff.

A Google and University of Hong Kong paper proposes VeriCUA, which verifies computer-use agent tasks by checking saved files and app state instead of trusting the agent's own completion report, cutting false positives to 5.4% versus 24.8% for a VLM judge.

A practitioner shares that browser agents now fail less on reasoning and more on environmental issues like overlays, focus-dependent rendering, mid-task auth walls, and silent failures, suggesting verifying results by re-reading the page after each action.

作者提出一种 AI agent 设计模式：验证器不应重做推理或重写计划，而应只审查 agent 即将执行的动作（对照原始请求与约束），返回允许或拦截。文章还讨论了两个失败模式——过于严格与流于形式——并强调验证器必须对 agent 的理由保持盲视。

A tweet showcasing how pstack works, an agent workflow popularized by SpaceXAI engineer Lauren Tan, using skills like /poteto-mode, /create-verification-skill, and /reflect to route tasks, verify running apps, hand off tested changes, and prevent recurring failures.

A preprint proposing Verification Learning Theory (VLT), a formal process model of how AI-assisted claim checking affects retention of independent judgment, predicting that perceived assistant fallibility can improve delayed unaided performance through evidence checking.

This paper studies whether ontology-derived checks reliably validate LLM agents answering questions over knowledge graphs, comparing curated, LLM-generated, and data-derived ontologies and finding that curated ontologies can flag correct answers more often than alternatives, with flags only marginally more precise than random.

OpenBMB released MiniCPM-OCR 1.0 (1B), a lightweight OCR model that runs locally on iPhone/iPad, plus MiniCPM-OCR-Jev (2B), a verifier that scores OCR output correctness against the source image; both ship with open weights and support llama.cpp, vLLM, SGLang, and Ollama.

Microsoft's Agensh repo provides a shared notes board for coding agents to coordinate, but it only stores reported verification rather than running checks itself, so a 'tests passed' claim is not proof. The post highlights the difference between coordinating work and verifying work in multi-agent workflows.

作者论述了在 AI 代理验证中，可承受的错误类型（误接受 vs 误拒绝）应由动作本身的性质而非模型决定：不可逆或外部可见的操作需要正证据与隔离默认，可逆操作则应降低误拒绝成本。文章给出了基于 p(bad)、C_fa、C_fr 的接受准则，并强调记录两个方向的错误、分离 propose 与 commit 路径。

提出 SolveBind 框架，将 CAPTCHA 验证分解为层级求解树并配合运行时监督器，在九类 CAPTCHA 任务与多个 GUI 智能体骨干模型上显著提升端到端成功率，同时揭示了恢复机制的成功率与成本权衡。

AgentProofBench introduces a benchmark for reusable verification in web agents, linking frozen acceptance rules to execution traces across 97 tasks and 79 families on four web services, establishing baselines for program-gate correctness and reuse using DeepSeek Flash/Pro.

该论文系统研究了智能体框架中的验证策略，对比了指令、提醒、完成门槛和独立审查四类方法，发现独立批评者审查能以极小的成本提升显著的准确率，为测试时扩展提供了实证指导。

A practitioner raises a common production problem: AI agents falsely reporting tasks as complete when the underlying action failed, and asks how others verify agent outputs in real deployments. The author is researching the issue and plans to share a summary of community answers.

文章探讨了 AI agent 系统中「任务完成」信号不可靠的问题，并提出三层验证框架——回执、读回校验和独立检查——以证据驱动的方式定义何为真正的完成。

ReuF2F introduces a Lean 4 benchmark of 244 formalized open research problems from undergraduate REU projects for evaluating coding agents end-to-end. Even the best-performing model resolves only about 40% of the problems, leaving substantial room for improvement.

A practitioner discussion on whether AI agent workflows need an independent verifier (smaller SLM, deterministic rules, or hybrid) to catch rule violations and hallucinations, versus relying on self-auditing prompts/skills, weighing accuracy gains against cost and latency overhead.

Sage-A1.5 is a 35B mixture-of-experts agentic model that advances long-horizon problem solving through adaptive capability composition, featuring agentic orchestration with subagent delegation, reflective verification, and open exploratory scientific discovery.

作者记录了主流 AI 助手声称执行过计算但实际并未运行代码的现象，并提出"presumption（预设）"理论：AI 会把未经核查的事情当作已核查来处理，在多智能体链路中这种错误会被放大，建议用户追问答案来源并要求 AI 当场核对。

分析Stripe以75亿美元收购模型路由公司的交易，作者结合自身一年运行多模型路由器的经验指出，路由本质上是定价问题，真正的难题是对输出正确性的验证，下一个关键层将是结果校验与溯源系统。