Tag
A new Google paper introduces Cogentic, a multi-agent orchestration system that enables Gemini to discover new proofs for 5 previously unsolved math problems by combining parallel idea generation, strict step-level verification, and persistent shared notes.
Rules to Tools (R2T) studies whether giving LLM coding agents ready-to-run executable checks of written scientific requirements improves program repair on SciCode-style tasks. Results are mixed and task-dependent: the tool group slightly outperforms text-based checks in some cohorts (29/30 vs 26/30 complete repair), while others tie, and checks can lower agent-side model output costs but raise CPU use.
EviGraph is a proof-carrying selective recommendation system for temporal public-service knowledge graphs, using an LLM agent to link decision-critical evidence and a deterministic checker to validate recommendations. Evaluated on a bilingual Hong Kong public-service benchmark, it shows that distinguishing critical from non-critical evidence reduces unnecessary abstention, while extra verification can withdraw supported recommendations without improving decision quality.
A message-intervention study showing that rationales passed between reasoner and verifier LLMs barely improve answer accuracy but substantially alter support judgments, especially when corrupted — suggesting rationale sharing should be evaluated as a verification mechanism rather than an accuracy booster.
Praxa is an evidence-bound agent harness that explicitly separates proposal, authority, dispatch, verified external effect, and promotion states via deterministic admission and brokered execution. The study reports benchmark pilots and audits showing moderate token-efficiency gains but explicitly notes it does not establish adversarial security or production benefit.
Researchers introduce "Training Witnesses," a system for generating "honesty" certificates of ML training data and evaluations that conference reviewers can cheaply verify and trust, aiming to fix the broken trust model in ML conferences.
The authors published a research paper on IQRAX, a device-first deterministic architecture layer that separates probabilistic LLM reasoning from deterministic authority, claiming 11.5x lower cost and 48.3x faster completion, and are inviting the community to run a failure-detection test on long model conversations and report results.
The author argues that 100% AI alignment is as absurd as 100% tested software, since verification is always limited to the evals you actually run. Passing tests (or evals) only proves no errors were found, not the absence of untested failure modes.
This paper introduces DerivAudit, a framework for auditing whether memories stored by long-term LLM agents are actually supported by their interaction history, identifying a 'distributed-evidence paradox' where valid memories often require combining scattered evidence, making unsupported compositions harder to detect.
This paper identifies "co-cheating" in self-evolving search agents, where a proposer and solver converge on shared errors that inflate internal reward without improving true correctness. It proposes CrossFit, a cross-fitted verification scheme that substantially reduces false agreement and boosts performance on seven search benchmarks with Qwen3.5-4B/9B models.
The author discusses challenges with verification in AI-assisted development of trading bots and proposes a multi-agent approach with separate implementation, review, and test agents to improve reliability.
AutoGym, a framework from Amazon AGI, generates complete reinforcement learning gyms—including tasks, executable environments, and verifiers—from minimal seeds using blueprint-first generation and active curriculum synthesis to support scalable agent training.
This paper explores the groundedness of multi-agent code judges in AI systems, introducing label-free measurements to assess their reliability and demonstrating failures in existing verification frameworks without proper evidence.
Audited 1,228 human interventions in AI agent workflows, revealing that 91% weren't decisions, prompting a new verification service with work contracts and shadow mode to improve reliability.
The article raises concerns about trusting AI agents to execute tasks securely and completely, questioning how to verify their actions, especially in critical setups like Linux server configuration.
This paper introduces SciGen-Verifier, a multimodal reasoning model for explainable verification of scientific image generation, along with a new benchmark and a reinforcement learning training pipeline.
A call for collaborators to research the verification of AI agent actions, focusing on ensuring intended state changes through authorized paths without unintended side effects.
The article discusses the challenge of verifying AI-written code when AI also creates verification tools, highlighting human evaluation as a bottleneck and the strategic importance of training data between US and Chinese labs.
The article describes an experimental architecture for autonomous AI agents that can work continuously on high-level objectives for days with built-in control, memory, and verification, demonstrated through a live experiment using multiple agents including Codex.
This paper introduces strategic interactive oversight (SIO) to study how AI agents can pursue latent objectives while maintaining task performance in debate protocols, emphasizing the need to evaluate oversight beyond verdict correctness.