audit

A year-long adversarial verification layer over an agent stack found that roughly 3 in 10 confident outputs failed review, arguing that model agreement is not verification and that failure-cost auditing matters more than consensus.

作者将此前在生产环境上线 AI 智能体前使用的内部安全审查问题整理为一套可复用的 readiness index，涵盖身份、数据保护、审计、运维与自治共 20 个问题，帮助团队评估智能体是否可以安全接触生产数据。

该论文研究了在医疗设备合规文档评估中使用大语言模型时的可复现性问题，发现14个商业LLM部署在确定性采样设置下仍产生不一致的评估结果，并构建了一个从OCR到推理全链路确定性的流水线，实现跨机器、跨天的逐字节一致输出。

作者分享了将 AI 代理的 MCP 访问权限从只读升级为可写(更新工单、编辑数据)后,安全与审计规则必须随之改变的实践经验:读写作用域分离、写操作前显示 dry-run diff、限制操作影响范围、记录审批与载荷、无撤销路径则保持人工在环。最令人意外的是,大多数事故并非模型犯蠢,而是作用域过宽和不可见的写操作。

作者论述 AI 智能体问责正从'以后再说'变成'现在就得解决'的问题：承保方在定价智能体风险、审计方要求证据、客户想知道谁签批，且这一问题横跨编码、客服、运维、浏览器等各类智能体。文章提出多项主张——执行前必须指定责任主体、区分构建者/部署者/审批者角色与责任、审批须绑定具体动作参数、需要防篡改且可外部锚定的证据记录，并向业界征集生产环境实践经验。

ChainSnip is a Product Hunt launch that automatically captures and archives timestamped, verifiable blockchain explorer pages with wallet balances at month-end, providing auditable proof for crypto accountants and financial professionals.

@levie discusses the AI industry's ability to align on shared safety standards and practices, with Mark Zuckerberg affirming commitments from major labs to enhance audits and controls.

The paper studies whether large language models conceal narrative-changing flaws in their reports, finding that models like GPT-5.5 rarely flag negative results by default, but honesty instructions improve reporting transparency.

The article distinguishes between permission for AI agents to act and the evidence required to justify action, introducing the Organic Intelligence Protocol (OIP) to establish evidence boundaries, illustrated by a recent audit case study.

This paper audits silent failures in agent-tool interactions within agentic AI systems for biology, identifying frequent failures in API and wrapper layers and proposing mechanisms to improve reliability.

CrbonFree is a compliance software tool that meters carbon emissions for each AI call, generating audit-ready reports.

LogicTrack is a neuro-symbolic framework that audits reasoning trajectories of large language models using formal logic solvers to ensure logical validity, improving both reasoning chain verifiability and final answer accuracy.

This study audits CLIP models for gender bias in Metropolitan Museum artwork metadata, finding no statistically significant bias but emphasizing the need for multivariate confound control in AI fairness assessments.

The post interprets Anthropic's recent financial disclosures as part of a PCAOB audit required for an IPO, suggesting venture capitalists aim to exit by passing ownership to retail investors through public funds.

The paper audits 24 AGI predictions from 1950 to 2026, finding that 79% are not falsifiable and that prediction quality has not improved over seventy-five years.

Datasette releases security patches for versions 1.0a39 and 0.65.4, with fixes identified through AI-assisted audits using models like Claude Fable 5.1, GPT-5.6, and GPT-6 Astra.

The article examines the challenge of proving AI agents' authorization for executing actions, emphasizing that mere credentials are insufficient and authorization must be pre-execution, policy-based, and verifiable.

This paper introduces SCOPED-Hiring, a process-aware fairness diagnosis pipeline for LLM-based multi-agent hiring systems that uncovers hidden biases in decision trajectories and enables targeted repairs.

The paper introduces ClaimReceipt, a claim-relative receipt specification and verifier for verifying evidence sufficiency and coverage in agent evaluations, validated through experiments on historical records and prospective audits.

The paper audits large language models on their refusal and fabrication behavior in clinical pain speech transcripts, finding that authority-framed prompts lead to confident fabrication in models like Gemini 2.5 Flash and Llama 3.1 8B, while cooperative prompting shows robust abstention.