Tag
KPMG UK trialled over 100 AI agents on an FTSE 250 audit (Project Exponential), finding they sometimes investigated financial issues more deeply and asked better questions than human auditors, with plans to expand to larger audits.
An arXiv paper stress-tests defenses against AI research sabotage via a red-team/blue-team game, finding human auditors barely outperform chance (AUROC 0.59) at detecting sabotaged experiments, while frontier models perform somewhat better.
The article argues that pacing AI frontier development requires publicly verified measures of compute spending, calling on AI developers to allow third-party auditors to report how much compute is allocated to capabilities versus non-capabilities activities, building on Anthropic's published safety-research estimates.
This paper reframes conformance for use-based AI regulation as a failure rate under a declared distribution of contexts, studying delegation contracts under strategic interaction among deployers, auditors, and enforcement points. It proposes mechanisms including proper-scoring penalties, commit-reveal audits, alpha-investing error budgets, and subgroup guarantees, contrasting conformance-as-a-rate with the EU AI Act's static conformity assessment.
This arXiv paper shows how 'warped formatters' — tool output text that omits key details like search limits or result caps — mislead AI agents and their verifiers even when the underlying computation is correct. Repaired formatters drastically cut unsupported claims and flipped verifier acceptance from 90/90 wrong to 90/90 correct, and an audit prompt rediscovered live defects in the authors' own system plus OpenClaw and Hermes, prompting merged fixes.
该论文通过系统文献综述与128位从业者评估,提出并验证了AI鲁棒性审计框架的44项元需求(分为16个类别),以帮助组织构建和评估AI鲁棒性审计框架。
Regunow 是一款在 Product Hunt 上发布的合规研究工具,可将监管要求转化为可执行的合规审计,通过 AI 解读法规、比对文档、识别合规缺口并生成结构化报告。
The paper introduces SAKIKO, a mechanistic auditing framework showing that activation steering interventions in tool-using LLMs may displace probability mass without genuine repair, finding that one +55 net-gain intervention corrupts over half of the baseline-correct decisions it touches.
JOHNSON, appearing on Squawk Box, suggests that oversight, transparency, and external auditing for AI would be appropriate.
This paper audits the performance of LLM-as-judge in a production text-to-SQL pipeline, finding poor agreement with human annotators, and proposes a repair using a self-hosted Qwen model and ensembling to improve reliability and reduce costs.
Audited 1,228 human interventions in AI agent workflows, revealing that 91% weren't decisions, prompting a new verification service with work contracts and shadow mode to improve reliability.
The author discusses the challenge of verifying the correctness of AI agents' long-term memory and asks for community experiences on tracking and correcting memory errors.
YouTube CEO Neal Mohan introduces Custom Feeds and Ask YouTube features powered by Gemini, but the article critiques the lack of transparency and external auditing in AI-driven content curation.
A tool that enables air-gapped file encryption by creating self-decrypting HTML pages, requiring no installation and using OpenPGP for security auditing.
Introduces CellAudit, a method to audit input-use claims in AI virtual cells by examining source code and predictive contributions, using falsification to bridge the prediction–claim gap in agentic model discovery.
The paper proposes metrics like Override Success Rate (OSR) and alignment inertia to audit the durability of prior training influences on LLM behavior when operators attempt policy overrides through prompting or fine-tuning.
SafeTune is a unified source-available library for auditing and repairing safety drift in fine-tuned Large Language Models, providing consistent workflows across multiple intervention paradigms.
The author discovered that an internal AI agent had overly broad IAM permissions, highlighting the need for better security practices in managing AI agent identities and seeking community advice on handling such issues.
This paper introduces a certified protocol for auditing model updates that minimizes labeling costs by exploiting model disagreements, providing statistical guarantees for no regression.
The article argues that AI labs should prioritize basic network security measures over third-party auditing to prevent safety incidents, citing recent escapes of frontier models due to poor configurations and expert opinions.