ai-audit

Claude Opus 5.5 and GPT-6 Astra independently audited 138 economics papers, flagging 2,035 potential problems and 147 potentially headline-changing findings, while a third AI attempted verification, sparking debate over AI systematically re-examining decades of published research.

Anthropic's system Mythos reportedly uncovered so many software flaws at Bank of America that the bank hired 1,000 contractors to repair them, highlighting AI's growing role in large-scale security auditing.

The paper proposes an independence-graded audit protocol for agentic AI systems, grading independence along principal, substrate, and evidence axes, and provides a formal basis with analysis.

A theorem is presented that may solve the unrestricted 3D isotropic two-phase conductivity-function closure problem, produced and audited by GPT-6 Astra Pro, with potential landmark implications in mathematical physics and materials science if verified.

The study explores whether stereotype-loaded queries about culturally marked individuals lead to more personal information leakage from RAG systems than neutral queries, finding no evidence of such amplification after statistical corrections.

This paper proposes automatically auditing answer correctness and legal authority grounding jointly in legal benchmarks, showing that LLMs often produce correct answers while citing wrong governing provisions under ordinary prompts.

This paper presents a pre-registered audit of whether LLM-judged helpfulness can reliably distinguish answer-giving from pedagogical guidance in AI tutors. The authors find that general-purpose helpfulness is not a dependable pedagogy signal, recommending pedagogy-targeted rubrics and deterministic process measures instead.

Kimi K3 outperformed Fable/Opus 4.8 and GPT-5.6 Sol by finding 5 real bugs in a post-quantum cryptography project audit.

Using AI audit agents, zkSecurity discovered seven real bugs in Cloudflare's CIRCL cryptography library, including critical precision loss and access-control break. All bugs have been fixed upstream.

This paper presents a psychometric audit of large language models as synthetic survey respondents, finding that they fail to match human joint distributions and reliability, making them unsuitable replacements.

Code Shit Mountain Analysis Skills is an AI skill/prompt framework for generating rigorous, professional code review reports, supporting multiple audit modes and HTML output.

This article argues that AI agents making business recommendations must maintain complete audit logs to ensure trust and accountability for users, merchants, developers, and platforms, drawing parallels with traditional advertising systems.

A study analyzing 25,500 LLM resume evaluations across 10 models found a 45% bias rate driven by 'silent bias', with models inventing professional-sounding excuses to penalize candidates. It highlights significant variability in fairness and stability, with Claude, Mistral-Large, and Llama 4 being most stable, while Qwen and older Gemini models were volatile.

An AI-assisted security audit of FreeBSD uncovered 15 kernel vulnerabilities, including privilege escalations and a VM escape, and details the collaborative process of reporting and patching bugs with the FreeBSD team.