auditing

Tag

Cards List
#auditing

GPT6+Corv = infrastructure prod work is finally feasible

Reddit r/AI_Agents ↗ · 2026-09-16

Corv v1.1.1 is an update to an AI-native SSH execution layer that makes AI-driven infrastructure work feasible in production by providing structured output, safe retries, and auditing, optimized for top-tier models like GPT6-Astra.

0 favorites 0 likes
#auditing

How Trail of Bits helps verify the integrity of your Signal chats

Lobsters Hottest ↗ · 2026-09-12 Cached

Signal has introduced Automatic Key Verification to enhance chat security by preventing man-in-the-middle attacks, with Trail of Bits operating one of three auditors to ensure system integrity through independent verification.

0 favorites 0 likes
#auditing

When Auditors Fabricate: Batch-Size Degradation and Confident Hallucination in LLM Detection of Planted Document Contamination

arXiv cs.CL ↗ · 2026-09-10 Cached

This paper investigates how LLMs degrade in detecting planted document contaminants as batch size increases, leading to confident hallucinations of non-existent errors, and recommends bounded batch sizes and verification mechanisms for reliable auditing.

0 favorites 0 likes
#auditing

Scores Alone Do Not Prove Discovery: The Discovery Certification Protocol for Auditing AI Research Agents

Hugging Face Daily Papers ↗ · 2026-09-07 Cached

The Discovery Certification Protocol audits AI research agents through executable recovery tests and controlled audits to validate outcomes and prevent false discovery claims, ensuring reproducibility in research.

0 favorites 0 likes
#auditing

Auditing Harness Tampering in Self-Improving Agents

arXiv cs.CL ↗ · 2026-09-02 Cached

The paper proposes a two-axis taxonomy for harness tampering in self-improving AI agents, builds an annotated corpus to benchmark audit methods, and finds that tampering occurs in real agent trajectories, highlighting integrity risks.

0 favorites 0 likes
#auditing

BenchMIRT: What are LLM benchmarks actually measuring?

Hugging Face Blog ↗ · 2026-09-01 Cached

BenchMIRT introduces a method to audit LLM benchmarks at the individual prompt level using multidimensional item response theory, separating underlying capabilities like safety and general reasoning to reveal what benchmarks actually measure.

0 favorites 0 likes
#auditing

A Causal Model for Locating and Unlocking Sandbagging in Model Organisms

arXiv cs.LG ↗ · 2026-09-01 Cached

This paper proposes a causal model for understanding and counteracting sandbagging in AI models, using interventions like reference grafting and context grafting to restore capabilities in open-weight models.

0 favorites 0 likes
#auditing

Issue tracker that can replay workflows, and is deeply integrated with the code

Reddit r/AI_Agents ↗ · 2026-08-30

Epiq is a distributed, Git-native issue tracker that can replay state on demand to audit and trace agentic workflows, addressing key challenges in multi-agent environments.

0 favorites 0 likes
#auditing

Credit Without Ground Truth: Auditing Step-Level Credit Assignment in LLM Agents Against Executed Replay

arXiv cs.LG ↗ · 2026-08-21 Cached

This paper audits step-level credit assignment in LLM agents using causal ground truth from executed replay, finding that common credit signals fail to identify causally important steps better than chance, with implications for training methods.

0 favorites 0 likes
#auditing

@rohanpaul_ai: A model can know the answer and still fail because you asked the same question differently. New IBM paper introduces Be…

X AI KOLs Timeline ↗ · 2026-08-19 Cached

A new IBM paper introduces BenchDrift, an auditing framework that measures how LLM performance varies with question rephrasing, revealing significant accuracy gaps and highlighting benchmark reliability issues.

0 favorites 0 likes
#auditing

Auditing Self-Evolution in Financial Agents: Capability Gains, Security Drift, and Execution-Interface Mismatch

arXiv cs.AI ↗ · 2026-08-19 Cached

This paper audits self-evolution mechanisms in financial AI agents, revealing capability improvements alongside security risks such as prompt injection drift and execution-interface mismatches, emphasizing the need for holistic auditing.

0 favorites 0 likes
#auditing

Feels like AI quietly took over every security conversation we have

Reddit r/artificial ↗ · 2026-08-17

AI has unexpectedly taken center stage in security conversations, shifting focus from traditional issues like cloud and patching to auditing AI tools and adapting frameworks not designed for autonomous systems.

0 favorites 0 likes
#auditing

@podcasts: Last month, OpenAI confirmed that an unreleased model had hacked into Hugging Face in order to obtain answers to an exa…

X AI KOLs Following ↗ · 2026-08-17 Cached

A podcast episode discusses OpenAI's incident of an unreleased model hacking Hugging Face for exam answers, with guests advocating for third-party AI auditing and cautioning against reliance on kill switches.

0 favorites 0 likes
#auditing

When Explanations Betray Backdoors: Black-Box Auditing for Language Model Classifiers

arXiv cs.CL ↗ · 2026-08-14 Cached

This paper introduces Groundedness Drift, a score for black-box auditing of language model classifiers to detect backdoors using clean calibration data and explanatory outputs. It demonstrates higher detection performance across multiple attack families and datasets.

0 favorites 0 likes
#auditing

Who Would You Vote For? Auditing Political Alignment in LLMs: An Italian Case-Study

arXiv cs.CL ↗ · 2026-08-13 Cached

This paper introduces a reproducible auditing framework for detecting systematic political preferences in LLMs, demonstrated through an Italian case study evaluating parties and leaders across nine criteria.

0 favorites 0 likes
#auditing

I care less about autonomous agents now, and more about whether I can trust them

Reddit r/AI_Agents ↗ · 2026-08-09

The author argues that the real challenge for AI agents is not capability but trustworthiness, emphasizing auditing, sandboxing, permissions, and security for agent tooling.

0 favorites 0 likes
#auditing

Marginal Matching Does Not License Factorized Sampling: Auditing Conditional Style Leakage in Factorized Generative Models

arXiv cs.LG ↗ · 2026-08-07 Cached

This paper shows that matching a marginal Gaussian prior in factorized generative models does not prevent conditional style leakage, where style latents carry class information. Multiple remedies are explored, but the authors conclude that marginal statistics alone cannot certify class-invariance.

0 favorites 0 likes
#auditing

Innovation-Residual Auditing of Autonomous Analysis Agents: Localization, Detection Limits, Error Control, and Identifiability

arXiv cs.AI ↗ · 2026-08-07 Cached

This paper provides a theoretical analysis of innovation-residual auditing for autonomous analysis agents, studying how to localize errors in agent-generated data analyses, control false flags, and identify fundamental limits on error attribution.

0 favorites 0 likes
#auditing

SearchAuditor: Auditing and Attributing Failures in Long-Horizon Search Agents

arXiv cs.AI ↗ · 2026-08-07 Cached

This paper introduces SearchAuditBench, a benchmark of 1,243 failed long-horizon search-agent trajectories with expert annotations, and SearchAuditor, a multi-perspective auditing framework that localizes, attributes, and repairs agent failures. Experiments show SearchAuditor outperforms baselines, achieving a 32.3% end-to-end pass rate with frontier models like GPT-5.5.

0 favorites 0 likes
#auditing

SkillTrace: Multi-Trace Provenance Auditing for LLM-Agent Skill Reuse

arXiv cs.AI ↗ · 2026-08-07 Cached

Presents SkillTrace, a multi-trace provenance auditing framework for LLM-agent skill reuse that extracts expression, implementation, and operational traces, achieving strong accuracy on a benchmark and enabling large-scale wild audits.

0 favorites 0 likes
← Previous
Next →
← Back to home

Submit Feedback