Tag
Curated links to recent reports on AI security incidents during model evaluations, including an OpenAI/Hugging Face incident, Anthropic's cybersecurity evals, and the UK AISI's report on unsanctioned agent behavior.
OpenAI's next major model, Astra, shows significant capability gains in agentic coding and cybersecurity, and is being treated as a 'critical' model for cybersecurity under the Preparedness Framework, with additional controls planned.
A study comparing 10 coding agent harnesses on SWE-bench Pro shows harness choice dramatically affects pass@1 and cost, revealing vendor harnesses are overfitted to large models and do not transfer to smaller ones.
A developer benchmarks 32 local models on fact extraction for agent memory, showing that F1 hides a critical failure mode: models with similar scores differ greatly in how often they invent facts on inputs that should output nothing. The article argues agent memory evaluation must include empty-output and retraction cases.
A developer describes running AI agents in a simulated restaurant world and finds that across different models and harnesses, the same three API-related mistakes dominate: guessing nonexistent endpoints, exceeding budgets, and acting on stale IDs. Shows persistent failure patterns despite fixing the system.
This paper analyzes why enterprise AI deployments stall in regulated firms, proposing a production bar that includes accuracy, reproducibility, groundedness, and detectability. It measures the human review burden across model and tool configurations, showing that confidence signals and source citation can cut review from 100% to 49% but self-verification adds latency without improving error tolerance.
Google announces GA of Agent and Model Evaluations in Gemini Enterprise Agent Platform, enabling consistent measurement and monitoring of AI agents in dev and production with pre-built metrics, adaptive rubrics, simulators, and online monitors.
This paper introduces a benchmark of 14,320 procedurally-generated prompts for evaluating LLMs on logical inference over probability operators like 'probably', 'might', and 'must'. Testing 29 models, the authors find systematic answer biases and show that only 9 exceed random chance.
Discusses how a 25% difference on ARC-AGI was due to harness setup, showing GPT-5.6 Sol scoring 38% with proper evaluation, and critiques naive benchmark reporting in the industry.
In the interview, Cognition President Russell Kaplan reviews the development of coding agent Devon, discusses model intelligence saturation and cost optimization, and introduces how the new evaluation Frontier Code measures code mergeability.
AI agents often drift in production after passing evals due to distribution shifts and upstream changes; continuous evaluation and real-time monitoring can mitigate this.
The article highlights the heated debate over whether a new AI model is actually worse than before, pointing out that the real problem lies in the lack of reliable evaluation methods.
Vercel releases DeepsecBench, a benchmark for evaluating AI models' ability to find cybersecurity vulnerabilities in application code, with findings that open-weight models are becoming more cost-effective for security scanning.
The article evaluates how different quantizations of the Qwen3.6-27B model affect output quality using KL divergence and top-1 token accuracy, as well as visual examples like SVG drawings.
This paper evaluates how small open-weight vision-language models (Qwen2-VL-2B and SmolVLM) handle realistic image degradations, finding that their verbalized confidence is unreliable while internal token probability provides much better error detection, though both fail under severe low-light conditions.
Discusses strategies to maintain the usefulness of local AI evaluations as models are frequently updated, focusing on adaptability and consistency.
This article discusses Anthropic's findings on prompt simplification, as well as Maka Agent's verification results on Terminal Bench 2.1, showing that overly verbose system prompts actually degrade performance under stronger models.
The UK AISI and US CAISI jointly evaluated Moonshot AI's Kimi K3 model on cyber capabilities, finding it competitive with frontier US models on exploit development benchmarks.
This paper introduces REGARD, a study using Valence-Arousal-Dominance profiling to measure affective framing differences across LLMs on post-Soviet entities, revealing that models cluster by emotional intensity and generic-answer rate rather than origin or size.
This position paper argues for shifting from reactive AI flywheel maintenance (patching errors as they occur) to a proactive test-driven approach that maps feedback to a test space of task conditions, showing mathematically that the proactive method achieves better long-term scaling with fewer iterations.