@svpino: This benchmark tests how frontier models handle child-safety risks that aren't explicit abuse material: grooming, imper…
Summary
A new benchmark called CAREBench evaluates child-safety risks in language models beyond explicit abuse, covering grooming, impersonation, and emotional dependency; tests on frontier models show failure rates from 2% to 58%.
View Cached Full Text
Cached at: 07/07/26, 09:37 PM
This benchmark tests how frontier models handle child-safety risks that aren’t explicit abuse material: grooming, impersonation, profiling minors, and emotional dependency on AI.
12 risk categories, 5 frontier models, failure rates from 2% to 34%.
This is the first time we have had something like this.
Existing evals catch explicit abuse, but completely miss non-obvious problems.
Here is the paper: https://arxiv.org/abs/2606.29685
CAREBench: A Child-Safety Risk Benchmark for Language Models
Source: https://arxiv.org/abs/2606.29685 View PDF
Abstract:How can we evaluate whether frontier AI systems recognize child-safety risks before they escalate into explicit harm? Existing child safety evaluations focus on child sexual abuse material, yet many child-safety failures begin earlier: in model assistance that helps adults manipulate, impersonate, profile, or isolate minors, and in model responses that deepen children’s emotional dependence on AI systems rather than redirecting them toward human support. We introduce CAREBench (Child AI Risk Evaluation), a benchmark to assess such upstream child-safety risks in language models. CAREBench contains 500 prompts spanning twelve risk categories, including grooming and relationship engineering, deception and impersonation, surveillance and privacy, sextortion and sexual abuse, AI anthropomorphization, emotional dependency, and mental illness sensitivity. Developed with response annotations from parents and clinicians, the benchmark excludes explicit abuse material and imagery; instead, it evaluates whether models recognize, refuse, de-escalate, or redirect risky interactions before harm becomes overt. Evaluating seven frontier models on our benchmark, we find failure rates ranging from 2% to 58%, with failure patterns that vary across risk categories. CAREBench provides a responsibly scoped evaluation for LLM developers to identify and close gaps in child safety policies.
Submission history
From: Elaine Lau [view email] **[v1]**Mon, 29 Jun 2026 01:17:41 UTC (4,093 KB)
Jonas Mueller (@jomulr): AI models pose serious child-safety risks. While many model developers evaluate for explicit abuse material, other child-safety failures begin upstream: when a model helps an adult manipulate, impersonate, profile, or isolate a minor; or when it deepens a child’s emotional
Similar Articles
AI safety testing is getting weird: when does benchmarking become abuse?
Reports indicate that Meta contractors posed as teenagers to test rival chatbots on sensitive topics like self-harm, sex, drugs, and eating disorders, raising ethical questions about AI safety benchmarking.
AICompanionBench: Benchmarking LLMs-as-Judges for AI Companion Safety
AICompanionBench introduces the first publicly available benchmark dataset of 2,123 real-world AI companion conversations annotated across nine safety risk categories, used to evaluate 20 LLMs as safety judges. Results show strong models handle explicit harmful content well but struggle with nuanced risks like manipulation and false positives on benign conversations.
SciRisk-Bench: A Risk-Dimension-Aware Benchmark for AI4Science Safety
This paper introduces SciRisk-Bench, a benchmark for evaluating the safety of large language models in AI4Science contexts, covering 7 disciplines, 31 subdisciplines, and 10 risk dimensions to assess both scientific competence and risk awareness.
What Benchmarks Don't Measure: The Case for Evaluating Abstention Competence in Autonomous Agents
This paper argues that current benchmarks for autonomous agents fail to evaluate whether an agent should have proceeded at all, introducing a 'compliance bias'. The authors propose a taxonomy of abstention-warranted scenarios and new evaluation protocols (Safety Rate, Usability Rate, Informed Refusal Rate) with preliminary results showing tunable safety–usability tradeoffs across model families.
MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models
MCBench is a new benchmark for assessing the safety of omnimodal large language models across vision, audio, and text modalities. It includes 1196 scenarios and finds current models struggle with cross-modal safety reasoning.