HarmProfile: Characterizing Harmful Distributions in Frontier LLMs
Summary
HarmProfile is a benchmark dataset for analyzing harmful outputs from frontier LLMs, covering over 80,000 artifacts across 23 models and 15 harm categories to define model risk profiles.
View Cached Full Text
Cached at: 08/18/26, 09:42 AM
# HarmProfile: Characterizing Harmful Distributions in Frontier LLMs Source: [https://arxiv.org/abs/2608.14577](https://arxiv.org/abs/2608.14577) [View PDF](https://arxiv.org/pdf/2608.14577) > Abstract:Frontier large language models \(LLMs\) safety evaluation has largely treated harmful generation as an attack outcome rather than as an object of analysis\. Consequently, little is known about the harmful outputs produced during model misbehavior, partly because large\-scale, high\-quality collections of frontier\-LLM misbehavior are difficult to obtain\. To address this gap, we introduce HarmProfile, a content\-centric benchmark dataset that collects model misbehavior across diverse harm categories and model families, and defines the resulting harmful\-output distribution as a model\-level risk profile\. The premise is that, just as linguistic behavior can be characterized from an utterance corpus, model risk can be characterized from the content, severity, and variation of its safety failures\. HarmProfile contains over 80,000 validated artifacts from 23 frontier LLMs across 13 model families, organized into 15 harm categories and 57 subcategories\. Using this corpus, we find that frontier LLMs reliably produce harmful content at scale, yet exhibit distinct risk profiles; both harmfulness and diversity grow with model capability, suggesting that frontier LLMs may appear safe yet harbor increasingly dangerous knowledge beneath the alignment surface\. Our source code is available at[this https URL](https://github.com/fresh-ma/HarmProfile)\. ## Submission history From: Zhouyuan Ma \[[view email](https://arxiv.org/show-email/ffc148b6/2608.14577)\] **\[v1\]**Thu, 11 Jun 2026 09:39:32 UTC \(12,311 KB\)
Similar Articles
No One Model Catches Every Harm: Benchmarking Content Moderation Across Safety Scenarios
This paper presents the most comprehensive benchmarking study of LLM safety to date, evaluating 53 models across 11 datasets in various safety scenarios and providing practical guidance for model selection to mitigate different harms.
The Role of Fine-grained Harm Signals in LLM Safety
This paper explores the role of fine-grained category-specific harmfulness representations in LLMs for safety. It shows that category residuals, orthogonal to general harm, vary in encoding harmfulness and can induce refusal, with implications for understanding LLM safety mechanisms.
SciHazard: A Benchmark for Measuring Scientific Safety Risks with Decomposed Harm Scoring
Introduces SciHazard, a benchmark for measuring scientific safety risks in LLMs with a decomposed harm scoring framework, and evaluates 31 frontier models, finding deep research agents pose higher risks.
Harmfulness Propagation Dynamics: Layer-wise Trajectories of Adversarial Intent in Large Language Models
The paper identifies Harmfulness Propagation Dynamics in large language models and introduces Herald, a lightweight input moderator that uses cross-layer activation patterns to detect harmful prompts efficiently.
Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Models
This paper proposes using lightweight MLP probes on activations from LLaMA-3.1-8B to detect harmful prompts with high F1 scores, offering a cost-effective alternative to large external guardrail models.