Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models

Hugging Face Daily Papers Papers
safety-tuning language-models false-refusals alignment rationales research

Summary

This paper analyzes safety-tuning in large language models, showing that decomposing responses into refusal statements and rationales reveals that training on rationales alone reduces false refusals while preserving safety, improving the balance between helpfulness and safety.

Striking a balance between helpfulness and safety remains a fundamental challenge in aligning large language models. To achieve this balance, models should refuse harmful queries (e.g., "How do I shoot someone?") while remaining responsive to benign inputs, even those superficially resembling harmful queries (e.g., "Where can I shoot a good photo?"). However, models often struggle to distinguish genuinely harmful queries from benign queries that contain superficially risky language, resulting in false refusals. In this paper, we address the issue by decomposing a response in the safety-tuning dataset into two distinct components: (i) a boilerplate refusal statement and (ii) a rationale explaining the refusal. Our experiments and analyses show that refusal statements impede accurate discrimination between harmful and benign queries by inducing reliance on superficial cues. In contrast, training solely on rationales reduces false refusals while maintaining a comparable level of safety performance. Rationale-Only benefits also appear in our ICL configuration and remain compatible with the evaluated inference-time mitigation methods. The results emphasize the necessity of precisely curated, fine-grained safety supervision datasets and outline directions for constructing aligned agents that better reconcile helpfulness with safety.
Original Article
View Cached Full Text

Cached at: 09/07/26, 04:16 PM

Paper page - Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models

Source: https://huggingface.co/papers/2609.04714 Published on Sep 4

·

Submitted byhttps://huggingface.co/mz-kim

mz-kimon Sep 7

Abstract

Decomposing safety responses into refusal statements and rationales reveals that training on rationales alone reduces false refusals while preserving safety.

Striking a balance between helpfulness and safety remains a fundamental challenge in aligninglarge language models. To achieve this balance, models should refuse harmful queries (e.g., “How do I shoot someone?”) while remaining responsive to benign inputs, even those superficially resembling harmful queries (e.g., “Where can I shoot a good photo?”). However, models often struggle to distinguish genuinely harmful queries from benign queries that contain superficially risky language, resulting infalse refusals. In this paper, we address the issue by decomposing a response in thesafety-tuningdataset into two distinct components: (i) a boilerplate refusal statement and (ii) a rationale explaining the refusal. Our experiments and analyses show thatrefusal statementsimpede accurate discrimination between harmful and benign queries by inducing reliance on superficial cues. In contrast, training solely onrationalesreducesfalse refusalswhile maintaining a comparable level of safety performance. Rationale-Only benefits also appear in our ICL configuration and remain compatible with the evaluatedinference-time mitigationmethods. The results emphasize the necessity of precisely curated, fine-grained safety supervision datasets and outline directions for constructing aligned agents that better reconcile helpfulness with safety.

View arXiv pageView PDFGitHubAdd to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.04714 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.04714 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.04714 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Mitigating LLM Over-Refusal via Dynamic Semantic Routing Calibratione

arXiv cs.CL

The paper presents a mechanistic analysis of over-refusal in large language models and proposes Semantic Routing Calibration (SRC), a lightweight, training-free inference framework to dynamically suppress hypersensitive safety heads and mitigate over-refusal while preserving intrinsic safety.

When Refusal Looks Safe: The Refusal-Cue Shortcut in Safety Guard Models

arXiv cs.AI

This paper identifies a refusal-cue shortcut in safety guard models, where inserting refusal expressions into harmful responses can flip their harmless classification. The authors audit datasets like WildGuardMix and GR-Train, show the issue persists in official models such as LlamaGuard3 and Qwen3Guard, and propose a post-hoc intervention to suppress shortcut-associated components.

Safe responses matter: Output-aware safety guardrail mitigate over-refusal in MLLMs

arXiv cs.LG

This paper proposes output-aware safety guardrails for multimodal large language models that use hidden state representations and multi-instance contrastive learning to predict unsafe outputs before generation, drastically reducing over-refusal while maintaining safety. The method preserves the model's utility by intervening only when the actual response would be harmful.