Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal

Hugging Face Daily Papers Papers
llm-safety self-distillation controlled-refusal alignment boundary-aware evaluation fine-tuning

Summary

This paper introduces boundary-aware self-distillation to improve LLM safety refusal by reducing false refusals on benign prompts while maintaining genuine refusals through controlled data composition.

Safety alignment is usually posed as a topic-level question: is this subject harmful? Deployments ask a narrower one. A civics tutor and a public-sector assistant may share a base model yet need different boundaries inside the same topic, refusing targeted political manipulation while still answering factual questions about the same election. We formulate this as narrow-boundary safety and introduce an offline self-generated framework combining controlled topic generation, coverage repair, in-distribution compensation data, and harmful-benign pairs for training and evaluation. Single-shot generation leaves 19.88% of prompts without accepted refusal traces, whereas escalating retries leave 0.20%. On political persuasion with Qwen3-8B, training on refusal data completed through Escalate increases target-domain refusal from 9.47% to 84.75% and reduces the mean unsafe-response rate across three broader harmfulness benchmarks from 26.26% to 0.14%, but increases XSTest over-refusal from 2.00% to 74.00%. In a separate matched comparison, replacing external responses with verified target-model responses reduces over-refusal from 15.20% to 5.20%. Boundary-pair data reduces comply-side over-refusal on held-out pairs from 32.94% to 4.16%, while harmful-side refusal decreases only from 91.88% to 87.72%. These results show that data composition controls the safety and usability trade-off, and that safety alignment should be evaluated on both sides of the intended refusal boundary.
Original Article
View Cached Full Text

Cached at: 09/08/26, 04:25 PM

Paper page - Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal

Source: https://huggingface.co/papers/2609.04482 Safety alignment usually asks a topic-level question: is this subject harmful? Real deployments ask a narrower one. A civics tutor and a public-sector assistant can share one base model, yet only one may need to refuse requests to write political manipulation, while both still answer factual questions about the same election. Refusing all of politics is not an option.

In our latest paper we train a model to refuse only that harmful subset, and we measure both sides of the boundary. The finding that matters: the configuration with the lowest harmful-response rate also refused 74% of plainly safe prompts. It looks safe only because nobody checked the benign side. Adding harmful-benign boundary pairs to the training data brings that false-refusal rate from 32.94% to 4.16%, while genuine refusals barely move.

A model that refuses more is not a model that is safer. Data composition controls the trade-off, and both sides of the boundary have to be evaluated.

Similar Articles

CR4T: Rewrite-Based Guardrails for Adolescent LLM Safety

arXiv cs.CL

This paper proposes CR4T, a model-agnostic safeguarding framework that rewrites unsafe or refusal-style LLM outputs into developmentally appropriate, guidance-oriented responses for adolescents, offering a more human-centered alternative to traditional refusal-centric guardrails.

Low-Agreeableness Persona Conditioning for Safe LLM Fine-Tuning

arXiv cs.CL

This paper introduces a persona-driven rewriting pipeline that conditions LLM fine-tuning on low agreeableness to reduce jailbreak susceptibility and harmful outputs while preserving conversational warmth, without requiring safety labels or changes to training objective.