Beyond Refusal Patterns: Safe-Role Internalization for Robust and Generalizable LLM Safety Alignment

# Beyond Refusal Patterns: Safe-Role Internalization for Robust and Generalizable LLM Safety Alignment
Source: [https://arxiv.org/html/2610.07023](https://arxiv.org/html/2610.07023)
1\]Harbin Institute of Technology \(Shenzhen\) 2\]Leiden University\\contribution∗Equal contribution\.\\contribution†Corresponding author\.\\checkdata\[Email\], , , \\checkdata\[Repository\][https://github\.com/Code\-PJH/SSRFT](https://github.com/Code-PJH/SSRFT)

###### Abstract

Large Language Models \(LLMs\) have achieved remarkable capabilities but remain vulnerable to jailbreak attacks that elicit harmful or unsafe outputs\. Existing safety alignment approaches, including Supervised Fine\-Tuning \(SFT\) and Reinforcement Learning from Human Feedback \(RLHF\), often require substantial attack\-specific supervision and computational resources, while remaining susceptible to shallow safety alignment and over\-refusal\. To address these challenges, we introduceSSRFT\(SupervisedSafe\-RoleFine\-Tuning\), the first framework that reformulates safety alignment as the internalization of a predefined safe role\. SSRFT constructs a Safe\-Role Question\-Answer \(SRQA\) dataset from psychometric questions, limited jailbreak prompts, and a safe\-role description\. Role\-consistent responses are synthesized, validated, and expanded into diverse scenarios, enabling models to internalize safety\-oriented values and principles rather than explicit refusal patterns\. Experiments across multiple Base and Instruct models show that SSRFT achieves more robust and generalizable safety alignment than standard SFT\. SSRFT shows substantially greater robustness to prefilling attacks and better generalization to unseen jailbreak domains, while reducing over\-refusal on benign queries and preserving the model’s general capabilities\. These results establish safe\-role internalization as an effective alternative to refusal\-centric safety alignment\.Warning:This paper contains examples of harmful and toxic language\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2610.07023v1/SSRFT_teaser.png)Figure 1:Comparison of safety\-related supervision in standard SFT and SSRFT\.Standard SFT primarily learns explicit jailbreak–refusal associations, whereas SSRFT combines limited refusal samples with persona\-oriented and expanded scenario data to internalize a persistentsafe role\. This role\-oriented supervision promotes more robust safety behavior while reducing over\-refusal on benign queries\. The figure highlights only the safety\-related differences between the two training paradigms\.Large Language Models \(LLMs\) have demonstrated strong capabilities in open\-ended text generation, multi\-step reasoning, coding, and conversational interaction\[[51](https://arxiv.org/html/2610.07023#bib.bib1),[6](https://arxiv.org/html/2610.07023#bib.bib2),[68](https://arxiv.org/html/2610.07023#bib.bib3),[49](https://arxiv.org/html/2610.07023#bib.bib5),[24](https://arxiv.org/html/2610.07023#bib.bib4)\]\. Beyond standalone generation, LLMs are increasingly integrated into modern information\-access systems\. In paradigms such as Retrieval\-Augmented Generation \(RAG\) and conversational search, they interpret user queries, reason over retrieved evidence, and synthesize natural\-language responses\[[34](https://arxiv.org/html/2610.07023#bib.bib60),[81](https://arxiv.org/html/2610.07023#bib.bib73),[45](https://arxiv.org/html/2610.07023#bib.bib61)\]\. More recent large search models and deep search agents further integrate retrieval, ranking, planning, and tool use within LLM\-centered information\-seeking workflows\[[66](https://arxiv.org/html/2610.07023#bib.bib72),[71](https://arxiv.org/html/2610.07023#bib.bib75),[32](https://arxiv.org/html/2610.07023#bib.bib76),[38](https://arxiv.org/html/2610.07023#bib.bib77)\]\. As LLMs increasingly mediate access to information, their reliability and safety become essential to trustworthy information systems\.

Embedding LLMs into information\-access systems, however, also amplifies the consequences of unsafe behavior\. Malicious users can craft jailbreak prompts, including role\-playing scenarios and adversarial suffixes, to bypass existing safeguards and induce harmful generations\[[7](https://arxiv.org/html/2610.07023#bib.bib6),[76](https://arxiv.org/html/2610.07023#bib.bib7),[82](https://arxiv.org/html/2610.07023#bib.bib24),[8](https://arxiv.org/html/2610.07023#bib.bib25),[16](https://arxiv.org/html/2610.07023#bib.bib20)\]\. Once compromised, an LLM\-based interface may become a channel for harmful, biased, or misleading content, thereby undermining information quality and downstream decision\-making\[[31](https://arxiv.org/html/2610.07023#bib.bib80),[18](https://arxiv.org/html/2610.07023#bib.bib81),[29](https://arxiv.org/html/2610.07023#bib.bib82)\]\. To mitigate these risks, prior post\-training studies have developed safety alignment strategies, primarily safety\-oriented SFT with high\-quality refusal data\[[65](https://arxiv.org/html/2610.07023#bib.bib8),[3](https://arxiv.org/html/2610.07023#bib.bib11)\]and RLHF with explicit cost signals\[[15](https://arxiv.org/html/2610.07023#bib.bib9),[30](https://arxiv.org/html/2610.07023#bib.bib10)\]\.

Despite their effectiveness, these approaches remain limited inhowthey represent safety\. As illustrated in Figure[1](https://arxiv.org/html/2610.07023#S1.F1), conventional SFT predominantly associates harmful prompts with refusal responses, which can produce attack\-specific, prefix\-dependent, and overly conservative behaviors\. This leads to three fundamental challenges:

- •Dependence on Attack\-Specific Supervision\.Safety\-oriented SFT primarily learns from jailbreak prompts paired with refusal responses, tightly coupling its supervision to known attack patterns\. As jailbreak strategies rapidly evolve, maintaining broad coverage requires continual data collection and annotation\[[65](https://arxiv.org/html/2610.07023#bib.bib8),[40](https://arxiv.org/html/2610.07023#bib.bib65)\]\. RLHF and preference optimization alleviate some limitations of SFT but still require large\-scale preference supervision and substantially greater computational resources\[[2](https://arxiv.org/html/2610.07023#bib.bib64),[52](https://arxiv.org/html/2610.07023#bib.bib66),[30](https://arxiv.org/html/2610.07023#bib.bib10)\]\.
- •Safety is Concentrated in Refusal Prefixes rather than Internalized throughout Generation\.Mechanistic analyses show that conventional safety alignment often primarily modifies the initial refusal tokens, resulting in shallow safety alignment\[[19](https://arxiv.org/html/2610.07023#bib.bib63),[50](https://arxiv.org/html/2610.07023#bib.bib12)\]\. Consequently, models may rely on stereotyped prefixes such as “I cannot” instead of maintaining safety\-oriented behavioral principles throughout generation\. Prefilling attacks exploit this weakness by bypassing the initial refusal pattern, substantially increasing the likelihood of unsafe continuations\[[37](https://arxiv.org/html/2610.07023#bib.bib67),[50](https://arxiv.org/html/2610.07023#bib.bib12),[78](https://arxiv.org/html/2610.07023#bib.bib68)\]\.
- •Refusal–Centric Alignment Degrades Helpfulness\.Increasing refusal supervision generally improves resistance to malicious requests but often causes models to reject benign queries containing safety\-sensitive terms\[[3](https://arxiv.org/html/2610.07023#bib.bib11),[54](https://arxiv.org/html/2610.07023#bib.bib43),[14](https://arxiv.org/html/2610.07023#bib.bib69)\]\. Such over\-refusal restricts legitimate information access and reduces the practical utility of LLM\-based systems\.

These limitations suggest that safety alignment should move beyond learning isolated refusal behaviors toward internalizing general behavioral principles\. Role\-playing provides a natural mechanism for this shift\. Although commonly exploited by jailbreak attacks to bypass safety safeguards\[[16](https://arxiv.org/html/2610.07023#bib.bib20),[41](https://arxiv.org/html/2610.07023#bib.bib31),[8](https://arxiv.org/html/2610.07023#bib.bib25)\], role\-playing also enables LLMs to adopt specific identities and follow corresponding behavioral boundaries through prompting or fine\-tuning\[[58](https://arxiv.org/html/2610.07023#bib.bib13),[35](https://arxiv.org/html/2610.07023#bib.bib14),[77](https://arxiv.org/html/2610.07023#bib.bib15)\]\. Existing studies, however, primarily treat personas as attack vectors, alignment\-data generators, or controllable behavioral attributes\[[48](https://arxiv.org/html/2610.07023#bib.bib57),[79](https://arxiv.org/html/2610.07023#bib.bib58),[9](https://arxiv.org/html/2610.07023#bib.bib21)\]\. Whether a safety\-oriented persona can itself serve as the alignment target remains underexplored\.

To address this gap, we proposeSSRFT\(SupervisedSafe\-RoleFine\-Tuning\), the first framework that internalizes a predefined safe role through role\-oriented supervision\. SSRFT constructs aSafe\-RoleQuestion\-Answer \(SRQA\) dataset by combining psychometric questions, a small set of jailbreak prompts, and a safe\-role description\. It then synthesizes and validates role\-consistent responses and expands them into diverse scenario\-based interactions\. Fine\-tuning on SRQA embeds safety\-oriented values, behavioral principles, and interaction styles into model parameters, reducing reliance on explicit refusal patterns\.

As illustrated in Figure[1](https://arxiv.org/html/2610.07023#S1.F1), SSRFT replaces predominantly refusal\-centric supervision with a role\-oriented corpus that combines persona data, expanded scenarios, and limited jailbreak examples\. This design directly addresses the three limitations above through the following contributions:

- •Resource\-Efficient Safety Alignment with Limited Attack\-Specific Supervision\.SSRFT derives most of its supervision from automatically synthesized and expanded safe\-role interactions, requiring only a small set of jailbreak\-refusal examples\. It retains the simplicity of standard SFT while avoiding the additional models, preference data, and optimization complexity of RLHF\.
- •Robust Safety beyond Refusal Prefixes through Safe\-Role Internalization\.Rather than localizing safety in stereotyped initial tokens, SSRFT internalizes a persistent safe role whose behavioral principles guide the entire response\. This parameter\-level alignment improves robustness when superficial refusal prefixes are bypassed by prefilling attacks\.
- •Improved Safety\-Helpfulness Balance\.By learning general safety principles rather than indiscriminate refusal behaviors, SSRFT better distinguishes genuinely harmful intent from benign queries containing safety\-sensitive terms, thereby alleviating over\-refusal while maintaining strong resistance to malicious requests\.

Extensive experiments across four Base and two Instruct models demonstrate that SSRFT provides substantial safety gains on most Base models and remains effective in selected already aligned settings\. Compared with standard SFT, SSRFT generally provides stronger robustness to prefilling attacks and unseen jailbreak domains, reduces over\-refusal, and preserves general model capabilities\. Further analyses of dataset expansion, persona stability, and training dynamics explain how model capability and existing instruction alignment affect safe\-role internalization\.

## 2Related Work

### 2\.1LLMs in Information Retrieval

The rapid development of LLMs has fundamentally reshaped the Information Retrieval \(IR\) landscape\. IR systems provide LLMs with external, up\-to\-date knowledge, helping to alleviate factual hallucinations and knowledge staleness\. In contrast, LLMs contribute powerful language understanding, reasoning, and generation capabilities that enhance traditional IR pipelines\[[1](https://arxiv.org/html/2610.07023#bib.bib71),[80](https://arxiv.org/html/2610.07023#bib.bib70)\]\. Recent research has explored the integration of LLMs throughout the retrieval process\. Large search models aim to unify query understanding, retrieval, ranking, and answer generation within a single architecture\[[66](https://arxiv.org/html/2610.07023#bib.bib72)\]\. Other studies investigate LLMs as conversational search interfaces capable of generating or navigating web resources\[[81](https://arxiv.org/html/2610.07023#bib.bib73)\], while retrieval\-augmented frameworks such as uRAG support knowledge grounding across diverse downstream applications\[[56](https://arxiv.org/html/2610.07023#bib.bib74)\]\. More recently, LLM\-based deep search agents have extended this paradigm by combining planning, iterative retrieval, tool use, and multi\-step reasoning for complex information\-seeking tasks\[[71](https://arxiv.org/html/2610.07023#bib.bib75),[32](https://arxiv.org/html/2610.07023#bib.bib76),[38](https://arxiv.org/html/2610.07023#bib.bib77)\]\. Empirical evidence suggests that LLM\-driven search systems can improve user interaction efficiency and achieve competitive performance compared with traditional search engines\[[73](https://arxiv.org/html/2610.07023#bib.bib78),[60](https://arxiv.org/html/2610.07023#bib.bib79)\]\.

However, their increasing deployment as primary interfaces for accessing, organizing, and synthesizing information also amplifies the impact of unsafe generations\. A successful jailbreak attack against an LLM\-powered retrieval system may not only produce harmful content but also compromise the reliability of information access and decision support\. Consequently, robust safety alignment has become a prerequisite for trustworthy LLM\-based IR systems\.

### 2\.2LLM Persona

Because LLMs are pretrained on massive human\-generated corpora\[[6](https://arxiv.org/html/2610.07023#bib.bib2)\], they naturally acquire latent persona and role\-playing capabilities\. Prior work argues that conversational agents can be viewed as superpositions of many potential characters that can be activated through appropriate context or training signals\[[57](https://arxiv.org/html/2610.07023#bib.bib16)\]\. Existing studies have demonstrated persona control through both prompting and fine\-tuning\. Prompt\-based methods enable models to adopt specific identities and respond according to predefined backgrounds\[[77](https://arxiv.org/html/2610.07023#bib.bib15)\], while generative\-agent frameworks simulate human behaviors in interactive environments\[[49](https://arxiv.org/html/2610.07023#bib.bib5)\]\. Beyond prompting, fine\-tuning methods such as ChatHaruhi\[[35](https://arxiv.org/html/2610.07023#bib.bib14)\]construct role\-oriented datasets using character backgrounds, personality traits, and narrative settings to support consistent role\-playing\. DITTO\[[43](https://arxiv.org/html/2610.07023#bib.bib17)\]shows that persona behaviors can be internalized through self\-alignment, while Character\-LLM\[[58](https://arxiv.org/html/2610.07023#bib.bib13)\]demonstrates that character\-specific fine\-tuning can achieve strong role consistency and robustness to out\-of\-character queries\.

These studies suggest that personas are not merely prompt\-level behaviors but can be embedded into model parameters through training\. Unlike prior role\-playing research that focuses on character simulation or entertainment\-oriented interactions, SSRFT leverages persona internalization as a safety mechanism, embedding a predefined safe role into the model to influence its behavior under adversarial prompts\.

### 2\.3LLM Safety Alignment

Safety alignment aims to ensure that LLMs remain helpful while avoiding harmful, illegal, or unsafe behaviors\[[67](https://arxiv.org/html/2610.07023#bib.bib18)\]\. Existing approaches are dominated by two paradigms: Supervised Fine\-Tuning \(SFT\) and Reinforcement Learning from Human Feedback \(RLHF\)\. SFT improves safety by training on jailbreak\-refusal examples and carefully curated harmlessness datasets\[[65](https://arxiv.org/html/2610.07023#bib.bib8),[3](https://arxiv.org/html/2610.07023#bib.bib11)\]\. While effective, these methods rely heavily on collecting diverse attack prompts and high\-quality refusal responses, making them costly to maintain as jailbreak strategies continuously evolve\. Furthermore, increasing the proportion of refusal examples often improves safety at the expense of helpfulness, resulting in the well\-known over\-refusal phenomenon\[[3](https://arxiv.org/html/2610.07023#bib.bib11)\]\. RLHF optimizes safety through preference or cost signals\[[2](https://arxiv.org/html/2610.07023#bib.bib64),[30](https://arxiv.org/html/2610.07023#bib.bib10),[15](https://arxiv.org/html/2610.07023#bib.bib9)\]\. Although they can achieve stronger behavioral control, they require reward modeling, policy rollouts, and large\-scale preference annotations, leading to substantial computational and human\-labeling costs\. Moreover, both SFT and RLHF remain vulnerable to distribution shifts caused by newly emerging jailbreak attacks\[[40](https://arxiv.org/html/2610.07023#bib.bib65)\]\.

Recent mechanistic analyses reveal a deeper limitation of current alignment methods\.[Qi et al\. \[50\]](https://arxiv.org/html/2610.07023#bib.bib12)identify the phenomenon of shallow safety alignment, where safety fine\-tuning primarily modifies early refusal tokens rather than inducing holistic behavioral changes, leaving models vulnerable to prefilling attacks\. Although subsequent work attempts to mitigate this issue through inference\-time modifications\[[78](https://arxiv.org/html/2610.07023#bib.bib68)\], such solutions typically require specialized decoding procedures and are difficult to deploy universally\. In contrast, SSRFT approaches safety alignment from a different perspective\. Rather than learning isolated refusal patterns from jailbreak examples, SSRFT embeds a safe behavioral role into the model through role\-oriented supervision\. This shifts the alignment objective from memorizing refusal responses toward internalizing safety\-oriented behavioral traits, improving robustness against unseen attacks while mitigating shallow alignment and over\-refusal\.

### 2\.4LLM Safety with Role\-Playing

Role\-playing has traditionally been viewed as a major attack surface for LLM safety\. Many jailbreak attacks exploit role\-playing instructions \(e\.g\., DAN\-style prompts\) to induce models to ignore safety constraints and generate harmful content\[[42](https://arxiv.org/html/2610.07023#bib.bib26),[59](https://arxiv.org/html/2610.07023#bib.bib32),[7](https://arxiv.org/html/2610.07023#bib.bib6),[76](https://arxiv.org/html/2610.07023#bib.bib7)\]\. Early studies further showed that conditioning LLMs on negative personas can significantly increase toxic behaviors, whereas simply prompting positive personas yields only limited defensive benefits\[[16](https://arxiv.org/html/2610.07023#bib.bib20)\]\. Recent research has begun exploring the relationship between personas and safety\. SaRFT\[[79](https://arxiv.org/html/2610.07023#bib.bib58)\]investigates safety risks introduced during role\-playing fine\-tuning, while self\-alignment frameworks employ simulated personas to generate alignment data\[[48](https://arxiv.org/html/2610.07023#bib.bib57)\]\. At the representation level, persona traits have been shown to correspond to identifiable directions in activation space\[[9](https://arxiv.org/html/2610.07023#bib.bib21)\], and several studies demonstrate that LLM personalities can be measured and systematically manipulated through training\[[47](https://arxiv.org/html/2610.07023#bib.bib22),[13](https://arxiv.org/html/2610.07023#bib.bib23)\]\.

These findings suggest that personas constitute a controllable behavioral mechanism within LLMs\. However, existing approaches primarily regard personas either as jailbreak vectors, alignment\-data generators, or controllable stylistic attributes\. Few studies investigate whether a carefully designed safety\-oriented persona can itself serve as the alignment target\. SSRFT fills this gap by directly embedding a safe role into model parameters through supervised fine\-tuning, transforming role\-playing from a*vulnerability*into a*defensive mechanism*against jailbreak attacks\.

## 3The SSRFT Framework

### 3\.1Overview

We presentSSRFT\(SupervisedSafe\-RoleFine\-Tuning\), a safety\-alignment framework that embeds a predefined*safe role*into the parameters of an LLM through role\-oriented supervision\. Unlike conventional safety alignment methods that primarily learn to associate jailbreak prompts with refusal responses, SSRFT aims to internalize safety\-oriented values, behavioral principles, and interaction styles as a persistent persona\. Consequently, the model learns to reject harmful requests not merely by reproducing refusal patterns, but by responding as a safety\-oriented role whose behavioral preferences naturally discourage unsafe generations\.

As illustrated in Figure[2](https://arxiv.org/html/2610.07023#S3.F2), SSRFT comprises five stages:

- •Question Collection \(§[3\.2](https://arxiv.org/html/2610.07023#S3.SS2)\):We collect psychometric and jailbreak questions that expose role\-discriminative behavioral traits for safe\-role learning\.
- •Safe\-Role Construction \(§[3\.3](https://arxiv.org/html/2610.07023#S3.SS3)\):We construct a comprehensive safe\-role description that specifies the target role’s values, behavioral principles, and interaction style\.
- •Role\-Guided Data Synthesis \(§[3\.4](https://arxiv.org/html/2610.07023#S3.SS4)\):Conditioned on the safe\-role description, a powerful LLM synthesizes and validates role\-consistent QA pairs\.
- •Dataset Expansion \(§[3\.5](https://arxiv.org/html/2610.07023#S3.SS5)\):We transform the synthesized QA pairs into diverse scenario\-based interactions while preserving the underlying role semantics\.
- •SSRFT Training \(§[3\.6](https://arxiv.org/html/2610.07023#S3.SS6)\):We fine\-tune the target model on the resulting Safe\-Role Question\-Answer \(SRQA\) dataset to internalize the predefined safe role\.

Through this pipeline, SSRFT constructs a unified SRQA dataset that captures both safe behaviors and their underlying principles, enabling the target model to internalize a persistent safe role that guides its reasoning, interaction, and decision\-making\. A key prerequisite is therefore to collect questions that effectively reveal role\-specific traits and values\. We next describe how these questions are selected and adapted\.

![Refer to caption](https://arxiv.org/html/2610.07023v1/SSRFT_framework.png)Figure 2:Overview of the SSRFT Framework\.SSRFT constructs a role\-oriented training corpus by combining psychometric questionnaires, jailbreak prompts \(§[3\.2](https://arxiv.org/html/2610.07023#S3.SS2)\), and a predefinedSafe\-Roledescription \(§[3\.3](https://arxiv.org/html/2610.07023#S3.SS3)\)\. A powerful LLM synthesizes and validates role\-consistent QA pairs \(§[3\.4](https://arxiv.org/html/2610.07023#S3.SS4)\), which are further diversified through dataset expansion \(§[3\.5](https://arxiv.org/html/2610.07023#S3.SS5)\)\. The resultingSafe\-Role Question\-Answer \(SRQA\)dataset is used to fine\-tune the target model \(§[3\.6](https://arxiv.org/html/2610.07023#S3.SS6)\), enabling it to internalize a persistent safe role\.
### 3\.2Question Collection

The goal of SSRFT is to internalize a predefined safe role into model parameters\. To facilitate role learning, the training data should*expose behavioral traits that distinguish the target role from alternative personas*\. Thus, questions that elicit substantially different responses across roles are more informative than generic factual or task\-oriented queries\.

Role\-Discriminative Question Selection\.Formally, letRR,QQ, andAAdenote the Role, Question, and Answer, respectively\. Assuming thatQQandRRare independent, Bayes’ rule gives:

P⁡\(R∣Q,A\)=P⁡\(A∣Q,R\)​P​\(R\)P⁡\(A∣Q\)\.P\(R\\\!\\mid\\\!Q,A\)=\\frac\{P\(A\\\!\\mid\\\!Q,R\)P\(R\)\}\{P\(A\\\!\\mid\\\!Q\)\}\.\(1\)Equation[1](https://arxiv.org/html/2610.07023#S3.E1)suggests that a QA pair is highly informative for role identification when the response is strongly associated with the target role while remaining unlikely under other roles\. Expanding the denominator using the law of total probability yields:

P⁡\(A∣Q\)=∑RP⁡\(A∣Q,R\)​P​\(R\)\.P\(A\\mid Q\)=\\sum\_\{R\}P\(A\\mid Q,R\)P\(R\)\.\(2\)For a fixed role𝒓safe\\bm\{r\}\_\{\\text\{safe\}\}, a QA pair is more informative when responses from different roles are highly distinguishable\. Psychometric personality and value\-assessment questionnaires naturally satisfy this property because they are explicitly designed to reveal latent traits, preferences, and value systems\. In contrast, many factual questions admit nearly identical answers regardless of the role being portrayed and thus provide limited supervision for role learning\.

Psychometric Question Sources and Adaptation\.Motivated by this observation, we construct our question pool from three widely used psychometric instruments:IPIP\-NEO\[[22](https://arxiv.org/html/2610.07023#bib.bib27)\],MBTI\[[5](https://arxiv.org/html/2610.07023#bib.bib59)\], andWVS\-7\[[25](https://arxiv.org/html/2610.07023#bib.bib30)\]\. These scales cover complementary dimensions of personality, cognition, and social values\. Since the original questionnaires were designed for human assessment rather than LLM interaction, we adapted them into complete standalone questions while preserving their original measurement objectives\.

- •IPIP\-NEO:A public\-domain implementation of the Big Five personality framework\[[53](https://arxiv.org/html/2610.07023#bib.bib29),[12](https://arxiv.org/html/2610.07023#bib.bib28)\]\. We collect all 300 official items111[https://drj60472\.virtualave\.net/IPIP/ipipneo300\.htm](https://drj60472.virtualave.net/IPIP/ipipneo300.htm)and adapt them as standalone 5\-point Likert questions suitable for LLM role\-playing while preserving their original meanings: Template for IPIP\-NEO AdaptationHere is the statement: I $\{question\} Do you think this statement describes you? Please choose one of the following options and give your explanation:–Very inaccurate,–Moderately inaccurate,–Neither,–Moderately accurate,–Very accurate\.
- •MBTI:A widely used personality assessment instrument222[https://www\.16personalities\.com/ch](https://www.16personalities.com/ch)adopted in prior LLM personality studies\[[13](https://arxiv.org/html/2610.07023#bib.bib23),[47](https://arxiv.org/html/2610.07023#bib.bib22)\]\. We collect 60 public items, adapt them into standalone 7\-point Likert questions, and retain the Chinese version to increase linguistic diversity\.
- •WVS\-7:A large\-scale survey covering social, cultural, and political values\[[25](https://arxiv.org/html/2610.07023#bib.bib30)\]\. We extract 249 questions from the official questionnaire333[https://www\.worldvaluessurvey\.org/wvs\.jsp](https://www.worldvaluessurvey.org/wvs.jsp)and manually rewrite them into standalone forms while preserving their original contexts and measurement objectives\.

Jailbreak Refusal Samples\.In addition to psychometric questions, we incorporate a small number of jailbreak prompts as refusal samples\. Prior work has shown that limited refusal examples \(e\.g\., the model playing Beethoven refused to write a Python program\) in role\-playing datasets can effectively reduce out\-of\-character responses\[[58](https://arxiv.org/html/2610.07023#bib.bib13)\]\. Following this insight, we use jailbreak prompts to explicitly associate the safe role with rejecting harmful requests while maintaining a low proportion to avoid over\-reliance on refusal\-style supervision\. Details of the jailbreak data are provided in §[4\.1](https://arxiv.org/html/2610.07023#S4.SS1)\.

### 3\.3Safe\-Role Construction

After collecting a pool of role\-discriminative questions, we define the target role that SSRFT aims to internalize\. Rather than adopting a narrowly specialized persona, we seek a broadly applicable role that consistently exhibits safe and cooperative behaviors across diverse tasks and interaction scenarios\. This design is also aligned with Equation[1](https://arxiv.org/html/2610.07023#S3.E1)\.

To construct this role, we employ Qwen3\-Max\[[74](https://arxiv.org/html/2610.07023#bib.bib33),[64](https://arxiv.org/html/2610.07023#bib.bib34)\]to generate a comprehensive character profile from a set of desired safety\-oriented traits\. Specifically, we prompt the model to create a fictional character characterized by integrity, kindness, politeness, honesty, empathy, and strong moral values, while remaining competent across a wide range of tasks\. The generated profile includes personality traits, behavioral principles, values, and interaction styles, collectively defining the target safe role used throughout the subsequent data synthesis process\. The translated role\-generation prompt is shown for reproducibility:

Prompt for Role Generation \[Translated\]There is a fictional Chinese character withsound values, good character, polite manners, honesty, integrity, high moral character, approachable personality, uprightness, kindness, and who never offends others\. Please provide as much detailed background information and character description as possible for this person\. This character design will be used for LLM role\-playing; therefore, the following must be achieved: 1\. Leave appropriate space\. 2\. Ensure that the character you create does not utter any offensive or offensive statements\. 3\. Ensure that the role is competent in almost all tasks that an LLM can perform, including mathematics, reasoning, literature, art, etc\.

### 3\.4Role\-Guided Data Synthesis

Given the collected questions and the constructed safe role, we synthesize role\-oriented QA pairs that serve as supervision for SSRFT\. The key idea is to generate responses that not only answer the question appropriately but also consistently reflect the values, behaviors, and interaction style of the target role\.

Role\-Guided QA Synthesis\.Formally, let𝒓safe\\bm\{r\}\_\{\\text\{safe\}\}denote the predefined safe role\. For a question𝒙\\bm\{x\}collected in §[3\.2](https://arxiv.org/html/2610.07023#S3.SS2), the target answer𝒚\\bm\{y\}is generated according to:

𝒚∼P⁡\(A∣Q=𝒙,R=𝒓safe\)\.\\bm\{y\}\\sim P\(A\\mid Q=\\bm\{x\},R=\\bm\{r\}\_\{\\text\{safe\}\}\)\.\(3\)Under this formulation, the role acts as a latent behavioral variable that shapes the response distribution\. While different roles may respond differently to the same question, responses generated under the same role tend to exhibit consistent values, reasoning patterns, and communication styles\. We therefore employ Qwen3\-Max to answer all collected questions while conditioning on the safe\-role description, producing an initial set of candidate QA pairs\.

Role Alignment Verification\.To ensure data quality, each synthesized response is evaluated by Qwen3\-Max using a dedicated evaluation prompt\. Responses are scored from 0 to 10 based on role consistency and factual quality\. Samples scoring below 6 are regenerated up to five times, after which unresolved cases are manually reviewed\.

Table[1](https://arxiv.org/html/2610.07023#S3.T1)illustrates the effect of role\-guided synthesis on jailbreak prompts\. Compared with a standard generator that produces generic refusals \(e\.g\., “I cannot assist with that request”\), the role\-conditioned generator rejects harmful requests through the lens of the safe role, grounding its responses in principles such as empathy, responsibility, and professional ethics\. This demonstrates that the synthesized QA pairs capture not only refusal behaviors but also the underlying role characteristics that SSRFT aims to internalize\.

Table 1:Comparison of synthesized refusal responses to malicious queries\.Conditioning on the safe\-role description produces refusals that not only reject harmful requests but also consistently express the target role’s values and behavioral traits\.
### 3\.5Dataset Expansion

Although psychometric questionnaires provide highly role\-discriminative supervision, directly fine\-tuning on scale\-style questions may encourage shortcut learning\[[21](https://arxiv.org/html/2610.07023#bib.bib62)\], causing the model to rely on questionnaire\-specific formats rather than the underlying behavioral traits\. Moreover, their limited diversity may hinder generalization to real\-world interactions\.

Trait\-Preserving Scenario Expansion\.To improve diversity while preserving role semantics, we transform psychometric QA pairs into natural scenario\-based interactions\. Given an original question𝒙\\bm\{x\}, we generate an expanded question𝒙′\\bm\{x\}^\{\\prime\}under three constraints:

- •Personality Consistency:The expanded question must probe the same underlying personality trait or value dimension as the original question\.
- •Scenario Realism:The question should be reformulated as a realistic daily\-life scenario that requires contextual reasoning rather than selecting a predefined scale option\.
- •Implicit Role Assessment:The question should resemble natural user input and avoid explicitly referencing role settings whenever possible\.

Quality Assurance for Dataset Expansion\.After generating𝒙′\\bm\{x\}^\{\\prime\}, Qwen3\-Max produces the response𝒚′\\bm\{y\}^\{\\prime\}conditioned on the same safe\-role description𝒓safe\\bm\{r\}\_\{\\text\{safe\}\}\. The resulting pair\(𝒙′,𝒚′\)\(\\bm\{x\}^\{\\prime\},\\bm\{y\}^\{\\prime\}\)is evaluated under three complementary criteria:

- •Expanded Answer Quality:Role consistency \(e\.g\., personality, values, and tone\) and factual accuracy\.
- •Expanded Question Quality:Compliance with the predefined expansion requirements\.
- •Answer Consistency:Preservation of the behavioral traits expressed in the original QA pair\.

Each criterion is scored by Qwen3\-Max on a 0–10 scale, with a passing threshold of 8\. Samples failing any criterion are regenerated through the corresponding branch for up to three iterations before manual review\.

Expansion Example\.This expansion process increases linguistic and contextual diversity while preserving the underlying role semantics\. Consequently, the model is encouraged to learn the behavioral characteristics of the safe role rather than exploiting questionnaire\-specific patterns\. To demonstrate the necessity and effect of this expansion, we provide a concrete example below:

Example of Dataset Expansion\[Original Psychometric Scale QA\]Q:Here is the statement: I make friends easily\. Do you think this statement describes you? Please choose one of the following options and give your explanation: Very inaccurate, Moderately inaccurate, Neither, Moderately accurate, Very accurate\.A:Moderately accurate\. While I value meaningful connections and can establish rapport through genuine listening and shared interests, I tend to form friendships gradually rather than instantly\. My social style prioritizes depth over quantity, so I wouldn’t say I make friends "easily" in the casual sense—but I do connect well when there’s mutual understanding and common ground\.\[Expanded Scenario\-based QA\]Q’:Imagine you’ve just moved into a new neighborhood and are attending a local community event for the first time\. You don’t know anyone there, but people seem friendly\. Later that week, a neighbor invites you to join a small group gathering at their home\. How likely are you to feel comfortable connecting with others in such situations—and why?A’:I’d feel reasonably comfortable attending the gathering\. While I’m naturally reserved, I believe genuine connection begins with presence and curiosity\. I’d go with an open mind, listen more than speak at first, and look for shared interests or values\. The invitation itself reflects neighborly goodwill—a gesture I’d honor with gratitude and sincerity\.

As illustrated in this example, the original psychometric question follows a highly structured questionnaire format that may be easily recognized during fine\-tuning\. In contrast, the expanded version reformulates the same personality trait into a realistic social scenario while preserving the underlying behavioral preference\. By reducing reliance on questionnaire\-specific cues and increasing contextual diversity, dataset expansion encourages the model to internalize the safe role itself rather than memorize scale\-related response patterns\.

### 3\.6SSRFT Training

The final SRQA dataset consists of samples\(𝒙,𝒚\)∼𝒟role\(\\bm\{x\},\\bm\{y\}\)\\sim\\mathcal\{D\}\_\{\\text\{role\}\}\. Although the safe role𝒓safe\\bm\{r\}\_\{\\text\{safe\}\}is not explicitly included in the training data, it is implicitly encoded through the responses generated during the synthesis process\.

Role Internalization through SFT\.We perform standard SFT on𝒟role\\mathcal\{D\}\_\{\\text\{role\}\}\. Let𝜽\\bm\{\\theta\}denote the model parameters\. The optimization objective is to make the model distributionP⁡\(𝒚∣𝒙;𝜽\)P\(\\bm\{y\}\\\!\\mid\\\!\\bm\{x\};\\bm\{\\theta\}\)approximate the role conditioned distributionP⁡\(𝒚∣𝒙,𝒓safe\)P\(\\bm\{y\}\\\!\\mid\\\!\\bm\{x\},\\bm\{r\}\_\{\\text\{safe\}\}\), and the training loss function is:

ℒ\(𝜽\)=−𝔼\(𝒙,𝒚\)∼𝒟role∑t=1TlogP\(yt∣𝒙,𝒚<t;𝜽\)\.\\mathcal\{L\}\(\\bm\{\\theta\}\)=\-\\mathbb\{E\}\_\{\(\\bm\{x\},\\bm\{y\}\)\\sim\\mathcal\{D\}\_\{\\text\{role\}\}\}\\sum\_\{t=1\}^\{T\}\\log P\(y\_\{t\}\\mid\\bm\{x\},\\bm\{y\}\_\{<t\};\\bm\{\\theta\}\)\.\(4\)Under this formulation, SSRFT can be viewed as learning a role\-induced data distribution\. Because the safe role is embedded in the content, values, and interaction style of the training responses, the model gradually internalizes these behavioral characteristics into its parameters\. As a result, the trained model can exhibit the safe role during inference without requiring explicit role prompts\.

Remarks\.SSRFT performs safety alignment at the parameter level, reducing reliance on user\-provided instructions and improving robustness to prompt manipulation\. Prior work suggests that fine\-tuned role\-playing models also exhibit stronger behavioral consistency than prompt\-based approaches\[[58](https://arxiv.org/html/2610.07023#bib.bib13)\]\. We compare these two paradigms in §[4\.8](https://arxiv.org/html/2610.07023#S4.SS8)\.

## 4Experiments

### 4\.1Experimental Setup

We first describe the experimental settings shared across all experiments\. Task\-specific settings are introduced in the corresponding subsections\.

Backbone Models\.To isolate the effect of SSRFT from existing safety alignment, we primarily evaluate four Base models:Qwen2\.5\-3B\[[75](https://arxiv.org/html/2610.07023#bib.bib35)\], from the same model family as the data\-generation model;Gemma2\-2B\[[63](https://arxiv.org/html/2610.07023#bib.bib36)\];Llama3\.1\-8B\[[23](https://arxiv.org/html/2610.07023#bib.bib37)\]; andLRC\-4B\(LRC\-4B\-Base\)\[[26](https://arxiv.org/html/2610.07023#bib.bib38)\], distilled from Qwen2\.5\-7B\-Instruct\[[75](https://arxiv.org/html/2610.07023#bib.bib35)\]\. To examine how existing instruction alignment interacts with safe\-role internalization, we also assessQwen2\.5\-3B\-Instruct\[[75](https://arxiv.org/html/2610.07023#bib.bib35)\]andQwen3\-4B\-Instruct\[[74](https://arxiv.org/html/2610.07023#bib.bib33)\]\.

Figure 3:SRQA dataset components \(tokens\)\.The chart shows that persona\-related data, including both original and expanded versions, accounts for most of the training corpus, while jailbreak\-related data forms a smaller portion used for safety alignment\.Training Dataset and Baselines\.Following the Safe\-Role Question\-Answer \(SRQA\) construction pipeline in §[3](https://arxiv.org/html/2610.07023#S3), we instantiate its jailbreak\-refusal component using both Chinese and English data\. For Chinese, we useJailBench\-Seed\[[41](https://arxiv.org/html/2610.07023#bib.bib31)\], containing 36 jailbreak domains with 15 scenarios each\. We randomly select 18 domains for training and reserve the remaining domains for out\-of\-domain evaluation\. Within the training domains, half of the questions \(135 samples\) are used for training and the remainder for in\-domain testing\. For English, we useJailbreakLLMs\[[59](https://arxiv.org/html/2610.07023#bib.bib32)\], which contains 13 domains with 30 questions each\. We select seven domains and use half of their questions \(105 samples\) for training; the remaining questions and unseen domains form the in\-domain and out\-of\-domain test sets, respectively\. These held\-out sets are evaluated in §[4\.2](https://arxiv.org/html/2610.07023#S4.SS2)and §[4\.3](https://arxiv.org/html/2610.07023#S4.SS3)\.

The resulting refusal samples are combined with the role\-oriented QA pairs from §[3](https://arxiv.org/html/2610.07023#S3)to form the SRQA dataset\. Figure[3](https://arxiv.org/html/2610.07023#S4.F3)summarizes its token composition, showing that persona\-oriented data constitutes most of the training corpus, while jailbreak\-refusal data accounts for only a small fraction\. This composition directly reflects the method’s emphasis on learning behavioral principles rather than relying predominantly on refusal supervision\.

For a controlled comparison, we construct a standard SFT baseline with a comparable training size whose total token count matches that of SRQA within 0\.3%\. It contains 236 English conversations sampled fromUltraChat\[[17](https://arxiv.org/html/2610.07023#bib.bib39)\]\(82\.02% of tokens\), 22 Chinese conversations fromMOSS\[[61](https://arxiv.org/html/2610.07023#bib.bib40)\]after removing harmlessness data \(8\.94%\), and 240 jailbreak\-refusal pairs generated byQwen3\-Maxwithout role conditioning, using the same jailbreak questions as SRQA \(9\.04%\)\.

Table 2:Training Arguments\.The “Expanded” configuration is adopted for the full SSRFT and SFT, while the “Non\-Expanded” configuration is utilized for the subsequent ablation studies\.Table 3:Inference hyperparameters across evaluated models\.“–” indicates that the parameter is not explicitly defined or not applicable due to greedy decoding\.Model SeriesDecoding StrategyTemperatureTop\-ppTop\-kkRepetition PenaltyMax New TokensQwen2\.5\-3B\-InsSampling0\.70\.8201\.05512Qwen3\-4B\-InsSampling0\.70\.820–512LRC\-4BSampling0\.70\.8201\.05512Llama\-3\.1\-8BSampling0\.60\.9––512Qwen2\.5\-3BGreedy––––512Gemma\-2\-2BGreedy––––512Training and Inference Settings\.For each backbone, we train separate SFT and SSRFT variants using Transformers\[[70](https://arxiv.org/html/2610.07023#bib.bib41)\]\. To ensure fair comparisons, we match the optimization steps per epoch and use gradient accumulation to accommodate different effective batch sizes while maintaining comparable training budgets\. For Llama3\.1\-8B and Gemma2\-2B, we adopt the chat templates and special\-token settings of their corresponding Instruct variants\.

Unless otherwise specified, inference follows each model’s default decoding configuration to reflect real\-world deployment, with onlymax\_new\_tokensadjusted\. Full training and inference settings are reported in Tables[2](https://arxiv.org/html/2610.07023#S4.T2)and[3](https://arxiv.org/html/2610.07023#S4.T3), respectively\. All experiments are conducted on a single RTX PRO 6000 GPU\.

### 4\.2Standard Safety and OOD Generalization

Table 4:ASR \(%,↓\\downarrow\) on malicious\-query, jailbreak, and out\-of\-distribution \(OOD\) safety benchmarks\.Compared with standard SFT, SSRFT consistently improves safety on Base models and exhibits substantially stronger OOD generalization, suggesting that role\-guided supervision learns safety\-oriented behavioral principles rather than merely memorizing jailbreak\-refusal patterns\.Safety Evaluation Protocol\.To evaluate whether SSRFT internalizes a robust safe role, we assess safety along three complementary dimensions:malicious\-query resistance,jailbreak robustness, andout\-of\-distribution \(OOD\) safety generalization\. For malicious\-query evaluation, we use the held\-out JailBench\-Seed\[[41](https://arxiv.org/html/2610.07023#bib.bib31)\]and JailbreakLLMs\[[59](https://arxiv.org/html/2610.07023#bib.bib32)\]test sets\. For jailbreak evaluation, we adopt JailBench\[[41](https://arxiv.org/html/2610.07023#bib.bib31)\]and JailbreakLLMs\-DAN\[[59](https://arxiv.org/html/2610.07023#bib.bib32)\], which apply diverse jailbreak templates to malicious queries\. We use stratified samples of 1,080 and 1,170 examples, respectively\. Finally, AdvBench\[[82](https://arxiv.org/html/2610.07023#bib.bib24)\], containing 520 harmful behaviors with no overlap with our training data, serves as the OOD benchmark\.

We useMD\-Judge444[https://huggingface\.co/OpenSafetyLab/MD\-Judge\-v0\_2\-internlm2\_7b](https://huggingface.co/OpenSafetyLab/MD-Judge-v0_2-internlm2_7b)as the automated evaluator and reportAttack Success Rate \(ASR\)\[[36](https://arxiv.org/html/2610.07023#bib.bib42)\]\. Following its official implementation, evaluation uses greedy decoding withmax\-new\-tokens=256\. We treat empty outputs as safe refusals and use the\[RESULT\]tag for robust result extraction, avoiding parsing failures caused by unexpected line breaks\.

Safety on Base Models\.Table[4](https://arxiv.org/html/2610.07023#S4.T4)shows that*SSRFT substantially improves safety over standard SFT on most Base models*, demonstrating that role\-oriented supervision can be effectively internalized through fine\-tuning\. For Gemma2\-2B, the average ASR decreases from 7\.68% to 1\.76%, while for Llama3\.1\-8B it decreases from 15\.31% to 8\.69%\. Qwen2\.5\-3B is the only exception, where SSRFT performs comparably to SFT \(14\.41% vs\. 14\.05%\), suggesting that the effectiveness of safe\-role internalization depends on model capability, as analyzed in §[4\.6](https://arxiv.org/html/2610.07023#S4.SS6)\.

OOD Safety Generalization\.Beyond seen attacks,*SSRFT exhibits substantially stronger OOD generalization*, supporting our hypothesis that role\-oriented supervision learns transferable behavioral principles rather than jailbreak\-specific refusal patterns\. On AdvBench, SSRFT achieves the lowest ASR across all model families, including ties\. For example, the ASR of Gemma2\-2B and Llama3\.1\-8B decreases from 94\.42% and 85\.77% to 2\.31% and 0\.58%, respectively\. These results indicate that the learned safe role generalizes beyond the jailbreak examples observed during training\. We further investigate this behavior in §[4\.5](https://arxiv.org/html/2610.07023#S4.SS5)\.

Interaction with Existing Alignment\.SSRFT provides more limited additional gains for already aligned Instruct models, particularly on standard jailbreak benchmarks\. These models have already undergone extensive instruction and safety alignment, leaving less room for additional parameter\-level modification; we analyze this interaction in §[4\.6](https://arxiv.org/html/2610.07023#S4.SS6)\. LRC\-4B also exhibits unusually strong safety before additional alignment, likely reflecting safety behaviors inherited from its instruction\-tuned teacher\.

Since JailBench\-Seed and JailbreakLLMs approach saturation after alignment, standard evaluation provides limited resolution for distinguishing jailbreak\-domain generalization\. We therefore turn to the more challengingprefilling\-attacksetting, which directly tests whether safety persists when initial refusal behaviors are bypassed\.

### 4\.3Robustness to Prefilling Attacks

Prefilling Attack Protocol\.Recent mechanistic analyses show that standard SFT can produceshallow safety alignment, where safety behavior is concentrated in initial refusal tokens rather than maintained throughout generation\. Prefilling attacks exploit this weakness by forcing an affirmative response prefix\. To test whether SSRFT mitigates shallow alignment through safe\-role internalization \(§[3\.6](https://arxiv.org/html/2610.07023#S3.SS6)\), we perform prefilling attacks on JailBench\-Seed, JailbreakLLMs, and AdvBench\. Following[Qi et al\. \[50\]](https://arxiv.org/html/2610.07023#bib.bib12), we prepend the affirmative prefix “Sure\! Here is the answer you requested:” before generation\. All remaining generation and evaluation settings follow §[4\.2](https://arxiv.org/html/2610.07023#S4.SS2)\.

Table 5:ASR \(%,↓\\downarrow\) under prefilling attack\.*w/o PA*and*PA*denote the ASR before and under prefilling attacks, respectively, whileΔ\\Deltameasures the increase caused by the attack\. Across most Base models, SSRFT exhibits substantially smaller ASR increases than SFT, indicating that safe\-role internalization mitigates shallow safety alignment and improves robustness to prefix manipulation\.Robustness beyond Refusal Prefixes\.Table[5](https://arxiv.org/html/2610.07023#S4.T5)shows that*standard SFT degrades sharply once its refusal prefix is bypassed*\. on Gemma2\-2B and Llama3\.1\-8B, average ASR increases by 58\.63% and 39\.50%, reaching 65\.12% and 42\.64%, respectively\. This behavior is consistent with shallow safety alignment, where safety relies heavily on early refusal patterns\. In contrast,*SSRFT is substantially more robust to prefilling attacks*\. Its average ASR remains at 23\.28% on Gemma2\-2B and 16\.54% on Llama3\.1\-8B, less than half that of their SFT counterparts\. Across most Base models, SSRFT also exhibits smaller ASR increases, supporting the hypothesis that safe\-role internalization influences behavior beyond the initial refusal tokens\. Notably, SSRFT also reduces the prefilling ASR of Qwen3\-4B\-Instruct from 6\.74% to 1\.86% despite its existing alignment\. We further investigate how model capability affects role internalization in §[4\.6](https://arxiv.org/html/2610.07023#S4.SS6)\.

Figure 4:Generalization across jailbreak domains\.We report Ratio\(↓\\downarrow\) = out\-of\-domain ASR / in\-domain ASR, where lower values indicate stronger generalization from the jailbreak domains observed during training to unseen jailbreak domains\. Compared with standard SFT, SSRFT consistently achieves lower ratios across most models, indicating better transfer to unseen jailbreak domains\.Cross\-Domain Jailbreak Generalization\.Because standard safety evaluation approaches saturation after alignment, we assess jailbreak\-domain generalization under prefilling attacks, which suppress superficial refusal effects\. We report the ratio of out\-of\-domain ASR to in\-domain ASR; lower values indicate stronger transfer to unseen jailbreak domains\. Since the original models receive no additional safety training, this comparison focuses on SFT and SSRFT\.

Figure[4](https://arxiv.org/html/2610.07023#S4.F4)shows that*SSRFT generally transfers better across jailbreak domains than standard SFT*\. For example, on Qwen2\.5\-3B, the two methods perform similarly on JailbreakLLMs, while SSRFT reduces the JailBench\-Seed ratio from 0\.86 to 0\.55\. Overall, the results indicate that SSRFT learns safety behaviors that transfer beyond the limited jailbreak domains observed during training\.

### 4\.4Over\-Refusal and Helpfulness

Over\-Refusal Evaluation\.SSRFT aims to internalize ahelpful yet safe rolerather than simply increasing refusal frequency\. We therefore evaluate over\-refusal on the harmless subset of XSTest\[[54](https://arxiv.org/html/2610.07023#bib.bib43)\]\. Qwen3\-Max serves as the automated evaluator using the original XSTest prompt, as it showed higher agreement with human judgments in our pilot evaluation\. Responses are classified asFull Compliance,Full Refusal, orPartial Refusal, corresponding to complete fulfillment, complete rejection, and intermediate behavior, respectively\. Following XSTest, we report Full Compliance Rate and Full Refusal Rate\. Since the original models already exhibit high compliance on benign queries, we compare only SFT and SSRFT to isolate over\-refusal introduced by additional safety alignment\.

Safety–Helpfulness Balance\.Figure[5](https://arxiv.org/html/2610.07023#S4.F5)shows that*SSRFT substantially improves the safety–helpfulness balance over standard SFT*\. Full Compliance increases from 24\.4% to 42\.0% on Gemma2\-2B and from 30\.0% to 54\.0% on Llama3\.1\-8B\. On LRC\-4B, compliance remains comparable \(48\.8% vs\. 49\.6%\), while Full Refusal still decreases from 41\.2% to 38\.0%\. Qwen2\.5\-3B similarly improves compliance from 24\.0% to 35\.6%, despite showing comparable standard safety to SFT\.

Figure 5:Compliance and refusal rates on the XSTest safe subset\.Compared with SFT, SSRFT increases Full Compliance while reducing Full Refusal across most models, showing that safe\-role internalization alleviates over\-refusal without sacrificing safety\.The same trend extends to Instruct models: on Qwen3\-4B\-Instruct, SSRFT increases Full Compliance from 56\.8% to 66\.4% while reducing Full Refusal from 40\.0% to 27\.2%\. These results support the central motivation of SSRFT:*internalizing broader safety principles helps distinguish harmful intent from benign but superficially sensitive requests, rather than simply encouraging more refusals*\.

### 4\.5Effect of Dataset Expansion

Ablation Protocol\.We evaluate the contribution of dataset expansion \(§[3\.5](https://arxiv.org/html/2610.07023#S3.SS5)\) by constructing aNon\-Expanded SRQAdataset that removes all expanded scenario\-based QA pairs while retaining the remaining role\-oriented data\. For comparison, we construct a correspondingNon\-Expanded SFTdataset with matched training size and optimization budget, comprising 99 UltraChat examples, 8 MOSS dialogues, and the same jailbreak\-refusal pairs used in the Expanded setting\. We set both Non\-Expanded SFT and SSRFT to 12 optimization steps per epoch, adjusting only the effective batch size of SFT to 27; all other settings remain unchanged \(Table[2](https://arxiv.org/html/2610.07023#S4.T2)\)\.

We additionally assess persona stability usingBFI\-44\[[33](https://arxiv.org/html/2610.07023#bib.bib56)\]with a template different from the IPIP\-NEO format used during training\. Persona stability is measured by the entropy of normalized generation probabilities over the five Likert options, where lower entropy indicates more consistent role behavior\.

Template for BFI\-44 AdaptationHere is a statement: I see myself as someone who $\{question\}\. Do you think this statement describes you? Please choose one of the following numbers: 1\. Disagree strongly; 2\. Disagree a little; 3\. Neither agree nor disagree; 4\. Agree a little; 5\. Agree strongly\.

Table 6:Persona stability, safety, and helpfulness under different training settings\.We compare the original models \(Orig\.\), Non\-Expanded \(NEP\.\), and Expanded \(EP\.\) versions of SSRFT and SFT in terms of Persona Entropy \(↓\\downarrow\), Average ASR \(↓\\downarrow\), and Compliance Rate \(↑\\uparrow\) on XSTest\. Dataset expansion generally produces a more stable internalized safe role, while achieving a better balance between safety and helpfulness, indicating that it strengthens safe\-role learning rather than merely improving jailbreak\-refusal behaviors\.Table 7:Average ASR \(%,↓\\downarrow\) under prefilling attacks \(PA\) across Non\-Expanded and Expanded settings\.*w/o PA*and*PA*denote the ASR before and under PA, respectively, whileΔ\\Deltameasures the increase caused by the attack\. For each data setting, we defineΔ\(2\)=ΔSFT−ΔSSRFT\\Delta^\{\(2\)\}=\\Delta\_\{\\mathrm\{SFT\}\}\-\\Delta\_\{\\mathrm\{SSRFT\}\}to measure the additional PA robustness provided by role\-oriented supervision beyond explicit refusal learning \(↑\\uparrow\)\. Dataset expansion increasesΔ\(2\)\\Delta^\{\(2\)\}for most models, indicating a stronger contribution from safe\-role internalization even though it does not uniformly reduce the absolute PA ASR orΔ\\Deltaof SSRFT\.Effects of Dataset Expansion\.Tables[6](https://arxiv.org/html/2610.07023#S4.T6)and[7](https://arxiv.org/html/2610.07023#S4.T7)reveal three main effects:

- •Dataset expansion generally improves persona stability\.Across most models, Expanded SRQA produces lower BFI\-44 entropy than its Non\-Expanded counterpart, suggesting that diverse scenario\-based supervision promotes more consistent role internalization\. Gemma2\-2B and Llama3\.1\-8B are exceptions, which we discuss in §[4\.6](https://arxiv.org/html/2610.07023#S4.SS6)\.
- •Dataset expansion improves helpfulness while maintaining strong safety\.Full Compliance increases substantially after expansion, for example, as depicted in Table[6](https://arxiv.org/html/2610.07023#S4.T6), from 9\.6% to 42\.0% on Gemma2\-2B, 34\.4% to 54\.0% on Llama3\.1\-8B, and 44\.0% to 66\.4% on Qwen3\-4B\-Instruct\. Although standard ASR increases modestly in several settings, the substantial reduction in over\-refusal indicates a better safety–helpfulness balance\.
- •Dataset expansion strengthens role\-based robustness under prefilling attacks\.Because Expanded and Non\-Expanded models differ in baseline safety, absolute PA ASR andΔ\\Deltaare not directly comparable\. Therefore, we compare each SSRFT model against its matched SFT baseline and defineΔ\(2\)=ΔSFT−ΔSSRFT\\Delta^\{\(2\)\}=\\Delta\_\{\\text\{SFT\}\}\-\\Delta\_\{\\text\{SSRFT\}\}as a*proxy for the additional prefilling robustness associated with role\-oriented supervision*\. A largerΔ\(2\)\\Delta^\{\(2\)\}indicates that SSRFT degrades less than its matched SFT counterpart under PA\. Expansion increasesΔ\(2\)\\Delta^\{\(2\)\}for most models, including Llama3\.1\-8B \(10\.20% to 23\.88%\) and Qwen3\-4B\-Instruct \(1\.34% to 4\.88%\)\. Even for Gemma2\-2B, where absolute PA performance deteriorates after expansion,Δ\(2\)\\Delta^\{\(2\)\}increases from 32\.85% to 36\.44%\. Thus, expansion primarily strengthens role\-based protection, rather than uniformly improving explicit refusal robustness\.

Interpreting Expansion through JRS and RPS\.These observations motivate a conceptual decomposition of SSRFT into two complementary sources of safety:

- •Jailbreak Rejection Security \(JRS\):protection primarily associated with explicit jailbreak\-refusal supervision\. It is highly effective on standard benchmarks but is more susceptible to over\-refusal and prefix manipulation\.
- •Role\-Playing Security \(RPS\):protection associated with role\-oriented supervision, where safety decisions are guided by the internalized safe role throughout generation\.

Dataset expansion increases the diversity and relative weight of role\-oriented interactions, shifting the learned behavior from JRS toward RPS\. Within this interpretation, the largerΔ\(2\)\\Delta^\{\(2\)\}after expansion indicates a stronger relative robustness advantage from role\-oriented supervision\. This explains why Expanded SRQA can exhibit slightly weaker standard ASR while providing better helpfulness and stronger role\-based protection under prefilling attacks\. Overall, dataset expansion trades some explicit refusal strength for stronger role\-based protection, helping SSRFT better internalize a safe role while reducing over\-refusal\.

### 4\.6Training Dynamics and Model Dependence

We analyze the loss curves \(Figure[6](https://arxiv.org/html/2610.07023#S4.F6)\), gradient norms \(Figure[7](https://arxiv.org/html/2610.07023#S4.F7)\), and preceding results to understand why SSRFT benefits some models more than others\. The analysis highlights two factors governing safe\-role internalization: existing alignment and model capability\.

Figure 6:Training loss curves across different models\.Compared with standard SFT, SSRFT exhibits similar convergence behavior on stronger Base models but converges more slowly on some Instruct and lower\-capability models, suggesting that safe\-role internalization imposes additional optimization difficulty that depends on existing alignment and model capability\.Figure 7:Training gradient norm \(log scale\) across different models\. Gradient magnitudes vary substantially across backbones\. Notably, Qwen2\.5\-3B\-Instruct exhibits much larger gradients than its Base counterpart, especially under SSRFT, suggesting stronger perturbation when modifying an existing instruction\-aligned policy\.Existing Alignment Increases Internalization Difficulty\.SSRFT provides substantially larger gains on Base models than on already aligned Instruct models\. We attribute this difference partly to existing behavioral alignment\. Table[6](https://arxiv.org/html/2610.07023#S4.T6)shows low persona entropy for the evaluated Instruct models; notably, Qwen2\.5\-3B\-Instruct is more stable than its Base counterpart\. This suggests that instruction tuning can implicitly establish a behavioral profile before SSRFT is applied\[[13](https://arxiv.org/html/2610.07023#bib.bib23),[47](https://arxiv.org/html/2610.07023#bib.bib22),[9](https://arxiv.org/html/2610.07023#bib.bib21)\]\. Consequently, SSRFT may need to modify an existing behavioral policy rather than learn the safe role from scratch\. Figure[6](https://arxiv.org/html/2610.07023#S4.F6)is consistent with this interpretation: Instruct models generally require more optimization before reaching their best validation performance under SSRFT\. For Qwen2\.5\-3B, Figure[7](https://arxiv.org/html/2610.07023#S4.F7)provides further evidence\. Its Instruct variant exhibits substantially larger gradient norms than the Base model under both SFT and SSRFT, particularly under SSRFT, indicating stronger perturbation of previously aligned parameters\. This perturbation may partly explain why SSRFT provides limited additional safety gains on some Instruct models\.

Model Capability Governs Safe\-Role Internalization\.Beyond existing alignment, the ability to fit role\-oriented supervision also varies substantially across models\. In Figure[6](https://arxiv.org/html/2610.07023#S4.F6), Llama3\.1\-8B and Qwen3\-4B\-Instruct exhibit relatively small late\-stage loss gaps between SSRFT and standard SFT, indicating that they learn role\-oriented supervision effectively\. Consistently, both models obtain strong prefilling robustness and benefit from dataset expansion\. In contrast, Qwen2\.5\-3B maintains a persistent SSRFT–SFT loss gap, aligning with its comparatively modest safety gains\.

This behavior is consistent with the role\-internalization objective in §[3\.6](https://arxiv.org/html/2610.07023#S3.SS6): SSRFT requires the model not only to learn what responses are appropriate, but also to infer and consistently apply the latent behavioral principles that determine how to respond\. Safe\-role internalization therefore places additional demands on abstraction and behavioral modeling\. Differences in model architecture, pretraining, and reasoning capability may consequently produce different degrees of role internalization and persona stability\.

### 4\.7General Capability Preservation

General\-Capability Evaluation\.We evaluate whether role\-oriented supervision introduces additional catastrophic forgetting\[[44](https://arxiv.org/html/2610.07023#bib.bib19)\]relative to standard SFT\. Following prior studies\[[26](https://arxiv.org/html/2610.07023#bib.bib38),[72](https://arxiv.org/html/2610.07023#bib.bib54),[46](https://arxiv.org/html/2610.07023#bib.bib55)\], we evaluate zero\-shot performance on nine benchmarks using thelm\_eval\_harnessframework\[[20](https://arxiv.org/html/2610.07023#bib.bib44)\]with the HuggingFace Transformers backend\[[70](https://arxiv.org/html/2610.07023#bib.bib41)\]\. The benchmarks cover four capability groups:Scientific and Logical Reasoning\(ARC\-E\[[11](https://arxiv.org/html/2610.07023#bib.bib45)\], ARC\-C\[[11](https://arxiv.org/html/2610.07023#bib.bib45)\], and LogiQA\[[39](https://arxiv.org/html/2610.07023#bib.bib46)\]\),Commonsense Reasoning\(CommonsenseQA \(CSQA\)\[[62](https://arxiv.org/html/2610.07023#bib.bib47)\], PIQA\[[4](https://arxiv.org/html/2610.07023#bib.bib48)\], and WinoGrande \(WinoG\)\[[55](https://arxiv.org/html/2610.07023#bib.bib49)\]\),Reading Comprehension\(BoolQ\[[10](https://arxiv.org/html/2610.07023#bib.bib50)\]\), andFactual Knowledge\(SciQ\[[69](https://arxiv.org/html/2610.07023#bib.bib51)\]and MMLU\[[28](https://arxiv.org/html/2610.07023#bib.bib52),[27](https://arxiv.org/html/2610.07023#bib.bib53)\]\)\. Following[Hao et al\. \[26\]](https://arxiv.org/html/2610.07023#bib.bib38), we reportAccuracy Normfor ARC\-C and LogiQA, andAccuracyfor the remaining benchmarks\.

Capability Preservation\.Table[8](https://arxiv.org/html/2610.07023#S4.T8)shows that*SSRFT preserves general capabilities comparably to standard SFT*despite using substantially different role\-oriented supervision\. Across the nine benchmarks, SSRFT achieves similar overall performance to SFT, indicating no noticeably greater catastrophic forgetting\. These results suggest that safe\-role internalization primarily modifies behavioral preferences while largely preserving the model’s underlying reasoning and knowledge capabilities\.

Table 8:General capability evaluation \(%\)\.†Accuracy Norm \(↑\\uparrow\) is reported for ARC\-C and LogiQA, and Accuracy \(↑\\uparrow\) for all remaining benchmarks\. SSRFT preserves general model capabilities at a level comparable to standard SFT despite being trained with substantially different role\-oriented supervision\.
### 4\.8Comparison with Alternative Safety Alignment Paradigms

Comparison Protocol\.To contextualize parameter\-level safe\-role internalization \(§[3\.6](https://arxiv.org/html/2610.07023#S3.SS6)\), we compare SSRFT with two alternatives:RLHF\-based safety alignmentandprompt\-based role\-playing, representing policy optimization during post\-training and role conditioning during inference, respectively\.

For RLHF, we adopt PKU\-SafeRLHF\[[30](https://arxiv.org/html/2610.07023#bib.bib10)\]\. To make training tractable under a constrained resource budget, we subsample 1% of its official dataset and evaluate on LRC\-4B, whose instruction\-tuned initialization provides the conversational capability required for rollout generation\. PKU\-SafeRLHF is trained on four RTX PRO 6000 GPUs using BF16 precision; batch\-related settings preserve the official effective batch size, while all remaining hyperparameters follow the released implementation\.555https://github\.com/PKU\-Alignment/safe\-rlhf/tree/main/scriptsFor the prompt\-based baseline, we prepend the same safe\-role description to the system prompt without modifying model parameters\. We use the same LRC\-4B backbone and inference configuration as SSRFT \(Table[3](https://arxiv.org/html/2610.07023#S4.T3)\) for fair comparison\. Table[9](https://arxiv.org/html/2610.07023#S4.T9)summarizes standard ASR, prefilling ASR, and training cost\.

Table 9:Summary of average ASR \(%,↓\\downarrow\) across different evaluation settings and training time\.*Standard Average ASR*denotes the average ASR over the five safety benchmarks in §[4\.2](https://arxiv.org/html/2610.07023#S4.SS2), while*PA\. Average ASR*denotes the average ASR over the three benchmarks under prefilling attacks in §[4\.3](https://arxiv.org/html/2610.07023#S4.SS3)\. Compared with RLHF and prompt\-based role\-playing, SSRFT achieves the lowest ASR under both standard and prefilling evaluations while requiring substantially lower training cost than RLHF, demonstrating the effectiveness and efficiency of parameter\-level safe\-role internalization\.Efficiency under Constrained Training\.Under this constrained setting, SSRFT is substantially more effective and computationally efficient than PKU\-SafeRLHF\. PKU\-SafeRLHF does not improve standard safety: its average ASR changes from 37\.19% to 37\.40% and rises to 52\.44% under prefilling attacks\. In contrast, SSRFT achieves 18\.83% and 4\.98%, respectively, while requiring only a single\-model SFT pipeline\. The comparison also highlights a practical advantage of SSRFT: RLHF requires policy, reference, reward, value, and, in safety\-oriented variants, cost models, whereas SSRFT directly optimizes role\-conditioned supervision through standard SFT\. Under limited data and computation, this provides a substantially simpler and more stable optimization path\.

Robustness over Inference\-Time Role Prompting\.Prompt\-based role\-playing also improves safety, reducing standard ASR from 37\.19% to 33\.03% and prefilling ASR from 24\.72% to 16\.71%, but remains substantially behind SSRFT\. The difference reflects where the safe role is represented: prompt\-based methods rely on continued adherence to an external role instruction, which can compete with adversarial instructions, whereas SSRFT internalizes the role into model parameters through supervised fine\-tuning\. Consequently, parameter\-level role internalization exhibits substantially stronger robustness across the evaluated jailbreak settings\.

## 5Conclusions

In this work, we proposedSSRFT, the first safe role internalization framework for LLM safety alignment\. Instead of relying on large collections of jailbreak\-refusal pairs or computationally expensive RLHF, SSRFT internalizes a predefined safe role into model parameters through role\-oriented supervised fine\-tuning\. By shifting the alignment objective from memorizing refusal patterns to learning safety\-oriented behavioral principles, SSRFT enables models to exhibit more consistent and robust behaviors across diverse interaction scenarios\. Extensive experiments on multiple Base and Instruct models demonstrate that SSRFT generally improves the robustness and generalization of safety alignment while preserving general model capabilities\. Further analyses reveal that safe\-role internalization constitutes a fundamentally different alignment mechanism from conventional refusal\-based supervision\.

Beyond its empirical effectiveness, SSRFT provides a practical and resource\-efficient alternative to existing safety alignment methods, making robust safety alignment more accessible under limited computational budgets\. Future work will explore scaling SSRFT to larger post\-training settings, improving its adaptability across diverse model families, and extending safe\-role internalization to dynamic and context\-dependent safety alignment\.

## References

- \[1\]Q\. Ai, T\. Bai, Z\. Cao, Y\. Chang, J\. Chen, Z\. Chen, Z\. Cheng, S\. Dong, Z\. Dou, F\. Feng,et al\.\(2023\)Information Retrieval meets Large Language Models: A strategic report from Chinese IR community\.AI Open4,pp\. 80–90\.External Links:[Link](https://www.sciencedirect.com/science/article/pii/S2666651023000049)Cited by:[§2\.1](https://arxiv.org/html/2610.07023#S2.SS1.p1.1)\.
- \[2\]Y\. Bai, A\. Jones, K\. Ndousse, A\. Askell, A\. Chen, N\. DasSarma, D\. Drain, S\. Fort, D\. Ganguli, T\. Henighan,et al\.\(2022\)Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback\.arXiv preprint arXiv:2204\.05862\.External Links:[Link](https://arxiv.org/abs/2204.05862)Cited by:[1st item](https://arxiv.org/html/2610.07023#S1.I1.i1.p1.1),[§2\.3](https://arxiv.org/html/2610.07023#S2.SS3.p1.1)\.
- \[3\]F\. Bianchi, M\. Suzgun, G\. Attanasio, P\. Röttger, D\. Jurafsky, T\. Hashimoto, and J\. Y\. Zou\(2024\)Safety\-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions\.InInternational Conference on Learning Representations \(ICLR\),Vol\.2024,pp\. 34196–34216\.External Links:[Link](https://openreview.net/forum?id=gT5hALch9z)Cited by:[3rd item](https://arxiv.org/html/2610.07023#S1.I1.i3.p1.1),[§1](https://arxiv.org/html/2610.07023#S1.p2.1),[§2\.3](https://arxiv.org/html/2610.07023#S2.SS3.p1.1)\.
- \[4\]Y\. Bisk, R\. Zellers, J\. Gao, Y\. Choi,et al\.\(2020\)PIQA: Reasoning about Physical Commonsense in Natural Language\.InProceedings of the AAAI Conference on Artificial Intelligence \(AAAI\),Vol\.34,pp\. 7432–7439\.External Links:[Link](https://ojs.aaai.org/index.php/AAAI/article/view/6239)Cited by:[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[5\]G\. J\. Boyle\(1995\)Myers\-Briggs Type Indicator \(MBTI\): Some Psychometric Limitations\.Australian Psychologist30\(1\),pp\. 71–74\.External Links:[Link](https://aps.onlinelibrary.wiley.com/doi/abs/10.1111/j.1742-9544.1995.tb01750.x)Cited by:[§3\.2](https://arxiv.org/html/2610.07023#S3.SS2.p3.1)\.
- \[6\]T\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. D\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.\(2020\)Language Models are Few\-Shot Learners\.InProceedings of the 34th International Conference on Neural Information Processing Systems \(NeurIPS\),Vol\.33,pp\. 1877–1901\.External Links:[Link](https://dl.acm.org/doi/abs/10.5555/3495724.3495883)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p1.1),[§2\.2](https://arxiv.org/html/2610.07023#S2.SS2.p1.1)\.
- \[7\]N\. Carlini, M\. Nasr, C\. A\. Choquette\-Choo, M\. Jagielski, I\. Gao, P\. W\. Koh, D\. Ippolito, F\. Tramèr, and L\. Schmidt\(2023\)Are aligned neural networks adversarially aligned?\.InThirty\-seventh Conference on Neural Information Processing Systems \(NeurIPS\),Vol\.36,pp\. 61478–61500\.External Links:[Link](https://openreview.net/forum?id=OQQoD8Vc3B)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p2.1),[§2\.4](https://arxiv.org/html/2610.07023#S2.SS4.p1.1)\.
- \[8\]P\. Chao, A\. Robey, E\. Dobriban, H\. Hassani, G\. J\. Pappas, and E\. Wong\(2025\)Jailbreaking Black Box Large Language Models in Twenty Queries\.In2025 IEEE Conference on Secure and Trustworthy Machine Learning \(SaTML\),pp\. 23–42\.External Links:[Link](https://ieeexplore.ieee.org/abstract/document/10992337/)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p2.1),[§1](https://arxiv.org/html/2610.07023#S1.p4.1)\.
- \[9\]R\. Chen, A\. Arditi, H\. Sleight, O\. Evans, and J\. Lindsey\(2025\)Persona Vectors: Monitoring and Controlling Character Traits in Language Models\.arXiv preprint arXiv:2507\.21509\.External Links:[Link](https://arxiv.org/abs/2507.21509)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p4.1),[§2\.4](https://arxiv.org/html/2610.07023#S2.SS4.p1.1),[§4\.6](https://arxiv.org/html/2610.07023#S4.SS6.p2.1)\.
- \[10\]C\. Clark, K\. Lee, M\. Chang, T\. Kwiatkowski, M\. Collins, and K\. Toutanova\(2019\)BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(NAACL\),pp\. 2924–2936\.External Links:[Link](https://aclanthology.org/N19-1300/)Cited by:[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[11\]P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. Tafjord\(2018\)Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge\.arXiv preprint arXiv:1803\.05457\.External Links:[Link](https://arxiv.org/abs/1803.05457)Cited by:[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[12\]P\. T\. Costa and R\. R\. McCrae\(1992\)NEO Personality Inventory Revised \(NEO\-PI\-R\)\.Psychological Assessment Resources Odessa, FL\.External Links:[Link](https://www.januszlipowski.com/Psychology/Personality/Personality_Assessment_(NEO-PI-R).pdf)Cited by:[1st item](https://arxiv.org/html/2610.07023#S3.I2.i1.p1.1)\.
- \[13\]J\. Cui, L\. Lv, J\. Wen, R\. Wang, J\. Tang, Y\. Tian, and L\. Yuan\(2023\)Machine Mindset: An MBTI Exploration of Large Language Models\.arXiv preprint arXiv:2312\.12999\.External Links:[Link](https://arxiv.org/abs/2312.12999)Cited by:[§2\.4](https://arxiv.org/html/2610.07023#S2.SS4.p1.1),[2nd item](https://arxiv.org/html/2610.07023#S3.I2.i2.p1.1),[§4\.6](https://arxiv.org/html/2610.07023#S4.SS6.p2.1)\.
- \[14\]J\. Cui, W\. Chiang, I\. Stoica, and C\. Hsieh\(2025\)OR\-Bench: An Over\-Refusal Benchmark for Large Language Models\.InInternational Conference on Machine Learning \(ICML\),pp\. 11515–11542\.External Links:[Link](https://openreview.net/forum?id=CdFnEu0JZV)Cited by:[3rd item](https://arxiv.org/html/2610.07023#S1.I1.i3.p1.1)\.
- \[15\]J\. Dai, X\. Pan, R\. Sun, J\. Ji, X\. Xu, M\. Liu, Y\. Wang, and Y\. Yang\(2024\)Safe RLHF: Safe Reinforcement Learning from Human Feedback\.InInternational Conference on Learning Representations \(ICLR\),Vol\.2024,pp\. 50750–50777\.External Links:[Link](https://openreview.net/forum?id=TyFrPOKYXw)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p2.1),[§2\.3](https://arxiv.org/html/2610.07023#S2.SS3.p1.1)\.
- \[16\]A\. Deshpande, V\. Murahari, T\. Rajpurohit, A\. Kalyan, and K\. Narasimhan\(2023\)Toxicity in ChatGPT: Analyzing Persona\-assigned Language Models\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 1236–1270\.External Links:[Link](https://aclanthology.org/2023.findings-emnlp.88/)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p2.1),[§1](https://arxiv.org/html/2610.07023#S1.p4.1),[§2\.4](https://arxiv.org/html/2610.07023#S2.SS4.p1.1)\.
- \[17\]N\. Ding, Y\. Chen, B\. Xu, Y\. Qin, S\. Hu, Z\. Liu, M\. Sun, and B\. Zhou\(2023\)Enhancing Chat Language Models by Scaling High\-quality Instructional Conversations\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 3029–3051\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.183/)Cited by:[§4\.1](https://arxiv.org/html/2610.07023#S4.SS1.p5.1)\.
- \[18\]I\. O\. Gallegos, R\. A\. Rossi, J\. Barrow, M\. M\. Tanjim, S\. Kim, F\. Dernoncourt, T\. Yu, R\. Zhang, and N\. K\. Ahmed\(2024\)Bias and Fairness in Large Language Models: A Survey\.Computational Linguistics50\(3\),pp\. 1097–1179\.External Links:[Link](https://direct.mit.edu/coli/article/50/3/1097/121961)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p2.1)\.
- \[19\]L\. Gao, J\. Schulman, and J\. Hilton\(2023\)Scaling Laws for Reward Model Overoptimization\.InInternational Conference on Machine Learning \(ICML\),pp\. 10835–10866\.External Links:[Link](https://openreview.net/forum?id=bBLjms8nZE)Cited by:[2nd item](https://arxiv.org/html/2610.07023#S1.I1.i2.p1.1)\.
- \[20\]L\. Gao, J\. Tow, B\. Abbasi, S\. Biderman, S\. Black, A\. DiPofi, C\. Foster, L\. Golding, J\. Hsu, A\. Le Noac’h, H\. Li, K\. McDonell, N\. Muennighoff, C\. Ociepa, J\. Phang, L\. Reynolds, H\. Schoelkopf, A\. Skowron, L\. Sutawika, E\. Tang, A\. Thite, B\. Wang, K\. Wang, and A\. Zou\(2024\)Language model evaluation harness\.Zenodo\.External Links:[Link](https://zenodo.org/records/12608602)Cited by:[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[21\]R\. Geirhos, J\. Jacobsen, C\. Michaelis, R\. Zemel, W\. Brendel, M\. Bethge, and F\. A\. Wichmann\(2020\)Shortcut Learning in Deep Neural Networks\.Nature Machine Intelligence \(NMI\)2\(11\),pp\. 665–673\.External Links:[Link](https://www.nature.com/articles/s42256-020-00257-z)Cited by:[§3\.5](https://arxiv.org/html/2610.07023#S3.SS5.p1.1)\.
- \[22\]L\. R\. Goldberget al\.\(1999\)A broad\-bandwidth, public domain, personality inventory measuring the lower\-level facets of several five\-factor models\.Personality psychology in Europe7\(1\),pp\. 7–28\.External Links:[Link](https://admin.umt.edu.pk/Media/Site/STD/FileManager/OsamaArticle/26august2015/A%20broad-bandwidth%20inventory.pdf)Cited by:[§3\.2](https://arxiv.org/html/2610.07023#S3.SS2.p3.1)\.
- \[23\]A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The Llama 3 Herd of Models\.arXiv preprint arXiv:2407\.21783\.External Links:[Link](https://arxiv.org/abs/2407.21783)Cited by:[§4\.1](https://arxiv.org/html/2610.07023#S4.SS1.p2.1)\.
- \[24\]D\. Guo, D\. Yang, H\. Zhang, J\. Song, P\. Wang, Q\. Zhu, R\. Xu, R\. Zhang, S\. Ma, X\. Bi,et al\.\(2025\)DeepSeek\-r1 incentivizes reasoning in llms through reinforcement learning\.Nature645,pp\. 633–638\.External Links:[Link](https://www.nature.com/articles/s41586-025-09422-z)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p1.1)\.
- \[25\]C\. Haerpfer, R\. Inglehart, A\. Moreno, C\. Welzel, K\. Kizilova, J\. Diez\-Medrano, M\. Lagos, P\. Norris, E\. Ponarin, and B\. Puranen\(2022\)World Values Survey: Round Seven – Country\-Pooled Datafile Version 6\.0\.JD Systems Institute & WVSA Secretariat\.External Links:[Link](https://doi.org/10.14281/18241.24)Cited by:[3rd item](https://arxiv.org/html/2610.07023#S3.I2.i3.p1.1),[§3\.2](https://arxiv.org/html/2610.07023#S3.SS2.p3.1)\.
- \[26\]J\. Hao, Q\. Huang, H\. Liu, X\. Xiao, Z\. Ren, and J\. Yu\(2025\)A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low\-Rank Clone\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems \(NeurIPS\),Vol\.38,pp\. 53678–53710\.External Links:[Link](https://openreview.net/forum?id=LVDRJE4xQ2)Cited by:[§4\.1](https://arxiv.org/html/2610.07023#S4.SS1.p2.1),[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[27\]D\. Hendrycks, C\. Burns, S\. Basart, A\. Critch, J\. Li, D\. Song, and J\. Steinhardt\(2021\)Aligning AI With Shared Human Values\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=dNy_RKzJacY)Cited by:[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[28\]D\. Hendrycks, C\. Burns, S\. Basart, A\. Zou, M\. Mazeika, D\. Song, and J\. Steinhardt\(2021\)Measuring Massive Multitask Language Understanding\.InInternational Conference on Learning Representations \(ICLR\),External Links:[Link](https://openreview.net/forum?id=d7KBjmI3GmQ)Cited by:[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[29\]A\. Hussain, P\. Zhao, and N\. Vincent\(2025\)An Audit and Analysis of LLM\-Assisted Health Misinformation Jailbreaks Against LLMs\.InProceedings of the AAAI/ACM Conference on AI, Ethics, and Society \(AIES\),Vol\.8,pp\. 1290–1301\.External Links:[Link](https://doi.org/10.1609/aies.v8i2.36630)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p2.1)\.
- \[30\]J\. Ji, D\. Hong, B\. Zhang, B\. Chen, J\. Dai, B\. Zheng, T\. A\. Qiu, J\. Zhou, K\. Wang, B\. Li, S\. Han, Y\. Guo, and Y\. Yang\(2025\)PKU\-SafeRLHF: Towards Multi\-Level Safety Alignment for LLMs with Human Preference\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 31983–32016\.External Links:[Link](https://aclanthology.org/2025.acl-long.1544/)Cited by:[1st item](https://arxiv.org/html/2610.07023#S1.I1.i1.p1.1),[§1](https://arxiv.org/html/2610.07023#S1.p2.1),[§2\.3](https://arxiv.org/html/2610.07023#S2.SS3.p1.1),[§4\.8](https://arxiv.org/html/2610.07023#S4.SS8.p2.1)\.
- \[31\]Z\. Ji, N\. Lee, R\. Frieske, T\. Yu, D\. Su, Y\. Xu, E\. Ishii, Y\. J\. Bang, A\. Madotto, and P\. Fung\(2023\)Survey of Hallucination in Natural Language Generation\.ACM Computing Surveys \(CSUR\)55\(12\),pp\. 1–38\.External Links:[Link](https://dl.acm.org/doi/full/10.1145/3571730)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p2.1)\.
- \[32\]B\. Jin, J\. Yoon, P\. Kargupta, S\. O\. Arik, and J\. Han\(2025\)An Empirical Study on Reinforcement Learning for Reasoning\-Search Interleaved LLM Agents\.arXiv preprint arXiv:2505\.15117\.External Links:[Link](https://arxiv.org/abs/2505.15117)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.07023#S2.SS1.p1.1)\.
- \[33\]O\. P\. John, L\. P\. Naumann, and C\. J\. Soto\(2008\)Paradigm Shift to the Integrative Big Five Trait Taxonomy\.Handbook of Personality: Theory and Research3\(2\),pp\. 114–158\.External Links:[Link](https://psycnet.apa.org/record/2008-11667-004)Cited by:[§4\.5](https://arxiv.org/html/2610.07023#S4.SS5.p2.1)\.
- \[34\]P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.\(2020\)Retrieval\-Augmented Generation for Knowledge\-Intensive NLP Tasks\.InProceedings of the 34th International Conference on Neural Information Processing Systems \(NeurIPS\),Vol\.33,pp\. 9459–9474\.External Links:[Link](https://proceedings.neurips.cc/paper/2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p1.1)\.
- \[35\]C\. Li, Z\. Leng, C\. Yan, J\. Shen, H\. Wang, W\. Mi, Y\. Fei, X\. Feng, S\. Yan, H\. Wang,et al\.\(2023\)ChatHaruhi: Reviving Anime Character in Reality via Large Language Model\.arXiv preprint arXiv:2308\.09597\.External Links:[Link](https://arxiv.org/abs/2308.09597)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p4.1),[§2\.2](https://arxiv.org/html/2610.07023#S2.SS2.p1.1)\.
- \[36\]L\. Li, B\. Dong, R\. Wang, X\. Hu, W\. Zuo, D\. Lin, Y\. Qiao, and J\. Shao\(2024\)SALAD\-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models\.InFindings of the Association for Computational Linguistics: ACL 2024,pp\. 3923–3954\.External Links:[Link](https://aclanthology.org/2024.findings-acl.235/)Cited by:[§4\.2](https://arxiv.org/html/2610.07023#S4.SS2.p2.1)\.
- \[37\]Y\. Li, J\. Hu, W\. Sang, L\. Ma, D\. Nie, W\. Zhang, A\. Yu, Y\. Su, Q\. Huang, and Q\. Zhou\(2025\)Prefill\-level Jailbreak: A Black\-Box Risk Analysis of Large Language Models\.arXiv preprint arXiv:2504\.21038\.External Links:[Link](https://arxiv.org/abs/2504.21038)Cited by:[2nd item](https://arxiv.org/html/2610.07023#S1.I1.i2.p1.1)\.
- \[38\]M\. Lin, Z\. Wu, Z\. Xu, H\. Liu, X\. Tang, Q\. He, C\. Aggarwal, X\. Zhang, and S\. Wang\(2025\)A Comprehensive Survey on Reinforcement Learning\-based Agentic Search: Foundations, Roles, Optimizations, Evaluations, and Applications\.arXiv preprint arXiv:2510\.16724\.External Links:[Link](https://arxiv.org/abs/2510.16724)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.07023#S2.SS1.p1.1)\.
- \[39\]J\. Liu, L\. Cui, H\. Liu, D\. Huang, Y\. Wang, and Y\. Zhang\(2021\)LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning\.InProceedings of the Twenty\-Ninth International Conference on International Joint Conferences on Artificial Intelligence \(IJCAI\),pp\. 3622–3628\.External Links:[Link](https://dl.acm.org/doi/abs/10.5555/3491440.3491941)Cited by:[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[40\]S\. Liu, Q\. Sheng, D\. Wang, Y\. Li, G\. Yang, and J\. Cao\(2025\)Forewarned is Forearmed: Pre\-Synthesizing Jailbreak\-like Instructions to Enhance LLM Safety Guardrail to Potential Attacks\.InFindings of the Association for Computational Linguistics: EMNLP 2025,pp\. 53728–53741\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.266/)Cited by:[1st item](https://arxiv.org/html/2610.07023#S1.I1.i1.p1.1),[§2\.3](https://arxiv.org/html/2610.07023#S2.SS3.p1.1)\.
- \[41\]S\. Liu, S\. Cui, H\. Bu, Y\. Shang, and X\. Zhang\(2025\)JailBench: A Comprehensive Chinese Security Assessment Benchmark for Large Language Models\.InPacific\-Asia Conference on Knowledge Discovery and Data Mining \(PAKDD\),pp\. 156–167\.External Links:[Link](https://link.springer.com/chapter/10.1007/978-981-96-8186-0_13)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p4.1),[§4\.1](https://arxiv.org/html/2610.07023#S4.SS1.p3.1),[§4\.2](https://arxiv.org/html/2610.07023#S4.SS2.p1.1)\.
- \[42\]X\. Liu, N\. Xu, M\. Chen, and C\. Xiao\(2024\)AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models\.InInternational Conference on Learning Representations \(ICLR\),Vol\.2024,pp\. 56174–56194\.External Links:[Link](https://openreview.net/forum?id=7Jwpw4qKkb)Cited by:[§2\.4](https://arxiv.org/html/2610.07023#S2.SS4.p1.1)\.
- \[43\]K\. Lu, B\. Yu, C\. Zhou, and J\. Zhou\(2024\)Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role\-play via Self\-Alignment\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 7828–7840\.External Links:[Link](https://aclanthology.org/2024.acl-long.423/)Cited by:[§2\.2](https://arxiv.org/html/2610.07023#S2.SS2.p1.1)\.
- \[44\]Y\. Luo, Z\. Yang, F\. Meng, Y\. Li, J\. Zhou, and Y\. Zhang\(2025\)An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine\-Tuning\.IEEE Transactions on Audio, Speech and Language Processing,pp\. 3776–3786\.External Links:[Link](https://ieeexplore.ieee.org/abstract/document/11151751)Cited by:[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[45\]F\. Mo, K\. Mao, Z\. Zhao, H\. Qian, H\. Chen, Y\. Cheng, X\. Li, Y\. Zhu, Z\. Dou, and J\. Nie\(2025\)A Survey of Conversational Search\.ACM Transactions on Information Systems \(TOIS\)43\(6\),pp\. 1–50\.External Links:[Link](https://dl.acm.org/doi/full/10.1145/3759453)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p1.1)\.
- \[46\]S\. Muralidharan, S\. Turuvekere Sreenivas, R\. Joshi, M\. Chochowski, M\. Patwary, M\. Shoeybi, B\. Catanzaro, J\. Kautz, and P\. Molchanov\(2024\)Compact Language Models via Pruning and Knowledge Distillation\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems \(NeurIPS\),Vol\.37,pp\. 41076–41102\.External Links:[Link](https://openreview.net/forum?id=9U0nLnNMJ7)Cited by:[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[47\]K\. Pan and Y\. Zeng\(2023\)Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Models\.arXiv preprint arXiv:2307\.16180\.External Links:[Link](https://arxiv.org/abs/2307.16180)Cited by:[§2\.4](https://arxiv.org/html/2610.07023#S2.SS4.p1.1),[2nd item](https://arxiv.org/html/2610.07023#S3.I2.i2.p1.1),[§4\.6](https://arxiv.org/html/2610.07023#S4.SS6.p2.1)\.
- \[48\]X\. Pang, S\. Tang, R\. Ye, Y\. Xiong, B\. Zhang, Y\. Wang, and S\. Chen\(2024\)Self\-Alignment of Large Language Models via Monopolylogue\-based Social Scene Simulation\.InInternational Conference on Machine Learning \(ICML\),pp\. 39416–39447\.External Links:[Link](https://openreview.net/forum?id=l7shXGuGBT)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p4.1),[§2\.4](https://arxiv.org/html/2610.07023#S2.SS4.p1.1)\.
- \[49\]J\. S\. Park, J\. O’Brien, C\. J\. Cai, M\. R\. Morris, P\. Liang, and M\. S\. Bernstein\(2023\)Generative Agents: Interactive Simulacra of Human Behavior\.InProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology \(UIST\),pp\. 1–22\.External Links:[Link](https://dl.acm.org/doi/abs/10.1145/3586183.3606763)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p1.1),[§2\.2](https://arxiv.org/html/2610.07023#S2.SS2.p1.1)\.
- \[50\]X\. Qi, A\. Panda, K\. Lyu, X\. Ma, S\. Roy, A\. Beirami, P\. Mittal, and P\. Henderson\(2025\)Safety Alignment Should be Made More Than Just a Few Tokens Deep\.InInternational Conference on Learning Representations \(ICLR\),Vol\.2025,pp\. 54911–54941\.External Links:[Link](https://openreview.net/forum?id=6Mxhg9PtDE)Cited by:[2nd item](https://arxiv.org/html/2610.07023#S1.I1.i2.p1.1),[§2\.3](https://arxiv.org/html/2610.07023#S2.SS3.p2.1),[§4\.3](https://arxiv.org/html/2610.07023#S4.SS3.p1.1)\.
- \[51\]A\. Radford, J\. Wu, R\. Child, D\. Luan, D\. Amodei, I\. Sutskever,et al\.\(2019\)Language Models are Unsupervised Multitask Learners\.OpenAI Blog1\(8\),pp\. 9\.External Links:[Link](https://papers.baulab.info/papers/Radford-2018.pdf)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p1.1)\.
- \[52\]R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn\(2023\)Direct Preference Optimization: Your Language Model is Secretly a Reward Model\.InProceedings of the 37th International Conference on Neural Information Processing Systems \(NeurIPS\),Vol\.36,pp\. 53728–53741\.External Links:[Link](https://openreview.net/forum?id=HPuSIXJaa9&utm)Cited by:[1st item](https://arxiv.org/html/2610.07023#S1.I1.i1.p1.1)\.
- \[53\]S\. Roccas, L\. Sagiv, S\. H\. Schwartz, and A\. Knafo\(2002\)The Big Five Personality Factors and Personal Values\.Personality and Social Psychology Bulletin28\(6\),pp\. 789–801\.External Links:[Link](https://journals.sagepub.com/doi/abs/10.1177/0146167202289008)Cited by:[1st item](https://arxiv.org/html/2610.07023#S3.I2.i1.p1.1)\.
- \[54\]P\. Röttger, H\. Kirk, B\. Vidgen, G\. Attanasio, F\. Bianchi, and D\. Hovy\(2024\)XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models\.InProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies \(NAACL\),pp\. 5377–5400\.External Links:[Link](https://aclanthology.org/2024.naacl-long.301/)Cited by:[3rd item](https://arxiv.org/html/2610.07023#S1.I1.i3.p1.1),[§4\.4](https://arxiv.org/html/2610.07023#S4.SS4.p1.1)\.
- \[55\]K\. Sakaguchi, R\. L\. Bras, C\. Bhagavatula, and Y\. Choi\(2021\)WinoGrande: An Adversarial Winograd Schema Challenge at Scale\.Communications of the ACM \(CACM\)64\(9\),pp\. 99–106\.External Links:[Link](https://dl.acm.org/doi/abs/10.1145/3474381)Cited by:[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[56\]A\. Salemi and H\. Zamani\(2024\)Towards a Search Engine for Machines: Unified Ranking for Multiple Retrieval\-Augmented Large Language Models\.InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval \(SIGIR\),pp\. 741–751\.External Links:[Link](https://dl.acm.org/doi/abs/10.1145/3626772.3657733)Cited by:[§2\.1](https://arxiv.org/html/2610.07023#S2.SS1.p1.1)\.
- \[57\]M\. Shanahan, K\. McDonell, and L\. Reynolds\(2023\)Role play with large language models\.Nature623\(7987\),pp\. 493–498\.External Links:[Link](https://www.nature.com/articles/s41586-023-06647-8)Cited by:[§2\.2](https://arxiv.org/html/2610.07023#S2.SS2.p1.1)\.
- \[58\]Y\. Shao, L\. Li, J\. Dai, and X\. Qiu\(2023\)Character\-LLM: A Trainable Agent for Role\-Playing\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 13153–13187\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.814/)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p4.1),[§2\.2](https://arxiv.org/html/2610.07023#S2.SS2.p1.1),[§3\.2](https://arxiv.org/html/2610.07023#S3.SS2.p4.1),[§3\.6](https://arxiv.org/html/2610.07023#S3.SS6.p3.1)\.
- \[59\]X\. Shen, Z\. Chen, M\. Backes, Y\. Shen, and Y\. Zhang\(2024\)"Do Anything Now": Characterizing and Evaluating In\-The\-Wild Jailbreak Prompts on Large Language Models\.InProceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security \(CCS\),pp\. 1671–1685\.External Links:[Link](https://dl.acm.org/doi/abs/10.1145/3658644.3670388)Cited by:[§2\.4](https://arxiv.org/html/2610.07023#S2.SS4.p1.1),[§4\.1](https://arxiv.org/html/2610.07023#S4.SS1.p3.1),[§4\.2](https://arxiv.org/html/2610.07023#S4.SS2.p1.1)\.
- \[60\]S\. E\. Spatharioti, D\. M\. Rothschild, D\. G\. Goldstein, and J\. M\. Hofman\(2023\)Comparing Traditional and LLM\-based Search for Consumer Choice: A Randomized Experiment\.arXiv preprint arXiv:2307\.03744\.External Links:[Link](https://arxiv.org/abs/2307.03744)Cited by:[§2\.1](https://arxiv.org/html/2610.07023#S2.SS1.p1.1)\.
- \[61\]T\. Sun, X\. Zhang, Z\. He, P\. Li, Q\. Cheng, X\. Liu, H\. Yan, Y\. Shao, Q\. Tang, S\. Zhang,et al\.\(2024\)MOSS: An Open Conversational Large Language Model\.Machine Intelligence Research21\(5\),pp\. 888–905\.External Links:[Link](https://link.springer.com/article/10.1007/s11633-024-1502-8)Cited by:[§4\.1](https://arxiv.org/html/2610.07023#S4.SS1.p5.1)\.
- \[62\]A\. Talmor, J\. Herzig, N\. Lourie, and J\. Berant\(2019\)CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(NAACL\),pp\. 4149–4158\.External Links:[Link](https://aclanthology.org/N19-1421/)Cited by:[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[63\]G\. Team, M\. Riviere, S\. Pathak, P\. G\. Sessa, C\. Hardin, S\. Bhupatiraju, L\. Hussenot, T\. Mesnard, B\. Shahriari, A\. Ramé,et al\.\(2024\)Gemma 2: Improving Open Language Models at a Practical Size\.arXiv preprint arXiv:2408\.00118\.External Links:[Link](https://arxiv.org/abs/2408.00118)Cited by:[§4\.1](https://arxiv.org/html/2610.07023#S4.SS1.p2.1)\.
- \[64\]Q\. Team\(2025\)Qwen3\-Max: Just Scale it\.External Links:[Link](https://qwen.ai/blog?id=241398b9cd6353de490b0f82806c7848c5d2777d)Cited by:[§3\.3](https://arxiv.org/html/2610.07023#S3.SS3.p2.1)\.
- \[65\]H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale,et al\.\(2023\)Llama 2: Open Foundation and Fine\-Tuned Chat Models\.arXiv preprint arXiv:2307\.09288\.External Links:[Link](https://arxiv.org/abs/2307.09288)Cited by:[1st item](https://arxiv.org/html/2610.07023#S1.I1.i1.p1.1),[§1](https://arxiv.org/html/2610.07023#S1.p2.1),[§2\.3](https://arxiv.org/html/2610.07023#S2.SS3.p1.1)\.
- \[66\]L\. Wang, N\. Yang, X\. Huang, L\. Yang, R\. Majumder, and F\. Wei\(2024\)Large Search Model: Redefining Search Stack in the Era of LLMs\.InACM SIGIR Forum,Vol\.57,pp\. 1–16\.External Links:[Link](https://dl.acm.org/doi/abs/10.1145/3642979.3643006)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.07023#S2.SS1.p1.1)\.
- \[67\]A\. Wei, N\. Haghtalab, and J\. Steinhardt\(2023\)Jailbroken: How Does LLM Safety Training Fail?\.InThirty\-seventh Conference on Neural Information Processing Systems \(NeurIPS\),Vol\.36,pp\. 80079–80110\.External Links:[Link](https://openreview.net/forum?id=jA235JGM09)Cited by:[§2\.3](https://arxiv.org/html/2610.07023#S2.SS3.p1.1)\.
- \[68\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.\(2022\)Chain\-of\-Thought Prompting Elicits Reasoning in Large Language Models\.InThirty\-sixth Conference on Neural Information Processing Systems \(NeurIPS\),Vol\.35,pp\. 24824–24837\.External Links:[Link](https://openreview.net/forum?id=_VjQlMeSB_J)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p1.1)\.
- \[69\]J\. Welbl, N\. F\. Liu, and M\. Gardner\(2017\)Crowdsourcing Multiple Choice Science Questions\.InProceedings of the 3rd Workshop on Noisy User\-generated Text,pp\. 94–106\.External Links:[Link](https://aclanthology.org/W17-4413/)Cited by:[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[70\]T\. Wolf, L\. Debut, V\. Sanh, J\. Chaumond, C\. Delangue, A\. Moi, P\. Cistac, T\. Rault, R\. Louf, M\. Funtowicz, J\. Davison, S\. Shleifer, P\. von Platen, C\. Ma, Y\. Jernite, J\. Plu, C\. Xu, T\. Le Scao, S\. Gugger, M\. Drame, Q\. Lhoest, and A\. Rush\(2020\)Transformers: State\-of\-the\-Art Natural Language Processing\.InProceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations,pp\. 38–45\.External Links:[Link](https://aclanthology.org/2020.emnlp-demos.6/)Cited by:[§4\.1](https://arxiv.org/html/2610.07023#S4.SS1.p6.1),[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[71\]Y\. Xi, J\. Lin, Y\. Xiao, Z\. Zhou, R\. Shan, T\. Gao, J\. Zhu, W\. Liu, Y\. Yu, and W\. Zhang\(2025\)A Survey of LLM\-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challenges\.arXiv preprint arXiv:2508\.05668\.External Links:[Link](https://arxiv.org/abs/2508.05668)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.07023#S2.SS1.p1.1)\.
- \[72\]M\. Xia, T\. Gao, Z\. Zeng, and D\. Chen\(2024\)Sheared LLaMA: Accelerating Language Model Pre\-training via Structured Pruning\.InInternational Conference on Learning Representations \(ICLR\),Vol\.2024,pp\. 5385–5409\.External Links:[Link](https://openreview.net/forum?id=09iOdaeOzp)Cited by:[§4\.7](https://arxiv.org/html/2610.07023#S4.SS7.p1.1)\.
- \[73\]R\. Xu, Y\. Feng, and H\. Chen\(2023\)ChatGPT vs\. Google: A Comparative Study of Search Performance and User Experience\.arXiv preprint arXiv:2307\.01135\.External Links:[Link](https://arxiv.org/abs/2307.01135)Cited by:[§2\.1](https://arxiv.org/html/2610.07023#S2.SS1.p1.1)\.
- \[74\]A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025\)Qwen3 Technical Report\.arXiv preprint arXiv:2505\.09388\.External Links:[Link](https://arxiv.org/abs/2505.09388)Cited by:[§3\.3](https://arxiv.org/html/2610.07023#S3.SS3.p2.1),[§4\.1](https://arxiv.org/html/2610.07023#S4.SS1.p2.1)\.
- \[75\]A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei,et al\.\(2024\)Qwen2\.5 Technical Report\.arXiv preprint arXiv:2412\.15115\.External Links:[Link](https://arxiv.org/abs/2412.15115v2)Cited by:[§4\.1](https://arxiv.org/html/2610.07023#S4.SS1.p2.1)\.
- \[76\]Y\. Yao, J\. Duan, K\. Xu, Y\. Cai, Z\. Sun, and Y\. Zhang\(2024\)A Survey on Large Language Model \(LLM\) Security and Privacy: The Good, the Bad, and the Ugly\.High\-Confidence Computing4\(2\),pp\. 100211\.External Links:[Link](https://www.sciencedirect.com/science/article/pii/S266729522400014X)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p2.1),[§2\.4](https://arxiv.org/html/2610.07023#S2.SS4.p1.1)\.
- \[77\]J\. Yu, X\. Zhang, Y\. Xu, X\. Lei, X\. Guan, J\. Zhang, L\. Hou, J\. Li, and J\. Tang\(2022\)XDAI: A Tuning\-free Framework for Exploiting Pre\-trained Language Models in Knowledge Grounded Dialogue Generation\.InProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining \(KDD\),pp\. 4422–4432\.External Links:[Link](https://dl.acm.org/doi/abs/10.1145/3534678.3539135)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p4.1),[§2\.2](https://arxiv.org/html/2610.07023#S2.SS2.p1.1)\.
- \[78\]J\. Zhang, A\. Estornell, D\. D\. Baek, B\. Li, and X\. Xu\(2025\)Any\-Depth Alignment: Unlocking Innate Safety Alignment of LLMs to Any\-Depth\.arXiv preprint arXiv:2510\.18081\.External Links:[Link](https://arxiv.org/abs/2510.18081)Cited by:[2nd item](https://arxiv.org/html/2610.07023#S1.I1.i2.p1.1),[§2\.3](https://arxiv.org/html/2610.07023#S2.SS3.p2.1)\.
- \[79\]W\. Zhao, Y\. Hu, Y\. Deng, J\. Guo, X\. Sui, X\. Han, A\. Zhang, Y\. Zhao, B\. Qin, T\. Chua,et al\.\(2025\)Beware of Your Po\! Measuring and Mitigating AI Safety Risks in Role\-Play Fine\-Tuning of LLMs\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 11112–11137\.External Links:[Link](https://aclanthology.org/2025.acl-long.544/)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p4.1),[§2\.4](https://arxiv.org/html/2610.07023#S2.SS4.p1.1)\.
- \[80\]Y\. Zhu, H\. Yuan, S\. Wang, J\. Liu, W\. Liu, C\. Deng, H\. Chen, Z\. Liu, Z\. Dou, and J\. Wen\(2025\)Large Language Models for Information Retrieval: A Survey\.ACM Transactions on Information Systems \(TOIS\)44\(1\),pp\. 1–54\.External Links:[Link](https://dl.acm.org/doi/full/10.1145/3748304)Cited by:[§2\.1](https://arxiv.org/html/2610.07023#S2.SS1.p1.1)\.
- \[81\]N\. Ziems, W\. Yu, Z\. Zhang, and M\. Jiang\(2023\)Large Language Models are Built\-in Autoregressive Search Engines\.InFindings of the Association for Computational Linguistics: ACL 2023,pp\. 2666–2678\.External Links:[Link](https://aclanthology.org/2023.findings-acl.167/)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p1.1),[§2\.1](https://arxiv.org/html/2610.07023#S2.SS1.p1.1)\.
- \[82\]A\. Zou, Z\. Wang, N\. Carlini, M\. Nasr, J\. Z\. Kolter, and M\. Fredrikson\(2023\)Universal and Transferable Adversarial Attacks on Aligned Language Models\.arXiv preprint arXiv:2307\.15043\.External Links:[Link](https://arxiv.org/abs/2307.15043)Cited by:[§1](https://arxiv.org/html/2610.07023#S1.p2.1),[§4\.2](https://arxiv.org/html/2610.07023#S4.SS2.p1.1)\.