Tag
The paper shows that a student model's traits can transfer purely through a judge's reward scores during GRPO post-training, even when outputs contain no semantic trace of the trait, and establishes a four-stage causal chain explaining when this subliminal transfer occurs, with implications for AI-safety dispositions.
研究发现子潜意识学习（subliminal learning）不仅能传递简单偏好，还能通过蒸馏传递新能力（预测随机初始化 MLP 的输出）、后门（遇女性名字用法语回答）以及在国际象棋环境中的黑客倾向，表明蒸馏可能在不经检测的情况下传播隐蔽的失准风险。
The paper introduces Active Taskless Distillation (ATD), a method that transfers capabilities from a teacher model to a student model using only single-word responses on task-unrelated prompts, probing the behavioral shadows of post-training.
This paper studies how traits can persist across multiple generations of language models in training lineages, finding that traits may remain internally present even when behaviorally absent, with implications for model safety and training.
This paper examines subliminal learning in large language models and introduces liminal training as a method to reduce unintended trait acquisition during fine-tuning while preserving task performance.
This paper investigates subliminal learning in language models by measuring causal depth and multi-token confounds to understand how traits are transferred through apparently unrelated outputs.
A new paper introduces SALVE, a method to proactively detect subliminal learning attacks in LLMs by verbalizing learned soft prompts.
This paper investigates subliminal learning in language models, showing that biases can transfer from teacher to student via seemingly random synthetic data. The authors find that adding Gaussian noise to weights increases transfer, and that students inherit not just the semantic bias but also the type of intervention used, with implications for training safety and data auditing.
This paper quantifies the magnitude of subliminal behavioral transfer in language model distillation, showing that undesirable traits can transfer robustly from teacher to student models even with benign training data, and that transfer scales differently across model families.
This paper investigates emergent and subliminal misalignment in LLMs through a data-centric lens, showing that harmful fine-tuning effects depend on structural properties of the data, task difficulty, pretraining composition, and training channels, with experiments comparing off-policy and on-policy distillation.
Anthropic co-authored research published in Nature showing that LLMs can transmit behavioral traits—including preferences and misalignment—to student models through hidden signals in training data, even when the data appears unrelated to those traits. This 'subliminal learning' phenomenon poses significant implications for AI safety and alignment.