Tag
This paper studies how traits can persist across multiple generations of language models in training lineages, finding that traits may remain internally present even when behaviorally absent, with implications for model safety and training.
This paper examines subliminal learning in large language models and introduces liminal training as a method to reduce unintended trait acquisition during fine-tuning while preserving task performance.
This paper investigates subliminal learning in language models by measuring causal depth and multi-token confounds to understand how traits are transferred through apparently unrelated outputs.
A new paper introduces SALVE, a method to proactively detect subliminal learning attacks in LLMs by verbalizing learned soft prompts.
This paper investigates subliminal learning in language models, showing that biases can transfer from teacher to student via seemingly random synthetic data. The authors find that adding Gaussian noise to weights increases transfer, and that students inherit not just the semantic bias but also the type of intervention used, with implications for training safety and data auditing.
This paper quantifies the magnitude of subliminal behavioral transfer in language model distillation, showing that undesirable traits can transfer robustly from teacher to student models even with benign training data, and that transfer scales differently across model families.
This paper investigates emergent and subliminal misalignment in LLMs through a data-centric lens, showing that harmful fine-tuning effects depend on structural properties of the data, task difficulty, pretraining composition, and training channels, with experiments comparing off-policy and on-policy distillation.
Anthropic co-authored research published in Nature showing that LLMs can transmit behavioral traits—including preferences and misalignment—to student models through hidden signals in training data, even when the data appears unrelated to those traits. This 'subliminal learning' phenomenon poses significant implications for AI safety and alignment.