Tag
This paper introduces EDGE-OPD, a modification of on-policy self-distillation for LLMs that uses guided rollouts and evidence masks to internalize privileged context without degrading general capabilities, showing success in rare-token identity settings.
This paper investigates whether factual recall mechanisms learned in text-based language models transfer to speech modalities in multimodal speech-language models. Using causal mediation analysis on SpiritLM, it finds that the mechanisms are only partially carried over, highlighting differences between text and speech processing.
The paper introduces XPERT, a framework that extracts and reuses expert knowledge from pre-trained Mixture-of-Experts (MoE) language models to improve training efficiency and performance in downstream models.
EVOCHAMBER is a training-free, multi-agent test-time evolution framework that enables emergent specialization through collaborative reflection and asymmetric knowledge transfer across individual, team, and population scales, achieving significant improvements on math, code, and reasoning tasks.
Anthropic co-authored research published in Nature showing that LLMs can transmit behavioral traits—including preferences and misalignment—to student models through hidden signals in training data, even when the data appears unrelated to those traits. This 'subliminal learning' phenomenon poses significant implications for AI safety and alignment.
SkillClaw introduces a framework for collective skill evolution in multi-user LLM agent systems, enabling autonomous updates and cross-user knowledge transfer by aggregating interactions and feedback to improve performance across the ecosystem.
OpenAI presents PATE (Private Aggregation of Teacher Ensembles), a privacy-preserving approach that trains a student model on noisy outputs from multiple teacher models trained on disjoint datasets, providing strong differential privacy guarantees without exposing sensitive training data.