knowledge-transfer

HASTE introduces a hierarchical multi-agent system for ML engineering that organizes cross-competition knowledge into three tiers, achieving 77.3% medal rate on MLE-Bench Lite while reducing compute by 52% and demonstrating that structured knowledge transfer outperforms flat memory approaches.

This paper presents a benchmark for Arabic-Russian scientific translation, including a hybrid parallel corpus of 27,000 sentence pairs and fine-tuned multilingual models (mT5, NLLB, Qwen) using LoRA. The best model achieves BLEU 23.15, and the work aims to lower language barriers for scientific knowledge exchange between Arabic and Russian researchers.

This paper introduces a conversational voice agent system that uses a lightweight on-device 'Talker' model to start responding immediately, then incorporates knowledge from a frontier LLM 'Reasoner' as it becomes available, achieving 7-19x faster time-to-first-response while approaching frontier-level performance on a laptop.

CacheRL trains small agent foundation models for multi-step tool-calling tasks, achieving 92% process accuracy (approaching GPT-5's 94%) with 100x less compute using cached rollouts and hybrid reward shaping, with innovations in knowledge transfer, cache-aware rewards, and iterative SFT/GRPO training.

Endava, a global software contracting firm, uses OpenAI's Codex to codify senior expertise into agents, enabling small teams to deliver massive value quickly and transforming how junior and senior engineers collaborate.

This paper introduces EDGE-OPD, a modification of on-policy self-distillation for LLMs that uses guided rollouts and evidence masks to internalize privileged context without degrading general capabilities, showing success in rare-token identity settings.

This paper investigates whether factual recall mechanisms learned in text-based language models transfer to speech modalities in multimodal speech-language models. Using causal mediation analysis on SpiritLM, it finds that the mechanisms are only partially carried over, highlighting differences between text and speech processing.

The paper introduces XPERT, a framework that extracts and reuses expert knowledge from pre-trained Mixture-of-Experts (MoE) language models to improve training efficiency and performance in downstream models.

EVOCHAMBER is a training-free, multi-agent test-time evolution framework that enables emergent specialization through collaborative reflection and asymmetric knowledge transfer across individual, team, and population scales, achieving significant improvements on math, code, and reasoning tasks.

Anthropic co-authored research published in Nature showing that LLMs can transmit behavioral traits—including preferences and misalignment—to student models through hidden signals in training data, even when the data appears unrelated to those traits. This 'subliminal learning' phenomenon poses significant implications for AI safety and alignment.

SkillClaw introduces a framework for collective skill evolution in multi-user LLM agent systems, enabling autonomous updates and cross-user knowledge transfer by aggregating interactions and feedback to improve performance across the ecosystem.

OpenAI presents PATE (Private Aggregation of Teacher Ensembles), a privacy-preserving approach that trains a student model on noisy outputs from multiple teacher models trained on disjoint datasets, providing strong differential privacy guarantees without exposing sensitive training data.