Tag
研究训练小规模GPT-2模型于儿童日常语音转录（BabyView）和合成儿童导向对话（TinyDialogues）数据上，发现模型会显著特化于特定儿童的语言环境，多儿童数据能提升对混合留出语言的预测但对语法基准改进有限，表明小数据语言学习不仅取决于数据量，还取决于环境来源与匹配度。
ARMOR-DP is a modular framework for multilingual BabyLM that dynamically adjusts language sampling based on per-language validation loss, showing that the weakest language shifts during training and that dynamic protection outperforms fixed Chinese-protection baselines.
该论文研究了稀疏自编码器(SAEs)所需的最小激活数据量，发现功能行为(如重建)在较少数据下即可收敛，但特征级可复现性仍随数据持续提升，表明SAE训练可在匹配目标的前提下大幅降低数据成本。
YerevaNN researchers show that for image self-distillation, sample repetition effects are dataset-dependent and can violate standard scaling law assumptions — a 1.5% random subset of Maxar even outperforms the full 50M-image pool under equal compute — and propose a modified Chinchilla-style law accounting for unique samples and repetition.
该论文提出一种不确定性引导的渐进收缩框架，通过构建蒙特卡洛推理树评估过程级不确定性，并在SFT阶段进行在线路由，仅使用全量SFT约0.22%的优化预算（减少450倍以上的训练暴露）即可维持或提升LLM的推理能力。
A paper introducing MFPG-PPO, a multi-fidelity policy gradient method that uses cheap low-fidelity simulator data as a control variate to stabilize on-policy RL when high-fidelity real data is scarce, demonstrating stable learning on simulated robot locomotion and a physical Franka arm with only 4 real episodes per update.
The paper proposes Tex-Zero, a native 3D texture generation framework trained entirely on constructed 2D images rather than real 3D assets, by converting high-quality 2D images into 3D training samples via plane representations and patch-wise random rotations, showing high-fidelity texture synthesis is possible without 3D asset data.
DualRes is a compact oscillatory state-space model for machine fault diagnosis from vibration data, achieving state-of-the-art performance with limited labels and reduced computational requirements for edge deployment.
The paper introduces a teacher-guided curriculum learning approach for Reinforcement Learning with Verifiable Rewards (RLVR) to efficiently train language models on initially unsolvable mathematical problems, achieving significant data efficiency and expanding reasoning boundaries.
The paper establishes a theoretical framework showing that data and memory resources in autoregressive prediction are governed by a single predictive-energy spectrum, providing a joint scaling law.
This paper investigates structural transfer from non-language data to improve data efficiency in language modeling, finding that while it reduces next-token prediction loss, it does not reliably enhance downstream linguistic performance.
DE-Venus is a unified framework for data-efficient Reinforcement Learning with Verifiable Rewards in large language models, reducing annotation and training costs while maintaining performance.
The paper shows that multilayer perceptrons naturally develop monosemantic specialized neurons that improve data efficiency by learning local low-dimensional representations instead of a global one in regression problems with clustered data.
The article highlights the data efficiency gap where children outperform AI models in learning, explores research to close this gap, and covers emerging careers in space travel alongside other tech developments like AI data center backlash and humanoid robot records.
Children learn language with far less data than AI models like LLMs, and understanding this data efficiency gap could lead to more efficient AI systems and insights into human cognition.
This paper presents C-Guard, a constitution-grid instrument for data-efficient RL alignment of safety guards, using C-LIM learnability scores to prune, densify, amend, or expand training data. It reduces over-refusal from 22.4% to 12.8% and improves data efficiency.
Introduces ReliableTableQA, a framework for training LLMs to annotate statistical reliability of tabular QA results, showing that a small SFT set is sufficient and GRPO only helps when SFT is under-trained.
ILLUME-X is a unified multimodal model for free-form interleaved text-image generation, featuring improved data efficiency, stable training, and a comprehensive evaluation metric called ILScore. It outperforms previous models on tasks like style transfer, image decomposition, and storytelling.
CLI-Universe is a synthesis engine that generates verifiable terminal-agent tasks via multi-dimensional capability taxonomy and evidence-guided research, producing a distilled dataset of 6,000 trajectories. Fine-tuning Qwen3-32B on this dataset achieves 33.4% on Terminal-Bench 2.0, setting a new state-of-the-art for open-source models at or below 32B parameters.
This paper proposes offline preference-based trajectory evaluation for agentic systems, which compares trajectories via temporal preferences rather than binary success metrics. It shows that this approach reduces ties from roughly 75% to 35%, improving discriminative power and data efficiency across diverse benchmarks.