baby-lm

研究训练小规模GPT-2模型于儿童日常语音转录（BabyView）和合成儿童导向对话（TinyDialogues）数据上，发现模型会显著特化于特定儿童的语言环境，多儿童数据能提升对混合留出语言的预测但对语法基准改进有限，表明小数据语言学习不仅取决于数据量，还取决于环境来源与匹配度。

ARMOR-DP is a modular framework for multilingual BabyLM that dynamically adjusts language sampling based on per-language validation loss, showing that the weakest language shifts during training and that dynamic protection outperforms fixed Chinese-protection baselines.

An autonomous research program by Qiushi Engine conducted end-to-end research on BabyLM 2026 Strict-Small, improving data-efficient language models through principle-guided methods and achieving the highest score in the public snapshot.

This paper introduces Looped GPT-BERT, which uses depth-wise parameter sharing to train a small language model with fewer parameters, achieving comparable performance to baselines in the BabyLM 2026 Strict-small setting.