Tag
An autonomous research program by Qiushi Engine conducted end-to-end research on BabyLM 2026 Strict-Small, improving data-efficient language models through principle-guided methods and achieving the highest score in the public snapshot.
This paper introduces Looped GPT-BERT, which uses depth-wise parameter sharing to train a small language model with fewer parameters, achieving comparable performance to baselines in the BabyLM 2026 Strict-small setting.