Tag
Kyutai Labs trains 6B-parameter models on Common Crawl data ordered sequentially from 2018 to 2025, showing that performance drop on recent years disappears, and open-sources the checkpoints for continual learning research.