@kyutai_labs: We train 6B-param models on Common Crawl ordered sequentially from 2018 to 2025, so that the freshest data is seen last…
Summary
Kyutai Labs trains 6B-parameter models on Common Crawl data ordered sequentially from 2018 to 2025, showing that performance drop on recent years disappears, and open-sources the checkpoints for continual learning research.
View Cached Full Text
Cached at: 05/27/26, 07:03 AM
We train 6B-param models on Common Crawl ordered sequentially from 2018 to 2025, so that the freshest data is seen last, and show that the performance drop on more recent years disappears.
We open-source these checkpoints for future research on continual learning. https://t.co/8fFwpUesIz
Similar Articles
@kyutai_labs: We show that LLM accuracy drops off on time-sensitive questions like “Who won the Champions League in year X” - even wi…
Kyutai Labs demonstrates that LLM accuracy degrades on time-sensitive questions within the knowledge cutoff and introduces the KairosQA dataset. They train models that avoid this issue by temporally sorting the training data.
@KaiZhang_CS: Check out one of the best open-source search agents trained by @jianxie_ !! glad to see early experience methods work o…
Yu Su's team trained a frontier Deep Research Agent on an academic budget using 8K synthetic samples and RL, releasing fully open training infrastructure and models from 2B to 35B parameters.
@tom_doerr: Fully open sources training data for 30B scale search agents https://github.com/PolarSeeker/OpenSeeker…
OpenSeeker fully open-sources training data and models for 30B-scale ReAct-based search agents, achieving state-of-the-art performance on multiple benchmarks including BrowseComp and Humanity's Last Exam. It is the first purely academic project to reach frontier search benchmark performance while releasing complete training data.
A 4b model is now beating 30b ones at web research and the reason is not size
A 4 billion parameter open model from the Apodex family outperforms 30 billion parameter models on web research benchmarks, attributed to careful training data and self-verification techniques rather than raw scale, suggesting a more democratic trajectory for AI capability.
@nickscamara_: New discoveries are gonna come from models that can reason over the latest science The rate of scientific progress beco…
Firecrawl released a state-of-the-art research index for AI/ML papers, claiming 18% better recall on arXivQA than competitors, designed for autonomous research agents.