@QuixiAI: Announcing QuixiAI/SYN-1B - A synthetic pretraining dataset designed to teach models to track rules, update beliefs aft…

X AI KOLs Timeline Tools
synthetic-dataset pretraining rule-tracking belief-updating long-context open-source huggingface

Summary

QuixiAI releases SYN-1B, a synthetic pretraining dataset designed to teach models rule tracking, belief updating, and evidence preservation over long contexts, available on Hugging Face.

Announcing QuixiAI/SYN-1B - A synthetic pretraining dataset designed to teach models to track rules, update beliefs after corrections, preserve evidence over long contexts, ignore plausible distractors, and transfer abstract state-update patterns beyond memorized surface forms. 🆀 published on @huggingface
Original Article

Similar Articles

@neural_avb: https://x.com/neural_avb/status/2072294078805684613

X AI KOLs Timeline

This paper introduces Autodata, a method that uses an agentic 'data scientist' AI to automate the creation of high-quality synthetic datasets through iterative generation, verification, and refinement, specifically optimized for reinforcement learning (GRPO) to improve reasoning in language models.

Introducing SimpleQA

OpenAI Blog

OpenAI introduces SimpleQA, a new factuality benchmark dataset with 4,326 short fact-seeking questions designed to evaluate frontier language models on their ability to provide accurate answers without hallucination. The dataset achieves high quality through dual independent annotation, rigorous criteria, and achieves only ~3% estimated error rate, with GPT-4o scoring less than 40%.

Making a synthetic dataset for fine-tuning

Reddit r/LocalLLaMA

The author proposes a pipeline for generating diverse synthetic reasoning training data for LLMs using formal solvers, and asks for existing work and advice on avoiding repetitive templates.