Tag
A tool that lets you check if your data is included in the training sets of large language models, exploring the concept of digital immortality in AI.
Article questions why frontier AI labs like OpenAI and Anthropic do not disclose the size of their training data, suggesting that improvements may come from data volume rather than genuine intelligence.
A new initiative called Trace Commons aims to collect coding agent traces into an open CC-BY-4.0 dataset to help train open-weight and open-source models, countering the data advantage of proprietary models from Anthropic and OpenAI.
DeMix is a novel framework that detects erroneous training samples and identifies their specific error types (label errors, feature errors, spurious correlations) by analyzing influence vectors, achieving a 22.61% improvement in debugging F1-score and 9.32% gain in task performance after data repair.
This paper introduces ISE, a three-stage synthesis paradigm for generating multi-turn OS-agent trajectories with grounded execution, demonstrating that fine-tuning on the resulting ISE-Trace dataset significantly improves agent performance on ClawEval.
This primer paper explores how reasoning models improve after training, arguing that effective reasoning data relies more on checkable training evidence than raw data size. It categorizes reasoning data by verification methods and emphasizes preserving messy agent data for learning signals.
This paper introduces LIMMT, a data-centric study showing that training with high-quality, minimal subsets of motion data (under 3% of AMASS) outperforms using the full dataset for physics-based humanoid motion tracking, defining motion data quality through physics feasibility, diversity, and complexity.
Anyscale demonstrates a 20x speedup in cross-region training data reads by using Alluxio NVMe caching with Ray Data, showing warm cache reads drop from 4,241 to 208 seconds for 1TB.
This article explores model collapse not as a technical bug but as an epistemic problem: when an AI model's outputs become its own inputs, the model's representation of reality gradually flattens into a self-referential average, raising questions about how we distinguish a model that models the world from one that models only itself.
Asuka Zheng argues that the 'running out of training data' panic is misplaced; the real scarcity is a lack of imagination in collecting diverse, long-horizon data, illustrated by her SRE replacement project and broader research trends.
This paper identifies harmful continuations in answer-correct long chain-of-thought training traces for LLM SFT, characterized by uncertainty-geometry mismatches, and proposes a lightweight boundary proxy method to remove them.
This paper diagnoses the low diversity in LLM-generated stories, finding that 88.3% of sampled stories contain one of 11 common words (e.g., Elias, lighthouse) across models, and traces this homogeneity to post-training data and alignment rather than prevalence in pre-training data.
This paper investigates whether positional bias in dense retrievers originates from architecture or training data, finding that training data distribution strongly influences bias and that balanced training can reduce sensitivity by up to 87% while maintaining retrieval performance.
The article discusses concerns that as AI tools generate increasing amounts of code, future models trained on this synthetic code may suffer from reduced quality and originality, and asks how major AI labs like OpenAI, Anthropic, and GitHub plan to address this issue.
This paper investigates brain-LLM alignment across English, Chinese, and French using fMRI data and multiple LLMs, finding that training-language dominance and typological distance, not an inherent English advantage, drive alignment patterns.
Workers in India are increasingly using head-mounted cameras to collect video data for training humanoid robots, highlighting a growing trend in AI data collection.
An experiment comparing three Supervised Fine-Tuning data formats (demonstrations, first-person statements, synthetic documents) for injecting a C-3PO persona into Qwen3-4B, finding first-person statements best for generalization and synthetic documents best for factual knowledge.
Training Data - AI Microgames is a product where users play microgames to help gather training data for AI.
New research suggests that with sufficient compute, filtering training data for language models may be unnecessary, and models can benefit from low-quality data.
Terminal-World introduces a fully automated pipeline that uses agent skills to synthesize high-quality training data for terminal agents, enabling models to outperform baselines with only 1.2% of the training data. The method co-derives task instructions, environments, and teacher trajectories from skill primitives.