data-quality

DataPrep-Bench is a unified benchmark evaluating LLMs' capabilities in training data construction and quality evaluation across six domains, including a skill-guided agent (Data-Construction-Skill) and a distribution-based evaluator (DAS) that achieves strong cross-model correlation.

The article argues that for effective agent memory, it is crucial to forget bad examples, as retaining them degrades performance.

AgentFAIR is a multi-agent framework that uses LLM evaluators and a critic to assess FAIR compliance of geospatial datasets, achieving sub-principle agreement of 89% and Fleiss' κ=0.71 in expert studies, at a cost of $0.054 per dataset.

Gartner predicts 60% of AI projects will be abandoned through 2026. The author, working in enterprise content management, argues the real blocker is messy unstructured legacy content and inconsistent metadata, not the AI models themselves.

UltraX proposes a function-calling refinement framework for large-scale pre-training data that introduces insertion alongside deletion and modification, enabling fine-grained instance-level editing. It builds a reliable program-supervision generation pipeline and demonstrates improved data efficiency and model performance when pretraining 1B models from scratch.

The article outlines four foundational elements of AI architecture—data quality, context engineering, governance, and human expertise—that IT leaders should prioritize to scale AI systems reliably as models evolve.

Feyn introduces Pulpie, a family of Pareto-optimal models for extracting main content from HTML pages, achieving near state-of-the-art quality at one twentieth the cost. The smallest model, pulpie-orange-small, matches leading extractors in ROUGE-5 F1 while being smaller and much faster, enabling scalable web cleaning for pre-training and inference.

This paper identifies and corrects label errors and test-train overlap in the RVL-CDIP document classification dataset, finding 12% label errors and 35% duplication. Correction improves classification accuracy and out-of-distribution generalization.

AI has great potential in agriculture, but its effectiveness depends on clean and complete data foundations; the industry faces unique data challenges from IoT devices, weather feeds, and land-specific variables.

Meta FAIR's latest paper proposes the Autodata method, which uses an intelligent data scientist Agent to autonomously generate and optimize high-quality data, enabling a 4B small model to defeat a 397B large model on legal reasoning tasks. This indicates that data quality can bridge the gap in parameter count, providing new insights for data pipelines and scaling.

This paper investigates how training dynamics of neural networks for software defect prediction are affected by coupled data-quality issues such as class imbalance and overlap, proposing an interaction-aware empirical protocol.

The author argues that AI analysis quality is limited more by data access and reliability than by reasoning, and that structured datasets dramatically improve outputs.

Google's World Cup 2026 match schedule widget displays incorrect flags for countries like Norway and England due to likely data mapping or asset mismanagement, highlighting gaps in automated data quality checks.

An AI feature for support ticket triage failed not due to model issues but because of stale data from a pipeline change, highlighting the need for integrated monitoring across teams.

A 4 billion parameter open model from the Apodex family outperforms 30 billion parameter models on web research benchmarks, attributed to careful training data and self-verification techniques rather than raw scale, suggesting a more democratic trajectory for AI capability.

A worker at a FTSE100 company expresses frustration over AI adoption challenges, noting that despite pressure to use AI, the company struggles with basic data quality and user adoption, and questions if the transformation will actually happen.

A research paper proposing a unified agentic-retrieval framework for autonomous context-aware data quality assessment. It interprets natural-language usage descriptions, generates executable validation logic via multi-agent workflow, and uses feasibility validation to ensure reliability.

An opinion piece questioning whether we rely too heavily on confident agent recommendations (human or AI) when underlying data is often messy and incomplete, suggesting that agents should express uncertainty.

DeMix is a novel framework that detects erroneous training samples and identifies their specific error types (label errors, feature errors, spurious correlations) by analyzing influence vectors, achieving a 22.61% improvement in debugging F1-score and 9.32% gain in task performance after data repair.

The author reflects on why AI agents that perform well in demos often fail in real workflows, arguing that execution quality may be more tied to data issues (task examples, tool traces, evaluation sets) than to reasoning or planning alone, and notes that they are exploring this problem through the OpenDCAI/DataFlow project.