CuratorMAS: Automating Dataset Curation via Multi-Agent Orchestration

# CuratorMAS: Automating Dataset Curation via Multi-Agent Orchestration
Source: [https://arxiv.org/html/2610.07075](https://arxiv.org/html/2610.07075)
###### Abstract

High\-quality datasets are essential for reliable machine learning, but dataset curation remains costly and hard to generalize across domains\. Existing methods typically rely on manually designed heuristics or model\-dependent signals, limiting their applicability across tasks and user queries\. To address these limitations and automate data curation, we proposeCuratorMAS, a multi\-agent collaboration framework that orchestrates agents to evaluate and curate high\-quality datasets\. To achieve the goal of flexible curation, CuratorMAS decomposes the complex curation process into five programmable execution stages and forms a parallelizable workflow\. Specifically, CuratorMAS first performs dataset exploration to collect contextual information such as file structures and constraint cues, thereby developing a comprehensive understanding of the given task\. In order to acquire up\-to\-date information, CuratorMAS retrieves domain knowledge from online sources to augment the evaluation process\. Next, CuratorMAS derives the necessary evaluation criteria and computes the corresponding metrics\. Based on these results, CuratorMAS executes filtering accordingly\. Finally, an evolution module summarizes the evaluation outcomes and updates the relevant skills\. Extensive and comprehensive experiments demonstrate that CuratorMAS significantly reduces the noise rate by up to 36\.03 percentage points \(pp\) while also improving the F1 score of downstream models by up to 8\.88 pp\.

1University of Science and Technology of China

yixinzhang@mail\.ustc\.edu\.cn, fengwenjie@ustc\.edu\.cn

## Introduction

High\-quality data are fundamental to the success of modern machine learning systems, but dataset curation remains costly, domain\-specific, and difficult to scale\([Penedo et al\. 2024](https://arxiv.org/html/2610.07075#bib.bib8);[Soldaini et al\. 2024](https://arxiv.org/html/2610.07075#bib.bib7)\)\. As training datasets are increasingly collected from web sources, crowdsourced annotations, and synthetically generated samples, dataset curation faces three key challenges: \(i\) how to automatically construct task\-specific datasets; \(ii\) how to ensure the quality of the curated datasets; and \(iii\) how to satisfy diverse user requirements\. Failure to address these challenges can hinder downstream model training by slowing convergence, amplifying annotation bias, and degrading generalization performance\.

Large\-scale data curation pipelines typically rely on predefined heuristic rules, such as text\-length thresholds and toxicity filters\([Penedo et al\. 2024](https://arxiv.org/html/2610.07075#bib.bib8);[Soldaini et al\. 2024](https://arxiv.org/html/2610.07075#bib.bib7);[Raffel et al\. 2020](https://arxiv.org/html/2610.07075#bib.bib6)\)\. Although computationally efficient, these heuristics are often brittle and inflexible when the task or user query changes\. Another line of work estimates data quality via model\-dependent signals, including training dynamics, confident learning, and meta\-learned data valuation\([Northcutt et al\. 2021](https://arxiv.org/html/2610.07075#bib.bib10);[Li et al\. 2020](https://arxiv.org/html/2610.07075#bib.bib40);[Xiao et al\. 2023](https://arxiv.org/html/2610.07075#bib.bib39);[Qiao et al\. 2022](https://arxiv.org/html/2610.07075#bib.bib38);[Chong et al\. 2022](https://arxiv.org/html/2610.07075#bib.bib41);[Ethayarajh et al\. 2022](https://arxiv.org/html/2610.07075#bib.bib11);[Calian et al\. 2025](https://arxiv.org/html/2610.07075#bib.bib3)\)\. While these methods provide useful estimates of data quality, they often require task\-specific model training or access to model internals, limiting their applicability to diverse user\-provided datasets\. Recent advances in large language models \(LLMs\) have demonstrated strong capabilities in reasoning, tool use, and data analysis\([Zhu et al\. 2026](https://arxiv.org/html/2610.07075#bib.bib1);[Qiao et al\. 2026](https://arxiv.org/html/2610.07075#bib.bib21)\), making LLM\-based evaluation a promising alternative\([Zhang et al\. 2023](https://arxiv.org/html/2610.07075#bib.bib22);[Lin et al\. 2025](https://arxiv.org/html/2610.07075#bib.bib2)\)\. Nevertheless, a central question remains:How can agents automatically curate and evaluate datasets across diverse domains and user requirements?

![Refer to caption](https://arxiv.org/html/2610.07075v1/Figures/methodology.png)Figure 1:Overview of the CuratorMAS framework\.To this end, we proposeCuratorMAS, an automated multi\-agent collaboration framework that orchestrates four core curation agents—an Explorer Agent, a Researcher Agent, an Evaluator Agent, and a Filter Agent—to curate and evaluate high\-quality datasets\. Given a dataset and a user query, CuratorMAS first performs dataset comprehension to analyze the dataset structure and identify constraints imposed by downstream models and user preferences\. It then conducts knowledge\-augmented quality evaluation, during which the researcher retrieves up\-to\-date domain knowledge to enrich the system’s evaluation skills, while the evaluator derives appropriate quality criteria and computes the corresponding metrics\. Based on the results, the filter agent makes criterion\-aware filtering decisions and finally produces a curated dataset for downstream use\. Unlike static rule\-based and model\-dependent approaches, CuratorMAS adapts its evaluation criteria to different datasets while providing transparent and auditable evaluation results\. In addition, its evolution module enables the system to continuously incorporate up\-to\-date knowledge across runs\.

Our primary contributions are as follows: \(i\) We explicitly break down the data curation process into an executable and programmable workflow, providing a structured paradigm for automated data curation\. \(ii\) We present CuratorMAS, a multi\-agent collaboration framework that coordinates agents to flexibly evaluate and curate datasets across different domains\. \(iii\) We evaluate CuratorMAS against six baselines on two datasets spanning different modalities and domains, including rule\-based, model\-based, and prompt\-based methods\. CuratorMAS reduces the noise rate by up to 36\.03 pp, while improving the F1 scores of the downstream models by up to 8\.9 pp through our effective workflow\.

## Related Work

#### Dataset Curation\.

Large\-scale dataset construction typically relies on computationally efficient heuristics, such as length and toxicity filtering\([Penedo et al\. 2024](https://arxiv.org/html/2610.07075#bib.bib8);[Soldaini et al\. 2024](https://arxiv.org/html/2610.07075#bib.bib7);[Raffel et al\. 2020](https://arxiv.org/html/2610.07075#bib.bib6)\), but these rules are dataset\-specific and require manual redesign across modalities, tasks, and user needs\. Model\-based techniques, including confident learning, influence\-based valuation, and meta\-learned utility estimation\([Swayamdipta et al\. 2020](https://arxiv.org/html/2610.07075#bib.bib9);[Northcutt et al\. 2021](https://arxiv.org/html/2610.07075#bib.bib10);[Ethayarajh et al\. 2022](https://arxiv.org/html/2610.07075#bib.bib11);[Calian et al\. 2025](https://arxiv.org/html/2610.07075#bib.bib3)\), provide quality estimates but are often tied to specific task domains and are difficult to generalize\. Building on recent work using multimodal LLMs for quality assessment\([Lin et al\. 2025](https://arxiv.org/html/2610.07075#bib.bib2)\), our framework employs LLMs for query\- and domain\-aware dataset evaluation and curation\.

#### Multi\-Agent Collaboration\.

Prior studies on multi\-agent systems have demonstrated the benefits of role specialization, debate, and workflow decomposition\. Role\-playing frameworks assign agents distinct responsibilities for collaborative reasoning\([Li et al\. 2023](https://arxiv.org/html/2610.07075#bib.bib12);[Zhang et al\. 2025](https://arxiv.org/html/2610.07075#bib.bib17)\), while debate\-based methods can improve factuality by enabling agents to critique one another\([Du et al\. 2024](https://arxiv.org/html/2610.07075#bib.bib13)\)\. Engineering\-oriented frameworks such as AutoGen\([Wu et al\. 2023](https://arxiv.org/html/2610.07075#bib.bib14)\), MetaGPT\([Hong et al\. 2024](https://arxiv.org/html/2610.07075#bib.bib15)\), and ChatDev\([Qian et al\. 2024](https://arxiv.org/html/2610.07075#bib.bib16)\)support the decomposition of complex tasks into workflows\. CuratorMAS extends these principles to dataset curation by decomposing the curation process into specialized stages coordinated through a structured multi\-agent workflow\.

#### Self\-Evolution\.

Agent self\-evolution enables agents to improve by transforming experience into persistent memories, reusable skills, or executable tools\. Reflexion\([Shinn et al\. 2023](https://arxiv.org/html/2610.07075#bib.bib18)\)stores linguistic feedback in episodic memory, while ExpeL\([Zhao et al\. 2024](https://arxiv.org/html/2610.07075#bib.bib19)\)distills reusable insights from past trajectories\. Recent studies have explored skill evolution, long\-term memory, and the automated construction of reusable tools\([Qiu et al\. 2025b](https://arxiv.org/html/2610.07075#bib.bib5);[Xia et al\. 2026](https://arxiv.org/html/2610.07075#bib.bib4);[Qiu et al\. 2025a](https://arxiv.org/html/2610.07075#bib.bib20)\)\. Unlike prior work on general reasoning and programming, CuratorMAS applies self\-evolution to dataset curation by refining domain skills and developing reusable tools\.

## CuratorMAS

In this section, we present CuratorMAS, a multi\-agent collaboration framework for curating and evaluating high\-quality datasets, as illustrated in Figure[1](https://arxiv.org/html/2610.07075#Sx1.F1)\. CuratorMAS organizes its five\-stage workflow into three core components\. 1\)Dataset Comprehension: Given an input dataset and a user query, the Explorer Agent analyzes the dataset structure, summarizes its key characteristics, and decomposes the user query into constraints such as model requirements and user preferences\. 2\)Knowledge\-Augmented Evaluation: In this open\-domain setting, the Researcher Agent identifies gaps in the evaluation criteria of the current domain skill and retrieves up\-to\-date domain knowledge from online sources to augment the evaluation\. Subsequently, the Evaluator Agent performs multi\-granularity evaluation, after which the Filter Agent filters noisy samples accordingly to produce a curated dataset\. 3\)Experience\-Based Self\-Evolution: Based on the evaluation results, the Refiner Agent summarizes the evaluation outcomes and updates the domain skills by refining the criteria and distilling usable scripts\. The overall workflow of CuratorMAS is formalized in Algorithm[1](https://arxiv.org/html/2610.07075#alg1)\.

### Dataset Comprehension

To handle dataset heterogeneity across modalities and domains, we first employ an Explorer Agentπexplorer\\pi\_\{\\mathrm\{explorer\}\}to develop a comprehensive understanding of the target datasetDD\. The Explorer Agent analyzes the dataset’s structure, parses data files, and performs essential preprocessing\. It then summarizes the dataset into a structured document𝒰D\\mathcal\{U\}\_\{D\}, which contains a structural descriptionSDS\_\{D\}, a sample overviewODO\_\{D\}, and a data\-loading protocolLDL\_\{D\}\. This document further provides guidance for subsequent agent collaboration\. The Explorer Agent also decomposes the user queryqqinto model constraintsCmodelC\_\{\\mathrm\{model\}\}and data preferencesRdataR\_\{\\mathrm\{data\}\}\. Together, these components define the evaluation constraint set𝒞q=\{Cmodel,Rdata\}\\mathcal\{C\}\_\{q\}=\\\{C\_\{\\mathrm\{model\}\},R\_\{\\mathrm\{data\}\}\\\}, which bounds the retrieval and evaluation procedures\. By providing downstream agents with a shared representation of the dataset and user requirements, this comprehension stage promotes consistent reasoning and reduces the context overhead caused by repeated exploration\.

### Knowledge\-Augmented Evaluation

#### External Knowledge Retrieval\.

According to the Hugging Face taxonomy, datasets span 50 domains and more than nine modalities, each potentially requiring distinct quality criteria\. This diversity makes it impractical to define a fixed set of universally applicable evaluation metrics in advance\. Therefore, we introduce an online knowledge retrieval stage to support adaptive evaluation\. Specifically, guided by the query\-specific constraints𝒞q\\mathcal\{C\}\_\{q\}, the Researcher Agentπresearcher\\pi\_\{\\mathrm\{researcher\}\}reviews the current domain skillSdomainS\_\{\\mathrm\{domain\}\}and identifies potential quality risks that are not yet captured\. It then formulatesNNsearch queries\{xi\}i=1N\\\{x\_\{i\}\\\}\_\{i=1\}^\{N\}for online information retrieval\. For each query, the Researcher Agent retrieves the top\-KKmost relevant documents\{di,j\}j=1K\\\{d\_\{i,j\}\\\}\_\{j=1\}^\{K\}and spawns subagents to summarize them as defined in Equation \([1](https://arxiv.org/html/2610.07075#Sx3.E1)\):

\{si,j\}j=1K\\displaystyle\\\{s\_\{i,j\}\\\}\_\{j=1\}^\{K\}∼πsubagent\(⋅∣\{di,j\}j=1K\)\\displaystyle\\sim\\pi\_\{\\mathrm\{subagent\}\}\\left\(\\cdot\\mid\\\{d\_\{i,j\}\\\}\_\{j=1\}^\{K\}\\right\)\(1\)whereπsubagent\\pi\_\{\\mathrm\{subagent\}\}denotes the summarization policy used by each spawned subagent, andsi,js\_\{i,j\}denotes the summary of the retrieved documentdi,jd\_\{i,j\}\. We setN=4N=4andK=5K=5to balance retrieval quality and efficiency\. Finally, the Researcher Agent synthesizes these summaries and derives a set of complementary evaluation dimensionsQcompQ\_\{\\mathrm\{comp\}\}, designed to cover previously identified quality risks that are not adequately addressed by the existing criteria, as follows:

Qcomp∼πresearcher\(⋅∣𝒰D,\{si,j\}j=1K,Sdomain\)\.Q\_\{\\mathrm\{comp\}\}\\sim\\pi\_\{\\mathrm\{researcher\}\}\\left\(\\cdot\\mid\\mathcal\{U\}\_\{D\},\\\{s\_\{i,j\}\\\}\_\{j=1\}^\{K\},S\_\{\\mathrm\{domain\}\}\\right\)\.\(2\)This stage enables CuratorMAS to adapt to diverse domains by incorporating up\-to\-date domain knowledge, thereby enhancing its adaptability and evaluation robustness\.

Algorithm 1End\-to\-end workflow of CuratorMASInput: DatasetDD, queryqq, skill library𝒮\\mathcal\{S\}, tool library𝒯\\mathcal\{T\} Output: Curated datasetDcuratedD\_\{\\mathrm\{curated\}\}, rule\-wise noise setsDnoise\(ℓ\)D\_\{\\mathrm\{noise\}\}^\{\(\\ell\)\}, updatedSdomain′S^\{\\prime\}\_\{\\mathrm\{domain\}\}andTdomain′T^\{\\prime\}\_\{\\mathrm\{domain\}\}

1:

⊳\\trianglerightDataset Comprehension

2:

\(𝒰D,𝒞q\)∼πexplorer\(⋅∣D,q\)\(\\mathcal\{U\}\_\{D\},\\mathcal\{C\}\_\{q\}\)\\sim\\pi\_\{\\mathrm\{explorer\}\}\(\\cdot\\mid D,q\)
3:

Sdomain←LoadSkill⁡\(𝒮,q\)S\_\{\\mathrm\{domain\}\}\\leftarrow\\mathrm\{LoadSkill\}\(\\mathcal\{S\},q\)
4:

Tdomain←LoadTool⁡\(𝒯,q\)T\_\{\\mathrm\{domain\}\}\\leftarrow\\mathrm\{LoadTool\}\(\\mathcal\{T\},q\)
5:

⊳\\trianglerightExternal Knowledge Retrieval

6:

\{xi\}i=1N∼πresearcher\(⋅∣Sdomain\)\\\{x\_\{i\}\\\}\_\{i=1\}^\{N\}\\sim\\pi\_\{\\mathrm\{researcher\}\}\(\\cdot\\mid S\_\{\\mathrm\{domain\}\}\)
7:for

i=1i=1to

NNdo

8:

\{di,j\}j=1K←TopK⁡\(Search⁡\(xi\),K\)\\\{d\_\{i,j\}\\\}\_\{j=1\}^\{K\}\\leftarrow\\mathrm\{TopK\}\(\\mathrm\{Search\}\(x\_\{i\}\),K\)
9:

\{si,j\}j=1K∼πsubagent\(⋅∣\{di,j\}j=1K\)\\\{s\_\{i,j\}\\\}\_\{j=1\}^\{K\}\\sim\\pi\_\{\\mathrm\{subagent\}\}\(\\cdot\\mid\\\{d\_\{i,j\}\\\}\_\{j=1\}^\{K\}\)
10:endfor

11:

Qcomp∼πresearcher\(⋅∣𝒰D,\{si,j\},Sdomain\)Q\_\{\\mathrm\{comp\}\}\\sim\\pi\_\{\\mathrm\{researcher\}\}\(\\cdot\\mid\\mathcal\{U\}\_\{D\},\\\{s\_\{i,j\}\\\},S\_\{\\mathrm\{domain\}\}\)
12:

⊳\\trianglerightKnowledge\-Augmented Quality Evaluation

13:

𝒦aug←Qcomp⊕Sdomain\\mathcal\{K\}\_\{\\mathrm\{aug\}\}\\leftarrow Q\_\{\\mathrm\{comp\}\}\\oplus S\_\{\\mathrm\{domain\}\}
14:

Peval∼πevaluator\(⋅∣𝒰D,𝒦aug,𝒞q\)P\_\{\\mathrm\{eval\}\}\\sim\\pi\_\{\\mathrm\{evaluator\}\}\(\\cdot\\mid\\mathcal\{U\}\_\{D\},\\mathcal\{K\}\_\{\\mathrm\{aug\}\},\\mathcal\{C\}\_\{q\}\)
15:for all

tm∈Pevalt\_\{m\}\\in P\_\{\\mathrm\{eval\}\}in paralleldo

16:

om∼πworker\(⋅∣tm,𝒰D\)o\_\{m\}\\sim\\pi\_\{\\mathrm\{worker\}\}\(\\cdot\\mid t\_\{m\},\\mathcal\{U\}\_\{D\}\)
17:endfor

18:

ℬ←Buffer⁡\(\{om\}m=1M\)\\mathcal\{B\}\\leftarrow\\mathrm\{Buffer\}\(\\\{o\_\{m\}\\\}\_\{m=1\}^\{M\}\)
19:

⊳\\trianglerightAgentic Filtering and Dataset Export

20:

\{rℓ\}ℓ=1L∼πfilter\(⋅∣ℬ,𝒰D,𝒞q\)\\\{r\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}\\sim\\pi\_\{\\mathrm\{filter\}\}\(\\cdot\\mid\\mathcal\{B\},\\mathcal\{U\}\_\{D\},\\mathcal\{C\}\_\{q\}\)
21:for

ℓ=1\\ell=1to

LLdo

22:

Dnoise\(ℓ\)=\{x∈D:rℓ​\(x\)=1\}D\_\{\\mathrm\{noise\}\}^\{\(\\ell\)\}=\\\{x\\in D:r\_\{\\ell\}\(x\)=1\\\}
23:endfor

24:

Dnoise=⋃ℓ=1LDnoise\(ℓ\)D\_\{\\mathrm\{noise\}\}=\\displaystyle\\bigcup\_\{\\ell=1\}^\{L\}D\_\{\\mathrm\{noise\}\}^\{\(\\ell\)\}
25:

Dcurated=D∖DnoiseD\_\{\\mathrm\{curated\}\}=D\\setminus D\_\{\\mathrm\{noise\}\}
26:

⊳\\trianglerightExperience\-Based Self\-Evolution

27:

Spatch∼πrefiner\(⋅∣Peval,ℬ,Sdomain\)S\_\{\\mathrm\{patch\}\}\\sim\\pi\_\{\\mathrm\{refiner\}\}\(\\cdot\\mid P\_\{\\mathrm\{eval\}\},\\mathcal\{B\},S\_\{\\mathrm\{domain\}\}\)
28:

Sdomain′←Sdomain∪SpatchS^\{\\prime\}\_\{\\mathrm\{domain\}\}\\leftarrow S\_\{\\mathrm\{domain\}\}\\cup S\_\{\\mathrm\{patch\}\}
29:

Tpatch∼πrefiner\(⋅∣ℬ,Tdomain\)T\_\{\\mathrm\{patch\}\}\\sim\\pi\_\{\\mathrm\{refiner\}\}\(\\cdot\\mid\\mathcal\{B\},T\_\{\\mathrm\{domain\}\}\)
30:

Tdomain′←Tdomain∪𝕀gate​\(Tpatch\)T^\{\\prime\}\_\{\\mathrm\{domain\}\}\\leftarrow T\_\{\\mathrm\{domain\}\}\\cup\\mathbb\{I\}\_\{\\mathrm\{gate\}\}\(T\_\{\\mathrm\{patch\}\}\)
31:return

Dcurated,Dnoise\(ℓ\),Sdomain′,Tdomain′D\_\{\\mathrm\{curated\}\},D\_\{\\mathrm\{noise\}\}^\{\(\\ell\)\},S^\{\\prime\}\_\{\\mathrm\{domain\}\},T^\{\\prime\}\_\{\\mathrm\{domain\}\}

#### Knowledge\-Augmented Evaluation\.

The Evaluator Agentπevaluator\\pi\_\{\\mathrm\{evaluator\}\}constructs an augmented evaluation knowledge base𝒦aug=Sdomain⊕Qcomp\\mathcal\{K\}\_\{\\mathrm\{aug\}\}=S\_\{\\mathrm\{domain\}\}\\oplus Q\_\{\\mathrm\{comp\}\}, where⊕\\oplusdenotes a knowledge\-integration operator\. Based on𝒦aug\\mathcal\{K\}\_\{\\mathrm\{aug\}\}, the agent performs knowledge\-augmented evaluation\. Specifically, we adopt a two\-step process consisting of metric derivation and metric computation, thereby decoupling high\-level metric design from error\-prone computation\.

During metric derivation, the Evaluator Agent leverages the dataset context𝒰D\\mathcal\{U\}\_\{D\}, the augmented knowledge𝒦aug\\mathcal\{K\}\_\{\\mathrm\{aug\}\}, and the task constraints𝒞q\\mathcal\{C\}\_\{q\}to operationalize high\-level evaluation dimensions as concrete, computable metrics, as illustrated in Equation \([3](https://arxiv.org/html/2610.07075#Sx3.E3)\)\. It then organizes the derived metrics into a structured evaluation planPevalP\_\{\\mathrm\{eval\}\}spanning three granularities: a sample\-level componentPsampleP\_\{\\mathrm\{sample\}\}, a class\- or group\-level componentPclassP\_\{\\mathrm\{class\}\}, and a dataset\-level componentPdatasetP\_\{\\mathrm\{dataset\}\}\.

Peval\\displaystyle P\_\{\\mathrm\{eval\}\}∼πevaluator\(⋅∣𝒰D,𝒦aug,𝒞q\)\\displaystyle\\sim\\pi\_\{\\mathrm\{evaluator\}\}\\left\(\\cdot\\mid\\mathcal\{U\}\_\{D\},\\mathcal\{K\}\_\{\\mathrm\{aug\}\},\\mathcal\{C\}\_\{q\}\\right\)\(3\)During the computation, we dispatch multiple Worker Agents to execute evaluation tasks in parallel to ensure computational efficiency\. Each Worker Agent operates in an isolated workspace with a dedicated Python interpreter, thus preventing cross\-task context interference\. Upon receiving an assignmenttm∈Pevalt\_\{m\}\\in P\_\{\\mathrm\{eval\}\}, a Worker Agent loads the required files, performs the computation, and writes the results to the shared computation bufferℬ\\mathcal\{B\}, as formalized in Equation \([4](https://arxiv.org/html/2610.07075#Sx3.E4)\)\.

ℬ=Buffer\(\{om\|om∼πworker\(⋅∣tm,𝒰D\)\}m=1M\)\\mathcal\{B\}=\\operatorname\{Buffer\}\\\!\\left\(\\left\\\{o\_\{m\}\\,\\middle\|\\,o\_\{m\}\\sim\\pi\_\{\\mathrm\{worker\}\}\(\\cdot\\mid t\_\{m\},\\mathcal\{U\}\_\{D\}\)\\right\\\}\_\{m=1\}^\{M\}\\right\)\(4\)whereM=\|Peval\|M=\|P\_\{\\mathrm\{eval\}\}\|is the number of metrics in the evaluation plan andomo\_\{m\}denotes the output produced by tasktmt\_\{m\}\.

#### Agentic Filtering and Dataset Export\.

Using the results stored in the computation bufferℬ\\mathcal\{B\}, the Filter Agentπfilter\\pi\_\{\\mathrm\{filter\}\}performs a second\-stage analysis of the computed metrics and associated findings to identify poor\-quality samples\. It reviews the evaluation results and formulates a set of filtering rules\{rℓ\}ℓ=1L\\\{r\_\{\\ell\}\\\}\_\{\\ell=1\}^\{L\}based on the identified issues and the query\-specific constraints𝒞q\\mathcal\{C\}\_\{q\}\. Each rule is applied independently, and the samples flagged by each rule are recorded for traceability\. The union of the flagged samples is then removed from the original dataset\. Finally, the remaining samples are exported as the curated datasetDcuratedD\_\{\\mathrm\{curated\}\}for downstream use\. CuratorMAS also produces a detailed evaluation report and a set of visualizations summarizing the evaluation results\. The overall filtering process is formalized in Equation \([5](https://arxiv.org/html/2610.07075#Sx3.E5)\)\.

Dnoise\(ℓ\)\\displaystyle D\_\{\\mathrm\{noise\}\}^\{\(\\ell\)\}=\{x∈D:rℓ​\(x\)=1\},\\displaystyle=\\\{x\\in D:r\_\{\\ell\}\(x\)=1\\\},\(5\)Dcurated\\displaystyle D\_\{\\mathrm\{curated\}\}=D∖⋃ℓ=1LDnoise\(ℓ\)\.\\displaystyle=D\\setminus\\bigcup\_\{\\ell=1\}^\{L\}D\_\{\\mathrm\{noise\}\}^\{\(\\ell\)\}\.Here, each filtering rule is defined as an indicator functionrℓ:D→\{0,1\}r\_\{\\ell\}:D\\rightarrow\\\{0,1\\\}, whererℓ​\(x\)=1r\_\{\\ell\}\(x\)=1means that samplexxis identified as noisy, andrℓ​\(x\)=0r\_\{\\ell\}\(x\)=0otherwise\. Accordingly, the noisy subset identified byrℓr\_\{\\ell\}is formalized asDnoise\(ℓ\)D\_\{\\mathrm\{noise\}\}^\{\(\\ell\)\}\.

### Experience\-Based Self\-Evolution

#### Iterative Skill Refinement\.

As user requirements vary across domains, a static knowledge base may not adequately cover diverse curation scenarios and may therefore provide insufficient guidance\. To tackle this issue, CuratorMAS iteratively refines the domain skills to support continual learning\.

The Refiner Agentπrefiner\\pi\_\{\\mathrm\{refiner\}\}reviews the computed metrics and evaluation findings to identify capability gaps \(i\.e\., overly general criteria or insufficiently covered cases\) and failure modes in the current domain skillSdomainS\_\{\\mathrm\{domain\}\}\. It then derives a set of patchesSpatchS\_\{\\mathrm\{patch\}\}to remedy these deficiencies and update the domain skills as follows:

Sdomain′←Sdomain∪SpatchS^\{\\prime\}\_\{\\mathrm\{domain\}\}\\leftarrow S\_\{\\mathrm\{domain\}\}\\cup S\_\{\\mathrm\{patch\}\}\(6\)

#### Reusable Tool Distillation\.

During evaluation, the agents may generate a substantial number of execution scripts for common data\-processing operations\. To leverage these one\-off snippets, the Refiner Agent distills them into compact, reusable data\-processing tools\. We maintain a domain\-specific tool libraryTdomainT\_\{\\mathrm\{domain\}\}\. Rather than modifying the library directly, the Refiner Agent submits proposed changes as a set of CRUD operations, denoted byTpatchT\_\{\\mathrm\{patch\}\}\. BeforeTpatchT\_\{\\mathrm\{patch\}\}is applied toTdomainT\_\{\\mathrm\{domain\}\}, another Gate Agent𝕀gate\\mathbb\{I\}\_\{\\mathrm\{gate\}\}is invoked to review the proposed patches and reject redundant, overly specific, or invalid updates\. Only approved patches are committed to the tool library\. We further adopt a progressive\-disclosure strategy to avoid overwhelming the agent with a batch of raw code\. This gating mechanism improves the compactness, reusability, and general applicability of the tool library, as formalized in Equation \([7](https://arxiv.org/html/2610.07075#Sx3.E7)\):

Tdomain′\\displaystyle T^\{\\prime\}\_\{\\mathrm\{domain\}\}←Tdomain∪𝕀gate​\(Tpatch\)\.\\displaystyle\\leftarrow T\_\{\\mathrm\{domain\}\}\\cup\\mathbb\{I\}\_\{\\mathrm\{gate\}\}\(T\_\{\\mathrm\{patch\}\}\)\.\(7\)
Table 1:Performance of CuratorMAS across contamination levels\. Metrics \(%\) are reported as mean±\\pmstd across three runs, with contaminated samples as the positive class\.

## Experiments

### Experimental Settings

#### Datasets\.

We evaluate CuratorMAS on three datasets across the image and text modalities\. The text benchmarks are \(i\) JD: a sentiment classification dataset of product reviews from the JD platform\([DAMO\_NLP 2022](https://arxiv.org/html/2610.07075#bib.bib25)\)\. We selected and manually re\-annotated 1,000 samples to construct a clean subset, denoted as JD\_clean; \(ii\) NoisyAG\-News: a human\-annotated news\-topic classification dataset with naturally occurring label noise\([Huang et al\. 2024](https://arxiv.org/html/2610.07075#bib.bib24)\)\. We selected 3,000 samples with a noise rate of 19\.83%\. The image benchmark is \(iii\) CIFAR\-10N: a human\-annotated image classification dataset containing real\-world label noise\([Wei et al\. 2022](https://arxiv.org/html/2610.07075#bib.bib23)\)\. We selected 3,000 samples with a noise rate of 39\.67%\.

DatasetMethodFilterHitPrecisionRecallF1Noise LevelCIFAR\-10NOriginal\-\-\-\-\-39\.67%Rule\-based Method\(s\)Heuristic1,18855646\.80%46\.72%46\.76%34\.99%Model\-based Method\(s\)CleanLab87672782\.99%61\.09%70\.38%21\.80%DivideMix1,53992760\.23%77\.90%67\.94%18\.00%ProMix1,18083270\.51%69\.92%70\.21%19\.67%LossRank1,18877965\.57%65\.46%65\.52%22\.68%Prompt\-based Method\(s\)LLM\-as\-a\-Judge1,457104571\.72%87\.82%78\.96%9\.40%Agentic Method\(s\)Ours1,543113773\.69%95\.55%83\.21%3\.64%NoisyAG\-NewsOriginal\-\-\-\-\-19\.83%Rule\-based Method\(s\)Heuristic50720239\.84%33\.95%36\.66%15\.76%Model\-based Method\(s\)CleanLab998888\.89%14\.79%25\.36%17\.48%DivideMix13310881\.20%18\.15%29\.67%16\.99%ProMix66527541\.35%46\.22%43\.65%13\.70%LossRank59432454\.55%54\.45%54\.50%11\.26%Prompt\-based Method\(s\)LLM\-as\-a\-Judge24017773\.75%29\.75%42\.40%15\.14%Agentic Method\(s\)Ours42532275\.76%54\.12%63\.14%10\.60%Table 2:Overall performance comparison with baselines on CIFAR\-10N and NoisyAG\-News\.
#### Baselines\.

We compare CuratorMAS with six baselines across three categories: rule\-based, model\-based, and prompt\-based approaches\. For the rule\-based method, following previous work\([Penedo et al\. 2024](https://arxiv.org/html/2610.07075#bib.bib8);[Soldaini et al\. 2024](https://arxiv.org/html/2610.07075#bib.bib7);[Raffel et al\. 2020](https://arxiv.org/html/2610.07075#bib.bib6)\), we implement \(i\) Heuristic Rules, which comprises text\-integrity checks, anomalous\-image\-resolution checks, and other heuristics\. Model\-based methods include \(ii\) CleanLab\([Northcutt et al\. 2021](https://arxiv.org/html/2610.07075#bib.bib10)\), which uses confident learning to detect label noise; \(iii\) DivideMix\([Li et al\. 2020](https://arxiv.org/html/2610.07075#bib.bib40)\), which models the distribution of training losses to distinguish clean samples from noisy ones; \(iv\) ProMix\([Xiao et al\. 2023](https://arxiv.org/html/2610.07075#bib.bib39)\), which combines small\-loss selection, pseudo\-label expansion, and mixed\-sample augmentation; and \(v\) LossRank\([Chong et al\. 2022](https://arxiv.org/html/2610.07075#bib.bib41)\), which leverages loss as an indicator\. For the prompt\-based method, we evaluate \(vi\) LLM\-as\-a\-Judge, which uses GPT\-5\.4\([OpenAI 2026b](https://arxiv.org/html/2610.07075#bib.bib26)\)as a judge to classify whether a sample is noisy\.

#### Downstream Models\.

For image classification, we consider \(i\) ResNet\-18\([He et al\. 2016](https://arxiv.org/html/2610.07075#bib.bib29)\); \(ii\) MobileNetV2\([Sandler et al\. 2018](https://arxiv.org/html/2610.07075#bib.bib30)\); \(iii\) EfficientNet\-B0\([Tan and Le 2019](https://arxiv.org/html/2610.07075#bib.bib31)\); and \(iv\) GhostNet\([Han et al\. 2020](https://arxiv.org/html/2610.07075#bib.bib32)\)\. All four models are initialized with weights pretrained on ImageNet\-1K\([Deng et al\. 2009](https://arxiv.org/html/2610.07075#bib.bib37)\)\. For text classification, we leverage four pretrained language models, including \(v\) BERT\-base\-uncased\([Devlin et al\. 2019](https://arxiv.org/html/2610.07075#bib.bib33)\); \(vi\) DeBERTa\-v3\-small\([He et al\. 2023](https://arxiv.org/html/2610.07075#bib.bib34)\); \(vii\) MobileBERT\([Sun et al\. 2020](https://arxiv.org/html/2610.07075#bib.bib35)\); and \(viii\) BERT\-tiny\([Turc et al\. 2019](https://arxiv.org/html/2610.07075#bib.bib36)\)\.

#### Implementation Details\.

For the primary results, we use GPT\-5\.4 by default and also evaluate GPT\-5\.3\-Codex\([OpenAI 2026a](https://arxiv.org/html/2610.07075#bib.bib27)\)and Claude Sonnet\-4\.6\([Anthropic 2026](https://arxiv.org/html/2610.07075#bib.bib28)\)\. All LLM\-based agents follow the ReAct reasoning\-and\-action paradigm\([Yao et al\. 2023](https://arxiv.org/html/2610.07075#bib.bib43)\)\. All experiments were conducted on Ubuntu 22\.04 with an NVIDIA RTX 4090 GPU\.

![Refer to caption](https://arxiv.org/html/2610.07075v1/Figures/image-downstream-bar-aaai.png)\(a\)Model Performance on CIFAR\-10N\.
![Refer to caption](https://arxiv.org/html/2610.07075v1/Figures/text-downstream-bar-aaai.png)\(b\)Model Performance on NoisyAG\-News

Figure 2:Downstream performance of models trained on the original noisy and CuratorMAS\-curated datasets\.![Refer to caption](https://arxiv.org/html/2610.07075v1/Figures/cifar-example-aaai.png)Figure 3:Examples of different noise types in CIFAR\-10N\.

### Controlled Contamination

We synthetically inject five predefined noise types into JD\_clean at varying contamination levels\. Specifically, we consider five representative types of noise that may arise during data collection and annotation: \(i\)Label Flipping, where the original label is changed to the opposite class; \(ii\)Off\-Domain, where the original review is replaced with irrelevant text from an unrelated corpus, such as THUCNews\([Sun et al\. 2016](https://arxiv.org/html/2610.07075#bib.bib42)\); \(iii\)Ambiguous, where an LLM rewrites the review to make its sentiment ambiguous; \(iv\)Broken Text, where the review is corrupted by encoding errors, HTML residues, or other textual corruption; and \(v\)Duplication, where a randomly selected review is duplicated to replace another sample\. These noise types are injected in equal proportions to produce overall noise rates of 10%, 20%, and 30%\. For each level, we perform a single filtering pass\.

As shown in Table[1](https://arxiv.org/html/2610.07075#Sx3.T1), CuratorMAS achieves favorable performance across noise levels while maintaining a low false\-positive rate \(FPR\)\. At the 30% level, CuratorMAS achieves a recall of 83\.00% and a precision of 92\.72%, with a relatively low FPR of 3\.44%\. As the noise increases from 10% to 30%, CuratorMAS remains effective, with the F1 score rising from 78\.22% to 87\.13%, whereas the FPR increases only moderately from 2\.80% to 3\.44%\. This trend may be attributed to the increased prevalence of noisy samples, which provides more detectable positive instances\. These results indicate that the programmable curation workflow can identify and filter multiple types of noise across noise levels\.

### Real\-World Label Noise

To evaluate the performance under real\-world noise, we compare CuratorMAS with six classic baselines across two noisy datasets: CIFAR\-10N and NoisyAG\-News\. We apply two rounds of filtering to CIFAR\-10N and one round to NoisyAG\-News\. As shown in Table[2](https://arxiv.org/html/2610.07075#Sx4.T2), CuratorMAS achieves the best overall performance among all evaluated methods\.

On CIFAR\-10N, CuratorMAS reaches 95\.55% recall, a 7\.73\-point improvement over the prompt\-based method, while maintaining a precision of 73\.69% and reducing the residual noise rate to 3\.64%\. Compared with model\-based baselines, CuratorMAS drastically reduces the noise rate with consistently higher recall and precision\. Although LLMs are effective judges \(71\.72% precision and 87\.82% recall\), CuratorMAS decomposes the complex curation procedure into a programmable workflow, thereby further improving its overall performance\. Representative examples are shown in Figure[3](https://arxiv.org/html/2610.07075#Sx4.F3)\. On NoisyAG\-News, CuratorMAS also demonstrates its robust capabilities with the lowest residual noise rate of 10\.60% and the highest precision and recall\.

These results underscore CuratorMAS’s consistent ability to foster more effective collaboration that translates directly into curation capability, regardless of the task domain and modality\. We further observe a precision–recall trade\-off: additional filtering rounds improve noise coverage but may introduce a higher FPR, indicating the need to balance automated filtering with the available human\-review budget\.

![Refer to caption](https://arxiv.org/html/2610.07075v1/Figures/selected-downstream-loss-group-aaai.png)Figure 4:Training and validation loss curves of downstream models trained on the original and CuratorMAS\-curated datasets\.
### Impact on Downstream Training

To further investigate how CuratorMAS benefits model training and generalization, we evaluate the downstream performance of models trained on the aforementioned CIFAR\-10N and NoisyAG\-News datasets curated by CuratorMAS\. Figure[2](https://arxiv.org/html/2610.07075#Sx4.F2)shows the overall performance gains across eight model architectures\. For instance, on CIFAR\-10N, all four models consistently achieve performance gains, with F1 scores improving by up to 8\.88 pp \(MobileNetV2\) and by 5\.99 pp on average\. Similarly, on NoisyAG\-News, all four BERT\-style models benefit from the curated dataset with an average F1 score improvement of 3\.64 pp\. These results indicate that CuratorMAS supports better downstream model performance across both text and image modalities\.

Beyond improving downstream performance, CuratorMAS also yields more favorable training dynamics\. As illustrated in Figure[4](https://arxiv.org/html/2610.07075#Sx4.F4), models trained on the curated datasets converge in fewer epochs and reach lower final training losses than those trained on the original noisy datasets\. The validation curves also exhibit smaller fluctuations and less evidence of overfitting, suggesting more stable optimization\. Together with the improvements in test\-set performance, these observations indicate that CuratorMAS improves optimization efficiency and downstream generalization\.

### Ablation Study

Table 3:Sensitivity of CuratorMAS to LLM Backbones on CIFAR\-10N and NoisyAG\-News\.#### Sensitivity Analysis\.

We conduct a sensitivity analysis using three different LLM backbones for our task\-execution agents\. Table[3](https://arxiv.org/html/2610.07075#Sx4.T3)reveals noticeable performance differences among the three models\. GPT\-5\.4 delivers the best overall performance, achieving a favorable trade\-off among precision, recall, and residual noise rate\. GPT\-5\.3\-Codex does not outperform either GPT\-5\.4 or Sonnet\-4\.6\. This result suggests that specialization for coding alone does not necessarily translate into stronger data\-curation capabilities, whereas general reasoning ability may play a more fundamental role in data curation\. This observation is broadly consistent with previous research\([Zhu et al\. 2026](https://arxiv.org/html/2610.07075#bib.bib1)\)\. These findings indicate that although CuratorMAS can effectively support automated data curation, its curation performance remains sensitive to the capabilities of the underlying LLM backbone\.

#### Component Ablation\.

We also analyze the contributions of two key components of CuratorMAS on NoisyAG\-News: external knowledge retrieval and skill enhancement\. To prevent data leakage and ensure a fair comparison, the domain skills are constructed using a held\-out subset\. As shown in Table[4](https://arxiv.org/html/2610.07075#Sx4.T4), both components contribute to the effectiveness of CuratorMAS\. Removing both leads to the largest degradation, resulting in the poorest curation performance\. External knowledge retrieval supports domain\-specific evaluation, whereas domain skills facilitate long\-term knowledge accumulation and stability\. These results demonstrate the complementary roles of the two components in CuratorMAS\.

Table 4:Component Ablation of CuratorMAS\.w/o KAuses only domain skills;w/o Skilluses only external knowledge retrieval;Neitherremoves both components;Fulldenotes the complete CuratorMAS\.

## Conclusion

We presented CuratorMAS, a multi\-agent collaboration framework that enables agents to flexibly evaluate and curate datasets across diverse domains\. By breaking down the complex data curation process into executable and programmable stages, CuratorMAS can adapt its evaluation criteria to different domains and dynamic user queries, thereby providing a structured paradigm for automated data curation\. Experiments on three datasets against six baselines demonstrate the effectiveness of CuratorMAS, which reduces the noise rate by up to 36\.03 pp\. Moreover, models trained on the curated datasets achieve improvements of up to 8\.9 pp in F1 score\. These results establish CuratorMAS as a practical and flexible solution to automated data curation across diverse real\-world scenarios\. Future work includes extending CuratorMAS to more complex tasks while further improving its generalization and stability for high\-stakes applications\.

## References

- Anthropic \(2026\)AnthropicIntroducing claude sonnet 4\.6\.Note:https://www\.anthropic\.com/news/claude\-sonnet\-4\-6Accessed: 2026\-07\-29Cited by:[Implementation Details\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px4.p1.1)\.
- Calianet al\.\(2025\)D\. A\. Calian, G\. Farquhar, I\. Kemaev, L\. Zintgraf, M\. Hessel, J\. Shar, J\. Oh, A\. György, T\. Schaul, J\. Dean, H\. van Hasselt, and D\. SilverDataRater: meta\-learned dataset curation\.InAdvances in Neural Information Processing Systems,Vol\.38,pp\. 20060–20092\.Cited by:[Introduction](https://arxiv.org/html/2610.07075#Sx1.p2.1),[Dataset Curation\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px1.p1.1)\.
- Chonget al\.\(2022\)D\. Chong, J\. Hong, and C\. D\. ManningDetecting label errors by using pre\-trained language models\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7\-11, 2022,Y\. Goldberg, Z\. Kozareva, and Y\. Zhang \(Eds\.\),pp\. 9074–9091\.External Links:[Link](https://doi.org/10.18653/v1/2022.emnlp-main.618),[Document](https://dx.doi.org/10.18653/V1/2022.EMNLP-MAIN.618)Cited by:[Introduction](https://arxiv.org/html/2610.07075#Sx1.p2.1),[Baselines\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px2.p1.1)\.
- DAMO\_NLP \(2022\)DAMO\_NLPProduct review sentiment prediction: damo\_nlp/jd\.Note:https://www\.modelscope\.cn/datasets/DAMO\_NLP/jdAccessed: 2026\-07\-29Cited by:[Datasets\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px1.p1.1)\.
- Denget al\.\(2009\)J\. Deng, W\. Dong, R\. Socher, L\. Li, K\. Li, and F\. LiImageNet: a large\-scale hierarchical image database\.In2009 IEEE Conference on Computer Vision and Pattern Recognition,pp\. 248–255\.External Links:[Document](https://dx.doi.org/10.1109/CVPR.2009.5206848)Cited by:[Downstream Models\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px3.p1.1)\.
- Devlinet al\.\(2019\)J\. Devlin, M\. Chang, K\. Lee, and K\. ToutanovaBERT: pre\-training of deep bidirectional transformers for language understanding\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL\-HLT 2019, Minneapolis, MN, USA, June 2\-7, 2019, Volume 1 \(Long and Short Papers\),J\. Burstein, C\. Doran, and T\. Solorio \(Eds\.\),pp\. 4171–4186\.External Links:[Link](https://doi.org/10.18653/v1/n19-1423),[Document](https://dx.doi.org/10.18653/V1/N19-1423)Cited by:[Downstream Models\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px3.p1.1)\.
- Duet al\.\(2024\)Y\. Du, S\. Li, A\. Torralba, J\. B\. Tenenbaum, and I\. MordatchImproving factuality and reasoning in language models through multiagent debate\.InICML,Proceedings of Machine Learning Research, Vol\.235,pp\. 11733–11763\.Cited by:[Multi\-Agent Collaboration\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px2.p1.1)\.
- Ethayarajhet al\.\(2022\)K\. Ethayarajh, Y\. Choi, and S\. SwayamdiptaUnderstanding dataset difficulty with*V*\-usable information\.InICML,Proceedings of Machine Learning Research, Vol\.162,pp\. 5988–6008\.Cited by:[Introduction](https://arxiv.org/html/2610.07075#Sx1.p2.1),[Dataset Curation\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px1.p1.1)\.
- Hanet al\.\(2020\)K\. Han, Y\. Wang, Q\. Tian, J\. Guo, C\. Xu, and C\. XuGhostNet: more features from cheap operations\.In2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13\-19, 2020,pp\. 1577–1586\.External Links:[Link](https://openaccess.thecvf.com/content/_CVPR/_2020/html/Han/_GhostNet/_More/_Features/_From/_Cheap/_Operations/_CVPR/_2020/_paper.html),[Document](https://dx.doi.org/10.1109/CVPR42600.2020.00165)Cited by:[Downstream Models\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px3.p1.1)\.
- Heet al\.\(2016\)K\. He, X\. Zhang, S\. Ren, and J\. SunDeep residual learning for image recognition\.In2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27\-30, 2016,pp\. 770–778\.External Links:[Link](https://doi.org/10.1109/CVPR.2016.90),[Document](https://dx.doi.org/10.1109/CVPR.2016.90)Cited by:[Downstream Models\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px3.p1.1)\.
- Heet al\.\(2023\)P\. He, J\. Gao, and W\. ChenDeBERTaV3: improving deberta using electra\-style pre\-training with gradient\-disentangled embedding sharing\.InThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1\-5, 2023,External Links:[Link](https://openreview.net/forum?id=sE7-XhLxHA)Cited by:[Downstream Models\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px3.p1.1)\.
- Honget al\.\(2024\)S\. Hong, M\. Zhuge, J\. Chen, X\. Zheng, Y\. Cheng, J\. Wang, C\. Zhang, Z\. Wang, S\. K\. S\. Yau, Z\. Lin, L\. Zhou, C\. Ran, L\. Xiao, C\. Wu, and J\. SchmidhuberMetaGPT: meta programming for A multi\-agent collaborative framework\.InICLR,Cited by:[Multi\-Agent Collaboration\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px2.p1.1)\.
- Huanget al\.\(2024\)H\. Huang, T\. Liang, X\. Sun, Z\. Jin, and Y\. YinNoisyAG\-news: A benchmark for addressing instance\-dependent noise in text classification\.External Links:2407\.06579Cited by:[Datasets\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px1.p1.1)\.
- Liet al\.\(2023\)G\. Li, H\. Hammoud, H\. Itani, D\. Khizbullin, and B\. GhanemCAMEL: communicative agents for "mind" exploration of large language model society\.InNeurIPS,Vol\.36,pp\. 51991–52008\.Cited by:[Multi\-Agent Collaboration\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px2.p1.1)\.
- Liet al\.\(2020\)J\. Li, R\. Socher, and S\. C\. H\. HoiDivideMix: learning with noisy labels as semi\-supervised learning\.In8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26\-30, 2020,External Links:[Link](https://openreview.net/forum?id=HJgExaVtwr)Cited by:[Introduction](https://arxiv.org/html/2610.07075#Sx1.p2.1),[Baselines\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px2.p1.1)\.
- Linet al\.\(2025\)L\. Lin, D\. Shi, A\. Han, F\. Chen, Q\. Chen, J\. Li, Z\. Li, J\. Zhang, Z\. Sun, and J\. GaoACT as human: multimodal large language model data annotation with critical thinking\.InAdvances in Neural Information Processing Systems,Vol\.38,pp\. 24107–24138\.Cited by:[Introduction](https://arxiv.org/html/2610.07075#Sx1.p2.1),[Dataset Curation\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px1.p1.1)\.
- Northcuttet al\.\(2021\)C\. G\. Northcutt, L\. Jiang, and I\. L\. ChuangConfident learning: estimating uncertainty in dataset labels\.J\. Artif\. Intell\. Res\.70,pp\. 1373–1411\.Cited by:[Introduction](https://arxiv.org/html/2610.07075#Sx1.p2.1),[Dataset Curation\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px1.p1.1),[Baselines\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px2.p1.1)\.
- OpenAI \(2026a\)OpenAIIntroducing gpt\-5\.3\-codex\.Note:https://openai\.com/index/introducing\-gpt\-5\-3\-codex/Accessed: 2026\-07\-29Cited by:[Implementation Details\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px4.p1.1)\.
- OpenAI \(2026b\)OpenAIIntroducing gpt\-5\.4\.Note:https://openai\.com/index/introducing\-gpt\-5\-4/Accessed: 2026\-07\-29Cited by:[Baselines\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px2.p1.1)\.
- Penedoet al\.\(2024\)G\. Penedo, H\. Kydlíček, L\. B\. Allal, A\. Lozhkov, M\. Mitchell, C\. A\. Raffel, L\. von Werra, and T\. WolfThe fineweb datasets: decanting the web for the finest text data at scale\.InNeurIPS,Vol\.37,pp\. 30811–30849\.Cited by:[Introduction](https://arxiv.org/html/2610.07075#Sx1.p1.1),[Introduction](https://arxiv.org/html/2610.07075#Sx1.p2.1),[Dataset Curation\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px1.p1.1),[Baselines\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px2.p1.1)\.
- Qianet al\.\(2024\)C\. Qian, W\. Liu, H\. Liu, N\. Chen, Y\. Dang, J\. Li, C\. Yang, W\. Chen, Y\. Su, X\. Cong, J\. Xu, D\. Li, Z\. Liu, and M\. SunChatDev: communicative agents for software development\.InACL \(1\),pp\. 15174–15186\.Cited by:[Multi\-Agent Collaboration\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px2.p1.1)\.
- Qiaoet al\.\(2022\)D\. Qiao, C\. Dai, Y\. Ding, J\. Li, Q\. Chen, W\. Chen, and M\. ZhangSelfMix: robust learning against textual label noise with self\-mixup training\.InProceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, October 12\-17, 2022,N\. Calzolari, C\. Huang, H\. Kim, J\. Pustejovsky, L\. Wanner, K\. Choi, P\. Ryu, H\. Chen, L\. Donatelli, H\. Ji, S\. Kurohashi, P\. Paggio, N\. Xue, S\. Kim, Y\. Hahm, Z\. He, T\. K\. Lee, E\. Santus, F\. Bond, and S\. Na \(Eds\.\),pp\. 960–970\.External Links:[Link](https://aclanthology.org/2022.coling-1.80)Cited by:[Introduction](https://arxiv.org/html/2610.07075#Sx1.p2.1)\.
- Qiaoet al\.\(2026\)S\. Qiao, Y\. Zhao, Z\. Qiu, X\. Wang, J\. Zhang, Z\. Bin, N\. Zhang, Y\. Jiang, P\. Xie, F\. Huang, and H\. ChenScaling generalist data\-analytic agents\.External Links:2509\.25084Cited by:[Introduction](https://arxiv.org/html/2610.07075#Sx1.p2.1)\.
- Qiuet al\.\(2025a\)J\. Qiu, X\. Qi, H\. Wang, X\. Juan, Y\. Wang, Z\. Zhao, J\. Geng, J\. Guo, P\. Li, J\. Shi, S\. Liu, and M\. WangAlita\-g: self\-evolving generative agent for agent generation\.External Links:2510\.23601Cited by:[Self\-Evolution\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px3.p1.1)\.
- Qiuet al\.\(2025b\)J\. Qiu, X\. Qi, T\. Zhang, X\. Juan, J\. Guo, Y\. Lu, Y\. Wang, Z\. Yao, Q\. Ren, X\. Jiang, X\. Zhou, D\. Liu, L\. Yang, Y\. Wu, K\. Huang, S\. Liu, H\. Wang, and M\. WangAlita: generalist agent enabling scalable agentic reasoning with minimal predefinition and maximal self\-evolution\.External Links:2505\.20286Cited by:[Self\-Evolution\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px3.p1.1)\.
- Raffelet al\.\(2020\)C\. Raffel, N\. Shazeer, A\. Roberts, K\. Lee, S\. Narang, M\. Matena, Y\. Zhou, W\. Li, and P\. J\. LiuExploring the limits of transfer learning with a unified text\-to\-text transformer\.J\. Mach\. Learn\. Res\.21\(140\),pp\. 1–67\.Cited by:[Introduction](https://arxiv.org/html/2610.07075#Sx1.p2.1),[Dataset Curation\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px1.p1.1),[Baselines\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px2.p1.1)\.
- Sandleret al\.\(2018\)M\. Sandler, A\. G\. Howard, M\. Zhu, A\. Zhmoginov, and L\. ChenMobileNetV2: inverted residuals and linear bottlenecks\.In2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18\-22, 2018,pp\. 4510–4520\.External Links:[Link](http://openaccess.thecvf.com/content/_cvpr/_2018/html/Sandler/_MobileNetV2/_Inverted/_Residuals/_CVPR/_2018/_paper.html),[Document](https://dx.doi.org/10.1109/CVPR.2018.00474)Cited by:[Downstream Models\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px3.p1.1)\.
- Shinnet al\.\(2023\)N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.InNeurIPS,Vol\.36,pp\. 8634–8652\.Cited by:[Self\-Evolution\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px3.p1.1)\.
- Soldainiet al\.\(2024\)L\. Soldaini, R\. Kinney, A\. Bhagia, D\. Schwenk, D\. Atkinson, R\. Authur, B\. Bogin, K\. R\. Chandu, J\. Dumas, Y\. Elazar, V\. Hofmann, A\. H\. Jha, S\. Kumar, L\. Lucy, X\. Lyu, N\. Lambert, I\. Magnusson, J\. Morrison, N\. Muennighoff, A\. Naik, C\. Nam, M\. E\. Peters, A\. Ravichander, K\. Richardson, Z\. Shen, E\. Strubell, N\. Subramani, O\. Tafjord, P\. Walsh, L\. Zettlemoyer, N\. A\. Smith, H\. Hajishirzi, I\. Beltagy, D\. Groeneveld, J\. Dodge, and K\. LoDolma: an open corpus of three trillion tokens for language model pretraining research\.InACL \(1\),pp\. 15725–15788\.Cited by:[Introduction](https://arxiv.org/html/2610.07075#Sx1.p1.1),[Introduction](https://arxiv.org/html/2610.07075#Sx1.p2.1),[Dataset Curation\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px1.p1.1),[Baselines\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px2.p1.1)\.
- Sunet al\.\(2016\)M\. Sun, J\. Li, Z\. Guo, Y\. Zhao, Y\. Zheng, X\. Si, and Z\. LiuTHUCTC: an efficient chinese text classifier\.Note:http://thuctc\.thunlp\.org/Accessed: 2026\-07\-29Cited by:[Controlled Contamination](https://arxiv.org/html/2610.07075#Sx4.SSx2.p1.1)\.
- Sunet al\.\(2020\)Z\. Sun, H\. Yu, X\. Song, R\. Liu, Y\. Yang, and D\. ZhouMobileBERT: a compact task\-agnostic BERT for resource\-limited devices\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5\-10, 2020,D\. Jurafsky, J\. Chai, N\. Schluter, and J\. R\. Tetreault \(Eds\.\),pp\. 2158–2170\.External Links:[Link](https://doi.org/10.18653/v1/2020.acl-main.195),[Document](https://dx.doi.org/10.18653/V1/2020.ACL-MAIN.195)Cited by:[Downstream Models\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px3.p1.1)\.
- Swayamdiptaet al\.\(2020\)S\. Swayamdipta, R\. Schwartz, N\. Lourie, Y\. Wang, H\. Hajishirzi, N\. A\. Smith, and Y\. ChoiDataset cartography: mapping and diagnosing datasets with training dynamics\.InEMNLP \(1\),pp\. 9275–9293\.Cited by:[Dataset Curation\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px1.p1.1)\.
- Tan and Le \(2019\)M\. Tan and Q\. V\. LeEfficientNet: rethinking model scaling for convolutional neural networks\.InProceedings of the 36th International Conference on Machine Learning, ICML 2019, 9\-15 June 2019, Long Beach, California, USA,K\. Chaudhuri and R\. Salakhutdinov \(Eds\.\),Proceedings of Machine Learning Research, Vol\.97,pp\. 6105–6114\.External Links:[Link](http://proceedings.mlr.press/v97/tan19a.html)Cited by:[Downstream Models\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px3.p1.1)\.
- Turcet al\.\(2019\)I\. Turc, M\. Chang, K\. Lee, and K\. ToutanovaWell\-read students learn better: on the importance of pre\-training compact models\.External Links:1908\.08962Cited by:[Downstream Models\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px3.p1.1)\.
- Weiet al\.\(2022\)J\. Wei, Z\. Zhu, H\. Cheng, T\. Liu, G\. Niu, and Y\. LiuLearning with noisy labels revisited: A study using real\-world human annotations\.InThe Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25\-29, 2022,External Links:[Link](https://openreview.net/forum?id=TBWA6PLJZQm)Cited by:[Datasets\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px1.p1.1)\.
- Wuet al\.\(2023\)Q\. Wu, G\. Bansal, J\. Zhang, Y\. Wu, B\. Li, E\. Zhu, L\. Jiang, X\. Zhang, S\. Zhang, J\. Liu, A\. H\. Awadallah, R\. W\. White, D\. Burger, and C\. WangAutoGen: enabling next\-gen LLM applications via multi\-agent conversation\.External Links:2308\.08155Cited by:[Multi\-Agent Collaboration\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px2.p1.1)\.
- Xiaet al\.\(2026\)P\. Xia, J\. Chen, H\. Wang, J\. Liu, K\. Zeng, Y\. Wang, S\. Han, Y\. Zhou, X\. Zhao, H\. Chen, Z\. Zheng, C\. Xie, and H\. YaoSkillRL: evolving agents via recursive skill\-augmented reinforcement learning\.External Links:2602\.08234Cited by:[Self\-Evolution\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px3.p1.1)\.
- Xiaoet al\.\(2023\)R\. Xiao, Y\. Dong, H\. Wang, L\. Feng, R\. Wu, G\. Chen, and J\. ZhaoProMix: combating label noise via maximizing clean sample utility\.InProceedings of the Thirty\-Second International Joint Conference on Artificial Intelligence, IJCAI 2023, 19th\-25th August 2023, Macao, SAR, China,pp\. 4442–4450\.External Links:[Link](https://doi.org/10.24963/ijcai.2023/494),[Document](https://dx.doi.org/10.24963/IJCAI.2023/494)Cited by:[Introduction](https://arxiv.org/html/2610.07075#Sx1.p2.1),[Baselines\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px2.p1.1)\.
- Yaoet al\.\(2023\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=WE_vluYUL-X)Cited by:[Implementation Details\.](https://arxiv.org/html/2610.07075#Sx4.SSx1.SSS0.Px4.p1.1)\.
- Zhanget al\.\(2025\)G\. Zhang, Y\. Yue, X\. Sun, G\. Wan, M\. Yu, J\. Fang, K\. Wang, T\. Chen, and D\. ChengG\-designer: architecting multi\-agent communication topologies via graph neural networks\.InICML,Proceedings of Machine Learning Research, Vol\.267,pp\. 76678–76692\.Cited by:[Multi\-Agent Collaboration\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2023\)R\. Zhang, Y\. Li, Y\. Ma, M\. Zhou, and L\. ZouLLMaAA: making large language models as active annotators\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 13088–13103\.External Links:[Link](https://aclanthology.org/2023.findings-emnlp.872/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.872)Cited by:[Introduction](https://arxiv.org/html/2610.07075#Sx1.p2.1)\.
- Zhaoet al\.\(2024\)A\. Zhao, D\. Huang, Q\. Xu, M\. Lin, Y\. Liu, and G\. HuangExpeL: LLM agents are experiential learners\.InAAAI,pp\. 19632–19642\.Cited by:[Self\-Evolution\.](https://arxiv.org/html/2610.07075#Sx2.SS0.SSS0.Px3.p1.1)\.
- Zhuet al\.\(2026\)Y\. Zhu, Y\. Zhong, J\. Zhang, Z\. Zhang, S\. Qiao, Y\. Luo, L\. Du, D\. Zheng, N\. Zhang, and H\. ChenWhy do open\-source llms struggle with data analysis? A systematic empirical study\.InAAAI,pp\. 35239–35247\.Cited by:[Introduction](https://arxiv.org/html/2610.07075#Sx1.p2.1),[Sensitivity Analysis\.](https://arxiv.org/html/2610.07075#Sx4.SSx5.SSS0.Px1.p1.1)\.