A Vocabulary for Multi-Agent Automated Research Systems

Hugging Face Daily Papers Papers
multi-agent automated-research vocabulary research-systems design-choices

Summary

This paper introduces a formal vocabulary for describing and comparing multi-agent automated research systems, covering design choices such as agent identity, operations, communication, and evaluation. It distinguishes between generative and evaluative taste and instantiates the vocabulary on recent systems.

We introduce a vocabulary for automated research systems built from one or more agents to make their design choices easier to describe and compare. The vocabulary specifies 1) who the agents are, 2) what operations are available in the system, 3) who may invoke them, 4) how agents communicate, 5) what information is visible within and across runs, 6) how the next action is chosen, 7) how a run begins, and 8) how outputs are evaluated. A trajectory records one run from the input task to the returned artifact. Because agents, operations, and initialization may be stochastic, repeated runs on the same task induce a distribution over trajectories rather than a single behavior. Our vocabulary turns structural design questions, such as when agents should communicate, gain or lose a capability, or carry information across runs, into testable choices. It also makes the evaluator a component of the system, since reported gains depend on how closely the proxy score matches true quality. That separation also splits the vague complaint that these systems lack taste into two failures with different solutions. Generative taste is the rate at which a system proposes novel trajectories before any score is observed, and evaluative taste is the gap between the proxy score and the quality it should match. We instantiate the vocabulary on recent autoresearch systems to illustrate that it covers designs that differ widely in structure.
Original Article
View Cached Full Text

Cached at: 07/28/26, 02:25 PM

Paper page - A Vocabulary for Multi-Agent Automated Research Systems

Source: https://huggingface.co/papers/2607.22682

Abstract

Weintroduceavocabularyforautomatedresearchsystemsbuiltfromoneormoreagentstomaketheirdesignchoiceseasiertodescribeandcompare.Thevocabularyspecifies1)whotheagentsare,2)whatoperationsareavailableinthesystem,3)whomayinvokethem,4)howagentscommunicate,5)whatinformationisvisiblewithinandacrossruns,6)howthenextactionischosen,7)howarunbegins,and8)howoutputsareevaluated.Atrajectoryrecordsonerunfromtheinputtasktothereturnedartifact.Becauseagents,operations,andinitializationmaybestochastic,repeatedrunsonthesametaskinduceadistributionovertrajectoriesratherthanasinglebehavior.Ourvocabularyturnsstructuraldesignquestions,suchaswhenagentsshouldcommunicate,gainorloseacapability,orcarryinformationacrossruns,intotestablechoices.Italsomakestheevaluatoracomponentofthesystem,sincereportedgainsdependonhowcloselytheproxyscorematchestruequality.Thatseparationalsosplitsthevaguecomplaintthatthesesystemslacktasteintotwofailureswithdifferentsolutions.Generativetasteistherateatwhichasystemproposesnoveltrajectoriesbeforeanyscoreisobserved,andevaluativetasteisthegapbetweentheproxyscoreandthequalityitshouldmatch.Weinstantiatethevocabularyonrecentautoresearchsystemstoillustratethatitcoversdesignsthatdifferwidelyinstructure.

View arXiv pageView PDFProject pageGitHub0Add to collection

Get this paper in your agent:

hf papers read 2607\.22682

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2607.22682 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2607.22682 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2607.22682 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Multi-Paradigm Agent Interaction in Practice:A Systematic Analysis of Generator-Evaluator, ReAct Loop,and Adversarial Evaluation in the buddyMe Framework

arXiv cs.AI

This paper presents a systematic analysis of three agent interaction paradigms (Generator-Evaluator, ReAct Loop, and Adversarial Evaluation) implemented in the buddyMe framework, with empirical case studies from real-world deployments. It formalizes a five-stage pipeline and a six-dimensional evaluation schema, offering practical design guidelines for multi-paradigm agent systems.

An Empirical Study of Automating Agent Evaluation

arXiv cs.CL

This paper introduces EvalAgent, a system that automates the evaluation of AI agents by encoding domain-specific expertise, addressing the limitations of standard coding assistants in this task. It also presents AgentEvalBench, a benchmark for testing evaluation pipelines, and demonstrates significant improvements in evaluation reliability.

Towards Autonomous Mechanistic Reasoning in Virtual Cells

Hugging Face Daily Papers

This paper introduces VCR-Agent, a multi-agent framework that enhances large language models for biological research by generating and validating mechanistic explanations using structured formalism and the VC-TRACES dataset. The approach improves factual precision in gene expression prediction through verified mechanistic reasoning in virtual cells.

DuMate-DeepResearch: An Auditable Multi-Agent System with Recursive Search and Rubric-Grounded Reasoning

arXiv cs.AI

This technical report introduces DuMate-DeepResearch, a multi-agent framework for deep research tasks that decouples the agent core from a tool ecosystem, and incorporates graph-based dynamic planning, recursive two-level execution, and rubric-based test-time optimization. The system achieves state-of-the-art results on two deep research benchmarks, demonstrating the value of auditable agent infrastructure.