CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents

Hugging Face Daily Papers Papers
llm-agents mixture-of-agents reinforcement-learning routing-mechanisms continual-learning efficiency

Summary

CERA-MoA introduces a co-evolving framework for mixture-of-agents systems that uses reinforcement learning to dynamically route queries and adapt agent capabilities, enhancing task performance and efficiency.

Current Mixture-of-Agents (MoA) paradigms generally treat query routing and agent fine-tuning as separate processes, limiting their ability to respond to evolving agent capabilities. This disconnect prevents routing strategies from adapting to evolving agent capabilities during post-training and prevents agents from achieving synergistic data-driven specialization. To resolve this, we introduce CERA-MoA (Co-Evolving Router with continually learning Agents for Mixture-of-Agents), an iterative reinforcement learning framework where the dynamic router and independent agent policies co-evolve. We design a predictive familiarity estimator that leverages mid-layer hidden states to evaluate semantic competence among agents, avoiding the overhead of full rollouts. Based on these familiarity scores, a cumulative-threshold adaptive routing mechanism dynamically activates a tailored minimal agent subset, achieving a trade-off between task performance and efficiency. By proactively allocating targeted training samples to agents based on their evolving competence, CERA-MoA promotes capability differentiation. Extensive experiments across various domains demonstrate that CERA-MoA outperforms state-of-the-art static-agent routing and fix-workflow fine-tuning baselines.
Original Article
View Cached Full Text

Cached at: 09/17/26, 02:54 PM

Paper page - CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents

Source: https://huggingface.co/papers/2609.18779

Abstract

CurrentMixture-of-Agents(MoA)paradigmsgenerallytreatqueryroutingandagentfine-tuningasseparateprocesses,limitingtheirabilitytorespondtoevolvingagentcapabilities.Thisdisconnectpreventsroutingstrategiesfromadaptingtoevolvingagentcapabilitiesduringpost-trainingandpreventsagentsfromachievingsynergisticdata-drivenspecialization.Toresolvethis,weintroduceCERA-MoA(Co-EvolvingRouterwithcontinuallylearningAgentsforMixture-of-Agents),aniterativereinforcementlearningframeworkwherethedynamicrouterandindependentagentpoliciesco-evolve.Wedesignapredictivefamiliarityestimatorthatleveragesmid-layerhiddenstatestoevaluatesemanticcompetenceamongagents,avoidingtheoverheadoffullrollouts.Basedonthesefamiliarityscores,acumulative-thresholdadaptiveroutingmechanismdynamicallyactivatesatailoredminimalagentsubset,achievingatrade-offbetweentaskperformanceandefficiency.Byproactivelyallocatingtargetedtrainingsamplestoagentsbasedontheirevolvingcompetence,CERA-MoApromotescapabilitydifferentiation.ExtensiveexperimentsacrossvariousdomainsdemonstratethatCERA-MoAoutperformsstate-of-the-artstatic-agentroutingandfix-workflowfine-tuningbaselines.

View arXiv pageView PDFGitHub0Add to collection

Get this paper in your agent:

hf papers read 2609\.18779

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper1

#### mj0530/CERA-MoA-Qwen3-4B-4agents Text Generation• Updatedabout 3 hours ago • 1

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.18779 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.18779 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

MERA: Model Evolution and Routing with Skill Adaptation for Agentic Systems at Scale

arXiv cs.LG

MERA introduces a multi-cycle adaptation approach for LLM agents that improves small models by distilling execution-verified demonstrations into a SkillBook and fine-tuning LoRA adapters, with router-based deployment and verifier-backed fallback. Experiments show Qwen2.5-Coder-1.5B improves from 28.7% to 49.7% pass on HumanEval++MBPP while retaining most quality at lower cost.

Attention-Aware Routing: Coupling Routing and Attention in MoEs

arXiv cs.AI

This paper introduces Attention-Aware Routing (AAR), a method that enhances Mixture-of-Experts language models by incorporating attention weights into the router, improving mathematical reasoning performance and revealing coupled dynamics between routing and attention.