ACLArena: Agent Continue Learning in Multi-stage Post-training

Hugging Face Daily Papers Papers
continual-learning agent-learning post-training distillation fine-tuning model-merging loRA

Summary

The paper presents ACLArena, a framework for evaluating Agent Continual Learning in multi-stage post-training, analyzing forgetting and generalization mechanisms, and proposing an improved ACL recipe using offline replay and LoRA experts.

Building general-purpose agents for industrial deployment requires integrating multiple capabilities, each typically acquired at a distinct stage of training. Yet there is currently no well-established recipe for Agent Continual Learning (ACL), with little understanding of the trade-offs among existing integration paradigms. To address this gap, we introduce ACLArena, a framework for comprehensively studying, analyzing, and evaluating ACL. We first build a sequential training pipeline and conduct an in-depth analysis that explains the mechanisms of forgetting and generalization from two complementary perspectives, the model level and the token level. Guided by these analyses, we systematically compare multi-teacher on-policy distillation, self-distilled fine-tuning, and model merging to assess their ability to recover previously learned capabilities while preserving newly acquired ones. Through extensive experiments, we develop a detailed understanding of how capabilities transfer across stages. Finally, we propose a new ACL recipe that combines offline replay over high-quality trajectories with a routed network of multiple LoRA experts each specialized via RL, substantially improving the agent's ability to learn across multiple domains. Comprehensive experiments on four reasoning and agentic tasks, evaluated under both in-domain and out-of-domain settings, demonstrate the value of our analysis and the effectiveness of our approach.
Original Article
View Cached Full Text

Cached at: 09/22/26, 11:29 PM

Paper page - ACLArena: Agent Continue Learning in Multi-stage Post-training

Source: https://huggingface.co/papers/2609.23989 Authors:

,

,

,

,

,

,

,

,

,

,

Abstract

Buildinggeneral-purposeagentsforindustrialdeploymentrequiresintegratingmultiplecapabilities,eachtypicallyacquiredatadistinctstageoftraining.Yetthereiscurrentlynowell-establishedrecipeforAgentContinualLearning(ACL),withlittleunderstandingofthetrade-offsamongexistingintegrationparadigms.Toaddressthisgap,weintroduceACLArena,aframeworkforcomprehensivelystudying,analyzing,andevaluatingACL.Wefirstbuildasequentialtrainingpipelineandconductanin-depthanalysisthatexplainsthemechanismsofforgettingandgeneralizationfromtwocomplementaryperspectives,themodellevelandthetokenlevel.Guidedbytheseanalyses,wesystematicallycomparemulti-teacheron-policydistillation,self-distilledfine-tuning,andmodelmergingtoassesstheirabilitytorecoverpreviouslylearnedcapabilitieswhilepreservingnewlyacquiredones.Throughextensiveexperiments,wedevelopadetailedunderstandingofhowcapabilitiestransferacrossstages.Finally,weproposeanewACLrecipethatcombinesofflinereplayoverhigh-qualitytrajectorieswitharoutednetworkofmultipleLoRAexpertseachspecializedviaRL,substantiallyimprovingtheagent’sabilitytolearnacrossmultipledomains.Comprehensiveexperimentsonfourreasoningandagentictasks,evaluatedunderbothin-domainandout-of-domainsettings,demonstratethevalueofouranalysisandtheeffectivenessofourapproach.

View arXiv pageView PDFGitHub1Add to collection

Get this paper in your agent:

hf papers read 2609\.23989

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.23989 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.23989 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.23989 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

CLAP: Closed-Loop Training, Evaluation, and Release Control for Domain Agent Post-training

arXiv cs.AI

CLAP proposes a closed-loop method for domain agent post-training that converts noisy business data into structured SFT and preference samples, integrates reward/KL diagnosis, offline gates, and application-chain replay to decide adapter release. Experiments on five manufacturing batches show modest average gains and highlight that regression and high KL risks require an integrated data-training-evaluation-release loop rather than relying on a single score.

From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning

arXiv cs.CL

This paper proposes the LLM-as-Environment-Engineer framework, where a policy model analyzes failures to automatically redesign the training environment for reinforcement learning, and introduces MAPF-FrozenLake as a controllable testbed. The framework, using Qwen3-4B, outperforms larger models like GPT and Gemini, showing that policy learning improves the model's ability to diagnose weaknesses.