Agent Priors-guided Policy Learning

Hugging Face Daily Papers Papers
robot-learning imitation-learning policy-learning compositional-generalization multi-agent maniskill metaworld

Summary

This paper introduces Agent Priors-guided Policy Learning (APPL), a framework where construction and runtime agents use structural priors—describing what a behavior depends on—as an interface between skill learning and skill composition, improving out-of-distribution skill generalization on MetaWorld and long-horizon ManiSkill tasks.

Robots that learn from a few demonstrations often require two forms of generalization. Compositional generalization recombines skills to solve new tasks, and skill generalization lets the learned policy behind each skill work in new situations. The two depend on each other, yet information is lost between composition and the skills it calls. Where a skill works is determined by the structure its policy is trained with, while composition sees the skill only through a separate description, such as a name, an instruction, or a symbolic operator, that omits this structure. Our key idea is to use each policy's structural prior as part of the interface between composition and the skill. A structural prior states what a behavior depends on, for example that a grasp depends only on the gripper's pose relative to the object. Built into training, it shapes where the policy generalizes; stated in language, it tells composition where the policy applies. We instantiate this idea in Agent Priors-guided Policy Learning (APPL). A construction agent segments complete demonstrations into reusable skills, proposes several structural priors for each skill, and trains and verifies one policy per prior. A runtime agent then selects among these prior-specific policies and composes them toward new task goals using their interfaces. Across MetaWorld and long-horizon ManiSkill tasks, APPL improves out-of-distribution skill generalization and enables previously unseen skill compositions; ablating the interface information substantially reduces performance. These results support the use of training-time structural assumptions as a bridge between skill learning and skill composition.
Original Article
View Cached Full Text

Cached at: 10/02/26, 04:26 AM

Paper page - Agent Priors-guided Policy Learning

Source: https://huggingface.co/papers/2609.35690

Abstract

Robotsthatlearnfromafewdemonstrationsoftenrequiretwoformsofgeneralization.Compositionalgeneralizationrecombinesskillstosolvenewtasks,andskillgeneralizationletsthelearnedpolicybehindeachskillworkinnewsituations.Thetwodependoneachother,yetinformationislostbetweencompositionandtheskillsitcalls.Whereaskillworksisdeterminedbythestructureitspolicyistrainedwith,whilecompositionseestheskillonlythroughaseparatedescription,suchasaname,aninstruction,orasymbolicoperator,thatomitsthisstructure.Ourkeyideaistouseeachpolicy’sstructuralprioraspartoftheinterfacebetweencompositionandtheskill.Astructuralpriorstateswhatabehaviordependson,forexamplethatagraspdependsonlyonthegripper’sposerelativetotheobject.Builtintotraining,itshapeswherethepolicygeneralizes;statedinlanguage,ittellscompositionwherethepolicyapplies.WeinstantiatethisideainAgentPriors-guidedPolicyLearning(APPL).Aconstructionagentsegmentscompletedemonstrationsintoreusableskills,proposesseveralstructuralpriorsforeachskill,andtrainsandverifiesonepolicyperprior.Aruntimeagentthenselectsamongtheseprior-specificpoliciesandcomposesthemtowardnewtaskgoalsusingtheirinterfaces.AcrossMetaWorldandlong-horizonManiSkilltasks,APPLimprovesout-of-distributionskillgeneralizationandenablespreviouslyunseenskillcompositions;ablatingtheinterfaceinformationsubstantiallyreducesperformance.Theseresultssupporttheuseoftraining-timestructuralassumptionsasabridgebetweenskilllearningandskillcomposition.

View arXiv pageView PDFProject pageGitHub2Add to collection

Get this paper in your agent:

hf papers read 2609\.35690

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.35690 in a model README.md to link it from this page.

Datasets citing this paper1

#### Oscattt/APPL-assets Updatedabout 1 hour ago • 18 • 1

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.35690 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

APPO: Agentic Procedural Policy Optimization

Hugging Face Daily Papers

APPO improves multi-turn tool-use in LLM agents by refining branching decisions and credit assignment using fine-grained decision points and procedure-level advantage scaling, outperforming baselines by 4 points on 13 benchmarks.

Learning Agentic Policy from Action Guidance

arXiv cs.CL

The paper proposes ActGuide-RL, a method for training agentic policies in LLMs by using human action data as guidance to overcome exploration barriers in reinforcement learning without extensive supervised fine-tuning.

Learning Safe Agent Behaviour from Human Preferences and Justifications via World Models

arXiv cs.AI

This paper introduces DROPJ, a human-centred method for safely training and deploying agent policies by learning a world model from real-world trajectories, then eliciting human preferences with justifications to train a reward model for model predictive control. Experiments show that using human-generated simulated trajectories and justifications improves safety and reduces computational cost.