LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks
Summary
This paper introduces Experiential Learning (EL), a method that repurposes an LLM-as-a-Judge into an LLM-as-a-Coach to provide rich textual feedback instead of scalar rewards, improving performance and generalization on open-ended non-verifiable tasks.
View Cached Full Text
Cached at: 07/21/26, 06:35 AM
Paper page - LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks
Source: https://huggingface.co/papers/2607.18110 Published on Jul 20
·
Submitted byhttps://huggingface.co/ytz20
ytzon Jul 21
Abstract
Reinforcementlearning(RL)onopen-endedtaskscompressesanLLM’srubric-basedevaluationintoascalarreward,discardingrichtextualfeedbackandconflatingresponseswithdistinctqualityprofiles.WeproposeExperientialLearning(EL),whichrepurposesthefeedbackmodelfromanLLM-as-a-JudgeintoanLLM-as-a-Coach.Thecoachdistillsitsassessmentofeachon-policyresponseintotransferableexperientialknowledge,whichconditionsateachermodelandisinternalizedbythepolicythroughon-policycontextdistillation.Comparedwithscalarrewards,thishigher-bandwidthfeedbackchannelprovidesdensesupervisionandpreservesfine-grainedpreferencesamonghigh-qualityresponses.Acrosstwopolicyfamilies,withfeedbackfromthepolicyitselforaproprietarymodel,ELconsistentlyoutperformsrubric-basedRLonheld-outandunseenopen-endedtasks.Notably,ELgeneralizesbetterbeyondthetrainingdistribution,andmitigatesrewardhacking.Thesefindingsestablishexperientialknowledgeasaricherandmoregeneralizablelearningsignalforpost-trainingonnon-verifiabletasks.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2607\.18110
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2607.18110 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2607.18110 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2607.18110 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL
LLM-as-a-Tutor introduces a framework that extends LLM's role from judge to tutor by dynamically adjusting prompt difficulty through pairwise comparison and constraint addition, improving instruction-following performance in reinforcement learning.
From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning
This paper introduces LLM-as-Environment-Engineer, a framework where LLMs design their own training environments for reinforcement learning in multi-agent reasoning tasks, enabling self-improving training that surpasses larger proprietary models.
LLM-as-Judge in Education: A Curriculum-Grounded Marking Pipeline
This paper presents a curriculum-grounded LLM-as-Judge pipeline for automated question-level marking in high-stakes exam preparation, using syllabus artefacts and marking guidelines to improve consistency and transparency, with preliminary evaluation showing outcomes comparable to human tutors.
Pushing the Limits of LLM Tool Calling via Experiential Knowledge Integration and Activation
This paper presents a systematic study on integrating experiential knowledge into LLM tool calling, proposing the KATE framework that combines knowledge-augmented data, width-expanded inference, and knowledge-aware training, achieving consistent improvements on BFCL-V3 and AppWorld benchmarks.
Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning
This paper presents MetaEvolve, a framework that uses reinforcement learning to train LLMs in self-evolution meta-skills for iterative refinement, achieving significant improvements on coding benchmarks.