Chain-of-Experience for Continual LLM Improvement

Hugging Face Daily Papers Papers
chain-of-experience llm continual-learning feedback-loops test-time-improvement token-efficiency academic-research

Summary

The paper introduces Chain-of-Experience, a method for continual improvement of large language models through iterative test-time feedback, demonstrating better performance and cost efficiency across various domains.

Humans continuously learn from experience, whereas conventional large language model (LLM) evaluations ignore the models' ability to improve through inference-time interaction. In this paper, we study how LLMs learn from iterative experience at test time, a setting we refer to as Chain-of-Experience (CoE), where models accumulate experiential traces through iterative interactions with self or environmental feedback to form a continual improvement loop beyond zero-shot inference. We instantiate CoE with diverse feedback mechanisms, including model self-feedback and environmental signals such as correctness or public coding test pass rates, and evaluate across math, coding, and knowledge domains using 8 LLMs, including GPT-5, Gemini-2.5 Pro, Claude-4.5 Sonnet. Our study shows that leveraging iterative experience consistently outperforms feedback-free baselines, achieving substantial gains with self feedback alone, alongside a 5.6% overall improvement and 19% lower API cost across tasks and models. We further show that combining complementary feedback channels (e.g., model and correctness signals) yields additional gains, and that CoE delivers higher accuracy per token than existing test-time strategies. We observe a positive correlation between LLM base ability and improvement capacity, and show that models remain robust under weak or spurious feedback, with different feedback contributing to distinct improvement aspects and most gains emerging early in the iterations.
Original Article
View Cached Full Text

Cached at: 08/21/26, 08:09 AM

Paper page - Chain-of-Experience for Continual LLM Improvement

Source: https://huggingface.co/papers/2608.18027

Abstract

Large language models improve continuously through iterative test-time feedback loops called Chain-of-Experience, outperforming zero-shot baselines with lower cost and higher token efficiency.

Humans continuously learn from experience, whereas conventional large language model (LLM) evaluations ignore the models’ ability to improve throughinference-time interaction. In this paper, we study how LLMs learn from iterative experience at test time, a setting we refer to asChain-of-Experience(CoE), where models accumulate experiential traces through iterative interactions with self orenvironmental feedbackto form acontinual improvement loopbeyond zero-shot inference. We instantiate CoE with diverse feedback mechanisms, including modelself-feedbackand environmental signals such as correctness or public coding test pass rates, and evaluate across math, coding, and knowledge domains using 8 LLMs, including GPT-5, Gemini-2.5 Pro, Claude-4.5 Sonnet. Our study shows that leveraging iterative experience consistently outperforms feedback-free baselines, achieving substantial gains with self feedback alone, alongside a 5.6% overall improvement and 19% lower API cost across tasks and models. We further show that combining complementary feedback channels (e.g., model and correctness signals) yields additional gains, and that CoE delivers higher accuracy per token than existingtest-time strategies. We observe a positive correlation between LLM base ability and improvement capacity, and show that models remain robust under weak or spurious feedback, with different feedback contributing to distinct improvement aspects and most gains emerging early in the iterations.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2608\.18027

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.18027 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.18027 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.18027 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Chain-of-Experience for Continual LLM Improvement

arXiv cs.CL

This paper introduces Chain-of-Experience (CoE), a framework for continual LLM improvement through iterative test-time interactions with self and environmental feedback, showing consistent gains and lower API costs across multiple domains and models.

Rethinking Continual Experience Internalization for Self-Evolving LLM Agents

arXiv cs.CL

This paper investigates why LLM agents suffer from progressive capability collapse under multi-iteration experience internalization and proposes a robust recipe addressing experience granularity, injection patterns, and training regime. Key findings include that principle-level experience, step-wise injection, and off-policy context-distillation yield more stable and sustainable continual learning.

ContinualSkillBench: Can LLM Agents Truly Evolve Their Capabilities?

Hugging Face Daily Papers

Introduces ContinualSkillBench, a dynamic evaluation framework for in-context continual skill learning in LLM agents, showing that while sequential execution improves performance, current methods struggle to consolidate experience into robust, transferable skills.

When Does Continual Learning Require Learning

arXiv cs.LG

This paper proposes a unified framework for continual learning in LLMs, disentangling change along space (new domains) and time (data drift). It evaluates various methods including prompting, supervised learning, reinforcement learning, and context compression under realistic sequential settings.