Paper page - Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses

Source: https://huggingface.co/papers/2608.08466 Published on Aug 9

·

Submitted byhttps://huggingface.co/TalentZHOU

Tailinon Aug 21

Abstract

Hierarchical Self-Improvement evolves task-specific execution harnesses for frozen LLM agents via hierarchical self-modification, yielding substantial gains on moderate tasks while being bounded by feedback quality and backbone limits.

Modern LLM agents are often improved by modifying prompts, tools, or workflows manually, while the executable scaffold surrounding the model---the harness---is typically treated as a fixed artifact after deployment. This work studies an alternative where the harness is task-specific and continuously evolvable: each task family maintains its own harness, which is hot-swapped across iterations through a fixedtask-injection seamand rewritten using environment feedback. We introduceHierarchical Self-Improvement(HSI), a framework in which a singlefrozen LLMM operates across three hierarchical scopes: a task harness H that executes tasks, an evolver that rewrites H, and ameta-evolverthat rewrites the evolver’s strategy code under a frozen outer anchor. A thinking-on/off design isolates the contribution ofharness evolutionby disabling reasoning during task execution while enabling it during self-modification. HSI is bounded by two factors: afeedback-fidelity bound, since evolution requires informative reward signals to guide selection, and abackbone capability bound, since harness redesign cannot overcome limitations of the frozen model. On BALROG with DeepSeek-V4-Flash-Preview as the frozen backbone, HSI achieves consistent gains over the initial harness on moderate-difficulty tasks (+39.3 on BabyAI, +33.0 on Crafter, +25.0 on TextWorld, and +15.0 on MiniHack, all in raw \% Progress), while obtaining strong held-out generalization on BabaIsAI sub-suites (0.98 best-test on BreakStop and 1.00 on GoTo from a 20% unseen split). On tasks beyond the backbone’s capability (NLE),harness evolutionprovides no improvement. These results demonstrate task-specificharness evolutionas a viable axis for improvingfrozen LLMagents under clear empirical limits. Code is available at https://github.com/TailinZhou/hsi.

View arXiv pageView PDFGitHub7Add to collection

Get this paper in your agent:

hf papers read 2608\.08466

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.08466 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.08466 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.08466 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.