DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents

Hugging Face Daily Papers Papers
agents self-supervised preference-optimization tool-use long-horizon dpo

Summary

DeLoop is a self-supervised preference optimization method that reduces repetitive tool-call loops in long-horizon language-model agents by using the agent's own rollouts as training pairs, cutting looping tasks from 57.3% to 6.1% under context compression and improving success rates on larger models.

Language-model agents that run long tasks often redo work they have already done: they re-issue a tool call they made earlier and get back exactly the same observation, a step that adds no new information. We identify this failure as a loop. Loops appear natively in Qwen, Llama, and Gemini agents and become far more common once the agent's history is compressed to fit its context window. To reduce these loops, we propose DeLoop, a self-supervised training method. DeLoop builds on a simple observation: the agent's own rollout can provide the supervision needed to learn from its loops. We use a looping action as a negative example and the first later non-looping action in the same rollout as a positive example. DeLoop learns from these pairs through direct preference optimization and does not require task rewards, gold answers, or LLM judges. On CodeGym under compression, DeLoop reduces the share of Qwen3-8B tasks containing loops from 57.3% to 6.1%. It also uses about one third as many steps per task, with similar task success. On Qwen3-32B, DeLoop reduces loops and steps while raising task success from 37.7% to 44.8%.
Original Article
View Cached Full Text

Cached at: 10/22/26, 03:40 PM

Paper page - DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents

Source: https://huggingface.co/papers/2610.19376 Authors:

,

,

,

,

,

,

,

,

,

Abstract

DeLoop is a self-supervised preference optimization method that reduces repetitive tool-use loops in language-model agents by learning from their own rollouts, improving efficiency and success under context compression.

Language-model agentsthat run long tasks often redo work they have already done: they re-issue atool callthey made earlier and get back exactly the sameobservation, a step that adds no new information. We identify this failure as aloop.Loops appear natively in Qwen, Llama, and Gemini agents and become far more common once the agent’s history is compressed to fit its context window. To reduce theseloops, we proposeDeLoop, aself-supervised trainingmethod.DeLoopbuilds on a simpleobservation: the agent’s ownrolloutcan provide the supervision needed to learn from itsloops. We use alooping action as a negative example and the first later non-looping action in the samerolloutas a positive example.DeLooplearns from these pairs throughdirect preference optimizationand does not require task rewards, gold answers, or LLM judges. OnCodeGymunder compression,DeLoopreduces the share of Qwen3-8B tasks containingloops from 57.3% to 6.1%. It also uses about one third as many steps per task, with similar task success. On Qwen3-32B,DeLoopreducesloops and steps while raising task success from 37.7% to 44.8%.

View arXiv pageView PDFProject pageGitHub0Add to collection

Get this paper in your agent:

hf papers read 2610\.19376

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.19376 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.19376 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.19376 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

@jasonzhou1993: https://x.com/jasonzhou1993/status/2067937943545897143

X AI KOLs Timeline

Loop engineering is the practice of designing systems where AI agents autonomously decide what to work on, execute, and iterate, going beyond manual prompting by building outer loops that compound across different domains. The article explains the two-layer agent harness and how sharing artifacts between loops creates compounding learning.

@dunik_7: https://x.com/dunik_7/status/2069079047510864322

X AI KOLs Timeline

LangChain's loop engineering playbook replaces prompt engineering with four nested loops - agent, verification, event-driven, and hill-climbing - enabling AI agents to automatically improve themselves over time. The post argues that building self-optimizing loops is now the key competitive advantage, not using better models.

@shmidtqq: https://x.com/shmidtqq/status/2068704187492221405

X AI KOLs Timeline

An in-depth guide to loop engineering for AI coding agents, explaining how to build automated loops that repeatedly prompt agents, verify results, and avoid runaway costs, illustrated with a case study of one engineer shipping 259 PRs in a month.