Opera: A Verbal Critic Framework for Long-horizon Coding Agents

Hugging Face Daily Papers Papers
coding-agents critic-framework long-horizon-agents reinforcement-learning swe-bench test-time-scaling

Summary

Opera is a verbal critic framework for long-horizon coding agents that treats each correction as a persistent note, tracking agent actions to distinguish mere compliance from actual resolution. It improves resolver rates by up to 15 percentage points across benchmarks and provides on-policy training data that boosts Qwen3.5-9B's SWE-Bench Pro performance without needing a critic at inference time.

Long-horizon coding agents need timely corrections, yet feedback can be ineffective or even harmful when it misjudges ongoing work or fails to address the underlying problem. Existing critics focus on evaluating trajectories and generating feedback, but rarely track what happens after feedback is delivered. We present Opera, a verbal critic framework that treats each correction as a persistent note, followed until the diagnosed problem is resolved. Opera decides when to review through periodic and event-driven triggers, diagnoses issues with typed operators, audits feedback against visible evidence before delivery, and tracks the agent's subsequent actions to distinguish mere compliance from actual resolution. As a test-time critic, Opera improves the resolve rate of non-critic agents by up to 12.4, 15.0, and 8.9 percentage points on Terminal-Bench 2.1, a SWE-Bench Pro subset, and DeepSWE v1.1, respectively, across four policy models, and achieves the highest mean resolve rate among competitive critic baselines on all three benchmarks, and also improves policy models when the policy critiques itself. Beyond inference, Opera-guided rollouts provide approximately on-policy training data: fine-tuning Qwen3.5-9B on them improves its resolve rate on held-out SWE-Bench Pro repositories by 10.2 percentage points without a critic at inference time, matching fine-tuning on rollouts from a stronger model, while preserving its performance when switching harness, i.e., from Openhands to Terminus-2, which the latter substantially degrades. Our code is available at: https://github.com/dongyuanjushi/Opera.
Original Article
View Cached Full Text

Cached at: 10/09/26, 10:23 PM

Paper page - Opera: A Verbal Critic Framework for Long-horizon Coding Agents

Source: https://huggingface.co/papers/2609.33987 Published on Oct 8

·

Submitted byhttps://huggingface.co/dongyuanjushi

KaiMeion Oct 9

Abstract

Long-horizoncodingagentsneedtimelycorrections,yetfeedbackcanbeineffectiveorevenharmfulwhenitmisjudgesongoingworkorfailstoaddresstheunderlyingproblem.Existingcriticsfocusonevaluatingtrajectoriesandgeneratingfeedback,butrarelytrackwhathappensafterfeedbackisdelivered.WepresentOpera,averbalcriticframeworkthattreatseachcorrectionasapersistentnote,followeduntilthediagnosedproblemisresolved.Operadecideswhentoreviewthroughperiodicandevent-driventriggers,diagnosesissueswithtypedoperators,auditsfeedbackagainstvisibleevidencebeforedelivery,andtrackstheagent’ssubsequentactionstodistinguishmerecompliancefromactualresolution.Asatest-timecritic,Operaimprovestheresolverateofnon-criticagentsbyupto12.4,15.0,and8.9percentagepointsonTerminal-Bench2.1,aSWE-BenchProsubset,andDeepSWEv1.1,respectively,acrossfourpolicymodels,andachievesthehighestmeanresolverateamongcompetitivecriticbaselinesonallthreebenchmarks,andalsoimprovespolicymodelswhenthepolicycritiquesitself.Beyondinference,Opera-guidedrolloutsprovideapproximatelyon-policytrainingdata:fine-tuningQwen3.5-9Bonthemimprovesitsresolverateonheld-outSWE-BenchProrepositoriesby10.2percentagepointswithoutacriticatinferencetime,matchingfine-tuningonrolloutsfromastrongermodel,whilepreservingitsperformancewhenswitchingharness,i.e.,fromOpenhandstoTerminus-2,whichthelattersubstantiallydegrades.Ourcodeisavailableat:https://github.com/dongyuanjushi/Opera.

View arXiv pageView PDFGitHub1Add to collection

Get this paper in your agent:

hf papers read 2609\.33987

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2609.33987 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2609.33987 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2609.33987 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles