CopT: Contrastive On-Policy Thinking with Continuous Spaces for General and Agentic Reasoning

Hugging Face Daily Papers Papers
contrastive-verification on-policy-thinking continuous-embeddings llm-reasoning agentic-reasoning token-efficiency

Summary

CopT introduces a contrastive on-policy thinking framework for LLMs that generates draft answers first, then uses contrastive verification and dynamic thinking to improve accuracy while reducing token consumption, achieving up to 23% higher accuracy and 57% lower token usage on math, coding, and agentic reasoning tasks.

Chain-of-thought (CoT) is a standard approach for eliciting reasoning capabilities from large language models (LLMs). However, the common CoT paradigm treats thinking as a prerequisite for answering, which can delay access to plausible answers and incur unnecessary token costs even when the model is able to identify an answer before extended thinking, a behavior known as performative reasoning. In this paper, we introduce CopT, a reformulated reasoning pipeline that reverses the usual order of thinking and answering. Instead of thinking before answering, CopT first elicits a draft answer and then invokes subsequent on-policy thinking conditioned on its own draft answer for reflection and correction. To assess whether the draft answer should be trusted, CopT recasts continuous embeddings as inference-time contrastive verifiers. Specifically, it contrasts the model's support for the same generated tokens under discrete-token inputs and continuous-embedding inputs, yielding a sequence-level reverse KL estimator for answer reliability. Our analysis shows that under certain assumptions, the expected estimate equals the mutual information between the unresolved latent state and the emitted answer token, explaining why it captures answer-relevant uncertainty rather than arbitrary uncertainty in the latent state. When the answer is deemed insufficiently reliable, CopT performs further on-policy thinking, where a second KL estimator dynamically controls draft-answer visibility, preserving useful partial information while reducing the risk of being misled by unreliable content. Across mathematics, coding, and agentic reasoning tasks, CopT improves peak accuracy by up to 23% and reduces token usage by up to 57% at comparable or higher accuracy, without any additional training. The code is available at https://github.com/sdc17/CopT.
Original Article
View Cached Full Text

Cached at: 05/20/26, 10:37 AM

Paper page - CopT: Contrastive On-Policy Thinking with Continuous Spaces for General and Agentic Reasoning

Source: https://huggingface.co/papers/2605.20075

Abstract

CopT introduces a novel reasoning framework for LLMs that generates draft answers first, then uses contrastive verification and dynamic thinking to improve accuracy while reducing token consumption.

Chain-of-thought(CoT) is a standard approach for eliciting reasoning capabilities fromlarge language models(LLMs). However, the common CoT paradigm treats thinking as a prerequisite for answering, which can delay access to plausible answers and incur unnecessary token costs even when the model is able to identify an answer before extended thinking, a behavior known asperformative reasoning. In this paper, we introduce CopT, a reformulated reasoning pipeline that reverses the usual order of thinking and answering. Instead of thinking before answering, CopT first elicits adraft answerand then invokes subsequenton-policy thinkingconditioned on its owndraft answerfor reflection and correction. To assess whether thedraft answershould be trusted, CopT recastscontinuous embeddingsasinference-time contrastive verifiers. Specifically, it contrasts the model’s support for the same generated tokens under discrete-token inputs and continuous-embedding inputs, yielding a sequence-levelreverse KL estimatorfor answer reliability. Our analysis shows that under certain assumptions, the expected estimate equals themutual informationbetween the unresolved latent state and the emitted answer token, explaining why it captures answer-relevant uncertainty rather than arbitrary uncertainty in the latent state. When the answer is deemed insufficiently reliable, CopT performs furtheron-policy thinking, where a secondKL estimatordynamically controls draft-answer visibility, preserving useful partial information while reducing the risk of being misled by unreliable content. Across mathematics, coding, and agentic reasoning tasks, CopT improves peak accuracy by up to 23% and reducestoken usageby up to 57% at comparable or higher accuracy, without any additional training. The code is available at https://github.com/sdc17/CopT.

View arXiv pageView PDFProject pageGitHub2Add to collection

Get this paper in your agent:

hf papers read 2605\.20075

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.20075 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.20075 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.20075 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.

Similar Articles

Crosslingual On-Policy Self-Distillation for Multilingual Reasoning

Hugging Face Daily Papers

The paper proposes Crosslingual On-Policy Self-Distillation (COPSD), a method to transfer high-resource language reasoning capabilities to low-resource languages using a shared student-teacher architecture. Experiments across 17 African languages show significant improvements in mathematical reasoning and answer-format adherence, outperforming Group Relative Policy Optimization (GRPO).

Continuous Audio Thinking for Large Audio Language Models

arXiv cs.AI

The paper introduces Continuous Audio Thinking (CoAT), a framework that equips large audio language models with a continuous latent workspace to organize acoustic information before generating textual responses, improving performance on audio reasoning, understanding, and transcription tasks without additional decoding cost.

Long-Context Reasoning Through Proxy-Based Chain-of-Thought Tuning

arXiv cs.CL

Proposes ProxyCoT, a training framework that improves long-context reasoning in large language models by first obtaining chain-of-thought reasoning traces on short proxy contexts (via reinforcement learning or distillation) and then grounding them in full long contexts through supervised fine-tuning. Experiments show consistent improvements over baselines with reduced computational cost.

Guidance Contrastive Token Credit Assignment for Discrete Policy Optimization

Hugging Face Daily Papers

This paper introduces Guidance Contrastive Policy Optimization (GCPO), a novel algorithm that enables per-token credit assignment in reinforcement learning by contrasting model predictions under positive and negative prompts, consistently outperforming GRPO and DAPO baselines on text-to-image generation and chain-of-thought reasoning benchmarks.

Confidence-Aware Alignment Makes Reasoning LLMs More Reliable

arXiv cs.AI

This paper introduces CASPO, a framework for aligning token-level confidence with step-wise logical correctness in large reasoning models using iterative Direct Preference Optimization. It also proposes Confidence-aware Thought (CaT) for dynamically pruning uncertain reasoning branches during inference to improve reliability and efficiency.