Logic-Regularized Verifier Elicits Reasoning from LLMs

# Logic-Regularized Verifier Elicits Reasoning from LLMs
Source: [https://arxiv.org/html/2605.05893](https://arxiv.org/html/2605.05893)
###### Abstract

Verifiers are crucial components for enhancing modern LLMs’ reasoning capability\. Typical verifiers require resource\-intensive supervised dataset construction, which is costly and faces limitations in data diversity\. In this paper, we proposeLoVer, an unsupervised verifier regularized by logical rules\.LoVertreats the verifier as a binary latent variable, utilizing internal activations and enforcing three logical constraints on multiple reasoning paths: negation consistency, intra\-group consistency, and inter\-group consistency \(grouped by the final answer\)\. By incorporating logical rules as priors,LoVercan leverage unlabeled examples and is directly compatible with any off\-the\-shelf LLMs\. Experiments on1010datasets demonstrate thatLoVersignificantly outperforms unsupervised baselines, achieving performance comparable to the supervised verifier \(reaching its 95% level on average\)\. The source code is publicly available at[https://github\.com/wangxinyufighting/llm\-lover](https://github.com/wangxinyufighting/llm-lover)\.

11affiliationtext:Department of Computer Science and Technology, East China Normal University22affiliationtext:Institute of Artificial Intelligence \(TeleAI\), China Telecom††affiliationtext:\{xinyu\_wang@stu, ybwu@cs, xlwang@cs\}\.ecnu\.edu\.cn††affiliationtext:\{czsun\}@chinatelecom\.cn††affiliationtext:\{dell\.z, xuelong\_li\}@ieee\.orgLogic\-Regularized Verifier Elicits Reasoning from LLMs

\*\*footnotetext:Equal contribution\.\#\#footnotetext:Work done while this author was an intern at TeleAI\.$\{\\dagger\}$$\{\\dagger\}$footnotetext:Corresponding authors\.## 1Introduction

Verifiers guide LLMs by providing feedback to optimize their parameters \(RL scaling\) or outputs \(inference scaling\), which greatly enhances models’ reasoning capabilitiesOuyanget al\.\([2022](https://arxiv.org/html/2605.05893#bib.bib22)\); Snellet al\.\([2024](https://arxiv.org/html/2605.05893#bib.bib5)\)\. Verifiers are usually trained with supervised learningCobbeet al\.\([2021](https://arxiv.org/html/2605.05893#bib.bib12)\); Yuet al\.\([2024](https://arxiv.org/html/2605.05893#bib.bib86)\), where they learn to classify reasoning outputs as true or false based on labeled data\. It presents two challenges: 1\) The verifier relies heavily on labeled data for training, which can be expensive to collect \(particularly in specialized or complex domains\)\. For example, annotating a single Olympiad\-level problem typically takes a significant amount of time, and adding fine\-grained step\-level process annotationsLightmanet al\.\([2023](https://arxiv.org/html/2605.05893#bib.bib8)\)further increases the workload; 2\) Relying on expert annotations may result in a lack of diversity in the solutionsBasileet al\.\([2021](https://arxiv.org/html/2605.05893#bib.bib88)\); Xuet al\.\([2024a](https://arxiv.org/html/2605.05893#bib.bib89)\), as annotators may favor familiar reasoning methods while overlooking equally valid but less intuitive ones\. For example, when annotating geometric problems, annotators may prefer the standard coordinate method and down\-vote the less obvious geometric observation\. While one can improve the supervision process in various aspectsYanget al\.\([2019](https://arxiv.org/html/2605.05893#bib.bib92)\)\(e\.g\., more experts with diverse mathematical backgrounds and education experiences\), LLMs themselves already compact large amounts of knowledge and abilities to sample diverse generationsMinaeeet al\.\([2024](https://arxiv.org/html/2605.05893#bib.bib90)\); Xuet al\.\([2024b](https://arxiv.org/html/2605.05893#bib.bib91)\), a natural question is*whether we could build verifiers without the supervision process?*

Table 1:Comparison between existing verifiers\.Paradigm: The verifier is trained using a supervised \(Sup\.\) or unsupervised \(Unsup\.\) learning paradigm\.Prior: The prior knowledge used in the verifier\.Annotation: The type of annotation data\.‘‘Outcome\-based’’ is solution\-level annotation\. ‘‘Process\-based’’ is step\-level annotation\.Input: The input data type of the verifier\.Model: The model architecture\.Scenario: Reasoning scenarios suitable for the verifier\. ‘‘General’’ typically refers to reasoning problems that have a correct answer\. ‘‘Yes\-No’’ indicates that the answer to the question is either Yes or No\.To address these challenges, recent research has focused on unsupervised verifiers to uncover the intrinsic reasoning capabilities of LLMs\. Typical works include: 1\) CoT\-DecodingWang and Zhou\([2024](https://arxiv.org/html/2605.05893#bib.bib1)\), which proposes a heuristic rule\-based verifier by observing the probabilities of the outputs of LLMs\. It selects the correct reasoning path based on the probability difference between the top and secondary tokens in the answer span\. In experiments, we observed that CoT\-Decoding is sensitive to the backbone LLM choice\. For example, when using llama\-7b on the GSM8K dataset, CoT\-Decoding is 4\.8% lower than the majority voting strategy\. 2\) CCSBurnset al\.\([2023](https://arxiv.org/html/2605.05893#bib.bib2)\)introduces an unsupervised verifier, which is essentially a linear probe optimized through logical consistency loss\. Unfortunately, CCS can only address Yes\-No questions and struggles to scale to general reasoning tasks\. A practical verifier should have fewer limitations on its target problems, enabling it to handle a broader range of reasoning scenarios and provide more flexibility in real\-world applications

In this paper, we propose a principled frameworkLoVer, an unsupervised probabilistic verifier regularized by logical rules\. For each reasoning path, we search for the implicit, internal ‘‘beliefs’’ or ‘‘knowledge’’ learned by the LLM to infer the truth value of the reasoning\.LoVerbegins by generating contrastive assertions through incorporating text templates\. It then takes the internal activations of these assertions from the LLM as inputs and produces a binary latent variable to indicate the truth value\. Furthermore,LoVerincorporates three logical constraints including negation consistency, intra\-group consistency, and inter\-group consistency \(with multiple reasoning paths grouped by the final answer\)\. To bridge the gap between discrete logical rules and continuous neural networks, we propose corresponding soft probabilistic objectives that support differentiable training\. Our contributions are summarized as follows:

- •We proposeLoVer, a scalable and principled framework for verifying the truth value of reasoning paths, leveraging intrisic knowledge learned by the LLM and regularized by logical rules\. Additionally,LoVeris fully compatible with any off\-the\-shelf LLMs\.
- •To combine discrete logical rules with neural networks, we propose soft probabilistic objectives that enableLoVerto be trained end\-to\-end, improving its scalability and performance\.
- •Our extensive experiments across diverse datasets, including mathematical reasoning, common sense reasoning, and various backbones, demonstrate the effectiveness of the proposed method\.

## 2Approach

In this section, we present the proposedLoVer, an unsupervised verifier designed to reason over the internal activations of LLMs\.

![Refer to caption](https://arxiv.org/html/2605.05893v1/figs/overview.png)Figure 1:An illustration of our proposedLoVer\. For any questionqq, we createxix\_\{i\}by combiningqqwith theii\-th solution fromNNsolutions\. We formxi\+x^\{\+\}\_\{i\}andxi−x^\{\-\}\_\{i\}by adding "This is a true/false answer\." toxix\_\{i\}, respectively\. Choosing the correct solution involves determining which assertion,xi\+x^\{\+\}\_\{i\}orxi−x^\{\-\}\_\{i\}, is correct\. The hidden states of LLMs are used to representxi\+x^\{\+\}\_\{i\}andxi−x^\{\-\}\_\{i\}, which are then input intoLoVerto predict the correctness probability of each assertion\. We extract the final answer from each solution and group assertions with identical answers together\. These assertions follow three natural logical constraints that guideLoVer’s unsupervised training\.Negation Consistencyensures that only one ofxi\+x^\{\+\}\_\{i\}orxi−x^\{\-\}\_\{i\}is correct\.Intra\-group Consistencyrequires that assertions in the same group have equal correctness probabilities\.Inter\-group Consistencyensures that only one group’sx\+x^\{\+\}assertion is correct across all groups\.##### Task Definition

Given an LLM and an input questionqq, we first generateNNcomplete solutions\{si\}i=1N\\\{s\_\{i\}\\\}\_\{i=1\}^\{N\}, with eachsis\_\{i\}representing a CoT path \(Sec\.[2\.1](https://arxiv.org/html/2605.05893#S2.SS1)\)\. We then select the best solution based on a learned verifier\. For each solutionsis\_\{i\}, we definexi=q⊕si,xi\+=xi⊕𝚃\+,xi−=xi⊕𝚃−x\_\{i\}=q\\oplus s\_\{i\},x\_\{i\}^\{\+\}=x\_\{i\}\\oplus\\mathtt\{T\}^\{\+\},x\_\{i\}^\{\-\}=x\_\{i\}\\oplus\\mathtt\{T\}^\{\-\}, where⊕\\oplusdenotes the text concatenation, and𝚃\+,𝚃−\\mathtt\{T\}^\{\+\},\\mathtt\{T\}^\{\-\}are text templates\. GivenNNreasoning paths, we group them intoMMsets \(M≤NM\\leq N\) based on the*final answer*\(extracted through rules from the answer token\)\.𝒜\\mathcal\{A\}represents the index set from11toNN, and𝒜k\\mathcal\{A\}\_\{k\}denotes thekk\-th group, with𝒜=∪k=1M𝒜k\\mathcal\{A\}=\\cup\_\{k=1\}^\{M\}\\mathcal\{A\}\_\{k\}\. The verifier models a probabilistic distributionpθ​\(𝒛\|x\)p\_\{\\theta\}\(\\bm\{z\}\|x\), wherex∈∪i=1N\{xi\+,xi−\}x\\in\\cup\_\{i=1\}^\{N\}\\\{x\_\{i\}^\{\+\},x\_\{i\}^\{\-\}\\\}and𝒛∈\{0,1\}\\bm\{z\}\\in\\\{0,1\\\}is a binary latent variable indicating whether the natural language statementxxis valid\. In this paper, bold letters indicate variables\.

Inspired by CoT\-DecodingWang and Zhou\([2024](https://arxiv.org/html/2605.05893#bib.bib1)\)and CCSBurnset al\.\([2023](https://arxiv.org/html/2605.05893#bib.bib2)\), to find the correct answer, we first augment each reasoning path to derive its correct and incorrect assertions, and then treat the truth values of the assertions as binary latent variables\. On the one hand, we leverage the internal activations of the LLM as input, enabling better utilization of the model’s intrinsic knowledge\. On the other hand, the logical constraints provide implicit supervision signals to update the verifier, significantly reducing the need for human supervision\.

Next, we first introduce LLM decoding strategy \(Sec\.[2\.1](https://arxiv.org/html/2605.05893#S2.SS1)\) and how to obtain contrastive assertions \(Sec\.[2\.2](https://arxiv.org/html/2605.05893#S2.SS2)\)\. Then we detail the latent verifier model \(Sec\.[2\.3](https://arxiv.org/html/2605.05893#S2.SS3)\) and describe the logical constraints imposed on the latent variables \(Sec\.[2\.4](https://arxiv.org/html/2605.05893#S2.SS4)\)\. Finally, we present the training and inference procedure \(Sec\.[2\.5](https://arxiv.org/html/2605.05893#S2.SS5)\)\. Fig\.[1](https://arxiv.org/html/2605.05893#S2.F1)shows an overview of our method\.

### 2\.1LLM Decoding Strategy

Given an input questionqqand a typical decode\-only LLM, there are various strategies to decodeNNsolutions, such as beam search, nucleus sampling, and others\. In this work, we follow the CoT\-DecodingWang and Zhou\([2024](https://arxiv.org/html/2605.05893#bib.bib1)\)\. Specifically, we keep the topNNtokens with the highest probabilities at decoding step0, and then continue with greedy decoding for each token, ultimately producingNNsolutions\. Compared to other strategies, this method is more likely to produce a natural CoT reasoning path and does not rely on complex prompt engineeringWang and Zhou\([2024](https://arxiv.org/html/2605.05893#bib.bib1)\)\. In the experiments, we also study the impact of different decoding strategies\(Table[4](https://arxiv.org/html/2605.05893#S3.T4)\)\.

### 2\.2Contrastive Assertions

For eachxi=q⊕six\_\{i\}=q\\oplus s\_\{i\}, we construct each contrastive assertions by appending the text templates𝚃\+\\mathtt\{T\}^\{\+\}and𝕋−\\mathbb\{T\}^\{\-\}\. Formally, this is denoted asxi\+=xi⊕𝚃\+x\_\{i\}^\{\+\}=x\_\{i\}\\oplus\\mathtt\{T\}^\{\+\}andxi−=xi⊕𝚃−x\_\{i\}^\{\-\}=x\_\{i\}\\oplus\\mathtt\{T\}^\{\-\}\. In this paper, we adopt𝚃\+=\\mathtt\{T\}^\{\+\}=‘‘This is a true answer\.’’ and𝚃−=\\mathtt\{T\}^\{\-\}=‘‘This is a false answer\.’’\. Importantly, rather than directly considering each reasoning pathxix\_\{i\}, we introduce contrastive assertionsxi\+x\_\{i\}^\{\+\}andxi−x\_\{i\}^\{\-\}, which help elicit the internal ‘‘beliefs’’ or ‘‘knowledge’’ learned by the modelBurnset al\.\([2023](https://arxiv.org/html/2605.05893#bib.bib2)\)\.

### 2\.3Latent Verifier Model

For each natural language assertionx∈∪i=1N\{xi\+,xi−\}x\\in\\cup\_\{i=1\}^\{N\}\\\{x\_\{i\}^\{\+\},x\_\{i\}^\{\-\}\\\}, we first compute the feature vector ofxx, denoted asϕ​\(x\)\\phi\(x\),111The default is the hidden representation of the last token in the middle layer, and we also explore other options\. For details, please refer to Table[5](https://arxiv.org/html/2605.05893#S3.T5)\.then pass it through a randomly initialized MLP, and finally map it to a probability value using the sigmoid function\. Formally, we define the probabilistic distribution of verifierpθ​\(𝒛\|x\)p\_\{\\theta\}\(\\bm\{z\}\|x\)where𝒛∈\{0,1\}\\bm\{z\}\\in\\\{0,1\\\}is a binary latent variable indicating whether the natural language statementxxis valid\. For simplicity, we usepθ​\(𝒛\)p\_\{\\theta\}\(\\bm\{z\}\)to representpθ​\(𝒛=1\|x\)p\_\{\\theta\}\(\\bm\{z\}=1\|x\):

pθ\(𝒛\)=pθ\(𝒛=1\|x\)=𝚂𝚒𝚐𝚖𝚘𝚒𝚍\(𝙼𝙻𝙿\(ϕ\(x\)\)\.\\displaystyle p\_\{\\theta\}\(\\bm\{z\}\)=p\_\{\\theta\}\(\\bm\{z\}=1\|x\)=\\mathtt\{Sigmoid\}\(\\mathtt\{MLP\}\(\\phi\(x\)\)\.Importantly,LoVerdoes not modify the weights of the LLM and it does not use labels\.

### 2\.4Logical Constraints

After introducing the binary latent variables∪i=1N\{𝒛\+,𝒛\+\}\\cup\_\{i=1\}^\{N\}\\\{\\bm\{z\}^\{\+\},\\bm\{z\}^\{\+\}\\\}, we observe that certain natural logical consistencies between them should be satisfied\. Let us look at three such logical consistency requirements\.

##### Negation Consistency

Given the contrastive assertionsxi\+x\_\{i\}^\{\+\}andxi−x\_\{i\}^\{\-\}, their corresponding binary latent variables𝒛i\+\\bm\{z\}\_\{i\}^\{\+\}and𝒛i−\\bm\{z\}\_\{i\}^\{\-\}should satisfy negation consistency:

𝒛i\+=1−𝒛i−,i∈𝒜\.\\displaystyle\\bm\{z\}\_\{i\}^\{\+\}=1\-\\bm\{z\}\_\{i\}^\{\-\},i\\in\\mathcal\{A\}\.To this end, we relax the logic with soft probabilityChenet al\.\([2022a](https://arxiv.org/html/2605.05893#bib.bib3)\); Burnset al\.\([2023](https://arxiv.org/html/2605.05893#bib.bib2)\)for differentiability in training and regularization of binary latent variables\. Inspired by CCSBurnset al\.\([2023](https://arxiv.org/html/2605.05893#bib.bib2)\), we aim for the contrastive assertionsxi\+x\_\{i\}^\{\+\}andxi−x\_\{i\}^\{\-\}to satisfy the following: 1\)\.the sum of their probabilities equals11\(probability normalization\); 2\)\.their probabilities differ significantly \(the law of excluded middle\)\.

ℒsum\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{sum\}\}~=∑i=1N\[pθ​\(𝒛i\+\)\+pθ​\(𝒛i−\)−1\]2,\\displaystyle=\\sum\_\{i=1\}^\{N\}\\left\[p\_\{\\theta\}\(\\bm\{z\}\_\{i\}^\{\+\}\)\+p\_\{\\theta\}\(\\bm\{z\}\_\{i\}^\{\-\}\)\-1\\right\]^\{2\},ℒdiff\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{diff\}\}~=∑i=1Nmin\{pθ\(𝒛i\+\),pθ\(𝒛i−\)\}2,\\displaystyle=\\sum\_\{i=1\}^\{N\}\\min\\left\\\{p\_\{\\theta\}\(\\bm\{z\}\_\{i\}^\{\+\}\),p\_\{\\theta\}\(\\bm\{z\}\_\{i\}^\{\-\}\)\\right\\\}^\{2\},ℒnega\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{nega\}\}~=ℒsum\+ℒdiff\.\\displaystyle=\\mathcal\{L\}\_\{\\mathrm\{sum\}\}\+\\mathcal\{L\}\_\{\\mathrm\{diff\}\}\.Note that both losses are necessary; using either one alone leads to a degenerate solutionBurnset al\.\([2023](https://arxiv.org/html/2605.05893#bib.bib2)\)\.

##### Intra\-group Consistency

For each group𝒜k\\mathcal\{A\}\_\{k\}of reasoning paths, they share the same answer, though their reasoning processes may differ\. Overall, we expect the corresponding binary latent variables to satisfy intra\-group consistency, i\.e\.,

𝒛i\+=𝒛j\+,𝒛i−=𝒛j−,i∈𝒜k,j∈𝒜k\.\\displaystyle\\bm\{z\}\_\{i\}^\{\+\}=\\bm\{z\}\_\{j\}^\{\+\},\\bm\{z\}\_\{i\}^\{\-\}=\\bm\{z\}\_\{j\}^\{\-\},~~~i\\in\\mathcal\{A\}\_\{k\},j\\in\\mathcal\{A\}\_\{k\}\.To achieve this goal, we use a simple squared loss:

ℒintra\+\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{intra\}\}^\{\+\}=∑k=1M∑i∈𝒜k,j∈𝒜k\[pθ​\(𝒛i\+\)−pθ​\(𝒛j\+\)\]2,\\displaystyle=\\sum\_\{k=1\}^\{M\}\\sum\_\{i\\in\\mathcal\{A\}\_\{k\},j\\in\\mathcal\{A\}\_\{k\}\}\\left\[p\_\{\\theta\}\(\\bm\{z\}\_\{i\}^\{\+\}\)\-p\_\{\\theta\}\(\\bm\{z\}\_\{j\}^\{\+\}\)\\right\]^\{2\},ℒintra−\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{intra\}\}^\{\-\}=∑k=1M∑i∈𝒜k,j∈𝒜k\[pθ​\(𝒛i−\)−pθ​\(𝒛j−\)\]2,\\displaystyle=\\sum\_\{k=1\}^\{M\}\\sum\_\{i\\in\\mathcal\{A\}\_\{k\},j\\in\\mathcal\{A\}\_\{k\}\}\\left\[p\_\{\\theta\}\(\\bm\{z\}\_\{i\}^\{\-\}\)\-p\_\{\\theta\}\(\\bm\{z\}\_\{j\}^\{\-\}\)\\right\]^\{2\},ℒintra\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{intra\}\}=ℒintra\+\+ℒintra−\.\\displaystyle=\\mathcal\{L\}\_\{\\mathrm\{intra\}\}^\{\+\}\+\\mathcal\{L\}\_\{\\mathrm\{intra\}\}^\{\-\}\.

##### Inter\-group Consistency

Among theNNreasoning paths, there areMMdistinct answers\. We assume that the LLM’s capabilities are sufficiently strong to ensure the presence of a correct answer\. We examine the GSM8k dataset and find that whenN=10N=10, theP​@​10P@10accuracy of qwen\-2\.5 can reach91\.43%91\.43\\%\. This confirms the validity of the above assumption\. Specifically, for each group𝒜k\\mathcal\{A\}\_\{k\}, we randomly select anaka\_\{k\}and hope that its corresponding binary latent variable satisfies:

∑k=1M𝒛ak\+=1,ak∈𝒜k\.\\displaystyle\\sum\_\{k=1\}^\{M\}\\bm\{z\}\_\{a\_\{k\}\}^\{\+\}=1,~~~a\_\{k\}\\in\\mathcal\{A\}\_\{k\}\.\(1\)To achieve this inter\-group consistency, we propose a soft probability solution\.

ℒintersum\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{inter\}\}^\{\\mathrm\{sum\}\}=\[∑k=1Mpθ​\(𝒛ak\+\)−1\]2\.\\displaystyle=\\left\[\\sum\_\{k=1\}^\{M\}p\_\{\\theta\}\(\\bm\{z\}\_\{a\_\{k\}\}^\{\+\}\)\-1\\right\]^\{2\}\.However, in the experiments, we observe that if only the lossℒintersum\\mathcal\{L\}\_\{\\mathrm\{inter\}\}^\{\\mathrm\{sum\}\}is used, theMMprobabilitiespθ​\(𝒛ak\+\)p\_\{\\theta\}\(\\bm\{z\}\_\{a\_\{k\}\}^\{\+\}\)tend to become uniform\. To address this issue, we propose an entropy regularization\. We introduce a probability distributionp^\\hat\{p\}which defines over theMMvariables\{𝒛ak\+\}k=1M\\\{\\bm\{z\}\_\{a\_\{k\}\}^\{\+\}\\\}\_\{k=1\}^\{M\}\.

ℒinterh\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{inter\}\}^\{\\mathrm\{h\}\}=ℋ​\[p^​\(⋅\)\],p^​\(𝒛ak\+\)=pθ​\(𝒛ak\+\)∑i=1Mpθ​\(𝒛ai\+\),\\displaystyle=\\mathcal\{H\}\\left\[\\hat\{p\}\(\\cdot\)\\right\],~~~\\hat\{p\}\(\\bm\{z\}\_\{a\_\{k\}\}^\{\+\}\)=\\frac\{p\_\{\\theta\}\(\\bm\{z\}\_\{a\_\{k\}\}^\{\+\}\)\}\{\\sum\_\{i=1\}^\{M\}p\_\{\\theta\}\(\\bm\{z\}\_\{a\_\{i\}\}^\{\+\}\)\},ℒinter\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{inter\}\}=ℒintersum\+ℒinterh,\\displaystyle=\\mathcal\{L\}\_\{\\mathrm\{inter\}\}^\{\\mathrm\{sum\}\}\+\\mathcal\{L\}\_\{\\mathrm\{inter\}\}^\{\\mathrm\{h\}\},whereℋ\\mathcal\{H\}denote the entropy function\. In addition, we also explore a soft logic\-based solution, which is encapsulated in Appendix[A](https://arxiv.org/html/2605.05893#A1)\.

### 2\.5Training and Inference

##### Training

The final loss function is the sum of three losses mentioned above, which is defined as:

ℒ=ℒnega\+ℒintra\+ℒinter\.\\displaystyle\\mathcal\{L\}=\\mathcal\{L\}\_\{\\mathrm\{nega\}\}\+\\mathcal\{L\}\_\{\\mathrm\{intra\}\}\+\\mathcal\{L\}\_\{\\mathrm\{inter\}\}\.LoVeris structured as a MLP with 2 hidden layers, and we select ReLU as the activation function\. We use AdamWLoshchilov and Hutter\([2019](https://arxiv.org/html/2605.05893#bib.bib71)\)as optimizer \(weight\_decay =0\.010\.01\), and set learning rate to1×e−51\\times e^\{\-5\}\.

##### Inference

Given an input questionqq, we first decodeNNcandidate solutions \(Sec\.[2\.1](https://arxiv.org/html/2605.05893#S2.SS1)\) to obtain\{xi\}i=1N\\\{x\_\{i\}\\\}\_\{i=1\}^\{N\}\. For eachxix\_\{i\}, we generate contrastive assertionsxi\+x\_\{i\}^\{\+\}andxi−x\_\{i\}^\{\-\}ofsis\_\{i\}\(Sec\.[2\.2](https://arxiv.org/html/2605.05893#S2.SS2)\) and compute corresponding probabilitypθ​\(𝒛i\+\)p\_\{\\theta\}\(\\bm\{z\}\_\{i\}^\{\+\}\)andpθ​\(𝒛i−\)p\_\{\\theta\}\(\\bm\{z\}\_\{i\}^\{\-\}\)based on the latent verifier model \(Sec\.[2\.3](https://arxiv.org/html/2605.05893#S2.SS3)\)\. Bothpθ​\(𝒛i\+\)p\_\{\\theta\}\(\\bm\{z\}\_\{i\}^\{\+\}\)and1−pθ​\(𝒛i−\)1\-p\_\{\\theta\}\(\\bm\{z\}\_\{i\}^\{\-\}\)should represent the probability that thexix\_\{i\}is correct\. we consequently take the average of theseBurnset al\.\([2023](https://arxiv.org/html/2605.05893#bib.bib2)\):

pθ​\(𝒛i\)=12​\[pθ​\(𝒛i\+\)\+\(1−pθ​\(𝒛i−\)\)\]\.\\displaystyle p\_\{\\theta\}\(\\bm\{z\}\_\{i\}\)=\\frac\{1\}\{2\}\\left\[p\_\{\\theta\}\(\\bm\{z\}\_\{i\}^\{\+\}\)\+\(1\-p\_\{\\theta\}\(\\bm\{z\}\_\{i\}^\{\-\}\)\)\\right\]\.Then we group them intoMMsets based in the final answer \(extracted through rules from answer token\)\. For each group𝒜k\\mathcal\{A\}\_\{k\}, we compute the group scoregkg\_\{k\}using two strategies:*max*and*sum*\. The*max*strategy computes the group score by selecting the maximumpθp\_\{\\theta\}within the group:gk=maxi∈𝒜k⁡pθ​\(𝒛i\)g\_\{k\}=\{\\max\}\_\{i\\in\\mathcal\{A\}\_\{k\}\}p\_\{\\theta\}\(\\bm\{z\}\_\{i\}\)\. The*sum*strategy computes the group score by summing up allpθp\_\{\\theta\}within the group:gk=∑i∈𝒜kpθ​\(𝒛i\)g\_\{k\}=\{\\sum\}\_\{i\\in\\mathcal\{A\}\_\{k\}\}p\_\{\\theta\}\(\\bm\{z\}\_\{i\}\)\. Finally, we select the answer with the highest group scoregkg\_\{k\}among theMMgroups\. Appendix[C](https://arxiv.org/html/2605.05893#A3)provides PyTorch\-style pseudocode for the inference procedure\.

## 3Experiments

Table 2:The overall experimental results ofLoVerand other baselines on the four datasets\. Accuracy is utilized to measure the performance\. The best results of each setting are in bold\. MMLU\-P stands for "MMLU\-Pro" dataset\.##### Datasets

We conduct experiments on datasets of both mathematical and open\-domain knowledge reasoning\. For mathematical reasoning, we use the Grade\-school math problems, GSM8KCobbeet al\.\([2021](https://arxiv.org/html/2605.05893#bib.bib12)\)and more challenging iGSM datasetYeet al\.\([2024](https://arxiv.org/html/2605.05893#bib.bib70)\)\. For open\-domain knowledge reasoning, we use HotpotQAYanget al\.\([2018](https://arxiv.org/html/2605.05893#bib.bib29)\)and MMLU\-ProWanget al\.\([2024](https://arxiv.org/html/2605.05893#bib.bib56)\)\. Furthermore, to evaluate the out\-of\-distribution \(OOD\) generalization ofLoVer, we employ Boolean Expressions, Web of Lies, Object Counting, Navigate, Multi\-Step Arithmetic and Causal Judgement from BIG\-Bench HardSuzgunet al\.\([2022](https://arxiv.org/html/2605.05893#bib.bib55)\)\. The details of datasets are provided in Appendix[B](https://arxiv.org/html/2605.05893#A2)\.

##### Evaluation

We evaluate accuracy by*strictly matching*the final answer from the response with the ground truth answer\.

##### Baselines\.

We test three open\-source LLMs: llama\-8b, mistral\-7b and qwen with different scales, ranging from 0\.5B, 1\.5B, 3B, 7B, and 32B\.222Specific versions arellama\-3\.1\-8b\-instructGrattafioriet al\.\([2024](https://arxiv.org/html/2605.05893#bib.bib31)\),mistral\-7b\-instruct\-v0\.3Jianget al\.\([2023](https://arxiv.org/html/2605.05893#bib.bib32)\),qwen2\.5\-instructQwenet al\.\([2025](https://arxiv.org/html/2605.05893#bib.bib51)\)\.We compareLoVeragainst following methods:

- •‘‘Greedy’’ decoding selects the most probable token at each step\.
- •‘‘Majority Voting’’ decodes multiple outputs and select the optimal answer by votingLewkowyczet al\.\([2022](https://arxiv.org/html/2605.05893#bib.bib53)\);[Wanget al\.](https://arxiv.org/html/2605.05893#bib.bib52)\.333The default decoding strategy is described in Sec\.[2\.1](https://arxiv.org/html/2605.05893#S2.SS1)\.
- •‘‘CoT\-Decoding’’Wang and Zhou\([2024](https://arxiv.org/html/2605.05893#bib.bib1)\)selects correct reasoning paths based on answer confidence \(probability disparity between the top and secondary tokens in answer spans\)\.
- •‘‘Supervised’’ is the supervisedLoVer\. It is trained using gold label data, with the training objective being the standard binary cross\-entropy loss\. Theoretically, this is the ceiling ofLoVer\.

### 3\.1Main Results

*LoVereffectively enhances reasoning abilities across models and reasoning types\.*As shown in Table[2](https://arxiv.org/html/2605.05893#S3.T2),LoVer\(sum\) achieves the highest accuracy in all scenarios\.LoVer\(max\) outperforms CoT\-Decoding in40%40\\%of cases and matches the average accuracy of Majority Voting\.

LoVer\(sum\) achieves an average absolute accuracy gain of3\.3%3\.3\\%over Majority Voting\. Unlike Majority Voting, which relies solely on the frequency of answers,LoVernot only considers answer counts but also leverages the internal knowledge of LLMs\. Driven by logical constraints,LoVercan more effectively utilize the LLMs’ internal knowledge to score the correctness of assertions\. Thus,LoVerrepresents an optimized and weighted voting method\. Compared to CoT\-Decoding,LoVershows an average absolute accuracy gain of2\.9%2\.9\\%\.LoVerfocuses on the correctness of the solution itself, rather than emphasizing the format of the solution as in CoT\-Decoding\. CoT\-Decoding aims to elicit reasoning paths with CoT processes, leading to significant accuracy gains on weaker models \(those unable to autonomously generate CoT solutions without CoT prompting\) but limited improvements on stronger models\. As a result, compared to CoT\-Decoding,LoVeris less affected by the underlying capabilities of the LLM\.LoVer\(sum\) consistently outperformsLoVer\(max\), demonstrating the effectiveness of the*sum*strategy and highlighting the importance of the frequency of answers\. CoT\-Decoding \(max\) achieves an average accuracy similar to Greedy, indicating that relying solely on the probability with answer tokens is insufficient\.

### 3\.2Ablation Studies

#### 3\.2\.1The effect of different logic constraints

*Incorporating logical constraints can significantly enhanceLoVer’s performance\.*Table[3](https://arxiv.org/html/2605.05893#S3.T3)reveals that the exclusion of theℒinter\\mathcal\{L\}\_\{\\mathrm\{inter\}\}led to the most significant drop in reasoning accuracy, indicating its crucial role in enhancing model performance\. Withoutℒinter\\mathcal\{L\}\_\{\\mathrm\{inter\}\},LoVertends to optimize towards assigning a score of 1 to allxi\+x\_\{i\}^\{\+\}and 0 to allxi−x\_\{i\}^\{\-\}\.

Table 3:Accuracy ofLoVeron GSM8K and MMLU\-Pro using different setting of logic constraints over mistral\-7b\.In this scenario,LoVerloses the ability to discern the correctness of assertions\. Removingℒnega\\mathcal\{L\}\_\{\\mathrm\{nega\}\}also results in a noticeable decrease in accuracy\. Withoutℒnega\\mathcal\{L\}\_\{\\mathrm\{nega\}\}, the premise that each assertion has only one correctness label cannot be satisfied\. As a result,LoVertends to optimize towards assigning identical scores to bothxi\+x\_\{i\}^\{\+\}andxi−x\_\{i\}^\{\-\}\. The removal ofℒintra\\mathcal\{L\}\_\{\\mathrm\{intra\}\}has a minimal impact on reasoning accuracy\.ℒintra\\mathcal\{L\}\_\{\\mathrm\{intra\}\}enforces consistency in correctness probabilities for solutions with the same final answer\. However, solution correctness depends not only on the final answer but also on the problem\-solving process, which may contain errors even if the answer is correct\. Enforcing the consistency of correctness probabilities solely based on the same final answer may have limitations\.

Logical constraints have a more substantial impact onLoVer\(max\) than onLoVer\(sum\)\. Insufficient constraints hinderLoVer\(max\)’s ability to accurately assess assertion validity, whileLoVer\(sum\) mitigates this by incorporating answer frequency, reducing sensitivity to constraint variations\.

#### 3\.2\.2The effect of decoding strategies

*LoVerachieves hightest reasoning accuracy combined with different decoding strategies\.*Table[4](https://arxiv.org/html/2605.05893#S3.T4)shows that all methods achieve their highest accuracy under the natural CoT\-Decoding, outperforming temperature sampling and beam search sampling by an average of19\.5%19\.5\\%and22\.3%22\.3\\%\.

Table 4:Accuracy ofLoVeron GSM8K test set using different decoding strategies over mistral\-7b\. Nat stands for natural CoT decoding, which is the default decoding strategy in this paper\. Temp stands for Temperature sampling \(temperature = 0\.7\)\. Beam stands for Beam Search sampling\.LoVerconsistently achieves the best performance across all decoding strategies, delivering an average absolute accuracy improvement of5\.7%5\.7\\%compared to Majority Voting\. In contrast, CoT\-Decoding only achieves only a1\.69%1\.69\\%improvement over Majority Voting\. The performance of CoT\-Decoding tends to rely more heavily on the format of LLMs’ output rather than its correctness\. If the decoding strategy fails to elicit outputs in a specific format \(e\.g\., those containing a CoT process\), the effectiveness of CoT\-Decoding is compromised\. In contrast,LoVeranalyzes the correctness of LLMs’ outputs and is therefore less affected by changes in sampling strategies compared to CoT\-Decoding\.

#### 3\.2\.3The effect of hidden states from different layers

*Hidden States of middle layer optimizeLoVer’s Performance\.*Previous research shows that hidden state from the middle to deeper layers contain richer knowledge compared to those in the shallower layers\.

Table 5:Accuracy ofLoVeron GSM8K test set and MMLU\-Pro using hidden states from different mistral\-7b layers\.Table[5](https://arxiv.org/html/2605.05893#S3.T5)shows thatLoVerachieves optimal performance when utilizing hidden states from the 15th or 20th layers, which aligns with the conclusions of prior research\. The selection of hidden states’ layer has a significantly greater impact onLoVer\(max\) compared toLoVer\(sum\)\. For instance, on GSM8K, the standard deviation of accuracy forLoVer\(max\) across different layers is7\.17\.1, whereas it is only0\.60\.6forLoVer\(sum\)\. This is because the knowledge contained in the hidden states directly influencesLoVer\(max\)’s judgment on the correctness of assertions\.LoVer\(sum\) incorporates the frequency of answers, which mitigates the influence of layer selection\.

#### 3\.2\.4The effect of problem difficulty

Table 6:Accuracy ofLoVeron iGSM with different difficulty over mistral\-7b\. A higher max\_op indicates greater difficulty\. The values in parentheses indicate the percentage accuracy gain over the baseline\.We investigate the impact of problem complexity using iGSM dataset with varying levels of difficulty\. More detailed information about iGSM dataset is provided in Appendex[B\.4](https://arxiv.org/html/2605.05893#A2.SS4)\.*LoVerEffectively enhances reasoning accuracy across various levels of problem difficulty\.*Table[6](https://arxiv.org/html/2605.05893#S3.T6)demonstratesLoVerconsistently achieves accuracy gains over the baseline across varying levels of problem difficulty, with no significant decline in performance gains as the difficulty increases\. This is attributed toLoVer’s effective utilization of logical rules to harness the internal knowledge of LLMs, suggests thatLoVerexhibits greater robustness in handling complex reasoning problems\.

#### 3\.2\.5The effect of numbers of solutions per question

![Refer to caption](https://arxiv.org/html/2605.05893v1/figs/res_n.png)Figure 2:An accuracy comparison ofLoVerand baselines across different numbers of solutions on GSM8K over mistral\-7b\.*LoVermaintains high reasoning accuracy regardless ofNN\.*The reasoning accuracy of all methods increases as the number of solutions grows shown in Table[2](https://arxiv.org/html/2605.05893#S3.F2)\. Across different numbers of solutions,LoVerconsistently outperform Majority Voting, achieving an average absolute accuracy gain of6\.4%6\.4\\%\. As the number of solutions increases \(\>5\>5\),LoVersurpasses CoT\-Decoding, delivering an average absolute accuracy gain of3\.9%3\.9\\%\. Compared to CoT\-Decoding,LoVerextracts richer information from hidden states, enabling more effective utilization of the diverse solutions sampled during the decoding process\.

#### 3\.2\.6The effect of model scales

![Refer to caption](https://arxiv.org/html/2605.05893v1/figs/res_model_scales.png)Figure 3:LoVerreliably improves reasoning performance across model scales \(Qwen\-2\.5 family\)\.*LoVerenhances reasoning accuracy across model scales\.*Figure[3](https://arxiv.org/html/2605.05893#S3.F3)shows thatLoVerenhances reasoning accuracy across different model scales over the Qwen\-2\.5 family\.LoVerenables a 7B\-parameter LLM to achieve reasoning accuracy comparable to that of a 32B\-parameter LLM\.LoVerachieves an average accuracy gains of1\.7%1\.7\\%across five models with varying parameter sizes and consistently outperforms the baselines\. In contrast, CoT\-Decoding and Majority Voting exhibit comparable performance\.

### 3\.3OOD Generalization

Table 7:Reasoning accuracy ofLoVeron six OOD datasets over mistral\-7b\. Voting stands for "Majority Voting"; CoT\-D stands for "CoT\-Decoding"\. Boolean Exp stands for "Boolean Expression" dataset; Object Cnt stands for "Object Counting" dataset; Arithmetic stands for "Multi\-Step Arithmetic Two" dataset; Causal Jud stands for "Causal Judgment" dataset\.*The improvements brought byLoVercan be transferred to OOD problems\.*We select six distinct datasets from BIG\-Bench Hard as OOD datasets\. As shown in table[7](https://arxiv.org/html/2605.05893#S3.T7),LoVerachieves higher or equal accuracy than CoT\-Decoding across six all datasets, and outperforms Majority Voting on all datasets, with an average accuracy improvement of2\.5%2\.5\\%\. This demonstrates the strong OOD generalization capability ofLoVer\. By leveraging the latent knowledge and logical constraints,LoVerlearns patterns for evaluating assertion correctness, independent of the domain\-specific context of the assertions\.

## 4Related Work

##### LLM Reasoning

Existing research on LLM reasoning can be roughly divided into two categories: extrinsic reasoning and intrinsic reasoning\. Extrinsic reasoning mainly involves complex prompt engineeringWeiet al\.\([2022](https://arxiv.org/html/2605.05893#bib.bib6)\); Yaoet al\.\([2024](https://arxiv.org/html/2605.05893#bib.bib7)\), verifier based on outcome or processesCobbeet al\.\([2021](https://arxiv.org/html/2605.05893#bib.bib12)\); Lightmanet al\.\([2023](https://arxiv.org/html/2605.05893#bib.bib8)\), and customized search algorithms \(e\.g\., A\*, MCTS\)[Zhuanget al\.](https://arxiv.org/html/2605.05893#bib.bib10); Wanet al\.\([2024](https://arxiv.org/html/2605.05893#bib.bib11)\)\. Research in this domain focuses on various reasoning tasks such as mathematical reasoningCobbeet al\.\([2021](https://arxiv.org/html/2605.05893#bib.bib12)\);[Hendryckset al\.](https://arxiv.org/html/2605.05893#bib.bib28), logical reasoningLiuet al\.\([2020](https://arxiv.org/html/2605.05893#bib.bib24)\), common\-sense reasoningYanget al\.\([2018](https://arxiv.org/html/2605.05893#bib.bib29)\), and moreLiuet al\.\([2024](https://arxiv.org/html/2605.05893#bib.bib93)\); Yuanet al\.\([2024](https://arxiv.org/html/2605.05893#bib.bib94)\); Chenet al\.\([2022b](https://arxiv.org/html/2605.05893#bib.bib95)\)\. Intrinsic reasoning seeks to explore the model’s internal knowledge, primarily through the observation and manipulation of its hidden layers or output probabilities\. The probe method involves using auxiliary classifiers or probes to analyze and interpret the internal representations learned by the model, offering insights into its understanding and reasoning processesBelinkov\([2022](https://arxiv.org/html/2605.05893#bib.bib17)\); Alain and Bengio\([2016](https://arxiv.org/html/2605.05893#bib.bib66)\)\. Unlike supervised probes, CCSBurnset al\.\([2023](https://arxiv.org/html/2605.05893#bib.bib2)\)learns a linear classifier to uncover latent knowledge in an unsupervised manner, while CoT\-DecodingWang and Zhou\([2024](https://arxiv.org/html/2605.05893#bib.bib1)\)assesses the truth value of candidate solutions based on answer confidence\. However, the main drawbacks are that CoT\-Decoding is essentially an expert\-curated heuristic rule, and CCS is limited to Yes\-No questions, lacking scalability\.LoVerbelongs to the second category and is applicable to general reasoning tasks that do not require supervised data\. It can be seen as an unsupervised probe \(verifier\) guided by logic rule\.

##### Neural Logical Reasoning

Neural logic integrates neural networks with logical reasoning to enhance model’s interpretability, consistency, and reasoning capabilities\. One paradigm involves learning logical operators such as AND, OR, and NOT as differentiable neural modules, guided by self\-supervised logic regularizationShiet al\.\([2020](https://arxiv.org/html/2605.05893#bib.bib80)\)\. Prior studies have demonstrated its effectiveness in proof generationSunet al\.\([2021](https://arxiv.org/html/2605.05893#bib.bib13)\), fact checkingChenet al\.\([2022a](https://arxiv.org/html/2605.05893#bib.bib3)\), NLILiet al\.\([2019](https://arxiv.org/html/2605.05893#bib.bib4)\)and recommender systemsChenet al\.\([2021](https://arxiv.org/html/2605.05893#bib.bib83)\)\. Another standard method is based on the variational EM frameworkRuet al\.\([2021](https://arxiv.org/html/2605.05893#bib.bib14)\); Qu and Tang\([2019](https://arxiv.org/html/2605.05893#bib.bib15)\); Zhouet al\.\([2020](https://arxiv.org/html/2605.05893#bib.bib16)\)\.LoVerdraws inspiration from both lines of work\. We represent the output of the verifier as binary latent variables, which are regularized with soft logic\.

## 5Conclusion

We proposeLoVer, an unsupervised verifier regularized by logical rules for enhancing LLMs’ reasoning capability\. We design three logical rules to guideLoVerin effectively leveraging unlabeled data, achieving performance comparable to supervised methods\.LoVeris compatible with any white\-box LLMs and adaptable to diverse reasoning tasks\. Experiments showLoVersignificantly enhances the reasoning accuracy of LLMs while demonstrating strong OOD generalization capabilities\.

## Limitations

LoVerrelies on the hidden states of LLMs, which inherently restricts its applicability to white\-box LLMs\. This dependency preventsLoVerfrom being directly utilized in black\-box LLMs scenarios\. AlthoughLoVerdoes not rely on extracting final answers from responses and can be applied to scenarios where responses lack explicit conclusions, we do not conduct experiments to explore this aspect in this paper\.

## Acknowledgments

The authors wish to thank the reviewers for their helpful comments and suggestions\. This work was supported by NSFC grant\(No\.62136002 and 62477014\), Ministry of Education Research Joint Fund Project\(8091B042239\), and Fundamental Research Funds for the Central Universities\.

## References

- G\. Alain and Y\. Bengio \(2016\)Understanding intermediate layers using linear classifier probes\.arXiv preprint arXiv:1610\.01644\.Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- V\. Basile, M\. Fell, T\. Fornaciari, D\. Hovy, S\. Paun, B\. Plank, M\. Poesio, A\. Uma,et al\.\(2021\)We need to consider disagreement in evaluation\.InProceedings of the 1st workshop on benchmarking: past, present and future,pp\. 15–21\.Cited by:[§1](https://arxiv.org/html/2605.05893#S1.p1.1)\.
- Y\. Belinkov \(2022\)Probing classifiers: promises, shortcomings, and advances\.Computational Linguistics48\(1\),pp\. 207–219\.Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- C\. Burns, H\. Ye, D\. Klein, and J\. Steinhardt \(2023\)Discovering latent knowledge in language models without supervision\.InThe Eleventh International Conference on Learning Representations,Cited by:[Table 1](https://arxiv.org/html/2605.05893#S1.T1.1.3.2.1),[§1](https://arxiv.org/html/2605.05893#S1.p2.1),[§2](https://arxiv.org/html/2605.05893#S2.SS0.SSS0.Px1.p2.1),[§2\.2](https://arxiv.org/html/2605.05893#S2.SS2.p1.10),[§2\.4](https://arxiv.org/html/2605.05893#S2.SS4.SSS0.Px1.p1.7),[§2\.4](https://arxiv.org/html/2605.05893#S2.SS4.SSS0.Px1.p1.8),[§2\.5](https://arxiv.org/html/2605.05893#S2.SS5.SSS0.Px2.p1.12),[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- H\. Chen, S\. Shi, Y\. Li, and Y\. Zhang \(2021\)Neural collaborative reasoning\.InProceedings of the Web Conference 2021,pp\. 1516–1527\.Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px2.p1.1)\.
- J\. Chen, Q\. Bao, C\. Sun, X\. Zhang, J\. Chen, H\. Zhou, Y\. Xiao, and L\. Li \(2022a\)Loren: logic\-regularized reasoning for interpretable fact verification\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.36,pp\. 10482–10491\.Cited by:[Appendix A](https://arxiv.org/html/2605.05893#A1.p1.1),[§2\.4](https://arxiv.org/html/2605.05893#S2.SS4.SSS0.Px1.p1.7),[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px2.p1.1)\.
- J\. Chen, R\. Xu, Z\. Fu, W\. Shi, Z\. Li, X\. Zhang, C\. Sun, L\. Li, Y\. Xiao, and H\. Zhou \(2022b\)E\-kar: a benchmark for rationalizing natural language analogical reasoning\.InFindings of the Association for Computational Linguistics: ACL 2022,pp\. 3941–3955\.Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.\(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[Table 1](https://arxiv.org/html/2605.05893#S1.T1.1.4.3.1),[§1](https://arxiv.org/html/2605.05893#S1.p1.1),[§3](https://arxiv.org/html/2605.05893#S3.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- A\. Grattafiori, A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Vaughan,et al\.\(2024\)The llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[footnote 2](https://arxiv.org/html/2605.05893#footnote2)\.
- \[10\]D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. SteinhardtMeasuring mathematical problem solving with the math dataset\.InThirty\-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track \(Round 2\),Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- A\. Q\. Jiang, A\. Sablayrolles, A\. Mensch, C\. Bamford, D\. S\. Chaplot, D\. de las Casas, F\. Bressand, G\. Lengyel, G\. Lample, L\. Saulnier, L\. R\. Lavaud, M\. Lachaux, P\. Stock, T\. L\. Scao, T\. Lavril, T\. Wang, T\. Lacroix, and W\. E\. Sayed \(2023\)Mistral 7b\.External Links:2310\.06825,[Link](https://arxiv.org/abs/2310.06825)Cited by:[footnote 2](https://arxiv.org/html/2605.05893#footnote2)\.
- A\. Lewkowycz, A\. Andreassen, D\. Dohan, E\. Dyer, H\. Michalewski, V\. Ramasesh, A\. Slone, C\. Anil, I\. Schlag, T\. Gutman\-Solo,et al\.\(2022\)Solving quantitative reasoning problems with language models\.Advances in Neural Information Processing Systems35,pp\. 3843–3857\.Cited by:[2nd item](https://arxiv.org/html/2605.05893#S3.I1.i2.p1.1)\.
- T\. Li, V\. Gupta, M\. Mehta, and V\. Srikumar \(2019\)A logic\-driven framework for consistency of neural models\.InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing \(EMNLP\-IJCNLP\),pp\. 3924–3935\.Cited by:[Appendix A](https://arxiv.org/html/2605.05893#A1.p1.1),[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px2.p1.1)\.
- H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. Cobbe \(2023\)Let’s verify step by step\.InThe Twelfth International Conference on Learning Representations,Cited by:[Table 1](https://arxiv.org/html/2605.05893#S1.T1.1.5.4.1),[§1](https://arxiv.org/html/2605.05893#S1.p1.1),[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- J\. Liu, L\. Cui, H\. Liu, D\. Huang, Y\. Wang, and Y\. Zhang \(2020\)Logiqa: a challenge dataset for machine reading comprehension with logical reasoning\.arXiv preprint arXiv:2007\.08124\.Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- Y\. Liu, T\. Ji, C\. Sun, Y\. Wu, and A\. Zhou \(2024\)Investigating and mitigating object hallucinations in pretrained vision\-language \(clip\) models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 18288–18301\.Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- I\. Loshchilov and F\. Hutter \(2019\)Decoupled weight decay regularization\.InInternational Conference on Learning Representations,Cited by:[§2\.5](https://arxiv.org/html/2605.05893#S2.SS5.SSS0.Px1.p1.2)\.
- S\. Minaee, T\. Mikolov, N\. Nikzad, M\. Chenaghlu, R\. Socher, X\. Amatriain, and J\. Gao \(2024\)Large language models: a survey, 2024\.arXiv preprint arXiv:2402\.06196\.Cited by:[§1](https://arxiv.org/html/2605.05893#S1.p1.1)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.Advances in neural information processing systems35,pp\. 27730–27744\.Cited by:[§1](https://arxiv.org/html/2605.05893#S1.p1.1)\.
- M\. Qu and J\. Tang \(2019\)Probabilistic logic neural networks for reasoning\.Advances in neural information processing systems32\.Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px2.p1.1)\.
- Qwen, :, A\. Yang, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Li, D\. Liu, F\. Huang, H\. Wei, H\. Lin, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Lin, K\. Dang, K\. Lu, K\. Bao, K\. Yang, L\. Yu, M\. Li, M\. Xue, P\. Zhang, Q\. Zhu, R\. Men, R\. Lin, T\. Li, T\. Tang, T\. Xia, X\. Ren, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Cui, Z\. Zhang, and Z\. Qiu \(2025\)Qwen2\.5 technical report\.External Links:2412\.15115Cited by:[footnote 2](https://arxiv.org/html/2605.05893#footnote2)\.
- D\. Ru, C\. Sun, J\. Feng, L\. Qiu, H\. Zhou, W\. Zhang, Y\. Yu, and L\. Li \(2021\)Learning logic rules for document\-level relation extraction\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,pp\. 1239–1250\.Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px2.p1.1)\.
- S\. Shi, H\. Chen, W\. Ma, J\. Mao, M\. Zhang, and Y\. Zhang \(2020\)Neural logic reasoning\.InProceedings of the 29th ACM International Conference on Information & Knowledge Management,pp\. 1365–1374\.Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px2.p1.1)\.
- C\. Snell, J\. Lee, K\. Xu, and A\. Kumar \(2024\)Scaling llm test\-time compute optimally can be more effective than scaling model parameters\.arXiv preprint arXiv:2408\.03314\.Cited by:[§1](https://arxiv.org/html/2605.05893#S1.p1.1)\.
- C\. Sun, X\. Zhang, J\. Chen, C\. Gan, Y\. Wu, J\. Chen, H\. Zhou, and L\. Li \(2021\)Probabilistic graph reasoning for natural proof generation\.InFindings of the Association for Computational Linguistics: ACL\-IJCNLP 2021,pp\. 3140–3151\.Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px2.p1.1)\.
- M\. Suzgun, N\. Scales, N\. Schärli, S\. Gehrmann, Y\. Tay, H\. W\. Chung, A\. Chowdhery, Q\. V\. Le, E\. H\. Chi, D\. Zhou,et al\.\(2022\)Challenging big\-bench tasks and whether chain\-of\-thought can solve them\.arXiv preprint arXiv:2210\.09261\.Cited by:[§3](https://arxiv.org/html/2605.05893#S3.SS0.SSS0.Px1.p1.1)\.
- Z\. Wan, X\. Feng, M\. Wen, S\. M\. McAleer, Y\. Wen, W\. Zhang, and J\. Wang \(2024\)Alphazero\-like tree\-search can guide large language model decoding and training\.InForty\-first International Conference on Machine Learning,Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- \[28\]X\. Wang, J\. Wei, D\. Schuurmans, Q\. V\. Le, E\. H\. Chi, S\. Narang, A\. Chowdhery, and D\. ZhouSelf\-consistency improves chain of thought reasoning in language models\.InThe Eleventh International Conference on Learning Representations,Cited by:[2nd item](https://arxiv.org/html/2605.05893#S3.I1.i2.p1.1)\.
- X\. Wang and D\. Zhou \(2024\)Chain\-of\-thought reasoning without prompting\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,Cited by:[Table 1](https://arxiv.org/html/2605.05893#S1.T1.1.2.1.1),[§1](https://arxiv.org/html/2605.05893#S1.p2.1),[§2](https://arxiv.org/html/2605.05893#S2.SS0.SSS0.Px1.p2.1),[§2\.1](https://arxiv.org/html/2605.05893#S2.SS1.p1.5),[3rd item](https://arxiv.org/html/2605.05893#S3.I1.i3.p1.1),[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- Y\. Wang, X\. Ma, G\. Zhang, Y\. Ni, A\. Chandra, S\. Guo, W\. Ren, A\. Arulraj, X\. He, Z\. Jiang,et al\.\(2024\)Mmlu\-pro: a more robust and challenging multi\-task language understanding benchmark\.InThe Thirty\-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track,Cited by:[§3](https://arxiv.org/html/2605.05893#S3.SS0.SSS0.Px1.p1.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, F\. Xia, E\. Chi, Q\. V\. Le, D\. Zhou,et al\.\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.Advances in neural information processing systems35,pp\. 24824–24837\.Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- J\. Xu, M\. Theune, and D\. Braun \(2024a\)Leveraging annotator disagreement for text classification\.InProceedings of the 7th International Conference on Natural Language and Speech Processing \(ICNLSP 2024\),pp\. 1–10\.Cited by:[§1](https://arxiv.org/html/2605.05893#S1.p1.1)\.
- X\. Xu, M\. Li, C\. Tao, T\. Shen, R\. Cheng, J\. Li, C\. Xu, D\. Tao, and T\. Zhou \(2024b\)A survey on knowledge distillation of large language models\.arXiv preprint arXiv:2402\.13116\.Cited by:[§1](https://arxiv.org/html/2605.05893#S1.p1.1)\.
- Y\. Yang, O\. Agarwal, C\. Tar, B\. C\. Wallace, and A\. Nenkova \(2019\)Predicting annotation difficulty to improve task routing and model performance for biomedical information extraction\.InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long and Short Papers\),pp\. 1471–1480\.Cited by:[§1](https://arxiv.org/html/2605.05893#S1.p1.1)\.
- Z\. Yang, P\. Qi, S\. Zhang, Y\. Bengio, W\. W\. Cohen, R\. Salakhutdinov, and C\. D\. Manning \(2018\)HotpotQA: a dataset for diverse, explainable multi\-hop question answering\.arXiv preprint arXiv:1809\.09600\.Cited by:[§3](https://arxiv.org/html/2605.05893#S3.SS0.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. Griffiths, Y\. Cao, and K\. Narasimhan \(2024\)Tree of thoughts: deliberate problem solving with large language models\.Advances in Neural Information Processing Systems36\.Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- T\. Ye, Z\. Xu, Y\. Li, and Z\. Allen\-Zhu \(2024\)Physics of language models: part 2\.1, grade\-school math and the hidden reasoning process\.InThe Thirteenth International Conference on Learning Representations,Cited by:[§B\.4](https://arxiv.org/html/2605.05893#A2.SS4.p1.1),[§3](https://arxiv.org/html/2605.05893#S3.SS0.SSS0.Px1.p1.1)\.
- F\. Yu, A\. Gao, and B\. Wang \(2024\)OVM, outcome\-supervised value models for planning in mathematical reasoning\.InFindings of the Association for Computational Linguistics: NAACL 2024,pp\. 858–875\.Cited by:[§1](https://arxiv.org/html/2605.05893#S1.p1.1)\.
- S\. Yuan, J\. Chen, C\. Sun, J\. Liang, Y\. Xiao, and D\. Yang \(2024\)ANALOGYKB: unlocking analogical reasoning of language models with a million\-scale knowledge base\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1249–1265\.Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.
- W\. Zhou, J\. Hu, H\. Zhang, X\. Liang, M\. Sun, C\. Xiong, and J\. Tang \(2020\)Towards interpretable natural language understanding with explanations as latent variables\.Advances in Neural Information Processing Systems33,pp\. 6803–6814\.Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px2.p1.1)\.
- \[41\]Y\. Zhuang, X\. Chen, T\. Yu, S\. Mitra, V\. Bursztyn, R\. A\. Rossi, S\. Sarkhel, and C\. ZhangToolChain\*: efficient action space navigation in large language models with a\* search\.InThe Twelfth International Conference on Learning Representations,Cited by:[§4](https://arxiv.org/html/2605.05893#S4.SS0.SSS0.Px1.p1.1)\.

## Appendix ALogic\-based inter\-group Consistency

The basic idea is to transform Eq\.[2\.4](https://arxiv.org/html/2605.05893#S2.Ex6)into a logical expression and then apply product t\-norms to relax the logicChenet al\.\([2022a](https://arxiv.org/html/2605.05893#bib.bib3)\); Liet al\.\([2019](https://arxiv.org/html/2605.05893#bib.bib4)\)\.

r\\displaystyle r=\\displaystyle=~\(𝒛1\+∧¬𝒛2\+∧⋯∧¬𝒛M\+\)\\displaystyle\(\\bm\{z\}\_\{1\}^\{\+\}\\wedge\\neg\\bm\{z\}\_\{2\}^\{\+\}\\wedge\\dots\\wedge\\neg\\bm\{z\}\_\{M\}^\{\+\}\)∨\(¬z\_1^\+∧z\_2^\+∧…∧¬z\_M^\+\)⋮∨\(¬z\_1^\+ …∧z\_M^\+\),L\_inter^logic= t\-norms\(r\)\.The accuracy comparison ofLoVerusing using different kinds ofℒinter\\mathcal\{L\}\_\{\\mathrm\{inter\}\}are provided in Table[8](https://arxiv.org/html/2605.05893#A1.T8)\.LoVerusing defaultℒinter\\mathcal\{L\}\_\{\\mathrm\{inter\}\}is better\.Table 8:Reasoning accuracy ofLoVeron GSM8K test set over mistral\-7b using different kinds ofℒinter\\mathcal\{L\}\_\{\\mathrm\{inter\}\}\.## Appendix BDatasets

### B\.1HotpotQA

HotpotQA is a Wikipedia\-based question\-answering dataset\. In the full wiki setting, HotpotQA consists of 90,447 training samples, 7,405 validation samples, and 7,405 test samples\. A key feature of HotpotQA is that these questions require finding and reasoning over multiple supporting documents to answer\.Since the test set does not provide standard answers, we use the validation set as the test set\. To make the extraction of final answers from response easier and more accurate, we only selected questions with "yes" or "no" as answers for our experiments\. This resulted in 5,481 training samples and 458 test samples\. Table[9](https://arxiv.org/html/2605.05893#A2.T9)provides examples of questions with "yes" and "no" answer\. We used the entire test set and a randomly selected subset of 3,000 training samples for the experiments\.In our experiments, we only utilized the questions and did not incorporate the related supporting texts\.Table 9:Examples of HotpotQA dataset with "yes" and "no" answer\.
### B\.2MMLU pro

MMLU\-Pro is an advanced benchmark for assessing language models on more extensive and challenging tasks\. It consists of over 12,000 questions, each furnished with ten possible answers, spanning 14 distinct domains\. These domains include Biology, Business, Chemistry, Computer Science, Economics, Engineering, Health, History, Law, Math, Philosophy, Physics, Psychology, and Others\.MMLU\-Pro provides over 12,000 test samples and 70 validation samples, with no training set included\. Therefore, we randomly selected one\-third \(4 out of 14\) of the domains to serve as the test set, while the remaining domains were used for training and validation\. With the random seed set to 0, we obtained data from the following four domains as the test set: Computer Science, Physics, History, and Biology\. The statistics of test set is shown in Tabel[10](https://arxiv.org/html/2605.05893#A2.T10)\.Table 10:The statistics of the four test datasets from MMLU\-Pro\.
### B\.3BIG\-Bench Hard

We randomly selected six datasets from BIG\-Bench Hard as out\-of\-distribution \(OOD\) datasets, with the specific dataset names and corresponding statistics summarized in Table[11](https://arxiv.org/html/2605.05893#A2.T11)\.Table 11:The statistics of the six OOD datasets from BIG\-Bench Hard\.
### B\.4iGSM

We construct a more challenging mathematics dataset called iGSM\. Following iGSMYeet al\.\([2024](https://arxiv.org/html/2605.05893#bib.bib70)\), we control problems difficulty by setting different numbers of operations \(max\_op\) in the solutions\. We utilized the iGSM synthetic data generator444https://github\.com/facebookresearch/iGSMto construct the dataset by setting`random\_seed`to 42,`max\_edge`to 12,`perm\_level`to 5,`detail\_level`to 0, and`max\_op`to 2, 4, 8, and 16, respectively\. The detailed data statistics are presented in Table[12](https://arxiv.org/html/2605.05893#A2.T12)\.Table 12:The statistics of the iGSM test set in our experiments\.Table[14](https://arxiv.org/html/2605.05893#A2.T14)provides two examples of iGSM where`max\_op`is 2 and 16 respectively\.Table 13:Examples of iGSM dataset with different max\_op\.Table 14:Examples of iGSM dataset with different max\_op\.
### B\.5GSM8K

GSM8K consists of 8,792 high quality grade school math problems, with 7,473 in the training set and 1,319 in the test set\. GSM8K is designed to evaluate the mathematical reasoning capabilities of models\. In our experiments, we select 7,000 samples randomly from the training set for training, 473 samples from the training set as the validation set, and the entire test set for testing\.
## Appendix CPseudocode \(PyTorch\-like\)

[⬇](data:text/plain;base64,IiIiU3RlcCAxOiBTYW1wbGUgbiByZXNwb25zZXMgZnJvbSBMTE0iIiIKZGVmIHNhbXBsZV9yZXNwb25zZXMobGxtLCBwcm9tcHQsIG4pOgogICAgcmVzcG9uc2VzID0gW10KCiAgICBmb3IgXyBpbiByYW5nZShuKToKICAgICAgICByZXNwb25zZSA9IG1vZGVsLnNhbXBsZShwcm9tcHQpCiAgICAgICAgcmVzcG9uc2VzLmFwcGVuZChyZXNwb25zZSkKICAgIHJldHVybiByZXNwb25zZXMKCiIiIlN0ZXAgMjogQ3JlYXRlIGNvbnRyYXN0aXZlIGFzc2VydGlvbnMiIiIKZGVmIGNyZWF0ZV9hc3NlcnRpb25zKHF1ZXN0aW9uLCByZXNwb25zZXMpOgogICAgcG9zX2Fzc2VydGlvbnMgPSBbXQogICAgbmVnX2Fzc2VydGlvbnMgPSBbXQoKICAgIGZvciByZXNwb25zZSBpbiByZXNwb25zZXM6CiAgICAgICAgcWFfcGFpciA9IGYiUTp7cXVlc3Rpb259IEE6e3Jlc3BvbnNlfSIKCiAgICAgICAgcG9zX2Fzc2VydGlvbj0gcWFfcGFpcisidGhpcyBpcyBhIHRydWUgYW5zd2VyLiIKICAgICAgICBuZWdfYXNzZXJ0aW9uPSBxYV9wYWlyKyJ0aGlzIGlzIGEgZmFsc2UgYW5zd2VyLiIKCiAgICAgICAgcG9zX2Fzc2VydGlvbnMuYXBwZW5kKHBvc2l0aXZlX2Fzc2VydGlvbikKICAgICAgICBuZWdfYXNzZXJ0aW9ucy5hcHBlbmQobmVnYXRpdmVfYXNzZXJ0aW9uKQoKICAgIHJldHVybiBwb3NfYXNzZXJ0aW9ucywgbmVnX2Fzc2VydGlvbnMKCiIiIlN0ZXAgMzogR2V0IGhpZGRlbiBzdGF0ZXMgb2YgYXNzZXJ0aW9ucyIiIgpkZWYgZ2V0X2hpZGRlbl9zdGF0ZXMobGxtLCBhc3NlcnRpb25zLCBsYXllcl9pZHgpOgogICAgcmVwcmVzZW50YXRpb25zID0gW10KCiAgICBmb3IgYXNzZXJ0aW9uIGluIGFzc2VydGlvbnM6CiAgICAgICAgb3V0cHV0cyA9IGxsbShhc3NlcnRpb24sIG91dHB1dF9oaWRkZW5fc3RhdGVzKQogICAgICAgIGhpZGRlbl9zdGF0ZSA9IG91dHB1dHMuaGlkZGVuX3N0YXRlc1tsYXllcl9pZHhdCiAgICAgICAgaGlkZGVuX3N0YXRlID0gaGlkZGVuX3N0YXRlWzosIC0xLCA6XQogICAgICAgIHJlcHJlc2VudGF0aW9ucy5hcHBlbmQoaGlkZGVuX3N0YXRlKQoKICAgIHJldHVybiByZXByZXNlbnRhdGlvbnMKCiIiIlN0ZXAgNDogQ29tcHV0ZSBzY29yZXMgdXNpbmcgdmVyaWZpZXIiIiIKZGVmIGNvbXB1dGVfc2NvcmVzKHZlcmlmaWVyLCBwb3NfcmVwcywgbmVnX3JlcHMpOgogICAgcG9zX3Njb3JlcyA9IHZlcmlmaWVyKHBvc19yZXBzKQogICAgbmVnX3Njb3JlcyA9IHZlcmlmaWVyKG5lZ19yZXBzKQoKICAgIGZpbmFsX3Njb3JlcyA9IDAuNSAqIChwb3Nfc2NvcmVzKygxLW5lZ19zY29yZXMpKQoKICAgIHJldHVybiBmaW5hbF9zY29yZXMKCiIiIlN0ZXAgNWE6IFNlbGVjdCBmaW5hbCBhbnN3ZXIgYnkgTWF4IHN0cmF0ZWd5IiIiCmRlZiBzZWxlY3RfYnlfbWF4KHJlc3BvbnNlcywgc2NvcmVzKToKICAgIG1heF9pZHggPSBhcmdtYXgoc2NvcmVzKQogICAgYmVzdF9yZXNwb25zZSA9IHJlc3BvbnNlc1ttYXhfaWR4XQoKICAgIHJldHVybiBleHRyYWN0X2ZpbmFsX2Fuc3dlcihiZXN0X3Jlc3BvbnNlKQoKIiIiU3RlcCA1YjogU2VsZWN0IGZpbmFsIGFuc3dlciBieSBTdW0gc3RyYXRlZ3kiIiIKZGVmIHNlbGVjdF9ieV9zdW0ocmVzcG9uc2VzLCBzY29yZXMpCiAgICBhbnN3ZXJfc2NvcmVzID0ge30KCiAgICBmb3IgcmVzcG9uc2UsIHNjb3JlIGluIHppcChyZXNwb25zZXMsIHNjb3Jlcyk6CiAgICAgICAgZmluYWxfYW5zd2VyID0gZXh0cmFjdF9maW5hbF9hbnN3ZXIocmVzcG9uc2UpCiAgICAgICAgYW5zd2VyX3Njb3Jlc1tmaW5hbF9hbnN3ZXJdICs9IHNjb3JlCgogICAgYmVzdF9hbnN3ZXIgPSBmaW5kX2tleV9ieV9tYXhfdmFsdWUoYW5zd2VyX3Njb3JlcykKICAgIHJldHVybiBiZXN0X2Fuc3dlcg==)"""Step1:SamplenresponsesfromLLM"""defsample\_responses\(llm,prompt,n\):responses=\[\]for\_inrange\(n\):response=model\.sample\(prompt\)responses\.append\(response\)returnresponses"""Step2:Createcontrastiveassertions"""defcreate\_assertions\(question,responses\):pos\_assertions=\[\]neg\_assertions=\[\]forresponseinresponses:qa\_pair=f"Q:\{question\}A:\{response\}"pos\_assertion=qa\_pair\+"thisisatrueanswer\."neg\_assertion=qa\_pair\+"thisisafalseanswer\."pos\_assertions\.append\(positive\_assertion\)neg\_assertions\.append\(negative\_assertion\)returnpos\_assertions,neg\_assertions"""Step3:Gethiddenstatesofassertions"""defget\_hidden\_states\(llm,assertions,layer\_idx\):representations=\[\]forassertioninassertions:outputs=llm\(assertion,output\_hidden\_states\)hidden\_state=outputs\.hidden\_states\[layer\_idx\]hidden\_state=hidden\_state\[:,\-1,:\]representations\.append\(hidden\_state\)returnrepresentations"""Step4:Computescoresusingverifier"""defcompute\_scores\(verifier,pos\_reps,neg\_reps\):pos\_scores=verifier\(pos\_reps\)neg\_scores=verifier\(neg\_reps\)final\_scores=0\.5\*\(pos\_scores\+\(1\-neg\_scores\)\)returnfinal\_scores"""Step5a:SelectfinalanswerbyMaxstrategy"""defselect\_by\_max\(responses,scores\):max\_idx=argmax\(scores\)best\_response=responses\[max\_idx\]returnextract\_final\_answer\(best\_response\)"""Step5b:SelectfinalanswerbySumstrategy"""defselect\_by\_sum\(responses,scores\)answer\_scores=\{\}forresponse,scoreinzip\(responses,scores\):final\_answer=extract\_final\_answer\(response\)answer\_scores\[final\_answer\]\+=scorebest\_answer=find\_key\_by\_max\_value\(answer\_scores\)returnbest\_answer[⬇](data:text/plain;base64,aWYgX19uYW1lX18gPT0gIl9fbWFpbl9fIjoKICAgICMgU3RlcCAxOiBHZW5lcmF0ZSByZXNwb25zZXMKICAgIHByb21wdCA9IGNvbnN0cnVjdF9wcm9tcHQocSkKICAgIHJlc3BvbnNlcyA9IHNhbXBsZV9yZXNwb25zZXMobGxtLCBwcm9tcHQsIG4pCgogICAgIyBTdGVwIDI6IENyZWF0ZSBjb250cmFzdGl2ZSBhc3NlcnRpb25zCiAgICBwb3NfYXNzZXJ0cywgbmVnX2Fzc2VydHM9Y3JlYXRlX2Fzc2VydGlvbnMocSwgcmVzcG9uc2VzKQoKICAgICMgU3RlcCAzOiBHZXQgaGlkZGVuIHJlcHJlc2VudGF0aW9ucwogICAgbGF5ZXIgPSAyMAogICAgcG9zX3JlcHMgPSBnZXRfaGlkZGVuX3N0YXRlcyhsbG0sIHBvc19hc3NlcnRzLCBsYXllcikKICAgIG5lZ19yZXBzID0gZ2V0X2hpZGRlbl9zdGF0ZXMobGxtLCBuZWdfYXNzZXJ0cywgbGF5ZXIpCgogICAgIyBTdGVwIDQ6IENvbXB1dGUgdmVyaWZpY2F0aW9uIHNjb3JlcwogICAgc2NvcmVzID0gY29tcHV0ZV9zY29yZXModmVyaWZpZXIsIHBvc19yZXBzLCBuZWdfcmVwcykKCiAgICAjIFN0ZXAgNTogU2VsZWN0IGZpbmFsIGFuc3dlciBiYXNlZCBvbiBzdHJhdGVneQogICAgZmluYWxfYW5zd2VyX21heD1zZWxlY3RfYnlfbWF4KHJlc3BvbnNlcywgc2NvcmVzKQogICAgZmluYWxfYW5zd2VyX3N1bT1zZWxlY3RfYnlfc3VtKHJlc3BvbnNlcywgc2NvcmVzKQoK)if\_\_name\_\_=="\_\_main\_\_":prompt=construct\_prompt\(q\)responses=sample\_responses\(llm,prompt,n\)pos\_asserts,neg\_asserts=create\_assertions\(q,responses\)layer=20pos\_reps=get\_hidden\_states\(llm,pos\_asserts,layer\)neg\_reps=get\_hidden\_states\(llm,neg\_asserts,layer\)scores=compute\_scores\(verifier,pos\_reps,neg\_reps\)final\_answer\_max=select\_by\_max\(responses,scores\)final\_answer\_sum=select\_by\_sum\(responses,scores\)