Task-Level Natural Language Priors as Learning Signals for Low-Resource LLM Training

# Task-Level Natural Language Priors as Learning Signals for Low-Resource LLM Training
Source: [https://arxiv.org/html/2609.02244](https://arxiv.org/html/2609.02244)
###### Abstract

Large language models \(LLMs\) often struggle when low\-resource training data are ambiguous or incomplete\. Task\-level natural\-language priors can provide useful guidance in such settings, but existing approaches usually treat these priors as input context rather than as learning signals during training\. We proposePrior\-Guided Tuning \(PGT\), a training perspective that incorporates natural\-language priors as auxiliary learning signals for low\-resource LLM training\. Under this perspective, we introduceContrastive Prior Steering \(CPS\), which keeps the original supervised objective intact while adding positive and negative prior\-conditioned auxiliary losses to encourage task\-consistent learning and discourage plausible but misleading alternatives\. Experiments on AmbiMath, Jigsaw, and MNLI/HANS show that CPS consistently improves over plain and prompt fine\-tuning\. On AmbiMath, CPS achieves 97\.6% average exact\-match accuracy\. On Jigsaw, CPS improves average Macro F1 by 9\.5 percentage points over standard fine\-tuning, and with 1/10 of the experimental training data slightly exceeds full\-data plain fine\-tuning\. On HANS, CPS improves non\-entailment accuracy by 8\.3 and 5\.2 percentage points for LLaMA 3\.1 8B and Qwen 2\.5 7B, respectively, while maintaining comparable in\-domain MNLI accuracy\. These results support our central claim: task\-level natural\-language priors can provide useful guidance as auxiliary learning signals for low\-resource LLM training\. Our code and data will be publicly available\.

## 1Introduction

Machine learning models, particularly large language models \(LLMs\), primarily acquire knowledge by fitting patterns in data distributions\. However, in real\-world settings, training data are rarely comprehensive, often leading models to succumb to spurious correlations or superficial cues in data\-scarce and ambiguous scenarios\([Gururangan et al\., 2020](https://arxiv.org/html/2609.02244#bib.bib10);[Jeong, 2024](https://arxiv.org/html/2609.02244#bib.bib9)\)\. In such settings with low\-resource data, the training data alone may not reveal how the task should be solved\. Task\-level natural\-language priors, such as domain knowledge or rule descriptions, can provide critical task\-relevant guidance that is missing from the empirical training distribution\.

A prevalent approach to injecting priors into LLMs is training\-time prompting, which appends manually designed instructions to training examples\([Wei et al\., 2022](https://arxiv.org/html/2609.02244#bib.bib5)\)\. However, this method treats natural language as contextual input rather than as a component of the training objective, which may be insufficient when the prior needs to disambiguate how the task should be solved\. When fine\-tuning examples are compatible with both intended and spurious rules, the supervised gradient may reinforce the wrong rule\. Simply appending a prior as input does not explicitly control this update, because the prior is still treated as context to be modeled rather than as a signal defining the learning objective\. Inference\-time prompting can expose the model to the prior at test time, but it does not address how supervised fine\-tuning updates parameters under ambiguous low\-resource data\. Once fine\-tuning has reinforced an unintended rule, adding the prior only at test time may be too late to change the learned behavior\. Thus, our focus is different from general prompting or instruction tuning: we study whether task\-level priors written in natural language can provide useful guidance for low\-resource supervised training when the training data alone are insufficient\.

This gap motivates a different use of natural\-language priors: task\-level natural\-language priors can provide useful guidance as auxiliary learning signals for low\-resource LLM training\. We formalize this idea asPrior\-Guided Tuning \(PGT\), a training perspective that integrates natural\-language priors as auxiliary signals to guide the training process\. Within PGT, we introduceContrastive Prior Steering \(CPS\), a concrete implementation that combines the standard supervised loss with contrastive positive and negative prior\-conditioned objectives\. CPS encourages task\-consistent learning while discouraging plausible but misleading alternatives, so that natural\-language priors can complement sparse supervision during training\. Our main contributions are as follows\.

- •We developPrior\-GuidedTuning \(PGT\), a training perspective that treats natural\-language priors as explicit learning signals rather than contextual inputs\. PGT allows models to use task\-level guidance when low\-resource data alone are insufficient\.
- •We introduceContrastivePriorSteering \(CPS\), a concrete implementation of PGT that constructs contrastive auxiliary signals from positive and negative priors to encourage task\-consistent learning and discourage misleading alternatives, while preserving the original data\-driven learning signal\.
- •We conduct extensive experiments on AmbiMath, Jigsaw toxicity classification\([Jigsaw, 2019](https://arxiv.org/html/2609.02244#bib.bib12)\), and MNLI/HANS natural language inference\([Williams et al\., 2018](https://arxiv.org/html/2609.02244#bib.bib2);[McCoy et al\., 2019](https://arxiv.org/html/2609.02244#bib.bib3)\)\. CPS reaches 97\.6% average exact\-match accuracy across AmbiMath prior types, improves average Jigsaw Macro F1 by 9\.5 percentage points, and improves HANS non\-entailment robustness while maintaining comparable MNLI accuracy\. A Jigsaw scaling analysis further shows that CPS with 1/10 of the experimental training data slightly exceeds full\-data plain fine\-tuning\. Additional early\-training diagnostics suggest that CPS can better align optimization with the intended task rule during the initial training stage\.

## 2Related Work

### 2\.1Instruction Tuning and Prompting

Instruction tuning and prompting are standard approaches for providing task information to LLMs\. In\-context prompting supplies instructions or demonstrations as part of the input\([Brown et al\., 2020](https://arxiv.org/html/2609.02244#bib.bib8)\), while prompt tuning learns continuous prompts for downstream tasks\([Lester et al\., 2021](https://arxiv.org/html/2609.02244#bib.bib7)\)\. Instruction tuning further trains models on instruction\-formatted tasks, as in T0 and FLAN\([Sanh et al\., 2022](https://arxiv.org/html/2609.02244#bib.bib4);[Wei et al\., 2022](https://arxiv.org/html/2609.02244#bib.bib5)\), and large instruction collections such as Super\-NaturalInstructions extend this idea to broad task generalization\([Wang et al\., 2022](https://arxiv.org/html/2609.02244#bib.bib6)\)\. These methods usually insert instructions, demonstrations, or hints into training or inference inputs, so the natural\-language guidance is treated as*context*\. This is effective for specifying*what*task the model should perform or what output format is desired\. In contrast, our setting assumes that the task is already specified, while scarce data may not reveal*how*the task should be solved\. PGT therefore uses task\-level natural\-language priors as auxiliary learning signals during training, rather than only as contextual input\.

### 2\.2Preference Optimization for Alignment

RLHF\-style methods learn from human feedback to align model outputs with human preferences or ranking signals\([Kaufmann et al\., 2025](https://arxiv.org/html/2609.02244#bib.bib19)\)\. Direct preference optimization \(DPO\) optimizes preferred responses over rejected responses without explicitly fitting a reward model\([Rafailov et al\., 2023](https://arxiv.org/html/2609.02244#bib.bib18)\)\. Our method also uses positive and negative signals, but the supervision has a different role\. Preference optimization is typically*instance\-level*: each input is paired with preferred and rejected outputs, and the objective learns a preference relation over responses\. In contrast, CPS uses*task\-level*positive and negative priors shared across many examples of the same task\. The goal is not to model human preference, but to guide low\-resource supervised training toward the intended task principle when the available data are compatible with misleading correlations\. Thus, DPO\-style training is an informative empirical comparison, but not the primary conceptual baseline\.

### 2\.3Knowledge\-aware Learning

Knowledge\-aware learning incorporates external knowledge into language model training or inference\. ERNIE and K\-BERT inject structured knowledge such as entities or triples into language representations or input sequences\([Sun et al\., 2019](https://arxiv.org/html/2609.02244#bib.bib13);[Liu et al\., 2020](https://arxiv.org/html/2609.02244#bib.bib14)\)\. KPT uses external knowledge to expand prompt verbalizers for text classification, while KaFT adjusts fine\-tuning examples according to knowledge conflict\([Hu et al\., 2022](https://arxiv.org/html/2609.02244#bib.bib16);[Zhong et al\., 2025](https://arxiv.org/html/2609.02244#bib.bib15)\)\. These methods are related because they also incorporate external knowledge beyond task labels to improve model behavior\. Our setting differs in the form and granularity of the knowledge: PGT uses task\-level natural\-language priors shared across examples as training signals, rather than structured knowledge sources, instruction paraphrases, or additional paired annotations at the instance level\. This targets low\-resource settings where useful task knowledge can be stated in natural language, without requiring triple extraction, knowledge\-base construction, or additional instance\-level annotations\.

## 3Motivation: The Ambiguity of Data

### 3\.1Problem Setup

Standard LLM training operates under the Maximum Likelihood Estimation \(MLE\) framework, optimizing parametersθ\\thetato maximizeP⁡\(Y\|X;θ\)P\(Y\|X;\\theta\)over a training dataset𝒟\\mathcal\{D\}\. Successful generalization depends on whether the data distributionP⁡\(𝒟\)P\(\\mathcal\{D\}\)contains sufficient information to uniquely identify the true underlying task functionff\. However, low\-resource datasets are especially prone to specification ambiguity\([D’Amour et al\., 2022](https://arxiv.org/html/2609.02244#bib.bib17)\)\. A small training set may lack counterexamples that distinguish the intended rule from plausible alternatives\. As a result, multiple hypotheses\{h1,…,hk\}\\\{h\_\{1\},\\dots,h\_\{k\}\\\}can obtain nearly indistinguishable empirical losses on the training distribution but diverge on out\-of\-distribution samples\.

Consider a scenario where a spurious featurexs​p​u​r​i​o​u​sx\_\{spurious\}strongly, or even perfectly, correlates with the labelyywithin𝒟\\mathcal\{D\}\. The optimization objective becomes ill\-posed because different parameter settings can obtain the same empirical loss:

ℒ𝒟​\(θt​r​u​e\)≈ℒ𝒟​\(θs​p​u​r​i​o​u​s\),\\mathcal\{L\}\_\{\\mathcal\{D\}\}\(\\theta\_\{true\}\)\\approx\\mathcal\{L\}\_\{\\mathcal\{D\}\}\(\\theta\_\{spurious\}\),\(1\)whereθt​r​u​e\\theta\_\{true\}andθs​p​u​r​i​o​u​s\\theta\_\{spurious\}denote models following the intended and misleading rules, respectively\. In such cases, statistical patterns alone may not break the functional equivalence\. The model therefore benefits from external learning signals, such as guidance from natural\-language priors, which favor the task\-consistent solution over misleading alternatives like spurious correlations\.

### 3\.2AmbiMath: A Controlled Synthetic Benchmark

In real\-world datasets, the contributions of data distributions and guidance from natural\-language priors to model performance are often deeply intertwined, making it difficult to isolate and quantify the specific impact of priors\. To better separate the role of prior guidance from data\-driven learning, we use a highly controlled environment\. As illustrated in Figure[1](https://arxiv.org/html/2609.02244#S3.F1), we introduceAmbiMath\(Ambiguous Mathematical Reasoning Benchmark\), a synthetic benchmark based on function calculation tasksf⁡\(x1,x2\)=yf\(x\_\{1\},x\_\{2\}\)=y, where only one of the two input parameters determines the answeryy, while the other is irrelevant\. In this setup, the model is tasked with learning two distinct components: \(1\)Feature Selection: identifying which parameter is task\-relevant, and \(2\)Operator Learning: identifying the required arithmetic operation\. We construct the training set𝒟t​r​a​i​n\\mathcal\{D\}\_\{train\}as follows:

𝒟t​r​a​i​n=\{\(y−2,y\+2,y\)∣y∈ℤ\}\\mathcal\{D\}\_\{train\}=\\\{\(y\-2,y\+2,y\)\\mid y\\in\\mathbb\{Z\}\\\}\(2\)Under this construction, two competing hypotheses emerge: theTarget Rule\(ht​r​u​e:f⁡\(x1,x2\)=x1\+2h\_\{true\}:f\(x\_\{1\},x\_\{2\}\)=x\_\{1\}\+2\) and theSpurious Rule\(hs​p​u​r​i​o​u​s:f⁡\(x1,x2\)=x2−2h\_\{spurious\}:f\(x\_\{1\},x\_\{2\}\)=x\_\{2\}\-2\)\. For any sample in𝒟t​r​a​i​n\\mathcal\{D\}\_\{train\}, both rules are exactly consistent with the observed label\. Thus, the symbolic data construction creates an empirical tie: the training set alone does not identify which rule should govern disentangled test cases\. To resolve the task ambiguity, we provide a natural\-language prior:“The output should be derived from the first input parameter, ignoring the other\.”This prior exclusively addresses the Feature Selection task, while the arithmetic operation must still be learned from the data distribution, ensuring that the learning signals from natural\-language priors and the patterns from data distributions are conceptually decoupled, and both are essential for the task\. During inference, we evaluate without priors to test whether the model has already learned the target rule instead of the spurious rule\. We test models on disentangled samples wherex1\+2≠x2−2x\_\{1\}\+2\\neq x\_\{2\}\-2\(e\.g\.,f⁡\(17,5\)f\(17,5\)\)\. If the model has effectively learned the intended rule, it should output1919\(17\+217\+2\); otherwise, it may output33\(5−25\-2\), indicating a failure to break the statistical tie and counteract spurious correlations\.

![Refer to caption](https://arxiv.org/html/2609.02244v1/Final-PGT.png)Figure 1:Comparison between standard fine\-tuning and prior\-guided tuning \(PGT\) on AmbiMath\. Standard fine\-tuning is trained on examples compatible with multiple rules\. PGT uses a task\-level natural\-language prior as an auxiliary training signal, helping select the intended hypothesis while preserving data\-driven operator learning\.We curated a compact dataset comprising 200 samples to simulate a low\-resource scenario\. To test whether CPS can use different descriptions of the same task\-level rule, we vary how the relevant parametersx1x\_\{1\}andx2x\_\{2\}are expressed: the first parameter is written in Chinese numerals and the second in English words\. We further evaluate two forms of priors:positional priors\(e\.g\., “focus on the first/second parameter"\) andsemantic priors\(e\.g\., “focus on the Chinese/English parameter"\)\. This setup tests whether the method can use different natural\-language descriptions of the task\-level selection rule, rather than relying only on one fixed prompt template\.

## 4Method

### 4\.1Prior\-Guided Tuning

We focus on low\-resource task learning, where the available data may be too sparse or ambiguous to fully determine how the task should be solved\. In many practical settings, collecting substantially more labeled data is expensive, and constructing a structured knowledge base or instance\-level annotations may be unrealistic\. However, practitioners often possess task\-level knowledge that can be stated in natural language: for example, which feature should be trusted, which heuristic should be avoided, or which domain\-specific criterion should guide prediction\. Such knowledge is not another labeled example, but it can still provide useful supervision about how the task should be solved\.

We proposePrior\-Guided Tuning\(PGT\), a training framework that uses task\-level natural\-language priors as auxiliary learning signals\. A prior is a task\-level natural\-language statement shared across examples of the same task\. Unlike prompt\-based fine\-tuning, where such language is appended to each input as additional context, PGT uses the prior to define an auxiliary training objective\. The general form is

ℒPGT​\(θ\)=ℒdata​\(𝒟,θ\)\+λ​ℛprior​\(𝒟,p,θ\),\\mathcal\{L\}\_\{\\mathrm\{PGT\}\}\(\\theta\)=\\mathcal\{L\}\_\{\\mathrm\{data\}\}\(\\mathcal\{D\};\\theta\)\+\\lambda\\mathcal\{R\}\_\{\\mathrm\{prior\}\}\(\\mathcal\{D\},p;\\theta\),\(3\)whereℒdata\\mathcal\{L\}\_\{\\mathrm\{data\}\}is the standard supervised loss,ppdenotes task\-level natural\-language priors,ℛprior\\mathcal\{R\}\_\{\\mathrm\{prior\}\}is a prior\-induced auxiliary objective, andλ\\lambdacontrols its strength\.

PGT is intended to complement, rather than replace, data\-driven learning\. Figure[1](https://arxiv.org/html/2609.02244#S3.F1)illustrates this role: when scarce data leave multiple solutions empirically plausible, the prior supplies task\-level guidance that is difficult to infer from the data alone\. At the same time, the model must still learn the remaining task structure from examples\. In AmbiMath, for instance, the prior specifies which input parameter is relevant, while the arithmetic operation must still be learned from the training distribution\. In real tasks, analogous priors may specify invariances, domain constraints, or misleading heuristics to avoid\. Thus, PGT provides a lightweight interface for incorporating task\-level knowledge into low\-resource training without requiring additional labeled examples for each instance\.

### 4\.2Contrastive Prior Steering

![Refer to caption](https://arxiv.org/html/2609.02244v1/Final-CPS.png)

Figure 2:Illustration of Contrastive Prior Steering \(CPS\) on AmbiMath\. CPS keeps the original training view and constructs two contrastive auxiliary views, turning task\-level natural\-language priors into learning signals\. The positive prior describes the intended task principle, while the negative prior describes a plausible but misleading alternative\. Their target\-token losses are combined with the original supervised loss\.We instantiate PGT withContrastive Prior Steering\(CPS\)\. CPS uses two task\-level priors: a positive priorp\+p\_\{\+\}that describes task\-consistent guidance, and a negative priorp−p\_\{\-\}that describes a misleading but plausible alternative\. Figure[2](https://arxiv.org/html/2609.02244#S4.F2)illustrates that for each labeled example\(Ii,xi,yi\)\(I\_\{i\},x\_\{i\},y\_\{i\}\), whereIiI\_\{i\}denotes the task instruction or prompt text, CPS constructs three views: the original example, the example conditioned on the positive prior, and the example conditioned on the negative prior\. We compute cross\-entropy only on the target tokensyiy\_\{i\}\. The original supervised loss is

ℒ0=ℓ⁡\(fθ​\(\[Ii;xi\]\),yi\)\.\\mathcal\{L\}\_\{0\}=\\ell\\\!\\left\(f\_\{\\theta\}\(\[I\_\{i\};x\_\{i\}\]\),y\_\{i\}\\right\)\.\(4\)The positive\-prior and negative\-prior losses are

ℒ\+=ℓ⁡\(fθ​\(\[p\+;Ii;xi\]\),yi\),ℒ−=ℓ⁡\(fθ​\(\[p−;Ii;xi\]\),yi\)\.\\mathcal\{L\}\_\{\+\}=\\ell\\\!\\left\(f\_\{\\theta\}\(\[p\_\{\+\};I\_\{i\};x\_\{i\}\]\),y\_\{i\}\\right\),\\qquad\\mathcal\{L\}\_\{\-\}=\\ell\\\!\\left\(f\_\{\\theta\}\(\[p\_\{\-\};I\_\{i\};x\_\{i\}\]\),y\_\{i\}\\right\)\.\(5\)CPS then optimizes

ℒCPS=ℒ0\+λ⁡\(ℒ\+−γ​ℒ−\),\\mathcal\{L\}\_\{\\mathrm\{CPS\}\}=\\mathcal\{L\}\_\{0\}\+\\lambda\\left\(\\mathcal\{L\}\_\{\+\}\-\\gamma\\mathcal\{L\}\_\{\-\}\\right\),\(6\)whereλ\\lambdabalances the prior\-induced objective with the original supervised objective, andγ\\gammacontrols the strength of the negative prior term\. This objective has two complementary effects\. The positive prior termℒ\+\\mathcal\{L\}\_\{\+\}encourages the model to predict the target when the input is paired with task\-consistent guidance\. The negative prior termℒ−\\mathcal\{L\}\_\{\-\}discourages fitting the target under a misleading prior, which is useful when scarce data alone do not rule out that alternative\. The original lossℒ0\\mathcal\{L\}\_\{0\}anchors training to the observed task distribution, so the prior objective guides the solution principle while preserving the supervised task signal\.

### 4\.3Training Procedure

CPS differs from standard prompt\-based fine\-tuning only in how the training objective is formed: instead of optimizing a single supervised view, CPS forms prior\-conditioned auxiliary views and combines their target\-token losses with the original supervised loss\. In implementation, we compute loss only on target tokens and apply global gradient\-norm clipping before the optimizer update\. The detailed training procedure is provided in Algorithm[1](https://arxiv.org/html/2609.02244#alg1)in the Appendix\.

## 5Results

Experiment Details\.We evaluate CPS as an implementation of PGT on three complementary settings\. First, AmbiMath tests whether task\-level priors can resolve a controlled ambiguity where low\-resource data are compatible with multiple solution rules\. Second, Jigsaw tests low\-resource toxicity classification in real user comments, where identity terms can be spuriously associated with toxicity labels\. Third, MNLI/HANS evaluates heuristic robustness in NLI, where models often rely on shallow lexical or syntactic heuristics\.

Together, these settings evaluate whether task\-level natural\-language priors can provide useful auxiliary learning signals for low\-resource LLM training across a synthetic ambiguity benchmark, real\-world identity\-related toxicity classification, and NLI heuristic robustness\. We compare plain fine\-tuning, prompt fine\-tuning, and CPS\. Plain fine\-tuning optimizes only task data\. Prompt fine\-tuning treats the same prior as input context\. CPS uses the prior to construct auxiliary learning signals\. We evaluate instruction\-tuned LLMs including LLaMA 3\.1 8B\([AI@Meta, 2024](https://arxiv.org/html/2609.02244#bib.bib11)\)and Qwen 2\.5 7B\([Qwen Team, 2025](https://arxiv.org/html/2609.02244#bib.bib1)\); the model used in each experiment is specified in the corresponding table\. The main tables use the same prior\-free inference inputs across methods to test whether training has changed the model behavior itself; Appendix Table[11](https://arxiv.org/html/2609.02244#A1.T11)reports a prompt\-retained diagnostic on AmbiMath\. We compute loss only on response tokens and use model\-specific chat templates consistently across methods\.

Table 1:Exact\-match \(EM\) accuracy \(%\) on AmbiMath using LLaMA 3\.1 8B\. Columns correspond to different task\-level priors specifying the relevant input parameter\.AmbiMath: Synthetic Benchmark for Rule Ambiguity\.Table[1](https://arxiv.org/html/2609.02244#S5.T1)shows that plain fine\-tuning and prompt fine\-tuning are unreliable when the training data do not distinguish competing rules\. Prompt fine\-tuning can change model behavior, but treating the prior as input context does not consistently bind the model to the intended rule\. CPS reaches 96\.3–100\.0 EM across both positional and semantic priors, while plain and prompt fine\-tuning remain substantially lower and less stable\. This result shows that CPS effectively instantiates the PGT idea: task\-level natural\-language priors can be converted into auxiliary learning signals that help resolve ambiguity in low\-resource scenarios\.

Table 2:Test accuracy and F1 scores on the Jigsaw toxicity subset by gender group\. F1\+ and F1\- denote F1 for toxic and non\-toxic labels, respectively\.Jigsaw: Low\-resource Toxicity Classification\.We next evaluate CPS on the Jigsaw toxicity dataset\([Jigsaw, 2019](https://arxiv.org/html/2609.02244#bib.bib12)\), which contains user comments annotated with toxicity labels and identity terms\. We focus on gender\-related identity groups and train with a low\-resource subset, creating a realistic setting where models may learn unreliable associations between identity mentions and toxicity labels\. Table[2](https://arxiv.org/html/2609.02244#S5.T2)shows that plain fine\-tuning achieves relatively high accuracy, but its F1\+ is substantially lower than F1\-, indicating that the model struggles to identify toxic comments in these identity\-related subsets\. Prompt fine\-tuning provides limited and inconsistent gains, suggesting that exposing the model to the prior as input context does not reliably change the learned classifier\. We also include a DPO\-style baseline for this binary classification task by pairing the correct label verbalization with an incorrect label verbalization as a contrastive comparison\. DPO\-style training improves some groups, but its gains are less consistent than CPS\. Relative to plain and prompt fine\-tuning, CPS consistently improves F1\+ and Macro F1 across identity groups for both LLaMA and Qwen\. Compared with the DPO\-style baseline, CPS is stronger in most groups and more consistent across model families\. These results suggest that task\-level priors help the model use toxicity\-relevant evidence more effectively in identity\-related comments\.

Table 3:Accuracy \(%\) on MNLI and HANS\. MNLI\-m/mm denote matched and mismatched MNLI validation splits\. HANS Ent\. and HANS N\-Ent\. denote the entailment and non\-entailment subsets of HANS; HANS N\-Ent\. is the key robustness metric for shallow heuristic reliance\.MNLI/HANS: Heuristic Robustness in NLI\.Natural language inference \(NLI\) is a sentence\-pair classification task: given a premise and a hypothesis, the model predicts whether the hypothesis is entailed, contradicted, or neutral with respect to the premise\. In our HANS\-compatible evaluation, contradiction and neutral cases are grouped as non\-entailment\. We fine\-tune models on a low\-resource MNLI subset and evaluate both in\-domain MNLI accuracy and diagnostic robustness on HANS\. HANS is constructed to test shallow NLI heuristics such as lexical overlap, subsequence matching, and constituent matching\. Its non\-entailment subset is especially important because heuristic\-based models often incorrectly predict entailment on these examples\. We use HANS only as a diagnostic evaluation set, not for model selection\. As shown in Table[3](https://arxiv.org/html/2609.02244#S5.T3), CPS improves HANS accuracy for both LLaMA and Qwen, with gains concentrated on HANS non\-entailment examples\. CPS improves non\-entailment accuracy from 39\.2% to 47\.5% on LLaMA and from 45\.1% to 50\.3% on Qwen, while maintaining comparable MNLI matched and mismatched accuracy\. These results suggest that task\-level priors can guide low\-resource learning toward more robust decision principles in a different task family, where the misleading rule takes the form of shallow lexical or syntactic heuristics\.

## 6Discussion

The experiments above evaluate CPS as a concrete implementation of PGT\. Here we analyze what the results imply beyond the headline accuracies: whether task\-level priors improve data efficiency, whether their effect is visible early in optimization, and whether the method depends on a single prior wording or on only one side of the contrastive objective\.

#### Data efficiency\.

A central motivation for using task\-level priors is that many low\-resource settings have limited labeled data, while useful task\-level knowledge may already be available in natural language\. CPS provides a way to use such knowledge without collecting additional instance\-level annotations\. We therefore conduct a scaling analysis on the Jigsaw gender\-associated subset, comparing CPS with plain fine\-tuning when using 1/30, 1/10, 1/3, and all of the training set\. As shown in Figure[3](https://arxiv.org/html/2609.02244#S6.F3), CPS consistently improves both accuracy and Macro F1 across data scales\. The advantage is largest in the lowest\-data regime: with only 1/30 of the experimental training set, CPS reaches roughly 0\.71 Macro F1, while plain fine\-tuning remains below 0\.50\. More importantly, CPS with only 1/10 of the data achieves 0\.767 Macro F1, slightly exceeding plain fine\-tuning trained on the full experimental training set \(0\.761\)\. This result directly supports the low\-resource motivation of PGT: task\-level priors do not replace task data, but they can make a small amount of data substantially more useful when relevant task guidance can be stated in natural language\. Appendix Table[12](https://arxiv.org/html/2609.02244#A1.T12)reports the corresponding cost profile\.

Figure 3:Accuracy and Macro F1 of CPS and plain fine\-tuning across data scales on the Jigsaw gender\-associated subset\. Fractions are relative to the experimental training set used in the Jigsaw study; shaded regions denote standard deviation over three random seeds\.
#### Early\-training optimization diagnostic\.

The empirical gains of CPS suggest that priors affect training rather than merely serving as additional context\. We test this on separately trained plain, prompt, and CPS trajectories in the first\-parameter AmbiMath setting, where the training examples are ambiguous but the disentangled diagnostic examples separate the target rule from the competing rule\. At each checkpoint along an actual training trajectory, we compute the method\-specific update directionggby averaging the gradient of that method’s training objective over all 200 AmbiMath training examples\. For plain fine\-tuning, this is the supervised gradient on the original examples; for prompt fine\-tuning, it is the supervised gradient on prior\-prefixed examples; for CPS, it is the gradient of the full CPS objective\.

We then compute two reference directions on the full disentangled diagnostic set\. The target\-rule referencegref\+g\_\{\\mathrm\{ref\}\+\}is the supervised gradient obtained when the answer follows the intended first\-parameter rule\. The competing\-rule referencegref−g\_\{\\mathrm\{ref\}\-\}is computed on the same inputs but with answers following the alternative second\-parameter rule\. Because the diagnostic inputs are constructed so that these two answers differ, these references provide two separable directions for the two plausible solution principles\. We report

M=cos⁡\(g,gref\+\)−cos⁡\(g,gref−\),M=\\cos\(g,g\_\{\\mathrm\{ref\}\+\}\)\-\\cos\(g,g\_\{\\mathrm\{ref\}\-\}\),\(7\)where larger values indicate that the current update direction is closer to the intended rule and farther from the competing rule\. Figure[4](https://arxiv.org/html/2609.02244#S6.F4)focuses on the first epoch, where the choice of solution principle is most decisive\. CPS rapidly changes both the update direction and the learned behavior: from 0\.2 to 1\.0 epoch, its alignment margin increases from−0\.094\-0\.094to0\.9660\.966, while exact\-match accuracy rises from 0\.0% to 94\.5%\. At the same 1\.0\-epoch checkpoint, plain fine\-tuning reaches only 1\.5% exact match with a negative alignment margin \(−0\.278\-0\.278\), and prompt fine\-tuning reaches 2\.5% exact match with a much smaller margin \(0\.1920\.192\)\. This diagnostic is not intended as a full mechanistic proof\. Rather, it provides quantitative evidence for the PGT interpretation: CPS can make a natural\-language prior affect the early training direction, precisely when low\-resource models are most vulnerable to committing to an unintended solution rule\.

Figure 4:Early\-training diagnostic on AmbiMath\. During the first epoch, CPS rapidly improves both the alignment marginMMand EM accuracy, while two baseline methods remain near zero accuracy\.
#### Prior formulation and component ablations\.

A practical concern is whether CPS works only for one carefully written prior or whether a one\-sided objective would already be sufficient\. Table[4](https://arxiv.org/html/2609.02244#S6.T4)addresses both questions on AmbiMath\. Positive\-Only CPS removes the negative\-prior term, while Negative\-Only CPS removes the positive\-prior term\. Both one\-sided variants improve the average score over plain and prompt fine\-tuning, but their gains are uneven across prior types; full CPS is substantially more reliable across all four prior types\. CPS also remains strong when the same task principle is expressed with different prior formulations: across the three prior variants in Table[4](https://arxiv.org/html/2609.02244#S6.T4), performance remains far above plain and prompt fine\-tuning, with most scores above 88% exact match and several near 100%\. This suggests that CPS is not merely exploiting a single prompt template; what matters is that the prior expresses the correct task\-level principle\. Additional appendix experiments test alternative negative\-prior formulations \(Table[9](https://arxiv.org/html/2609.02244#A1.T9)\), prompt\-retained inference \(Table[11](https://arxiv.org/html/2609.02244#A1.T11)\), CoT\-SFT diagnostics \(Appendix[A\.4](https://arxiv.org/html/2609.02244#A1.SS4)\), and operation\-specifying priors \(Table[5](https://arxiv.org/html/2609.02244#A1.T5)\), supporting the same conclusion: the gains are not explained by one handcrafted wording, by simply reintroducing the prior at test time, or by replacing the prior objective with generated rationale supervision\.

Table 4:Ablation and prior formulation results on AmbiMath using LLaMA 3\.1 8B\. Positive\-Only and Negative\-Only CPS remove one side of the contrastive prior objective\. Prior variants are manually rewritten positive and negative priors with the same task\-level meaning\.

## 7Conclusion

We introducedPrior\-Guided Tuning\(PGT\), a training perspective that uses task\-level natural\-language priors as auxiliary learning signals for low\-resource LLM training, and instantiated it withContrastive Prior Steering\(CPS\)\. The central idea is simple: when low\-resource training data do not provide enough evidence to determine how a task should be solved, task\-level priors can complement the data by supplying guidance about the intended solution principle\. Across AmbiMath, Jigsaw, and MNLI/HANS, CPS improves over plain and prompt fine\-tuning in settings involving synthetic rule ambiguity, identity\-related toxicity classification, and shallow NLI heuristics\.

Beyond the main performance gains, our data\-efficiency study shows that CPS can make limited supervision substantially more useful, while the early\-training diagnostic suggests that CPS affects the optimization trajectory at the stage where models first commit to a solution rule\. Ablations further show that both positive and negative priors contribute to robust ambiguity resolution, and that the method is not tied to a single prior wording\. Taken together, these results support a focused claim: task\-level natural\-language priors can provide useful guidance as auxiliary learning signals for low\-resource LLM training\. CPS offers one lightweight way to operationalize this idea without requiring instance\-level rationales or preference pairs\. The present work focuses on how to use a given task\-level prior during training; scaling the acquisition, validation, and automatic refinement of such priors remains an important direction for future work\.

## References

- AI@Meta \(2024\)AI@MetaThe llama 3 herd of models\.CoRRabs/2407\.21783\.External Links:[Link](https://doi.org/10.48550/arXiv.2407.21783),[Document](https://dx.doi.org/10.48550/ARXIV.2407.21783),2407\.21783Cited by:[§5](https://arxiv.org/html/2609.02244#S5.p2.1)\.
- Brownet al\.\(2020\)T\. B\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell, S\. Agarwal, A\. Herbert\-Voss, G\. Krueger, T\. Henighan, R\. Child, A\. Ramesh, D\. M\. Ziegler, J\. Wu, C\. Winter, C\. Hesse, M\. Chen, E\. Sigler, M\. Litwin, S\. Gray, B\. Chess, J\. Clark, C\. Berner, S\. McCandlish, A\. Radford, I\. Sutskever, and D\. AmodeiLanguage models are few\-shot learners\.InAdvances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6\-12, 2020, virtual,H\. Larochelle, M\. Ranzato, R\. Hadsell, M\. Balcan, and H\. Lin \(Eds\.\),External Links:[Link](https://proceedings.neurips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html)Cited by:[§2\.1](https://arxiv.org/html/2609.02244#S2.SS1.p1.1)\.
- D’Amouret al\.\(2022\)A\. D’Amour, K\. A\. Heller, D\. Moldovan, B\. Adlam, B\. Alipanahi, A\. Beutel, C\. Chen, J\. Deaton, J\. Eisenstein, M\. D\. Hoffman, F\. Hormozdiari, N\. Houlsby, S\. Hou, G\. Jerfel, A\. Karthikesalingam, M\. Lucic, Y\. Ma, C\. Y\. McLean, D\. Mincu, A\. Mitani, A\. Montanari, Z\. Nado, V\. Natarajan, C\. Nielson, T\. F\. Osborne, R\. Raman, K\. Ramasamy, R\. Sayres, J\. Schrouff, M\. Seneviratne, S\. Sequeira, H\. Suresh, V\. Veitch, M\. Vladymyrov, X\. Wang, K\. Webster, S\. Yadlowsky, T\. Yun, X\. Zhai, and D\. SculleyUnderspecification presents challenges for credibility in modern machine learning\.J\. Mach\. Learn\. Res\.23,pp\. 226:1–226:61\.External Links:[Link](https://jmlr.org/papers/v23/20-1335.html)Cited by:[§3\.1](https://arxiv.org/html/2609.02244#S3.SS1.p1.1)\.
- Gururanganet al\.\(2020\)S\. Gururangan, A\. Marasovic, S\. Swayamdipta, K\. Lo, I\. Beltagy, D\. Downey, and N\. A\. SmithDon’t stop pretraining: adapt language models to domains and tasks\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5\-10, 2020,D\. Jurafsky, J\. Chai, N\. Schluter, and J\. R\. Tetreault \(Eds\.\),pp\. 8342–8360\.External Links:[Link](https://doi.org/10.18653/v1/2020.acl-main.740),[Document](https://dx.doi.org/10.18653/V1/2020.ACL-MAIN.740)Cited by:[§1](https://arxiv.org/html/2609.02244#S1.p1.1)\.
- Huet al\.\(2022\)S\. Hu, N\. Ding, H\. Wang, Z\. Liu, J\. Wang, J\. Li, W\. Wu, and M\. SunKnowledgeable prompt\-tuning: incorporating knowledge into prompt verbalizer for text classification\.InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\), ACL 2022, Dublin, Ireland, May 22\-27, 2022,S\. Muresan, P\. Nakov, and A\. Villavicencio \(Eds\.\),pp\. 2225–2240\.External Links:[Link](https://doi.org/10.18653/v1/2022.acl-long.158),[Document](https://dx.doi.org/10.18653/V1/2022.ACL-LONG.158)Cited by:[§2\.3](https://arxiv.org/html/2609.02244#S2.SS3.p1.1)\.
- Jeong \(2024\)C\. JeongFine\-tuning and utilization methods of domain\-specific llms\.CoRRabs/2401\.02981\.External Links:[Link](https://doi.org/10.48550/arXiv.2401.02981),[Document](https://dx.doi.org/10.48550/ARXIV.2401.02981),2401\.02981Cited by:[§1](https://arxiv.org/html/2609.02244#S1.p1.1)\.
- Jigsaw \(2019\)JigsawJigsaw unintended bias in toxicity classification\.Note:Kaggle competition datasetExternal Links:[Link](https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification)Cited by:[3rd item](https://arxiv.org/html/2609.02244#S1.I1.i3.p1.1),[§5](https://arxiv.org/html/2609.02244#S5.p4.1)\.
- Kaufmannet al\.\(2025\)T\. Kaufmann, P\. Weng, V\. Bengs, and E\. HüllermeierA survey of reinforcement learning from human feedback\.Trans\. Mach\. Learn\. Res\.\.External Links:[Link](https://openreview.net/forum?id=f7OkIurx4b)Cited by:[§2\.2](https://arxiv.org/html/2609.02244#S2.SS2.p1.1)\.
- Lesteret al\.\(2021\)B\. Lester, R\. Al\-Rfou, and N\. ConstantThe power of scale for parameter\-efficient prompt tuning\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7\-11 November, 2021,M\. Moens, X\. Huang, L\. Specia, and S\. W\. Yih \(Eds\.\),pp\. 3045–3059\.External Links:[Link](https://doi.org/10.18653/v1/2021.emnlp-main.243),[Document](https://dx.doi.org/10.18653/V1/2021.EMNLP-MAIN.243)Cited by:[§2\.1](https://arxiv.org/html/2609.02244#S2.SS1.p1.1)\.
- Liuet al\.\(2020\)W\. Liu, P\. Zhou, Z\. Zhao, Z\. Wang, Q\. Ju, H\. Deng, and P\. WangK\-BERT: enabling language representation with knowledge graph\.InThe Thirty\-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty\-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7\-12, 2020,pp\. 2901–2908\.External Links:[Link](https://doi.org/10.1609/aaai.v34i03.5681),[Document](https://dx.doi.org/10.1609/AAAI.V34I03.5681)Cited by:[§2\.3](https://arxiv.org/html/2609.02244#S2.SS3.p1.1)\.
- McCoyet al\.\(2019\)R\. T\. McCoy, E\. Pavlick, and T\. LinzenRight for the wrong reasons: diagnosing syntactic heuristics in natural language inference\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,pp\. 3428–3448\.External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1334),[Link](https://doi.org/10.18653/v1/P19-1334)Cited by:[3rd item](https://arxiv.org/html/2609.02244#S1.I1.i3.p1.1)\.
- Qwen Team \(2025\)Qwen TeamQwen2\.5 technical report\.External Links:2412\.15115,[Link](https://arxiv.org/abs/2412.15115)Cited by:[§5](https://arxiv.org/html/2609.02244#S5.p2.1)\.
- Rafailovet al\.\(2023\)R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. FinnDirect preference optimization: your language model is secretly a reward model\.InAdvances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 \- 16, 2023,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),External Links:[Link](http://papers.nips.cc/paper/_files/paper/2023/hash/a85b405ed65c6477a4fe8302b5e06ce7-Abstract-Conference.html)Cited by:[§2\.2](https://arxiv.org/html/2609.02244#S2.SS2.p1.1)\.
- Sanhet al\.\(2022\)V\. Sanh, A\. Webson, C\. Raffel, S\. H\. Bach, L\. Sutawika, Z\. Alyafeai, A\. Chaffin, A\. Stiegler, A\. Raja, M\. Dey, M\. S\. Bari, C\. Xu, U\. Thakker, S\. S\. Sharma, E\. Szczechla, T\. Kim, G\. Chhablani, N\. V\. Nayak, D\. Datta, J\. Chang, M\. T\. Jiang, H\. Wang, M\. Manica, S\. Shen, Z\. X\. Yong, H\. Pandey, R\. Bawden, T\. Wang, T\. Neeraj, J\. Rozen, A\. Sharma, A\. Santilli, T\. Févry, J\. A\. Fries, R\. Teehan, T\. L\. Scao, S\. Biderman, L\. Gao, T\. Wolf, and A\. M\. RushMultitask prompted training enables zero\-shot task generalization\.InThe Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25\-29, 2022,External Links:[Link](https://openreview.net/forum?id=9Vrb9D0WI4)Cited by:[§2\.1](https://arxiv.org/html/2609.02244#S2.SS1.p1.1)\.
- Sunet al\.\(2019\)Y\. Sun, S\. Wang, Y\. Li, S\. Feng, X\. Chen, H\. Zhang, X\. Tian, D\. Zhu, H\. Tian, and H\. WuERNIE: enhanced representation through knowledge integration\.CoRRabs/1904\.09223\.External Links:[Link](http://arxiv.org/abs/1904.09223),1904\.09223Cited by:[§2\.3](https://arxiv.org/html/2609.02244#S2.SS3.p1.1)\.
- Wanget al\.\(2022\)Y\. Wang, S\. Mishra, P\. Alipoormolabashi, Y\. Kordi, A\. Mirzaei, A\. Naik, A\. Ashok, A\. S\. Dhanasekaran, A\. Arunkumar, D\. Stap, E\. Pathak, G\. Karamanolakis, H\. G\. Lai, I\. Purohit, I\. Mondal, J\. Anderson, K\. Kuznia, K\. Doshi, K\. K\. Pal, M\. Patel, M\. Moradshahi, M\. Parmar, M\. Purohit, N\. Varshney, P\. R\. Kaza, P\. Verma, R\. S\. Puri, R\. Karia, S\. Doshi, S\. K\. Sampat, S\. Mishra, S\. R\. A, S\. Patro, T\. Dixit, and X\. ShenSuper\-naturalinstructions: generalization via declarative instructions on 1600\+ NLP tasks\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7\-11, 2022,Y\. Goldberg, Z\. Kozareva, and Y\. Zhang \(Eds\.\),pp\. 5085–5109\.External Links:[Link](https://doi.org/10.18653/v1/2022.emnlp-main.340),[Document](https://dx.doi.org/10.18653/V1/2022.EMNLP-MAIN.340)Cited by:[§2\.1](https://arxiv.org/html/2609.02244#S2.SS1.p1.1)\.
- Weiet al\.\(2022\)J\. Wei, M\. Bosma, V\. Y\. Zhao, K\. Guu, A\. W\. Yu, B\. Lester, N\. Du, A\. M\. Dai, and Q\. V\. LeFinetuned language models are zero\-shot learners\.InThe Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25\-29, 2022,External Links:[Link](https://openreview.net/forum?id=gEZrGCozdqR)Cited by:[§1](https://arxiv.org/html/2609.02244#S1.p2.1),[§2\.1](https://arxiv.org/html/2609.02244#S2.SS1.p1.1)\.
- Williamset al\.\(2018\)A\. Williams, N\. Nangia, and S\. R\. BowmanA broad\-coverage challenge corpus for sentence understanding through inference\.InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 \(Long Papers\),pp\. 1112–1122\.External Links:[Document](https://dx.doi.org/10.18653/v1/N18-1101),[Link](https://doi.org/10.18653/v1/N18-1101)Cited by:[3rd item](https://arxiv.org/html/2609.02244#S1.I1.i3.p1.1)\.
- Zhonget al\.\(2025\)Q\. Zhong, L\. Ding, X\. Cai, J\. Liu, B\. Du, and D\. TaoKaFT: knowledge\-aware fine\-tuning for boosting llms’ domain\-specific question\-answering performance\.External Links:2505\.15480,[Link](https://arxiv.org/abs/2505.15480)Cited by:[§2\.3](https://arxiv.org/html/2609.02244#S2.SS3.p1.1)\.

## Appendix AAdditional Details

### A\.1Training Procedure and Implementation Details

For all fine\-tuning methods, we compute loss only on assistant response tokens\. User instructions, system prompts, and prior text are included in the input sequence when applicable, but their tokens are masked out of the supervised loss\. We use model\-specific chat templates for LLaMA 3\.1 and Qwen 2\.5 and keep the template fixed across plain fine\-tuning, prompt fine\-tuning, and CPS\.

For each training example\(Ii,xi,yi\)\(I\_\{i\},x\_\{i\},y\_\{i\}\), CPS constructs three views: the original example, a positive\-prior view, and a negative\-prior view\. The original supervised loss isℒ0\\mathcal\{L\}\_\{0\}, whileℒ\+\\mathcal\{L\}\_\{\+\}andℒ−\\mathcal\{L\}\_\{\-\}are the target\-token losses under the positive and negative priors\. The training objective is

ℒCPS=ℒ0\+λ⁡\(ℒ\+−γ​ℒ−\)\.\\mathcal\{L\}\_\{\\mathrm\{CPS\}\}=\\mathcal\{L\}\_\{0\}\+\\lambda\\left\(\\mathcal\{L\}\_\{\+\}\-\\gamma\\mathcal\{L\}\_\{\-\}\\right\)\.\(8\)We use global gradient\-norm clipping before the optimizer update:

𝐠~=𝐠⋅min⁡\(1,τ‖𝐠‖2\),𝐠=∇θℒCPS,\\tilde\{\\mathbf\{g\}\}=\\mathbf\{g\}\\cdot\\min\\left\(1,\\frac\{\\tau\}\{\\\|\\mathbf\{g\}\\\|\_\{2\}\}\\right\),\\quad\\mathbf\{g\}=\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\mathrm\{CPS\}\},\(9\)withτ=1\.0\\tau=1\.0in our experiments\. This is a standard optimization stabilizer and is applied after the combined objective is differentiated\.

Algorithm 1Contrastive Prior Steering1:Input:training set

𝒟\\mathcal\{D\}, priors

p\+,p−p\_\{\+\},p\_\{\-\}, model

fθf\_\{\\theta\}, learning rate

η\\eta, hyperparameters

λ,γ,τ\\lambda,\\gamma,\\tau
2:whilenot convergeddo

3:Sample a batch

\(I,x,y\)∼𝒟\(I,x,y\)\\sim\\mathcal\{D\}
4:Compute

ℒ0=ℓ⁡\(fθ​\(\[I;x\]\),y\)\\mathcal\{L\}\_\{0\}=\\ell\(f\_\{\\theta\}\(\[I;x\]\),y\)
5:Compute

ℒ\+=ℓ⁡\(fθ​\(\[p\+;I;x\]\),y\)\\mathcal\{L\}\_\{\+\}=\\ell\(f\_\{\\theta\}\(\[p\_\{\+\};I;x\]\),y\)
6:Compute

ℒ−=ℓ⁡\(fθ​\(\[p−;I;x\]\),y\)\\mathcal\{L\}\_\{\-\}=\\ell\(f\_\{\\theta\}\(\[p\_\{\-\};I;x\]\),y\)
7:

ℒCPS←ℒ0\+λ⁡\(ℒ\+−γ​ℒ−\)\\mathcal\{L\}\_\{\\mathrm\{CPS\}\}\\leftarrow\\mathcal\{L\}\_\{0\}\+\\lambda\(\\mathcal\{L\}\_\{\+\}\-\\gamma\\mathcal\{L\}\_\{\-\}\)
8:

𝐠←∇θℒCPS\\mathbf\{g\}\\leftarrow\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\mathrm\{CPS\}\}
9:

𝐠~←𝐠⋅min⁡\(1,τ/‖𝐠‖2\)\\tilde\{\\mathbf\{g\}\}\\leftarrow\\mathbf\{g\}\\cdot\\min\(1,\\tau/\\\|\\mathbf\{g\}\\\|\_\{2\}\)
10:Update

θ\\thetausing the optimizer with clipped gradient

𝐠~\\tilde\{\\mathbf\{g\}\}
11:endwhile

12:return

θ\\theta

Unless otherwise stated, we train LoRA adapters with AdamW\. We use rankr=8r=8, LoRA scalingα=16\\alpha=16, dropout0\.00\.0, and adapt the attention and MLP projection modulesq\_proj,k\_proj,v\_proj,o\_proj,gate\_proj,up\_proj, anddown\_proj\. We search learning rates in\{1×10−4,3×10−4,5×10−4\}\\\{1\{\\times\}10^\{\-4\},3\{\\times\}10^\{\-4\},5\{\\times\}10^\{\-4\}\\\}for all methods\. For CPS, we useγ=0\.1\\gamma=0\.1in the main experiments and searchλ\\lambdain\{0\.3,0\.5,1\.0\}\\\{0\.3,0\.5,1\.0\\\}\. The main AmbiMath results use a fixed training recipe before evaluation on the disentangled test sets; no disentangled validation examples are used to select the task rule or choose checkpoints\. For MNLI/HANS, HANS is used only for diagnostic evaluation and is not used for model selection\.

### A\.2Dataset and Prior Details

#### AmbiMath\.

AmbiMath contains function\-computation examples of the formf⁡\(x1,x2\)=yf\(x\_\{1\},x\_\{2\}\)=y\. In the main setting, training examples satisfyx1=y−2x\_\{1\}=y\-2andx2=y\+2x\_\{2\}=y\+2, so bothx1\+2x\_\{1\}\+2andx2−2x\_\{2\}\-2are consistent with all training labels\. Evaluation examples are disentangled so thatx1\+2≠x2−2x\_\{1\}\+2\\neq x\_\{2\}\-2\. Exact match is computed on the final numeric answer\. The main positive and negative priors for the first\-parameter setting are:

> Positive prior: The output of func should be derived from its first input parameter, ignoring the other\.

> Negative prior: The output of func should be derived from its second input parameter, ignoring the other\.

For the other AmbiMath columns, we replace “first” with “second”, “Chinese”, or “English” as appropriate\.

#### Jigsaw\.

For Jigsaw, we use gender\-associated toxicity examples and evaluate on gender\-related subsets\. The positive prior emphasizes toxicity\-relevant evidence rather than identity mentions, while the negative prior describes an identity\-term heuristic:

> Positive prior: You are a toxicity classification assistant\. Judge whether the comment is toxic based on its context and actual meaning, not merely on identity\-related terms\. Mentions of gender or other identity terms are not toxic by themselves\. If it is toxic, output 1; otherwise output 0\.

> Negative prior: Use identity\-related terms as the main signal\. If the comment mentions a gender or identity group, predict toxic; otherwise predict non\-toxic\.

#### MNLI/HANS\.

For MNLI/HANS, we train on a low\-resource subset of 1,000 MNLI training examples and evaluate on 1,200\-example MNLI matched and mismatched validation subsets, as well as the full 30,000\-example HANS diagnostic set\. Neutral and contradiction are grouped as non\-entailment to match the HANS binary setup\. The positive and negative priors are:

> Positive prior: Use the full meaning of the premise and hypothesis\. Do not predict entailment merely because the hypothesis words appear in the premise, or because the hypothesis is a subsequence or constituent of the premise\. Output entailment only when the premise logically guarantees the hypothesis; otherwise output non\-entailment\.

> Negative prior: Use shallow lexical heuristics\. If most hypothesis words appear in the premise, or the hypothesis is a subsequence or constituent of the premise, output entailment; otherwise output non\-entailment\.

#### DPO\-style baseline on Jigsaw\.

For the Jigsaw comparison, we additionally include a DPO\-style baseline because the binary labels make it possible to form simple preferred/rejected label pairs\. Each training input is paired with a preferred response containing the gold label and a rejected response containing the opposite label\. We train with TRL’s DPOTrainer using a PEFT reference\-model setup, whereref\_model=Nonelets the trainer use the base model with adapters disabled as the reference\. We search the DPOβ\\betaparameter over\{0\.1,0\.3,0\.5\}\\\{0\.1,0\.3,0\.5\\\}and use smaller learning rates than SFT, following common DPO practice\. This baseline is included as an empirical comparison because it also uses contrastive positive/negative label responses, but it differs from CPS in that it constructs instance\-level response preferences rather than task\-level natural\-language priors\.

### A\.3Additional AmbiMath Prior Variants

Table[5](https://arxiv.org/html/2609.02244#A1.T5)reports an additional AmbiMath setting that inverts the roles of data and priors\. In the main AmbiMath setting, the data reveal the arithmetic operation, while the prior specifies which input parameter should be used\. In this variant, the prior instead specifies the arithmetic operation, while the model must infer from the training distribution which parameter is consistent with that operation\. The training examples remain ambiguous in the same spirit as the main benchmark: the two input parameters are constructed so that both can produce the observed output under different arithmetic mappings, and the ambiguity is resolved only on disentangled test examples\.

Concretely, suppose a training example isfunc\(13, 17\)=15\. If the prior says that “the output offuncshould be obtained by applying\+2\+2to the correct input parameter,” the prior does not tell the model whether the first or second parameter is correct\. The model must still learn from the data that13\+2=1513\+2=15, so the first parameter is the one consistent with the prior\-specified operation\. Conversely, if the prior specifies a−2\-2operation, the same example supports the second parameter because17−2=1517\-2=15\. Thus, the prior supplies the operation, while the data distribution supplies the parameter choice\. At test time, we evaluate on disentangled examples where applying the prior\-specified operation to the two parameters yields different answers, so exact\-match accuracy reflects whether the model has combined the natural\-language prior with the learned data pattern\.

This setting checks that CPS is not limited to priors about parameter selection\. The results are consistent with the main AmbiMath results: CPS remains substantially stronger than plain and prompt fine\-tuning when the natural\-language prior supplies a different component of the task\. The released code includes the exact data\-generation script for this variant\.

Table 5:AmbiMath results when the prior specifies the arithmetic operation rather than the relevant input parameter\.
### A\.4CoT\-SFT Diagnostics

A natural question is whether explicit chain\-of\-thought \(CoT\) supervision can serve the same role as CPS by converting a task\-level prior into instance\-level rationales\. We treat this as a diagnostic rather than a broad claim about all CoT methods\. The comparison is useful because CoT\-SFT uses the prior in a very different way: the prior is first used to generate reasoning traces, and the student model then learns from those traces\. CPS instead keeps the original answer\-supervised objective and uses the prior as an auxiliary learning signal during optimization\.

We first evaluate CoT\-SFT on the same Jigsaw setting used in the main experiments\. In the no\-label setting, a strong instruction\-following LLM receives the task\-level prior and raw comment but not the gold toxicity label; it must generate both a rationale and a final label\. Even with up to three attempts, only 79\.9% of training examples yield a correct rationale/label pair\. We then train on the filtered correct CoT data\. We also evaluate a more favorable answer\-revealed setting, where the LLM is shown the gold label while generating the rationale\. As shown in Table[6](https://arxiv.org/html/2609.02244#A1.T6), both CoT\-SFT variants underperform CPS by a large margin\. This suggests that, when only task\-level guidance is available, forcing the prior through generated rationales can introduce a brittle intermediate supervision channel: erroneous rationales either reduce the usable training set through filtering or remain as noisy reasoning targets, while the final model still does not receive the prior as a live training signal\.

Table 6:CoT\-SFT diagnostic on the Jigsaw toxicity setting\. Metrics are percentages\. “No gold label” means the rationale generator sees only the task\-level prior and raw comment; “gold label revealed” gives the generator the answer when writing the rationale\. The CPS row averages the four gender categories reported in Table[2](https://arxiv.org/html/2609.02244#S5.T2)for LLaMA 3\.1 8B\.We also test CoT\-SFT on a harder AmbiMath construction where training examples satisfy2​a\+5=3​b−2=c2a\+5=3b\-2=c\. In the first\-input target setting, the held\-out answer is defined by2​a\+52a\+5; the second input remains a plausible alternative because both expressions agree on the training construction but are disentangled at test time\. Here, CoT rationales are generated with access to the gold answer, giving CoT\-SFT a favorable supervision format\. As shown in Table[7](https://arxiv.org/html/2609.02244#A1.T7), CoT\-SFT does not recover the target mapping, while CPS learns a substantial portion of the target rule under the same data construction\. This result is consistent with the Jigsaw diagnostic: explicitly asking the model to imitate generated reasoning is not necessarily a substitute for using the prior as an auxiliary learning signal, especially when the task still requires learning part of the mapping from the data distribution\.

Table 7:Diagnostic on a harder AmbiMath first\-input target setting with training rule2​a\+5=3​b−2=c2a\+5=3b\-2=c\. EM is computed against the intended first\-input rule2​a\+52a\+5on disentangled test examples\.
### A\.5Bounded Surrogate Diagnostic

CPS uses the negative prior by reducing the likelihood of the gold answer under a misleading prior\-conditioned view\. This makes the auxiliary contrastive term a direct training signal rather than a bounded surrogate\. In our intended low\-resource setting, we keep this objective controlled by retaining the original supervised loss, using a small negative\-prior coefficientγ=0\.1\\gamma=0\.1, and applying global gradient\-norm clipping\. Across the reported runs, we did not observe numerical divergence\.

We also tested whether a bounded surrogate could replace the direct negative\-prior term\. Specifically, we trained a variant of the form

ℒbnd=ℒ0\+λb​σ​\(ℒ\+−γb​ℒ−\),\\mathcal\{L\}\_\{\\mathrm\{bnd\}\}=\\mathcal\{L\}\_\{0\}\+\\lambda\_\{b\}\\,\\sigma\\\!\\left\(\\mathcal\{L\}\_\{\+\}\-\\gamma\_\{b\}\\mathcal\{L\}\_\{\-\}\\right\),\(10\)whereσ⁡\(⋅\)\\sigma\(\\cdot\)is the sigmoid function\. This keeps the auxiliary term bounded, but it can also saturate and provide a weaker corrective signal after the model begins separating the positive and negative prior views\. Table[8](https://arxiv.org/html/2609.02244#A1.T8)reports the best bounded run from our AmbiMath sweep\. The bounded variant improves some settings but is much less consistent than CPS, especially on the first\-parameter and Chinese\-prior conditions\. We therefore use the direct CPS objective in the main experiments, while viewing better bounded formulations as a useful direction for future work\.

Table 8:AmbiMath EM comparison between CPS and the best bounded surrogate run\. The bounded surrogate avoids direct negative\-loss subtraction but provides weaker and less consistent prior guidance\.
### A\.6Robustness to Negative Prior Formulation

The main Jigsaw experiments use the same positive prior shown in Appendix A\.2 and vary only the negative prior\. The default negative prior, Neg\-Prior1, describes the direct identity\-term shortcut: mention of a gender or identity group is treated as the main toxicity signal\. We also test two alternative ways of constructing a misleading negative signal\. Neg\-Prior2 replaces the direct shortcut with a vague subjective judgment based on rough associations with gender terms\. Neg\-Prior3 instead encourages under\-detection by asking the model to ignore toxic content unless the toxicity is explicit\. The concrete negative priors are:

> Neg\-Prior1 \(default\): Use identity\-related terms as the main signal\. If the comment mentions a gender or identity group, predict toxic; otherwise predict non\-toxic\.

> Neg\-Prior2: Judge toxicity based on a subjective impression\. If the text roughly feels associated with gender\-related groups, treat it as toxic; otherwise treat it as non\-toxic\.

> Neg\-Prior3: Be conservative about toxicity detection\. Unless the comment contains explicit toxic language, ignore potentially toxic content and predict non\-toxic\.

As shown in Table[9](https://arxiv.org/html/2609.02244#A1.T9), both alternative formulations retain strong performance, suggesting that CPS is not tied to a single handcrafted negative\-prior wording\.

Table 9:Jigsaw results with alternative negative priors for LLaMA 3\.1 8B\. F1\+ and F1\- denote toxic and non\-toxic labels\.
### A\.7Sensitivity to the Prior\-Loss Weight

We further sweep the CPS prior\-loss weight on the Jigsaw LLaMA 3\.1 8B setting while keeping other hyperparameters fixed\. Table[10](https://arxiv.org/html/2609.02244#A1.T10)shows that CPS has a useful operating range rather than a single narrow setting:λ=0\.3\\lambda=0\.3andλ=0\.5\\lambda=0\.5are nearly identical, andλ=1\.0\\lambda=1\.0remains close\. Very small or overly weak prior weighting is less effective, so we do not claim that CPS is hyperparameter\-free; the result instead suggests that the method does not rely on a fragile, cherry\-picked value\.

Table 10:Sensitivity to the CPS prior\-loss weight on Jigsaw with LLaMA 3\.1 8B\.
### A\.8Inference with and without Priors

Table[11](https://arxiv.org/html/2609.02244#A1.T11)compares AmbiMath evaluation with and without priors at inference time\. CPS remains strong when the prior is removed, while prompt fine\-tuning benefits more from reintroducing the prior at inference\. This supports the main paper’s distinction between using priors as training signals and using them only as input context\. We present this result as a diagnostic rather than as the main contribution\.

Table 11:AmbiMath exact\-match accuracy with and without priors during inference for LLaMA 3\.1 8B\.
### A\.9Compute Profile on Jigsaw

We profile the Jigsaw settings used for the data\-efficiency comparison with LLaMA 3\.1 8B\. Plain fine\-tuning at the 1/3 scale uses 6464 examples and 1616 optimization steps\. CPS at the 1/30 scale uses 646 examples and 162 optimization steps\. CPS is slower per step because it evaluates the original, positive\-prior, and negative\-prior views and the prior\-conditioned views are longer\. The relevant comparison here is therefore cost\-to\-target in the low\-resource regime, not per\-step cost\. In this profile, CPS reaches a higher Macro F1 than the larger plain fine\-tuning run with fewer optimization steps and lower measured wall\-clock training time\.

Table 12:Training cost profile on the Jigsaw data\-efficiency setting\. CPS is more expensive per step but uses fewer examples and optimization steps in this cost\-to\-target comparison\.
### A\.10Ethics and Broader Impacts

This work studies how to use task\-level natural\-language priors during low\-resource LLM training\. The main intended benefit is to reduce reliance on misleading correlations when task knowledge is available\. The same mechanism can also reflect errors in the supplied priors, so priors should be reviewed before deployment, especially in high\-stakes applications\. The Jigsaw experiments involve toxic or identity\-related language from a public benchmark; such content is used only for evaluation and does not express the authors’ views\. We do not collect private data or conduct new human\-subject studies\.

### A\.11Reproducibility Statement

We will release code and data needed to reproduce the main experiments, including AmbiMath generation scripts, training and evaluation scripts, prior prompts, model identifiers, and hyperparameter configurations\. All datasets used in the paper are public or generated synthetically\. Experiments were conducted by fine\-tuning pretrained language models on standard GPU servers; model\-specific chat templates and assistant\-only loss masking are used consistently across methods\.

### A\.12Use of Large Language Models

The authors used LLM\-based tools to assist with grammar checking, wording polish, and literature search during paper preparation\. The research questions, experimental design, implementation, result interpretation, and final text were reviewed and controlled by the authors\. LLM tools were not used to generate experimental results or make autonomous scientific decisions\.