Strategy-Induct: Task-Level Strategy Induction for Instruction Generation

# Task-Level Strategy Induction for Instruction Generation
Source: [https://arxiv.org/html/2605.20924](https://arxiv.org/html/2605.20924)
Po\-Chun Chen1Hen\-Hsen Huang2Hsin\-Hsi Chen1,3 1Department of Computer Science and Information Engineering, National Taiwan University, Taiwan 2Institute of Information Science, Academia Sinica, Taiwan 3AI Research Center \(AINTU\), National Taiwan University, Taiwan pcchen@nlg\.csie\.ntu\.edu\.tw, hhhuang@iis\.sinica\.edu\.tw, hhchen@ntu\.edu\.tw

###### Abstract

Designing effective task\-level prompts is crucial for improving the performance of Large Language Models \(LLMs\)\. While prior work on instruction induction demonstrates that LLMs can infer better instructions with limited examples, existing approaches often rely on input\-output pairs, where obtaining labeled answers can be difficult or costly\. To address this limitation, we proposeStrategy\-Induct, a framework that derives task\-level instructions solely from a small set of example questions without requiring labeled answers\. Our approach first prompts the model to generate explicit reasoning strategies for each question, forming \(strategy, question\) pairs\. These pairs are then used to induce a task instruction that guides reasoning\. Experiments across multiple tasks and model scales demonstrate thatStrategy\-Inductoutperforms state\-of\-the\-art methods in question\-only settings\. Furthermore, we observe that jointly utilizing LLMs and Large Reasoning Models across task instruction generation and inference may lead to further performance improvements\.

Strategy\-Induct: Task\-Level Strategy Induction for Instruction Generation

Po\-Chun Chen1Hen\-Hsen Huang2Hsin\-Hsi Chen1,31Department of Computer Science and Information Engineering,National Taiwan University, Taiwan2Institute of Information Science, Academia Sinica, Taiwan3AI Research Center \(AINTU\), National Taiwan University, Taiwanpcchen@nlg\.csie\.ntu\.edu\.tw,hhhuang@iis\.sinica\.edu\.tw, hhchen@ntu\.edu\.tw

![Refer to caption](https://arxiv.org/html/2605.20924v1/x1.png)Figure 1:Our proposedStrategy\-Inductframework for strategy\-based instruction induction\. The framework consists of three stages: \(1\)Strategy Stage, where the LLM generates strategies \(sis\_\{i\}\) for given inputs \(xix\_\{i\}\)\. \(2\)INDUCT Stage, where question\-strategy pairs \(\(si,xi\)\(s\_\{i\},x\_\{i\}\)\) are combined with meta prompts and short phrases to generate an induced prompt \(PStrategy\-InductP\_\{\\text\{\{Strategy\-Induct\}\}\}\)\. \(3\)Inference Stage, wherePStrategy\-InductP\_\{\\text\{\{Strategy\-Induct\}\}\}is used in inference to guide the LLM in solving a new question \(x\*\) in the same task\.## 1Introduction

Large language models \(LLMs\) have achieved remarkable progress in natural language processing, demonstrating strong capabilities in understanding, generating, and reasoningBrown et al\. \([2020](https://arxiv.org/html/2605.20924#bib.bib2)\); Wei et al\. \([2022a](https://arxiv.org/html/2605.20924#bib.bib36)\); Chen et al\. \([2023](https://arxiv.org/html/2605.20924#bib.bib8)\); Wei et al\. \([2023](https://arxiv.org/html/2605.20924#bib.bib38)\); OpenAI \([2023](https://arxiv.org/html/2605.20924#bib.bib26)\)\. Providing high\-quality task instructions is essential for enabling LLMs to effectively perform tasks, as well\-designed instructions can significantly enhance accuracy and consistency\. However, designing high\-quality task instructions is challenging, as it often requires domain expertise and extensive manual effort\.

Honovich et al\. \([2023](https://arxiv.org/html/2605.20924#bib.bib16)\)showed that LLMs can induce task instructions from input\-output pairs, a process known asinstruction induction\. Building on this concept, subsequent studies have explored automated task instruction generationZhou et al\. \([2022](https://arxiv.org/html/2605.20924#bib.bib46)\); Chen et al\. \([2024a](https://arxiv.org/html/2605.20924#bib.bib3)\); Hsieh et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib17)\)\. While effective, these methods often rely on large external resources or an initial instruction prompt, making them less practical for users with limited data or prompt\-writing skillsDesmond and Brachman \([2024](https://arxiv.org/html/2605.20924#bib.bib10)\)\. Moreover, their optimized instructions are often model\-specific, limiting generalization across architectures\.

To address these limitations, recent studies have explored low\-cost instruction induction methods that require only a small number of examplesChen et al\. \([2024b](https://arxiv.org/html/2605.20924#bib.bib7)\); Zhou et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib45)\); Aswani et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib1)\), allowing LLMs to induce effective task instructions to improve inference performance\. WhileChen et al\. \([2024b](https://arxiv.org/html/2605.20924#bib.bib7)\)demonstrates that LLMs can generate better instructions using limited examples and even exhibit cross\-model adaptability, their approach still heavily relies on input\-output pairs\. In real\-world applications, such reliance may be impractical, as obtaining labeled answers can often be difficult or costly\.

To overcome these challenges, we proposeStrategy\-Induct, a framework that generates task instructions solely from input questions, eliminating the need for labeled answers\. Instead of relying on input\-output pairs, it creates strategies for each example question and induces a task\-level instruction from these strategy\-question pairs\.

As mentioned earlier, general users often lack the prompt\-writing skills needed to communicate effectively with LLMs, making it challenging to craft well\-formed and complete instructions\. To address this, we adopt the Short Phrase Prompting concept proposed byChen et al\. \([2024b](https://arxiv.org/html/2605.20924#bib.bib7)\), where a brief task description helps convey task intent\. A Short Phrase can be used alongside a minimal user query when needed, or omitted if the question is already self\-explanatory\. The specific Short Phrases used in our experiments are listed in Appendix[D](https://arxiv.org/html/2605.20924#A4)\.

Strategy\-Inductprovides several advantages\. First, it enables instruction induction in question\-only settings, addressing the critical challenge of missing labeled outputs\. Second, unlike prompt optimization approaches,Strategy\-Inductgenerates instructions that generalize across different models without requiring task\-specific fine\-tuning\.

We evaluateStrategy\-Inductin question\-only scenarios, where each task provides a set of example questions without answers\. Our experiments cover multiple datasets, including BBH\-Induct, Evals\-Induct, and the Shift Cipher Task\. Our results show thatStrategy\-Inductsurpasses existing SOTA methods\. Furthermore, we analyze its effectiveness across different LLM families \(e\.g\., Llama, Mistral, GPT, Gemini\), model scales, and model types, including both LLMs and Large Reasoning Models \(LRMs\), such as GPT o3 mini, demonstrating its scalability and robustness\.

## 2Strategy\-InductFramework

Existing instruction induction methods such as INDUCT\-LEARNChen et al\. \([2024b](https://arxiv.org/html/2605.20924#bib.bib7)\)require labeled input\-output pairs to induce task instructions\. However, obtaining correct answers can be costly or infeasible in many practical scenarios\.Strategy\-Inductremoves this requirement by operating in a question\-only setting\. The key enabler is theStrategystage, which generates reasoning strategies for each question, providing the structured signal that labeled outputs would otherwise supply for the subsequent induction step\.

In this section, we formally describeStrategy\-Induct, a framework for task\-level strategy induction that induces task instructions from input questions without requiring labeled answers\. It consists of three stages: Strategy, Induct, and Inference\. The following subsections define each stage \(see Figure[1](https://arxiv.org/html/2605.20924#S0.F1)for an overview\)\.

### 2\.1StrategyStage

The goal of this stage is to generate a reasoning strategysis\_\{i\}for each input questionxix\_\{i\}\. Given a set ofNNinput questions from thesame task:

𝒳=\{x1,x2,…,xN\}\\mathcal\{X\}=\\\{x\_\{1\},x\_\{2\},\\dots,x\_\{N\}\\\}\(1\)
Eachxix\_\{i\}is a task\-specific question\. The LLM is prompted with a meta promptPSP\_\{\\text\{S\}\}, the questionxix\_\{i\}, and optionally a task\-level Short Phrasedd\(a brief task description\) to produce the corresponding reasoning strategy:

si=LLM​\(PS,d,xi\)s\_\{i\}=\\text\{LLM\}\(P\_\{\\text\{S\}\},d,x\_\{i\}\)\(2\)
Repeating this process for eachxi∈𝒳x\_\{i\}\\in\\mathcal\{X\}produces a set of strategy\-question pairs:

𝒮=\{\(s1,x1\),\(s2,x2\),…,\(sN,xN\)\}\\mathcal\{S\}=\\\{\(s\_\{1\},x\_\{1\}\),\(s\_\{2\},x\_\{2\}\),\\dots,\(s\_\{N\},x\_\{N\}\)\\\}\(3\)
The next stage takes𝒮\\mathcal\{S\}as input to induce a task\-level instruction prompt\.

### 2\.2InductStage

In this stage, we construct an induced promptPStrategy\-InductP\_\{\\text\{\{Strategy\-Induct\}\}\}using the collected strategy\-question pairs\. Specifically, the LLM is prompted with a meta promptPIP\_\{\\text\{I\}\}, the Short Phrasedd, and the set𝒮\\mathcal\{S\}to induce a task\-level instruction\.

PStrategy\-Induct=LLM​\(PI,d,𝒮\)P\_\{\\textsc\{Strategy\-Induct\}\}=\\text\{LLM\}\(P\_\{\\text\{I\}\},d,\\mathcal\{S\}\)\(4\)
The resultingPStrategy\-InductP\_\{\\text\{\{Strategy\-Induct\}\}\}is a reusable task\-level prompt applicable to other questions within the same task\.

### 2\.3Inference Stage

At inference time, the induced promptPStrategy\-InductP\_\{\\textsc\{Strategy\-Induct\}\}is used to solve new questions within the same task\. Given a new questionx∗x^\{\*\}, the LLM generates a response:

y∗=LLM​\(PStrategy\-Induct,x∗\)y^\{\*\}=\\text\{LLM\}\(P\_\{\\textsc\{Strategy\-Induct\}\},x^\{\*\}\)\(5\)
wherey∗y^\{\*\}is the model’s response\. Thus,PStrategy\-InductP\_\{\\textsc\{Strategy\-Induct\}\}can be reused across questions within the same task\.

## 3Experiments

### 3\.1Models

We conduct experiments using 18 models, spanning four LLM series and two LRMs, covering both open\-source and proprietary models\. Further details on model versions and configurations are provided in Appendix[B](https://arxiv.org/html/2605.20924#A2)\.

Table 1:Accuracy \(%\) of different models on BBH\-Induct, Evals\-Induct, and Shift Cipher in zero\-shot settings\. The highest overall accuracy is shown inbold\. “Induct” refers to the INDUCT baseline\.
### 3\.2Datasets

We adopt the datasets used in prior state\-of\-the\-art instruction induction methodsChen et al\. \([2024b](https://arxiv.org/html/2605.20924#bib.bib7)\), ensuring comparability with existing approaches\.

##### BBH\-Induct

This dataset is adapted from the BIG\-Bench Hard \(BBH\) benchmarkSuzgun et al\. \([2022](https://arxiv.org/html/2605.20924#bib.bib33)\), a suite of tasks that current LLMs struggle to match average human performance\. It comprises 23 tasks totaling 5,161 instances, covering diverse reasoning challenges including logical deduction, spatial reasoning, and language understanding\.

##### Evals\-Induct

Derived from the OpenAI Evals project, this dataset features 24 challenging tasks that are beyond the capabilities of state\-of\-the\-art models\. Each task contains approximately 91 to 378 examples, totaling 3,683 instances\.

##### Shift Cipher Task

We additionally include the Shift Cipher TaskPrabhakar et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib30)\)to assess the effectiveness of our approach on symbolic reasoning tasks\. This task involves decoding text encrypted with a shift cipher, where each letter is replaced by another a fixed number of positions earlier in the alphabet\. The dataset includes cases with shifts ranging fromk=1k=1to2525, requiring models to generalize across different rotation values\. Each example consists of a seven\-letter word encoded with a shift cipher\. FollowingPrabhakar et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib30)\), we use words from the highest\-probability bin \(Bin 1\), where words are ranked by GPT\-2 log probability and may include non\-standard forms that are frequent in pre\-training corpora but not necessarily valid English words\. For instance, a case with𝑠ℎ𝑖𝑓𝑡​\_​𝑙𝑒𝑣𝑒𝑙=3\\mathit\{shift\\\_level\}=3, also known as ROT\-3, requires decoding “fkrrvhg” back to “choosed” by shifting each letter 3 positions backward:f→c,k→h,r→o, and so on\.

Further details on all datasets are provided in Table[8](https://arxiv.org/html/2605.20924#A7.T8)to[10](https://arxiv.org/html/2605.20924#A7.T10)in the Appendix\.

### 3\.3Baselines

##### ZCoT

We apply Zero\-shot Chain\-of\-ThoughtKojima et al\. \([2022](https://arxiv.org/html/2605.20924#bib.bib18)\)using Short Phrase Instructions, where the model receives only a short phrase as guidance for reasoning\.

##### SCoT

Automatic Strategic Chain\-of\-Thought, proposed byWang et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib35)\), is a state\-of\-the\-art Chain\-of\-Thought \(CoT\) baseline that prompts the model to first identify an effective problem\-solving strategy before proceeding with step\-by\-step reasoning\. In our setting, we provide only Short Phrase Instructions as the task description\.

##### INDUCT

The instruction induction stage ofChen et al\. \([2024b](https://arxiv.org/html/2605.20924#bib.bib7)\)serves as the state\-of\-the\-art instruction induction baseline, which applies LLMs to infer task instructions from given examples\. In our setting, we provide the model with Short Phrase and Questions \(without Answers\) to induce task instructions\.

ZCoT and SCoT areinstance\-levelmethods that solve each question independently using only the Short Phrase as the task description\. INDUCT and ourStrategy\-Inductaretask\-levelmethods that additionally leverage a small set of example questions to induce a reusable task instruction prior to inference\. At inference time, all methods receive the same input: one instruction and one new question\. The distinction is that instance\-level methods use the Short Phrase directly as the instruction, whereas task\-level methods replace it with an induced instruction derived from the example questions\.

### 3\.4Other Details

All experiments follow the BBH\-Induct and Evals\-Induct settings, including a Short Phrase for every task or subtask as its task description\. The specific phrases for each dataset and subtask are listed in Appendix Tables[8](https://arxiv.org/html/2605.20924#A7.T8)–[10](https://arxiv.org/html/2605.20924#A7.T10)\.

##### N Setting

We setN=3N=3, soStrategy\-Inductgenerates a strategy for each of three questions and induces instructions from the resulting pairs\.

##### Experimental Configurations

We utilize API services from five LLM providers: OpenAI, Google, Mistral AI, SambaNova, and Together AI\. Model details are in Appendix[B](https://arxiv.org/html/2605.20924#A2)\. For cost considerations, we follow the sampling strategy ofChen et al\. \([2024b](https://arxiv.org/html/2605.20924#bib.bib7)\), randomly sampling 25 examples per task and setting temperature to 0 for deterministic outputs\.

![Refer to caption](https://arxiv.org/html/2605.20924v1/x2.png)Figure 2:Accuracy difference heatmap for the 23 tasks from BBH\-Induct across 18 models\. The accuracy delta is computed betweenStrategy\-Inductand the baseline method ZCoT \(blue/redindicates our method wins/loses\)\.![Refer to caption](https://arxiv.org/html/2605.20924v1/x3.png)Figure 3:The head\-to\-head comparison of the largest models within each LLM series on the 25 subtasks from Shift Cipher\. The accuracy delta indicates the accuracy difference betweenStrategy\-Inductand ZCoT \(blue/redindicates our method wins/loses\)\.![Refer to caption](https://arxiv.org/html/2605.20924v1/x4.png)Figure 4:The head\-to\-head comparison of LRMs on the 25 subtasks from Shift Cipher\. The accuracy delta represents the accuracy difference betweenStrategy\-Inductand ZCoT \(blue/redindicates our method wins/loses\)\.

## 4Results and Analysis

### 4\.1Quality of Strategy\-Induced Instructions

We evaluateStrategy\-Inducton BBH\-Induct, Evals\-Induct, and the Shift Cipher Task using LLMs and LRMs, measuring the accuracy of LLM responses given the question and generated instructions\. To assess effectiveness, we compare it with baselines across 60 settings \(20 per dataset\)\. As shown in Table[1](https://arxiv.org/html/2605.20924#S3.T1), our method outperforms existing methods in most settings\. Compared to ZCoT, it achieves a 50\-3\-7 win\-tie\-lose record\. Compared to INDUCTChen et al\. \([2024b](https://arxiv.org/html/2605.20924#bib.bib7)\), it attains 44\-3\-13, and compared to SCoT, it reaches 52\-0\-8\.

##### Performance on LLMs

Our results align with previous findingsHonovich et al\. \([2023](https://arxiv.org/html/2605.20924#bib.bib16)\); Chen et al\. \([2024b](https://arxiv.org/html/2605.20924#bib.bib7)\), showing that smaller LLMs such as Llama 3\.1 8B, Mistral Nemo 12B, Gemini 1\.5 Flash 8B, Gemini 2\.0 Flash Lite, and GPT\-4o mini struggle with instruction induction, making it difficult for them to derive effective task instructions\. However,Strategy\-Inductmitigates this issue, achieving a 10\-3\-2 win\-tie\-lose record against INDUCT on these small models\. For mid\-to\-large models,Strategy\-Inductis particularly effective\. Compared to ZCoT, it yields higher accuracy across all models, except for Mistral Large 2, which underperforms on Evals\-Induct, and Mistral Small 2, which produces a tie on Shift Cipher\. These results demonstrate the scalability of our method across different model sizes\. Additionally, we observe that SCoT struggles with maintaining output format consistency in the Shift Cipher Task, leading to performance degradation\. This issue is particularly evident in GPT o3 mini, where incorrect formatting negatively impacts accuracy\.

##### Performance on LRMs

A similar trend is observed in LRMs, where stronger models tend to benefit more from our approach\. For instance, the win\-tie\-lose record for GPT o3 mini improves from 2\-0\-1 \(lowreasoning effort\) to 2\-1\-0 \(highreasoning effort\)\. Overall,Strategy\-Inductimproves LLM and LRM performance across diverse tasks, outperforming prior state\-of\-the\-art methods\.

### 4\.2Breakdown of BBH\-Induct

We analyze the 23 tasks in BBH\-Induct to examine howStrategy\-Inductimpacts their performance\. Figure[2](https://arxiv.org/html/2605.20924#S3.F2)shows accuracy differences across 18 models, comparingStrategy\-Inductwith the baseline ZCoT\.

As model size increases, negative changes \(red regions\) decrease while positive improvements \(blue regions\) become more widespread\. This trend is consistent with the findings ofChen et al\. \([2024b](https://arxiv.org/html/2605.20924#bib.bib7)\)\. Some exceptions occur in simpler subtasks \(e\.g\., boolean expressions\), where strong models such as GPT\-4o, Llama 3\.1 405B, and LRMs already achieve near\-perfect accuracy, leaving little room for improvement\.

For traditional LLMs,Strategy\-Inductimproves performance on semantic understanding tasks, notablysnarks,ruin names, andsports understanding\. It also improves performance in ranking\-related tasks \(e\.g\.,word sorting\), particularly for smaller models\. Additionally, tasks involving geometric or spatial reasoning \(e\.g\.,geometric shapes and navigate\) also benefit\.

For LRMs,Strategy\-Inductimproves performance on semantic understanding tasks, such asdisambiguation QA,snarks,ruin names, andsports understanding\. This aligns with the design of LRMs, which primarily focus on mathematical, logical, and programming tasks rather than language understanding\.

Notably, several subtasks whereStrategy\-Inductshows strong gains—salient translation error detection,snarks, andsports understanding—are categorized as knowledge\-based rather than reasoning\-oriented bySuzgun et al\. \([2022](https://arxiv.org/html/2605.20924#bib.bib33)\)\. Our framework yields accuracy improvements of 8 to 60 percentage points onsnarksand win\-tie\-lose rates of 15\-3\-2 and 14\-3\-3 against ZCoT forsports understandingandsalient translation error detection, respectively\. Further analysis is provided in Appendix[E](https://arxiv.org/html/2605.20924#A5)and Appendix[F\.1](https://arxiv.org/html/2605.20924#A6.SS1)\.

We also observed errors in certain smaller models on specific subtasks\. For instance, Mistral Nemo, Mistral Small, and Gemini 1\.5 Flash 8B tend to classify objects instead of counting them in theobject countingtask, leading to misinterpretations and higher error rates\.

### 4\.3Breakdown of the Shift Cipher Task

To better understand the effects ofStrategy\-Induct, we analyze its performance across different shift values in the Shift Cipher task\. Prior work byPrabhakar et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib30)\)showed that even state\-of\-the\-art LLMs, such as GPT\-4 and Llama 3\.1 405B, perform well on commonly seen shift values like ROT\-1, ROT\-3 \(Caesar cipher\), and ROT\-13\. However, accuracy decreases for less frequent shifts \(rot\-kk, wherek≠1,3,13k\\neq 1,3,13\)\. Following the analytical approach ofPrabhakar et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib30)\), we select the largest model from each LLM series as a representative and include all available LRM models to ensure a comprehensive evaluation across different reasoning capabilities\.

As shown in Figure[3](https://arxiv.org/html/2605.20924#S3.F3), our method achieves greater improvements on less common shift values\. A possible explanation is thatStrategy\-Inductinduces prompts that explicitly guide LLMs to handle wrap\-around effects in letter positions, thereby reducing errors\. We also observe higher performance fluctuations for ROT\-10\. For instance, Llama 3\.1 405B shows noticeable accuracy gains, whereas GPT\-4o exhibits a drop\. Our analysis suggests that for ROT\-10, LLMs tend to be more sensitive to prompt variations\. Specifically, GPT\-4o frequently misaligns the positions of ’s’ and ’t’, the 19th and 20th letters of the alphabet, respectively, leading to errors\.

Overall,Strategy\-Inductenhances LLM performance on the Shift Cipher task \(Figure[3](https://arxiv.org/html/2605.20924#S3.F3)\) while also improving reasoning in weaker LRMs \(Figure[4](https://arxiv.org/html/2605.20924#S3.F4)\), such as Gemini 2\.0 Flash Thinking and GPT o3 mini \(low\)\.

## 5Analysis and Discussion

### 5\.1Effect of the Number of Example Questions

To analyze the impact of the number of example questions \(NN\), we conduct experiments withN=1,3,5N=1,3,5on BBH\-Induct\. As shown in Table[2](https://arxiv.org/html/2605.20924#S5.T2), performance generally improves fromN=1N=1toN=3N=3, withN=3N=3achieving the highest accuracy for most models \(19\-0\-1 win\-tie\-lose againstN=1N=1\)\. However, increasingNNto 5 does not consistently yield further gains\.

A likely explanation is that each example question is paired with a multi\-step strategy, so largerNNsubstantially increases the length and complexity of the induction prompt\. WhileN=3N=3offers sufficient diversity for effective induction,N=5N=5may exceed the capacity of some models to synthesize a concise instruction from the additional context\.

Table 2:Effect of the number of example questionsNNonStrategy\-Inductperformance across different models on BBH\-Induct\.![Refer to caption](https://arxiv.org/html/2605.20924v1/x5.png)Figure 5:Cross\-model generalization analysis on BBH\-Induct, where the prompt induction and inference models are different\. Accuracy is shown as a percentage, with darkerblueindicating greater improvement and darkerorangeindicating performance decline\.
### 5\.2Cross\-Model Generalization

To evaluate the cross\-model capability of our approach, we select recent small, medium, and large models from each LLM series and assess whether the prompts they generate enhance the performance of various inference models\. We analyze this generalization from four perspectives: \(1\) Effect of Model Scale on Prompt Quality, \(2\) Generalization of Prompts Across LLM Series, \(3\) Impact of Prompts on Different Model Scales, and \(4\) Cooperation Between LLMs and LRMs\. Figure[5](https://arxiv.org/html/2605.20924#S5.F5)presents our experimental results, which we analyze below\.

#### 5\.2\.1Effect of Model Scale on Prompt Quality

Smaller models, such as Llama 3\.1 8B and Mistral Nemo 12B, tend to generate less effective prompts, sometimes even reducing the performance of downstream models\. In contrast, prompts from larger models often lead to widespread performance improvements\. For example, those generated by Llama 3\.1 405B and Mistral Large 2 contribute to notable improvements\. Additionally, we observe that although Gemini 2\.0 Pro Exp is larger than Gemini 2\.0 Flash, their performance on BBH\-Induct is similar \(86\.72% vs\. 85\.45%\), resulting in prompts of comparable quality\. This suggests that prompt quality correlates more with task performance than with model size alone\.

#### 5\.2\.2Generalization of Prompts Across LLM Series

Our approach demonstrates strong cross\-series generalization\. The most capable models in each series \(Llama 3\.1 405B, Mistral Large 2, Gemini 2\.0 Pro Exp, and GPT\-4o\) generate prompts that improve the performance of models from other series\.

We observe that applying prompts across different models can sometimes yield higher performance gains\. For example, when GPT\-4o uses its own induced prompts on BBH\-Induct, it achieves a 3\.53% improvement over the baseline\.

However, when using prompts generated by Mistral Large 2 and Gemini 2\.0 Pro Exp, the improvement increases to 4\.63% and 5\.62%, respectively\. This suggests that prompts generated by different models may introduce alternative reasoning strategies, helping the original model refine its outputs, a pattern consistent across other model families\.

#### 5\.2\.3Impact of Prompts on Different Model Scales

Smaller models benefit more from detailed prompts, as they rely more on explicit guidance\. For example, in the Llama 3\.1 series, when using prompts from GPT\-4o, the performance gains for Llama 3\.1 8B, 70B, and 405B are 7\.88%, 5\.33%, and 3\.95%, respectively\. This pattern suggests that high\-quality prompts provide greater improvements for smaller models\.

Case Study: Snarks Task \(BBH\)Input Question and Generated Strategies:Input Question 1:
\(A\) Think of the poor man\! What will he do without his job?\(B\) Think of the poor man\! What will he do without his yacht?Generated Strategy 1:
Step 1: Analyze option \(A\) \[…\] Consider the common understanding of job loss as a serious issue\.Step 2: Analyze option \(B\) \[…\] Consider owning a yacht as a luxury, and the potential for sarcasm when expressing concern about someone losing a luxury item\. \[…\]Input Question 2:
\(A\) What a poor guy, years of work and he only got a $50,000 scholarship\(B\) What a poor guy, years of work and he only got a $50 scholarshipGenerated Strategy 2:
Step 1: Analyze option \(A\) for potential sarcasm\. Consider the implied meaning of “poor guy” in relation to receiving a $50,000 scholarship\. Evaluate if the statement conveys the opposite of its literal meaning\.Step 2: Analyze option \(B\) \[…\] Step 3: Compare the two options \(A\) and \(B\)\. \[…\]Input Question 3:
\(A\) Have you tried not being poor? It is much easier\(B\) Have you tried not being rude? It is much easierGenerated Strategy 3:
Step 1: Analyze option A for sarcastic intent\. Consider the inherent difficulty of overcoming poverty and whether the statement implies the opposite\.Step 2: Analyze option B for sarcastic intent\. Consider the ease of being polite and whether the statement implies the opposite\. \[…\]Final Induced Instruction \(Strategy\-Induct\):
Step 1: Analyze Sentence \(A\): Carefully examine sentence \(A\) for potential indicators of sarcasm\. \[…\] Look for elements such as:– Irony: A statement where the intended meaning is the opposite of the literal meaning\.– Hyperbole: Exaggeration for emphasis or effect\.– Satire: The use of humor, irony, exaggeration, or ridicule… \[…\]Step 2: Analyze Sentence \(B\): Repeat the analysis process from Step 1 for sentence \(B\)\.Step 3: Compare Sentences: Compare the two sentences based on the likelihood of sarcastic intent\. \[…\]Step 4: Select the More Sarcastic Sentence: Determine which sentence \(A or B\) is more likely to be sarcastic based on your analysis\.Result \(Gemini 2\.0 Flash\):SCoT: 0\.40→\\rightarrowStrategy\-Induct:0\.76Figure 6:Case study on thesnarkstask \(Gemini 2\.0 Flash\)\. Each instance\-level strategy captures one sarcasm cue; the induced instruction unifies them into a comprehensive reasoning framework\. Full details in Appendix\.
#### 5\.2\.4Cooperation Between LLMs and LRMs

LRMs sometimes gain less from their self\-generated prompts compared to those from stronger LLMs\. For instance, prompts generated by Llama 3\.1 405B, when used by GPT o3 mini \(low and medium\), lead to greater performance gains compared to prompts generated by GPT o3 mini itself\.

Meanwhile, some LRMs generate prompts that sometimes perform on par with, or even surpass, those from larger LLMs\. For example, prompts from Gemini 2\.0 Flash Thinking significantly boost Mistral Small 2 and Mistral Small 3, yielding gains of 8\.47% and 8\.81%, respectively\.

However, while Gemini 2\.0 Flash Thinking’s prompts improve most models, they negatively impact Gemini 2\.0 Flash Lite\. A possible explanation is that Flash Thinking tends to generate more structured, multi\-step strategies, which, while beneficial for many models, may be overly complex for Flash Lite\. For instance, in tasks likeTracking Shuffled Objects, where precise step tracking is required, Flash Lite may struggle to follow these detailed instructions, leading to errors\.

These findings suggest that LLMs and LRMs may complement each other, with LLMs potentially enhancing task instruction quality for LRMs, while certain LRMs may generate prompts that improve other models\.

### 5\.3Case Study: From Instance Strategies to Task Instructions

Figure[6](https://arxiv.org/html/2605.20924#S5.F6)illustrates howStrategy\-Inductabstracts instance\-level insights into a reusable task instruction on thesnarkstask\. Each individual strategy captures only one sarcasm cue \(e\.g\., irony or hyperbole\), whereas the induced instruction consolidates them into a comprehensive framework covering irony, hyperbole, satire, and other indicators\. This generalization leads to a substantial accuracy gain on Gemini 2\.0 Flash, from 0\.40 \(SCoT\) to 0\.76 \(Strategy\-Induct\)\.

## 6Related Work

##### Inductive Reasoning in LLMs

Inductive reasoning, a fundamental cognitive skill, has been widely examined in LLMs\. Prior studies show that LLMs can infer patterns and rules from limited observationsLiu et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib20)\); Yu et al\. \([2023](https://arxiv.org/html/2605.20924#bib.bib41)\); Li et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib19)\)\.He et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib15)\)found that while LLMs exhibit inherent inductive capabilities, interactive environments enhance their effectiveness\.Yang et al\. \([2022](https://arxiv.org/html/2605.20924#bib.bib40)\)investigated rule induction from factual data, whileZhu et al\. \([2023](https://arxiv.org/html/2605.20924#bib.bib47)\)explored generalization from examples\.Zhang et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib42)\)further incorporated inductive reasoning into instruction fine\-tuning, andChen et al\. \([2024c](https://arxiv.org/html/2605.20924#bib.bib9)\)applied it to enhance temporal reasoning\. These studies highlight the potential of leveraging induction to improve LLMs’ reasoning abilities\.

##### Task\-Level Instruction Generation

Instruction induction, where LLMs generate task instructions from input\-output examples, was introduced byHonovich et al\. \([2023](https://arxiv.org/html/2605.20924#bib.bib16)\)\. Subsequent studies have explored various techniques, including ranking candidate instructionsZhou et al\. \([2022](https://arxiv.org/html/2605.20924#bib.bib46)\); Zhang et al\. \([2023](https://arxiv.org/html/2605.20924#bib.bib43)\), iterative refinement using demonstrationsSun et al\. \([2023](https://arxiv.org/html/2605.20924#bib.bib32)\); Chen et al\. \([2024a](https://arxiv.org/html/2605.20924#bib.bib3)\), and dynamic reasoning modulesAswani et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib1)\)\. Additionally, evolutionary optimization has also been applied to improve instruction generationXu et al\. \([2022](https://arxiv.org/html/2605.20924#bib.bib39)\); Fernando et al\. \([2023](https://arxiv.org/html/2605.20924#bib.bib12)\); Guo et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib14)\)\. However, these methods often require substantial computational resources and are typically optimized for a single model, limiting their generalizability\.

More efficient alternatives focus on generating a single instruction with minimal overhead\.Zhou et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib45)\)constructed reasoning structures from predefined modules, whileChen et al\. \([2024b](https://arxiv.org/html/2605.20924#bib.bib7)\)proposed an approach that generates a single instruction with chain\-of\-thought \(CoT\) demonstrations to improve efficiency\. However, the approach ofChen et al\. \([2024b](https://arxiv.org/html/2605.20924#bib.bib7)\)still relies on labeled input\-output pairs for the induction process\. Our proposedStrategy\-Inductframework follows a similar efficiency\-driven approach but addresses a fundamentally different setting\. Unlike prior work that requires labeled input\-output pairs for induction,Strategy\-Inductderives task instructions from a small set of example questions alone, using strategy generation as the key mechanism to compensate for the absence of labeled answers\.

Additionally, our method is model\-agnostic, demonstrating strong cross\-model generalization\. Beyond text, this low\-cost induction paradigm has also been extended to multimodal settings, including visual question answeringChen et al\. \([2025b](https://arxiv.org/html/2605.20924#bib.bib5)\)and audio understandingChen et al\. \([2026](https://arxiv.org/html/2605.20924#bib.bib6)\)\.

##### Reasoning Methods

Chain\-of\-Thought \(CoT\) prompting improves LLMs’ problem\-solving by generating intermediate reasoning stepsWei et al\. \([2022b](https://arxiv.org/html/2605.20924#bib.bib37)\); Kojima et al\. \([2022](https://arxiv.org/html/2605.20924#bib.bib18)\); Nye et al\. \([2021](https://arxiv.org/html/2605.20924#bib.bib25)\)\. To enhance CoT effectiveness, prior work has explored diverse example selectionZhang et al\. \([2022](https://arxiv.org/html/2605.20924#bib.bib44)\)and structured reasoning strategies, including planning before answeringWang et al\. \([2023](https://arxiv.org/html/2605.20924#bib.bib34)\)and formulating strategies firstWang et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib35)\)\.Chen et al\. \([2025a](https://arxiv.org/html/2605.20924#bib.bib4)\)further proposed generating multiple diverse rationales per question and inducing them into a unified plan, improving zero\-shot reasoning without example questions\.

However, this approach operates at the instance level, requiring the full multi\-rationale induction process for every new question, which incurs higher inference cost\. These methods all require per\-instance reasoning, generating a reasoning path for each query\. In contrast,Strategy\-Inductoperates at the task level, deriving a reusable task instruction from a small set of example questions without answer labels, amortizing the induction cost across all questions within the same task\.

## 7Conclusion

We proposeStrategy\-Induct, a framework that derives task\-level instructions solely from example questions without requiring labeled answers\. By generating reasoning strategies for each question and inducing a unified instruction from these strategy\-question pairs, our approach enables effective instruction generation in question\-only settings\. By leveraging strategy generation as a substitute for labeled outputs,Strategy\-Inductbroadens the applicability of instruction induction to scenarios where annotated data is unavailable\. Experiments across multiple datasets and model families show thatStrategy\-Inductoutperforms existing methods across various LLMs and LRMs, demonstrating strong adaptability across model scales\. Our findings further suggest that combining LLMs and LRMs for task instruction generation and inference may yield additional performance gains\. The framework also exhibits robust cross\-model generalization, highlighting its broad applicability and potential for future extensions\.

## 8Limitations

##### Reliance on Instruction\-Following Models

Our approach relies on sufficiently strong instruction following models to generate responses in the desired format and to induce instructions that facilitate the extraction of effective strategies\. While modern instruction following models, such as Llama 3\.1 8B, Mistral NeMo 12B, Gemini 1\.5 Flash 8B, and GPT\-4o mini, are capable of handling complex instructions, earlier models, such as Llama 1 and Llama 2, may lack the necessary capability to adhere to structured formats\.

##### Generalization Across Models and Datasets

Our approach has been evaluated on multiple datasets and across 18 different models \(with 20 settings\), covering a diverse range of architectures and training paradigms\. While our results demonstrate strong generalization across these settings, its applicability to models and datasets beyond our experimental scope remains to be further explored\.

##### Dependence on API\-Based Models

Due to hardware limitations, our experiments rely on API\-based access to open\-source models provided by services such as Mistral AI, SambaNova, and Together AI\. Since we do not have full control over the internal configurations of these services, there may be variations in model behavior across different API providers\.

##### Task Scope

Our framework is designed for task\-level instruction induction, where a single reusable instruction is derived for an entire category of problems within established task definitions\. It excels at handling repetitive problems that share a common structure, such as those in BBH\-Induct, Evals\-Induct, and the Shift Cipher Task\. However, real\-world tasks can be diverse, open\-set, or combinations of multiple tasks, where preparing task descriptions and collecting representative example questions may be more challenging\. Extending our approach to such open\-ended scenarios remains an area for future work\.

## Acknowledgements

This work was supported by National Science and Technology Council, Taiwan, under grant NSTC 114\-2221\-E\-002 \-070 \-MY3, and by Ministry of Education \(MOE\), Taiwan, under grant NTU\-114L900901\.

## References

- Aswani et al\. \(2024\)Krishna Aswani, Huilin Lu, Pranav Patankar, Priya Dhalwani, Xue Tan, Jayant Ganeshmohan, and Simon Lacasse\. 2024\.[Auto\-evolve: Enhancing large language model‘s performance via self\-reasoning framework](https://doi.org/10.18653/v1/2024.findings-emnlp.774)\.In*Findings of the Association for Computational Linguistics: EMNLP 2024*, pages 13243–13257, Miami, Florida, USA\. Association for Computational Linguistics\.
- Brown et al\. \(2020\)Tom B\. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert\-Voss, Gretchen Krueger, T\. J\. Henighan, Rewon Child, Aditya Ramesh, Daniel M\. Ziegler, Jeff Wu, Clemens Winter, and 12 others\. 2020\.[Language models are few\-shot learners](https://api.semanticscholar.org/CorpusID:218971783)\.*ArXiv*, abs/2005\.14165\.
- Chen et al\. \(2024a\)Lichang Chen, Jiuhai Chen, Tom Goldstein, Heng Huang, and Tianyi Zhou\. 2024a\.[InstructZero: Efficient instruction optimization for black\-box large language models](https://proceedings.mlr.press/v235/chen24e.html)\.In*Proceedings of the 41st International Conference on Machine Learning*, volume 235 of*Proceedings of Machine Learning Research*, pages 6503–6518\. PMLR\.
- Chen et al\. \(2025a\)Po\-Chun Chen, Hen\-Hsen Huang, and Hsin\-Hsi Chen\. 2025a\.[Diverge to induce prompting: Multi\-rationale induction for zero\-shot reasoning](https://doi.org/10.18653/v1/2025.findings-ijcnlp.6)\.In*Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics*, pages 102–115, Mumbai, India\. The Asian Federation of Natural Language Processing and The Association for Computational Linguistics\.
- Chen et al\. \(2025b\)Po\-Chun Chen, Hen\-Hsen Huang, and Hsin\-Hsi Chen\. 2025b\.[Vqa\-induct: Instruction induction for visual question answering](https://doi.org/10.1145/3746252.3760963)\.In*Proceedings of the 34th ACM International Conference on Information and Knowledge Management*, CIKM ’25, page 4659–4664, New York, NY, USA\. Association for Computing Machinery\.
- Chen et al\. \(2026\)Po\-Chun Chen, Hen\-Hsen Huang, and Hsin\-Hsi Chen\. 2026\.[Task\-level instructions induction for audio question answering from few examples](https://doi.org/10.18653/v1/2026.eacl-short.18)\.In*Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 2: Short Papers\)*, pages 244–264, Rabat, Morocco\. Association for Computational Linguistics\.
- Chen et al\. \(2024b\)Po\-Chun Chen, Sheng\-Lun Wei, Hen\-Hsen Huang, and Hsin\-Hsi Chen\. 2024b\.[Induct\-learn: Short phrase prompting with instruction induction](https://doi.org/10.18653/v1/2024.emnlp-main.297)\.In*Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing*, pages 5204–5231, Miami, Florida, USA\. Association for Computational Linguistics\.
- Chen et al\. \(2023\)Wei\-Lin Chen, Cheng\-Kuang Wu, Yun\-Nung Chen, and Hsin\-Hsi Chen\. 2023\.[Self\-ICL: Zero\-shot in\-context learning with self\-generated demonstrations](https://doi.org/10.18653/v1/2023.emnlp-main.968)\.In*Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing*, pages 15651–15662, Singapore\. Association for Computational Linguistics\.
- Chen et al\. \(2024c\)Ziyang Chen, Dongfang Li, Xiang Zhao, Baotian Hu, and Min Zhang\. 2024c\.[Temporal knowledge question answering via abstract reasoning induction](https://doi.org/10.18653/v1/2024.acl-long.267)\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 4872–4889, Bangkok, Thailand\. Association for Computational Linguistics\.
- Desmond and Brachman \(2024\)Michael Desmond and Michelle Brachman\. 2024\.[Exploring prompt engineering practices in the enterprise](https://api.semanticscholar.org/CorpusID:268384789)\.*ArXiv*, abs/2403\.08950\.
- Dubey et al\. \(2024\)Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al\-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony S\. Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, and 510 others\. 2024\.[The llama 3 herd of models](https://api.semanticscholar.org/CorpusID:271571434)\.*ArXiv*, abs/2407\.21783\.
- Fernando et al\. \(2023\)Chrisantha Fernando, Dylan S\. Banarse, Henryk Michalewski, Simon Osindero, and Tim Rocktäschel\. 2023\.[Promptbreeder: Self\-referential self\-improvement via prompt evolution](https://api.semanticscholar.org/CorpusID:263310323)\.*ArXiv*, abs/2309\.16797\.
- Google \(2025\)Google\. 2025\.[Gemini 2\.0: Flash, flash\-lite and pro](https://developers.googleblog.com/en/gemini-2-family-expands/)\.FEB\. 5, 2025\.
- Guo et al\. \(2024\)Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang\. 2024\.[Connecting large language models with evolutionary algorithms yields powerful prompt optimizers](https://openreview.net/forum?id=ZG3RaNIsO8)\.In*The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7\-11, 2024*\. OpenReview\.net\.
- He et al\. \(2024\)Kaiyu He, Mian Zhang, Shuo Yan, Peilin Wu, and Zhiyu Zoey Chen\. 2024\.[Idea: Enhancing the rule learning ability of large language model agent through induction, deduction, and abduction](https://arxiv.org/abs/2408.10455)\.*Preprint*, arXiv:2408\.10455\.
- Honovich et al\. \(2023\)Or Honovich, Uri Shaham, Samuel R\. Bowman, and Omer Levy\. 2023\.[Instruction induction: From few examples to natural language task descriptions](https://doi.org/10.18653/v1/2023.acl-long.108)\.In*Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 1935–1952, Toronto, Canada\. Association for Computational Linguistics\.
- Hsieh et al\. \(2024\)Cho\-Jui Hsieh, Si Si, Felix Yu, and Inderjit Dhillon\. 2024\.[Automatic engineering of long prompts](https://doi.org/10.18653/v1/2024.findings-acl.634)\.In*Findings of the Association for Computational Linguistics ACL 2024*, pages 10672–10685, Bangkok, Thailand and virtual meeting\. Association for Computational Linguistics\.
- Kojima et al\. \(2022\)Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa\. 2022\.[Large language models are zero\-shot reasoners](https://api.semanticscholar.org/CorpusID:249017743)\.*ArXiv*, abs/2205\.11916\.
- Li et al\. \(2024\)Jiachun Li, Pengfei Cao, Zhuoran Jin, Yubo Chen, Kang Liu, and Jun Zhao\. 2024\.[Mirage: Evaluating and explaining inductive reasoning process in language models](https://api.semanticscholar.org/CorpusID:273345913)\.*ArXiv*, abs/2410\.09542\.
- Liu et al\. \(2024\)Emmy Liu, Graham Neubig, and Jacob Andreas\. 2024\.[An incomplete loop: Deductive, inductive, and abductive learning in large language models](https://api.semanticscholar.org/CorpusID:268889360)\.*ArXiv*, abs/2404\.03028\.
- MistralAI \(2024a\)MistralAI\. 2024a\.[Ai in abundance](https://mistral.ai/news/september-24-release)\.Sep 17, 2024\.
- MistralAI \(2024b\)MistralAI\. 2024b\.[Mistral nemo](https://mistral.ai/news/mistral-nemo)\.Jul 18, 2024\.
- MistralAI \(2024c\)MistralAI\. 2024c\.[Pixtral large](https://mistral.ai/news/pixtral-large)\.Nov 18, 2024\.
- MistralAI \(2025\)MistralAI\. 2025\.[Mistral small 3](https://mistral.ai/news/mistral-small-3)\.Jan 30, 2025\.
- Nye et al\. \(2021\)Maxwell Nye, Anders Andreassen, Guy Gur\-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena\. 2021\.[Show your work: Scratchpads for intermediate computation with language models](https://api.semanticscholar.org/CorpusID:244773644)\.*ArXiv*, abs/2112\.00114\.
- OpenAI \(2023\)OpenAI\. 2023\.[Gpt\-4 technical report](https://api.semanticscholar.org/CorpusID:257532815)\.*ArXiv*, abs/2303\.08774\.
- OpenAI \(2024a\)OpenAI\. 2024a\.[Gpt\-4o mini: advancing cost\-efficient intelligence](https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/)\.July 18, 2024\.
- OpenAI \(2024b\)OpenAI\. 2024b\.[Hello gpt\-4o](https://openai.com/index/hello-gpt-4o/)\.May 13, 2024\.
- OpenAI \(2025\)OpenAI\. 2025\.[Openai o3\-mini](https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/)\.January 31, 2025\.
- Prabhakar et al\. \(2024\)Akshara Prabhakar, Thomas L\. Griffiths, and R\. Thomas McCoy\. 2024\.[Deciphering the factors influencing the efficacy of chain\-of\-thought: Probability, memorization, and noisy reasoning](https://doi.org/10.18653/v1/2024.findings-emnlp.212)\.In*Findings of the Association for Computational Linguistics: EMNLP 2024*, pages 3710–3724, Miami, Florida, USA\. Association for Computational Linguistics\.
- Reid et al\. \(2024\)Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy P\. Lillicrap, Jean\-Baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, Ioannis Antonoglou, Rohan Anil, Sebastian Borgeaud, Andrew M\. Dai, Katie Millican, Ethan Dyer, Mia Glaese, Thibault Sottiaux, Ben jamin Lee, and 655 others\. 2024\.[Gemini 1\.5: Unlocking multimodal understanding across millions of tokens of context](https://api.semanticscholar.org/CorpusID:268297180)\.*ArXiv*, abs/2403\.05530\.
- Sun et al\. \(2023\)Hong Sun, Xue Li, Yi Xu, Youkow Homma, Qinhao Cao, Min man Wu, Jian Jiao, and Denis Xavier Charles\. 2023\.[Autohint: Automatic prompt optimization with hint generation](https://api.semanticscholar.org/CorpusID:259924936)\.*ArXiv*, abs/2307\.07415\.
- Suzgun et al\. \(2022\)Mirac Suzgun, Nathan Scales, Nathanael Scharli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V\. Le, Ed Huai hsin Chi, Denny Zhou, and Jason Wei\. 2022\.[Challenging big\-bench tasks and whether chain\-of\-thought can solve them](https://api.semanticscholar.org/CorpusID:252917648)\.In*Annual Meeting of the Association for Computational Linguistics*\.
- Wang et al\. \(2023\)Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka\-Wei Lee, and Ee\-Peng Lim\. 2023\.[Plan\-and\-solve prompting: Improving zero\-shot chain\-of\-thought reasoning by large language models](https://api.semanticscholar.org/CorpusID:258558102)\.In*Annual Meeting of the Association for Computational Linguistics*\.
- Wang et al\. \(2024\)Yu Wang, Shiwan Zhao, Zhihu Wang, Heyuan Huang, Ming Fan, Yubo Zhang, Zhixing Wang, Haijun Wang, and Ting Liu\. 2024\.[Strategic chain\-of\-thought: Guiding accurate reasoning in llms through strategy elicitation](https://api.semanticscholar.org/CorpusID:272423721)\.*ArXiv*, abs/2409\.03271\.
- Wei et al\. \(2022a\)Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed Huai hsin Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus\. 2022a\.[Emergent abilities of large language models](https://api.semanticscholar.org/CorpusID:249674500)\.*ArXiv*, abs/2206\.07682\.
- Wei et al\. \(2022b\)Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Huai hsin Chi, F\. Xia, Quoc Le, and Denny Zhou\. 2022b\.[Chain of thought prompting elicits reasoning in large language models](https://api.semanticscholar.org/CorpusID:246411621)\.*ArXiv*, abs/2201\.11903\.
- Wei et al\. \(2023\)Jerry W\. Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, and Tengyu Ma\. 2023\.[Larger language models do in\-context learning differently](https://api.semanticscholar.org/CorpusID:257378479)\.*ArXiv*, abs/2303\.03846\.
- Xu et al\. \(2022\)Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Yanggang Wang, Haiyu Li, and Zhilin Yang\. 2022\.[Gps: Genetic prompt search for efficient few\-shot learning](https://api.semanticscholar.org/CorpusID:253238066)\.In*Conference on Empirical Methods in Natural Language Processing*\.
- Yang et al\. \(2022\)Zonglin Yang, Li Dong, Xinya Du, Hao Cheng, E\. Cambria, Xiaodong Liu, Jianfeng Gao, and Furu Wei\. 2022\.[Language models as inductive reasoners](https://api.semanticscholar.org/CorpusID:254926851)\.*ArXiv*, abs/2212\.10923\.
- Yu et al\. \(2023\)Fei Yu, Hongbo Zhang, and Benyou Wang\. 2023\.[Natural language reasoning, a survey](https://api.semanticscholar.org/CorpusID:257766470)\.*ACM Computing Surveys*\.
- Zhang et al\. \(2024\)Huaqi Zhang, Lubao Wang, Haiming Shao, Mingxuan Wu, and Wei Ren\. 2024\.[Large model fine\-tuning method based on pre\-cognitive inductive reasoning \- pcir](https://doi.org/10.1109/AINIT61980.2024.10581497)\.In*2024 5th International Seminar on Artificial Intelligence, Networking and Information Technology \(AINIT\)*, pages 1218–1221\.
- Zhang et al\. \(2023\)Zhihan Zhang, Shuohang Wang, Wenhao Yu, Yichong Xu, Dan Iter, Qingkai Zeng, Yang Liu, Chenguang Zhu, and Meng Jiang\. 2023\.[Auto\-instruct: Automatic instruction generation and ranking for black\-box language models](https://doi.org/10.18653/v1/2023.findings-emnlp.659)\.In*Findings of the Association for Computational Linguistics: EMNLP 2023*, pages 9850–9867, Singapore\. Association for Computational Linguistics\.
- Zhang et al\. \(2022\)Zhuosheng Zhang, Aston Zhang, Mu Li, and Alexander J\. Smola\. 2022\.[Automatic chain of thought prompting in large language models](https://api.semanticscholar.org/CorpusID:252762275)\.*ArXiv*, abs/2210\.03493\.
- Zhou et al\. \(2024\)Pei Zhou, Jay Pujara, Xiang Ren, Xinyun Chen, Heng\-Tze Cheng, Quoc Le, Ed Chi, Denny Zhou, Swaroop Mishra, and Huaixiu Steven Zheng\. 2024\.[Self\-discover: Large language models self\-compose reasoning structures](https://api.semanticscholar.org/CorpusID:267500375)\.*ArXiv*, abs/2402\.03620\.
- Zhou et al\. \(2022\)Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba\. 2022\.[Large language models are human\-level prompt engineers](https://api.semanticscholar.org/CorpusID:253265328)\.*ArXiv*, abs/2211\.01910\.
- Zhu et al\. \(2023\)Zhaocheng Zhu, Yuan Xue, Xinyun Chen, Denny Zhou, Jian Tang, Dale Schuurmans, and Hanjun Dai\. 2023\.[Large language models can learn rules](https://api.semanticscholar.org/CorpusID:263834843)\.*ArXiv*, abs/2310\.07064\.

## Appendix APrompt Templates

We use three types of prompts: Strategy Generation, Induction \(including both Instruction Induction and Strategy Induction\), and Inference\.

To structure the reasoning process effectively, we adopt<deduction\>to encapsulate intermediate reasoning steps and<final\_answer\>to indicate the conclusive response\. During evaluation, we directly extract the content within<final\_answer\>for answer matching\. For detailed prompt structures, please refer to Figure[8](https://arxiv.org/html/2605.20924#A7.F8)to[13](https://arxiv.org/html/2605.20924#A7.F13)\.

## Appendix BDetailed Experimental Configurations

### B\.1Models Used in Experiments

We evaluate our framework using 18 models, including four series of large language models \(LLMs\) and reasoning models \(LRMs\), covering both open\-source and proprietary systems\.

##### Llama models

Llama 3\.1 8B, Llama 3\.1 70B, Llama 3\.1 405B, and Llama 3\.3 70BDubey et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib11)\)\.

##### Mistral models

Mistral Nemo 12B, Mistral Small 2, Mistral Small 3, and Mistral Large 2MistralAI \([2024b](https://arxiv.org/html/2605.20924#bib.bib22),[a](https://arxiv.org/html/2605.20924#bib.bib21),[c](https://arxiv.org/html/2605.20924#bib.bib23),[2025](https://arxiv.org/html/2605.20924#bib.bib24)\)\.

##### Gemini models

Gemini 1\.5 Flash 8B, Gemini 1\.5 Flash, Gemini 1\.5 Pro, Gemini 2\.0 Flash Lite Preview, Gemini 2\.0 Flash, and Gemini 2\.0 Pro ExpReid et al\. \([2024](https://arxiv.org/html/2605.20924#bib.bib31)\); Google \([2025](https://arxiv.org/html/2605.20924#bib.bib13)\)\.

##### GPT models

GPT\-4o mini and GPT\-4oOpenAI \([2024a](https://arxiv.org/html/2605.20924#bib.bib27),[b](https://arxiv.org/html/2605.20924#bib.bib28)\)\.

##### LRMs

Gemini 2\.0 Flash Thinking ExpGoogle \([2025](https://arxiv.org/html/2605.20924#bib.bib13)\)and GPT o3 miniOpenAI \([2025](https://arxiv.org/html/2605.20924#bib.bib29)\)withlow,medium, andhighreasoning effort settings\.

### B\.2Other Details

##### Model Version

Table[3](https://arxiv.org/html/2605.20924#A2.T3)lists the exact model versions used in our experiments, following the naming conventions used in the API\.

Table 3:Correspondence between model names and detailed versions\.Table 4:Estimated total costs for each model \(Google experimental models are free and are denoted by ’\-’ in the table, thus not included in the cost calculation\)\.
##### Safety Settings

We disable all safety settings for Gemini to prevent Google’s API from refusing to respond\.

##### Cost Details

Table[4](https://arxiv.org/html/2605.20924#A2.T4)shows the estimated total cost for all experiments, which amounted to approximately $972 with API services provided by SambaNova, Together AI, Mistral AI, Google, and OpenAI\.

##### Model Settings

We set the temperature to 0 and Top\-P to 1\. The maximum output length is set to 8,192 tokens, if supported by the model\.

## Appendix CUse of AI Assistants

In this paper, GPT\-4o and GPT o3 mini were used for grammar refinement and assistance in code development\. However, all outputs from the language models served only as references, and the final writing of the paper and code was reviewed and authored by the researcher\.

## Appendix DDetailed Dataset Information

This appendix provides the Short Phrases used for different tasks across all datasets in our study\. We adopt the Short Phrases fromChen et al\. \([2024b](https://arxiv.org/html/2605.20924#bib.bib7)\), making minor adjustments for clarity where necessary\. The specific Short Phrases for each task are listed in Table[8](https://arxiv.org/html/2605.20924#A7.T8)to Table[10](https://arxiv.org/html/2605.20924#A7.T10)\.

Table 5:Accuracy comparison on three non\-reasoning tasks from BBH\-Induct \(Salient Translation Error Detection, Snarks, and Sports\) across 18 models\. "Salient\." refers to the Salient Translation Error Detection task\. Bold numbers indicate the best performance for each model\.
## Appendix EAnalysis of Non\-Reasoning Tasks in BBH\-Induct

As discussed in Section[4\.2](https://arxiv.org/html/2605.20924#S4.SS2), although BBH is primarily a reasoning\-oriented benchmark, several subtasks—such assalient translation error detection,snarks, andsports understanding—are classified as knowledge\-based tasks bySuzgun et al\. \([2022](https://arxiv.org/html/2605.20924#bib.bib33)\)\.

Table[5](https://arxiv.org/html/2605.20924#A4.T5)presents a direct comparison among Z\-CoT, the Induct baseline, and our framework on these subtasks\. Our method achieves a win\-tie\-lose rate of 30\-10\-20 compared to Induct, and 49\-6\-5 compared to Z\-CoT\. These results indicate that strategy induction can be beneficial for a range of non\-reasoning tasks, in addition to its strengths on reasoning\-oriented benchmarks\. Further experiments and results on additional non\-reasoning tasks are presented in the following section\.

## Appendix FAdditional Experimental Result

Our current setting follows Induct\-Learn\(Chen et al\.,[2024b](https://arxiv.org/html/2605.20924#bib.bib7)\), where the authors constructed the BBH\-Induct dataset by extracting task\-relevant context from the original BBH questions, and removing them from the original question\. This added context, referred to as theShort Phrase\. The following is an example of BBH\-Induct:

Example from Short Phrase TaskShort Phrase:Similar Movie RecommendationQuestion:Minority Report, Total Recall, Inside Out, Forrest GumpOptions:\(A\) Phenomena\(B\) Lilting\(C\) Catwoman\(D\) Edge of Tomorrow

Without the Short Phrase, the question appears as just a list of movie names and options, and the task objective becomes ambiguous\. This is a specific characteristic of the BBH\-Induct dataset\. However, in many real\-world settings \(even in the original BBH dataset\), user queries tend to include the task objective explicitly\. In such cases, the Short Phrase becomes optional\.

### F\.1MMLU\-PRO Result

To further investigate this, we conducted additional experiments comparing the performance with and without Short Phrases\. We chose the MMLU\-PRO dataset for this comparison, as it consists of knowledge\-based questions that better reflect non\-reasoning tasks, which also allows us to evaluate the generalizability of our framework beyond reasoning\-oriented benchmarks\. This design allowed us to systematically examine both issues within a single experimental setting\.

We first present the performance comparison on the MMLU\-PRO dataset with Short Phrases included, to highlight how our framework performs against baselines under this setting\.

Due to computational constraints, we sampled 25 examples from each of the 14 sub\-tasks in MMLU\-PRO, resulting in a total of 350 instances\. We evaluated our framework across 15 different models, and compared it against the current state\-of\-the\-art Induct\-Learn \(Induct\) and Zero\-Shot Chain of Thought \(ZCoT\)\. The experimental results are presented in Table[6](https://arxiv.org/html/2605.20924#A6.T6)\.

ModelZCoTInductOurLlama 3\.1 8B41\.71%44\.00%42\.00%Llama 3\.3 70B70\.29%69\.14%71\.43%Mistral Small 249\.71%47\.14%46\.86%Mistral Small 364\.86%67\.43%66\.29%Mistral Large 268\.57%68\.57%68\.86%Gemini 1\.5 Flash 8B52\.29%53\.71%54\.57%Gemini 1\.5 Flash66\.00%65\.14%65\.71%Gemini 1\.5 Pro74\.86%73\.43%76\.29%Gemini 2\.0 Flash Lite68\.29%68\.29%67\.71%Gemini 2\.0 Flash63\.14%62\.86%70\.29%Gemini 2\.0 Pro Exp82\.29%84\.29%82\.86%GPT\-4o mini62\.29%63\.14%60\.57%GPT\-4o75\.14%76\.00%76\.57%Gemini 2\.0 Flash Thinking75\.43%75\.14%75\.71%GPT o3 mini \(low\)76\.57%74\.00%77\.14%GPT o3 mini \(medium\)77\.71%77\.43%78\.57%GPT o3 mini \(high\)79\.71%79\.14%80\.00%Table 6:Accuracy \(%\) of different models on MMLU\-PRO in zero\-shot settings\. The highest overall accuracy is shown in bold\. “Induct” refers to the INDUCT baseline\.
### F\.2Effect of Short Phrases

To further examine the role of Short Phrases, we compare the performance of our framework with and without Short Phrases using the same experimental protocol described above\. Table[7](https://arxiv.org/html/2605.20924#A6.T7)presents the results for all evaluated models under both settings\.

We observe that the performance differences across models are generally minor\. This outcome can be attributed to the nature of the MMLU\-PRO dataset: the questions are already well\-structured and self\-explanatory, so the inclusion of an explicit Short Phrase offers limited additional benefit\.

These results indicate that, for benchmarks like MMLU\-PRO where questions themselves provide sufficient task context, Short Phrases are optional and do not significantly impact performance\.

ModelNon SPSPLlama 3\.1 8B47\.43%42\.00%Llama 3\.3 70B70\.86%71\.43%Mistral Small 249\.43%46\.86%Mistral Small 366\.00%66\.29%Mistral Large 269\.71%68\.86%Gemini 1\.5 Flash 8B54\.00%54\.57%Gemini 1\.5 Flash66\.00%65\.71%Gemini 1\.5 Pro76\.00%76\.29%Gemini 2\.0 Flash Lite69\.71%67\.71%Gemini 2\.0 Flash68\.29%70\.29%Gemini 2\.0 Pro Exp84\.00%82\.86%GPT\-4o mini60\.29%60\.57%GPT\-4o76\.00%76\.57%Gemini 2\.0 Flash Thinking76\.29%75\.71%GPT o3 mini \(low\)78\.29%77\.14%GPT o3 mini \(medium\)78\.57%78\.57%GPT o3 mini \(high\)79\.71%80\.00%Table 7:Accuracy comparison of our framework on the MMLU\-PRO dataset with and without Short Phrases\. Bold values indicate the better performance for each model\. SP denotes Short Phrase\.

## Appendix GCase Study – BBH\-Induct Snarks

We present a case study on the Snarks \(sarcasm detection\) task from BBH\-Induct\. Strategies generated from individual questions are typically much shorter and less detailed compared to those obtained after induction\. As a result, applying a single strategy with S\-CoT on Gemini 2\.0 Flash yields an accuracy of only 0\.40\. After our induction stage consolidates these strategies, the resulting instruction is more comprehensive and effective \(see Figure[7](https://arxiv.org/html/2605.20924#A7.F7)\), leading to a substantial improvement in accuracy to 0\.76\.

Short PhraseSnarks \(Sarcasm Detection\)Question 1\(A\) Think of the poor man\! What will he do without his job?\(B\) Think of the poor man\! What will he do without his yacht?Strategy 1Step 1: Analyze option \(A\) to determine if the statement "Think of the poor man\! What will he do without his job?" is likely to be sarcastic\. Consider the common understanding of job loss as a serious issue\.Step 2: Analyze option \(B\) to determine if the statement "Think of the poor man\! What will he do without his yacht?" is likely to be sarcastic\. Consider the common understanding of owning a yacht as a luxury, and the potential for sarcasm when expressing concern about someone losing a luxury item\.Step 3: Compare the two options and determine which statement is more likely to be sarcastic based on the context and the implied meaning\.Step 4: Select the option \(A or B\) that is more likely to be sarcastic and output the corresponding letter\.Question 2\(A\) What a poor guy, years of work and he only got a $50,000 scholarship\(B\) What a poor guy, years of work and he only got a $50 scholarshipStrategy 2Step 1: Analyze option \(A\) for potential sarcasm\. Consider the implied meaning of "poor guy" in relation to receiving a \+$50,000 scholarship\. Evaluate if the statement is intended to convey the opposite of its literal meaning\.Step 2: Analyze option \(B\) for potential sarcasm\. Consider the implied meaning of "poor guy" in relation to receiving a $50 scholarship\. Evaluate if the statement is intended to convey the opposite of its literal meaning\.Step 3: Compare the two options \(A\) and \(B\)\. Determine which option is more likely to be sarcastic based on the discrepancy between the expected outcome \(years of work\) and the actual reward \($50,000 vs\. $50\)\.Step 4: Select the option \(A or B\) that demonstrates a greater degree of incongruity between the situation and the speaker’s expressed sentiment, indicating a higher likelihood of sarcasm\.Step 5: Output the corresponding letter \(A or B\) of the selected option\.Question 3\(A\) Have you tried not being poor? It is much easier\(B\) Have you tried not being rude? It is much easierStrategy 3Step 1: Analyze option A for sarcastic intent\. Consider the inherent difficulty of overcoming poverty and whether the statement implies the opposite\.Step 2: Analyze option B for sarcastic intent\. Consider the ease of being polite and whether the statement implies the opposite\.Step 3: Compare the two options and determine which statement is more likely to be interpreted as sarcastic based on common understanding and social context\.Step 4: Select the option \(A or B\) that exhibits a higher degree of sarcasm\.Step 5: Output the corresponding letter \(A or B\) as the answer\.Inducted Strategy \(STRATEGY\-INDUCT\)1\.Analyze Sentence \(A\):Carefully examine sentence \(A\) for potential indicators of sarcasm\. Consider the context and whether the statement could be interpreted as meaning the opposite of what it says, with the intention to criticize or be humorous\. Look for elements such as:– Irony: A statement where the intended meaning is the opposite of the literal meaning\.– Hyperbole: Exaggeration for emphasis or effect\.– Satire: The use of humor, irony, exaggeration, or ridicule to expose and criticize people’s stupidity or vices\.– Witty remarks: Cleverly humorous or ironic expressions\.– Ambivalent remarks: Statements that have mixed feelings or contradictory ideas\.2\.Analyze Sentence \(B\):Repeat the analysis process from Step 1 for sentence \(B\)\.3\.Compare Sentences:Compare the two sentences based on the likelihood of sarcastic intent\. Consider which sentence is more likely to be used to express the opposite of its literal meaning in a critical or humorous way, based on the definition of sarcasm\.4\.Select the More Sarcastic Sentence:Determine which sentence \(A or B\) is more likely to be sarcastic based on your analysis\.5\.Output the Answer:Output the letter corresponding to the sentence identified as more likely to be sarcastic\.Figure 7:Example strategies and the induced instruction fromStrategy\-Inducton the snarks \(sarcasm detection\) task in BBH\-Induct \(Gemini 2\.0 Flash\)\. The induced instruction unifies and formalizes pragmatic cues such as irony and exaggeration from diverse strategies\.You are tasked with designing a strategy to solve a given question\. Your goal is to carefully analyze the question and formulate a clear and effective plan outlining the steps needed to solve it\. Remember, you should not actually solve the question or provide a final answer\.Additionally, make sure to base your strategy on the details provided in<task\_information\>,<answer\_format\>, and<question\>for the most relevant approach\.Please follow these steps:1\. Carefully read and analyze the question\.2\. Identify the key components and challenges within the question\.3\. Develop a step\-by\-step strategy to address the question\.4\. Outline your strategy using numbered steps\.Present your strategy in the following format:<strategy\>Step 1: \[Brief description of the first step\]Step 2: \[Brief description of the second step\]Step 3: \[Brief description of the third step\]\[Continue with additional steps as needed\]</strategy\>\*\*Important:\*\* Focus solely on creating a strategy to solve the question\. Do not attempt to solve the question or provide a final answer\. Your strategy should outline the approach to solving the question, not the solution itself\.Here is the task\_information, answer\_format and the question you need to create a strategy for:<task\_information\>\{ \}</task\_information\><answer\_format\>\{ \}</answer\_format\><question\>\{ \}</question\>Figure 8:Prompt Template for Strategy Generation \(Ours\)\.You are an expert in the field of NLP \(Natural Language Processing\), possessing exceptional data observation and analysis skills\. Your expertise includes extracting significant rules from complex datasets and formulating precise task instructions based on these rules\.Currently, you are focusing on analyzing a specific set of examples, deriving insights to formulate a clear and detailed task instruction\. This task description will serve as an instruction set, guiding the execution of the related tasks\.The task instruction should include the following elements:1\. \*\*Task Content\*\*: Clearly define the purpose of the task and the specific activities required to be completed\.2\. \*\*Input Format\*\*: Provide detailed descriptions of the types of data accepted, their formats, and how to process these data effectively\.3\. \*\*Operational Steps\*\*: Detail the specific step\-by\-step procedures required to complete the task\. Remember to ensure that the final output format adheres to the "answer\_format" specification\.Ensure that your task instruction is concise, clear, and easily understandable by users\. It should provide all necessary information for someone to successfully complete the task\.Present your task instruction in the following format:<task\_instruction\>\{Your task instruction here, with each element under its own subheading\}</task\_instruction\>Remember to focus solely on creating the task instruction based on the given information and examples\. Do not attempt to complete the task itself\.Here is thetask\_information,answer\_formatand the examples you need to create a Task Instruction for:<task\_information\>\{\}</task\_information\><answer\_format\>\{\}</answer\_format\><examples\>\{\}</examples\>Figure 9:Prompt Template for INDUCT \(Instruction Induction\)You are an expert in the field of NLP \(Natural Language Processing\), with outstanding data observation and analysis skills\.Your expertise includes inductively deriving better solution rules \(or methods\) from various question\-solving strategies and question\-pair examples within the same task\. Based on these rules, you can formulate precise task instructions\.Currently, you are focused on analyzing a set of specific question\-solving strategies and question\-pair examples to gain insights for creating clear and detailed task instruction\. This task instruction will serve as instruction sets to guide the execution of related tasks\.The task instruction should include the following elements:1\. \*\*Task Content\*\*: Clearly define the purpose of the task and the specific activities required to be completed\.2\. \*\*Input Format\*\*: Provide detailed descriptions of the types of data accepted, their formats, and how to process these data effectively\.3\. \*\*Operational Steps\*\*: Detail the specific step\-by\-step procedures required to complete the task\. Remember to ensure that the final output format adheres to the "answer\_format" specification\.Ensure your task instruction are concise, clear, and easy to understand for users\. They should provide all the necessary information for someone to successfully complete the task\.Present your task instruction in the following format:<task\_instruction\>\{Your task instruction here, with each element under its own subheading\}</task\_instruction\>Please remember to focus solely on creating the task instruction based on the given information and examples\. Do not attempt to complete the task itself\.Here is thetask\_information,answer\_formatand the examples you need to create a Task Instruction for:<task\_information\>\{\}</task\_information\><answer\_format\>\{\}</answer\_format\><examples\>\{\}</examples\>Figure 10:Prompt Template for Strategy Induction\(Ours\)\[Task Instruction\]\{\}\*Output Format\*\*:\{\}Follow these steps carefully:1\. Provide step\-by\-step deduction that answers the question<deduction\>\[Your step\-by\-step deduction here\]</deduction\>2\. Based on your deduction, provide the final answer according to the rules specified in the "Output Format" section\.If unsure and "Output Format" is option, guess the closest option\. Present your final answer in the following format:<final\_answer\>\[Your final answer here\]</final\_answer\>NoteDo not use programming or code to solve this question\.Figure 11:Prompt Template for Zero\-shot Chain\-of\-Thought \(ZCoT\)\.\[Task Instruction\]\{\}\*Output Format\*\*:\{\}Follow these steps carefully:1\. Carefully consider the problem and generate the strategic knowledge that would best guide the problem\-solving process\.<strategy\>\[Your strategy here\]</strategy\>2\. Provide step\-by\-step deduction that answers the question\.<deduction\>\[Your step\-by\-step deduction here\]</deduction\>3\. Based on your deduction, provide the final answer according to the rules specified in the "Output Format" section\.If unsure and "Output Format" is option, guess the closest option\. Present your final answer in the following format:<final\_answer\>\[Your final answer here\]</final\_answer\>NoteDo not use programming or code to solve this question\.Figure 12:Prompt Template for Automatic Strategic Chain\-of\-Thought \(SCoT\)\[Task Instruction\]\{\}\*Output Format\*\*: \{\}Follow these steps carefully:1\. Provide step\-by\-step deduction that answers the question<deduction\>\[Your step\-by\-step deduction here\]</deduction\>2\. Based on your deduction, provide the final answer according to the rules specified in the "Output Format" section\.If unsure and "Output Format" is option, guess the closest option\. Present your final answer in the following format:<final\_answer\>\[Your final answer here\]</final\_answer\>NoteDo not use programming or code to solve this question\.Figure 13:Prompt Template for INDUCT andStrategy\-Induct\(Ours\)Strategy\-Induct Prompt\[Task Instruction\]\#\#\# Task Content:The purpose of this task is to detect sarcasm in a given set of options and select the correct option that exhibits sarcasm\. Sarcasm often involves irony, exaggeration, or statements that contradict reality or common expectations\. The task requires careful analysis of the tone, context, and implied meaning of the options to identify the sarcastic statement\.\#\#\# Input Format:\- You will be provided with a question containing two or more options \(e\.g\., A, B, etc\.\)\.\- Each option is a statement or phrase that needs to be analyzed for sarcasm\.\- The input will be presented in a format where each option is labeled with a letter \(e\.g\., A, B\)\.\#\#\# Operational Steps:1\.Understand the Task Requirement: Recognize that the goal is to detect sarcasm in the given options and select the one that best represents sarcasm\.2\.Analyze the Context of Each Option:\- Carefully read each option\.\- Identify the tone, context, and any potential mismatch between the literal meaning and the implied meaning\.3\.Identify Indicators of Sarcasm:\- Look for exaggeration, irony, or statements that seem to contradict reality or common expectations\.\- Pay attention to phrases that imply the opposite of what is being said or highlight an absurdity\.4\.Compare the Options:\- Evaluate each option to determine which one contains a sarcastic tone\.\- Consider the likelihood of the statement being ironic, exaggerated, or mocking\.5\.Select the Correct Option:\- Choose the single letter \(e\.g\., A, B, etc\.\) corresponding to the option that best represents sarcasm\.6\.Verify the Choice:\- Double\-check your reasoning to ensure the selected option aligns with the task of sarcasm detection\.7\.Output the Answer:\- Provide the output as a single letter \(e\.g\., A, B, etc\.\) corresponding to the chosen option, adhering to the specified answer format\.Prompt TemplateOutput Format:The output is a single letter corresponding to the correct option from the list \(e\.g\., A, B, C, etc\.\)\.Follow these steps carefully:1\. Provide step\-by\-step deduction that answers the question<deduction\>\[Your step\-by\-step deduction here\]</deduction\>2\. Based on your deduction, provide the final answer according to the rules specified in the "Output Format" section\.If unsure and "Output Format" is option, guess the closest option\. Present your final answer in the following format:<final\_answer\>\[Your final answer here\]</final\_answer\>New Question\[Question\]\(A\) The best way to defeat a stronger/larger and better regimented force is by fighting on their terms\(B\) The best way to defeat a stronger/larger and better regimented force is by fighting on your terms\[Answer\]Let’s refer to the "Operational Steps" and think step by step\.Model Response<deduction\>1\.Understand the Task Requirement: The task is to detect sarcasm in the given options and select the one that best represents sarcasm\. Sarcasm often involves irony, exaggeration, or a contradiction to reality or common expectations\.2\.Analyze the Context of Each Option:\-Option A: "The best way to defeat a stronger/larger and better regimented force is by fighting on their terms\." This statement suggests that one should fight a stronger and better\-organized force by adhering to their strengths and strategies\. This seems counterintuitive and ironic because it contradicts common sense; fighting on the enemy’s terms would likely lead to defeat\.\-Option B: "The best way to defeat a stronger/larger and better regimented force is by fighting on your terms\." This statement aligns with common sense and strategy\. It suggests that one should fight in a way that plays to their own strengths, which is logical and not sarcastic\.3\.Identify Indicators of Sarcasm:\-Option Acontains irony because it suggests doing something that is clearly counterproductive \(fighting on the enemy’s terms\)\. This is a sarcastic way of pointing out the absurdity of such an approach\.\-Option Bis straightforward and lacks any ironic or sarcastic tone\.4\.Compare the Options:\- Option A is sarcastic because it uses irony to highlight the absurdity of fighting on the enemy’s terms\.\- Option B is not sarcastic; it is a logical and straightforward statement\.5\.Select the Correct Option:\- Based on the analysis, Option A is the sarcastic statement\.6\.Verify the Choice:\- The reasoning aligns with the task of sarcasm detection\. Option A uses irony to make its point, making it the correct choice\.</deduction\><final\_answer\>A</final\_answer\>Figure 14:Example of Strategy\-Induct Prompting on theSnarkstask in the BBH\-Induct dataset, using GPT\-4o, where the correct answer is A\.Zero Shot CoT Prompt\[Task Instruction\]Snarks \(Sarcasm Detection\)Prompt TemplateOutput Format: The output is a single letter corresponding to the correct option from the list \(e\.g\., A, B, C, etc\.\)\.Follow these steps carefully:1\. Provide step\-by\-step deduction that answers the question<deduction\>\[Your step\-by\-step deduction here\]</deduction\>2\. Based on your deduction, provide the final answer according to the rules specified in the "Output Format" section\.If unsure and "Output Format" is option, guess the closest option\. Present your final answer in the following format:<final\_answer\>\[Your final answer here\]</final\_answer\>Note: Do not use programming or code to solve this question\.New Question\[Question\]\(A\) The best way to defeat a stronger/larger and better regimented force is by fighting on their terms\(B\) The best way to defeat a stronger/larger and better regimented force is by fighting on your terms\[Answer\]Let’s think step by step\.Model Response<deduction\>1\. The question revolves around the strategy to defeat a stronger, larger, and better\-regimented force\. This implies that the opponent has an advantage in terms of size, strength, and organization\.2\. Option \(A\) suggests fighting on the opponent’s terms\. This would mean engaging in a way that plays to the opponent’s strengths, which is counterintuitive\. If the opponent is stronger and better organized, fighting on their terms would likely lead to defeat\.3\. Option \(B\) suggests fighting on your own terms\. This implies leveraging your own strengths and avoiding the opponent’s advantages\. Historically and strategically, this approach is more effective when facing a superior force\. For example, guerrilla warfare or asymmetric tactics are often used to counteract a stronger opponent\.4\. Based on logic and historical precedent, fighting on your own terms \(Option B\) is the more effective strategy\.</deduction\><final\_answer\>B</final\_answer\>Figure 15:Example of Zero\-shot CoT Prompting on theSnarkstask in the BBH\-Induct dataset, using GPT\-4o, where the correct answer is A\.Table 8:Mapping between tasks, subtasks, and their corresponding short phrases in BBH\-Induct\.Table 9:Mapping between tasks, subtasks, and their corresponding short phrases in Evals\-Induct\.Table 10:Mapping between tasks, subtasks, and their corresponding short phrases in Shift Cipher\.