Refine Thought: A Test-Time Inference Method for Embedding Model Reasoning

# Refine Thought: A Test-Time Inference Method for Embedding Model Reasoning
Source: [https://arxiv.org/html/2511.13726](https://arxiv.org/html/2511.13726)
###### Abstract

We propose RT \(Refine Thought\), a method that can enhance the semantic reasoning ability of text embedding models\. The method obtains the final semantic representation by running multiple forward passes of the text embedding model\. Experiments show that RT achieves significant improvements on semantic reasoning tasks in BRIGHT\[[17](https://arxiv.org/html/2511.13726#bib.bib17)\]and the person job matching benchmark PJBenchmark111PJBenchmark \(Person\-Job Benchmark\) is a test set for recruitment matching scenarios\., while maintaining consistent performance on general\-purpose semantic understanding tasks such as C\-MTEB\. Our results indicate that RT is effective because it further activates the semantic reasoning ability learned during pretraining by decoder\-only text embedding models\(e\.g\., Qwen3\-Embedding\-8B\[[20](https://arxiv.org/html/2511.13726#bib.bib20)\]\)\. RT can be seen as a test\-time\[[16](https://arxiv.org/html/2511.13726#bib.bib16)\]inference method\.

## 1 Introduction

Neural networks can be seen as both representation learners that automatically acquire hierarchical features and differentiable universal computers\[[7](https://arxiv.org/html/2511.13726#bib.bib7)\]\. Reasoning is essentially a multi\-step computation involving sequential decisions\. It includes arithmetic, symbolic, semantic, and spatial reasoning\[[6](https://arxiv.org/html/2511.13726#bib.bib6)\]\. Among these, semantic reasoning relies both on multi\-step computation and on hierarchical language representations\. Most text embedding models perform self\-supervised learning on large\-scale text via pretraining objectives, which significantly improves hierarchical language representations\.Although this greatly boosts performance on general semantic tasks, limitations remain on semantic reasoning tasks that require multi\-step computation\[[17](https://arxiv.org/html/2511.13726#bib.bib17)\]\.

In text generation, semantic reasoning has been extensively studied\. There are two ways to improve reasoning through training: one is structural unrolling, which deepens the network to allow more layers of functional composition and thereby enhances its expressive power\[[11](https://arxiv.org/html/2511.13726#bib.bib11)\]; the other is temporal unrolling, which does not increase the depth but instead reuses the same structure across multiple time steps, thereby allowing the model to progressively refine its internal computation\[[18](https://arxiv.org/html/2511.13726#bib.bib18),[14](https://arxiv.org/html/2511.13726#bib.bib14)\]\.

Most text embedding models\[[19](https://arxiv.org/html/2511.13726#bib.bib19)\]obtain semantic representations by encoding the query and the document each with a single forward pass, and then use nearest\-neighbor similarity for retrieval or matching\. However, when a query requires multi\-step computation—such as multiple constraints, cross\-sentence integration, or sequential reasoning—a single forward pass fails to capture intermediate reasoning representations and lacks sufficient computational depth\.

Research on semantic reasoning within text embedding models is still limited\. With the growing demand for RAG\[[3](https://arxiv.org/html/2511.13726#bib.bib3)\]and Agent systems\[[10](https://arxiv.org/html/2511.13726#bib.bib10)\], LLMs increasingly need to retrieve and reason over internal information\. As a result, enhancing the semantic reasoning ability of text embedding models has become a key research challenge\. To address this limitation, we propose RT for text embedding models\. RT executes multiple forward passes on the query side to obtain semantic reasoning at test time, without modifying model parameters\. Across BRIGHT, C\-MTEB, and PJBenchmark, our experiments show that RT significantly improves semantic reasoning tasks while remaining stable on general\-purpose semantic understanding tasks\.

Our contributions are summarized as follows:

1. 1\.We propose RT, a method that improves semantic reasoning in text embedding models without extra training, and provide algorithmic details\.
2. 2\.We systematically analyze the relationship between the performance gain and task complexity\. The results show that the more complex the task, the more pronounced the gains from RT, while performance on simple tasks remains largely unchanged\.
3. 3\.We validated RT across multiple benchmarks, indicating strong potential for real\-world applications\.

## 2 Related Work

Neural Turing Machine research shows that when models fail to reason, the problem is usually not logical but computational — their computation graphs are not unrolled long enough in time for information to pass through\. Complex reasoning requires recursive, variable\-length computation over multiple time steps\[[4](https://arxiv.org/html/2511.13726#bib.bib4)\]\. Chain\-of\-Thought \(CoT\) increases text output length, enabling more time steps for multi\-step computation\[[18](https://arxiv.org/html/2511.13726#bib.bib18)\]\. DeepSeek\-R1 improves reasoning by training models via reinforcement learning to force chain\-of\-thought outputs\[[1](https://arxiv.org/html/2511.13726#bib.bib1)\]\.

Recent theoretical work provides a formal basis for the statement that “temporal unrolling improves arithmetic and symbolic reasoning”\[[8](https://arxiv.org/html/2511.13726#bib.bib8)\]\. With fixed network depth and limited precision, allowing a model to generate and exploit a CoT sequence of lengthTTduring inference effectively trades time steps for computational depth\. This implies that reasoning ability can be systematically improved by extending time steps without altering the network structure\.

## 3 Method

### 3\.1Problem Definition

Given a queryxqx\_\{q\}a set of candidate documentsC=xdC=\{x\_\{d\}\}, text encoders usually compute embeddingshq​\(xq\)h\_\{q\}\(x\_\{q\}\)andhd​\(xd\)h\_\{d\}\(x\_\{d\}\)through a single forward pass, and then rank documents according to their similarity:

d^=arg⁡maxd∈C⁡sim​\(hq​\(xq\),hd​\(xd\)\)\\hat\{d\}=\\arg\\max\_\{d\\in C\}\\;\\mathrm\{sim\}\\\!\\big\(h\_\{q\}\(x\_\{q\}\),\\,h\_\{d\}\(x\_\{d\}\)\\big\)
This approach is computationally efficient, however, when the queryxqx\_\{q\}involves multiple constraints, cross\-sentence information integration, or sequential reasoning, a single forward pass often fails to preserve the underlying reasoning trajectory\. We refer to such cases as semantic reasoning tasks\. The reasoning process forxqx\_\{q\}can be formalized as:

ht=\{f​\(x\)if​t=1f​\(x,h1,…,ht−1\)if​t∈\{2,…,T\}h\_\{t\}=\\begin\{cases\}f\(x\)&\\text\{if \}t=1\\\\\[6\.45831pt\] f\\\!\\left\(x,\\,h\_\{1\},\\ldots,h\_\{t\-1\}\\right\)&\\text\{if \}t\\in\\\{2,\\dots,T\\\}\\end\{cases\}
In this process, the model passes through multiple states to obtain the final semantic representation\.

### 3\.2The Refine Thought \(RT\) Method

We use a decoder\-only text embedding model that first performs a forward pass for the queryxqx\_\{q\}\. The representations of all tokens before the \[EOS\] token are mean\-pooled to obtain the intermediate representationht−1h\_\{t\-1\}for thett\-th step\. Based on this, the process is iteratively repeated forTTsteps, and the intermediate representations\{h1,h2,…,hT−1\}\\\{h\_\{1\},h\_\{2\},\\ldots,h\_\{T\-1\}\\\}collectively form the reasoning trajectory\. The detailed procedure is as follows:

Listing 1:RT \(Refine Thought\) Method – Python Pseudocodedefrefine\_thought\(x\_q,T,encoder\):

h\_0=encoder\(x\_q\)

states=\[h\_0\]

fortinrange\(1,T\+1\):

prev\_state=states\[\-1\]

x\_q\_with\_all\_states=concatenate\(x\_q,states\)

h\_t=encoder\(x\_q\_with\_all\_states\)

states\.append\(h\_t\)

returnstates\[\-1\]

This process effectively adds extra computation steps during inference, improving the model’s reasoning depth\. In contrast to the explicit CoT approach\[[18](https://arxiv.org/html/2511.13726#bib.bib18)\], RT performs these “time steps” implicitly in the hidden space\[[21](https://arxiv.org/html/2511.13726#bib.bib21)\], without generating additional text or altering the document representations or indexes\. The schematic of the RT method is shown in Figure[1](https://arxiv.org/html/2511.13726#S3.F1)\.

![Refer to caption](https://arxiv.org/html/2511.13726v2/images/image1.png)Figure 1:RT\(Refine Thought\)method overviewThe step countTTis set based on task complexity, with more complex tasks requiring largerTT\.

## 4 Experiments

### 4\.1Experimental Setup

We evaluate RT on BRIGHT, C\-MTEB, and PJBenchmark, and choose Qwen3\-Embedding\-8B\[[20](https://arxiv.org/html/2511.13726#bib.bib20)\]and bge\-large\-zh\-v1\.5\[[19](https://arxiv.org/html/2511.13726#bib.bib19)\]to represent decoder\-only and encoder\-only text embedding architectures, respectively\.

We evaluate three types of tasks: \(i\) Semantic Textual Similarity \(STS\) tasks from C\-MTEB, used to verify the robustness of RT on general\-purpose semantic understanding; \(ii\) Semantic reasoning subtasks from BRIGHT \(e\.g\., Psy, TheoT\), used to assess its gains on multi\-step reasoning; \(iii\) Recruitment matching tasks from PJBenchmark \(JD2CV, JD2JD, etc\.\), used to examine its effectiveness in real\-world semantic reasoning scenarios\.

RT is applied only on the queryxqx\_\{q\}; documentsxdx\_\{d\}is fixed\. We set the number of iterative steps toT∈\{1,2,…,10\}T\\\!\\in\\\!\\\{1,2,\\ldots,10\\\}, whereT=1T\{=\}1is the single\-step baseline, and larger values correspond to deeper temporal unrolling\. All experiments use the same hardware and batch settings to ensure fairness and comparability\.

### 4\.2Performance and Ablation Experiments

Overall, RT remains stable on general\-purpose semantic understanding tasks and achieves consistent gains on semantic reasoning tasks\. On C\-MTEB STS subtasks, RT is essentially on par with the baseline[1](https://arxiv.org/html/2511.13726#S4.T1), indicating that temporal unrolling does not degrade representation quality in standard similarity benchmarks\. Across BRIGHT subtasks, RT improves the overall average from 22\.9 to 23\.1 \(≈\\approx\+1%, Table[2](https://arxiv.org/html/2511.13726#S4.T2)\), indicating that additional reasoning steps bring marginal benefits to complex retrieval\. On PJBenchmark, RT achieves substantial gains in the Algorithm domain \(e\.g\., JD2CV: \+19%, CV2CV: \+21%\), while performance in the Finance domain slightly declines by 2%[3](https://arxiv.org/html/2511.13726#S4.T3), suggesting that its benefits depend on task structure and domain noise\.

Table 1:Performance on C\-MTEB STS subtasksTable 2:Performance on BRIGHTTable 3:Performance on PJBenchmarkWe examine the effect of the iteration step numberTT[2](https://arxiv.org/html/2511.13726#S4.F2)on performance\. Results indicate that the most notable gains occur atT=2T=2–33, after which performance saturates forT\>3T\>3, aligning with the notion of trading time steps for effective depth\. On general semantic understanding tasks such as STS\-B in C\-MTEB, RT performs on par with the baseline, introducing no additional noise\. In contrast, for high\-complexity semantic reasoning tasks such as BRIGHT and PJBenchmark, RT brings substantial improvements, suggesting that temporal unrolling is particularly beneficial in scenarios requiring multi\-step or compositional reasoning\.

![Refer to caption](https://arxiv.org/html/2511.13726v2/images/image2.png)Figure 2:Performance of the RT Method across Tasks and Iteration StepsWe further compare encoder\-only and decoder\-only architectures under the same RT setup on the PJBenchmark JD2CV task \(Table[4](https://arxiv.org/html/2511.13726#S4.T4)\)\. Qwen3\-Embedding\-8B improves from 62\.77 to 74\.33, whereas bge\-large\-zh\-v1\.5 rises only modestly from 21\.66 to 23\.06\. These results suggest that RT’s temporal unrolling mechanism aligns more naturally with autoregressive \(decoder\-only\) architectures\. In contrast, encoder\-only models can simulate iteration via an outer loop, but their architectural constraints and state\-update mechanisms limit performance gains\. This supports the hypothesis that recursion and variable\-length computation are key to complex reasoning, and that decoder\-only embedding models possess a natural advantage for temporal unrolling\.

Table 4:PJBenchmark JD2CV task comparison
### 4\.3Analysis and Discussion

The RT method achieves strong improvements on semantic reasoning tasks while remaining comparable on general\-purpose semantic understanding benchmarks\. The gains peak atT=2T=2–33and level off for largerTT, supporting the idea that additional time steps increase effective reasoning depth\. Decoder\-only models benefit more than encoder\-only ones, showing that autoregressive architectures more effectively integrate previously generated reasoning trajectories\. The improvement scales with task complexity, nearly zero for STS\-B but substantial for BRIGHT and PJBenchmark, showing that temporal unrolling helps the model compose reasoning more effectively in hidden space\.

Although the RT method performs well on complex tasks, it also has notable limitations\. First, RT is an inference\-only extension and lacks explicit credit assignment\[[15](https://arxiv.org/html/2511.13726#bib.bib15)\]\. As a result, the model cannot assess the effectiveness of each computation step, and the additional iterations only make limited adjustments within the existing parameter space, without forming new strategies or long\-term memory\. Second, RT shows little improvement on shallow or noisy tasks, and its reasoning trajectories tend to drift on long or loosely organized inputs, mainly due to the absence of explicit structural constraints and memory management\[[13](https://arxiv.org/html/2511.13726#bib.bib13)\]\. Finally, RT is sensitive to both the number of iterations and the state aggregation scheme: too many steps increase latency, while too few reduce reasoning depth\. Adaptive step scheduling and early\-stopping mechanisms are therefore needed in practice to maintain stability and efficiency\.

## 5 Conclusion and Future Work

We propose RT, a test\-time method that enhances the reasoning capability of text representation models, addressing their limited computational depth in semantic reasoning tasks\. Experiments show that RT significantly improves performance on semantic reasoning benchmarks such as BRIGHT and PJBenchmark, while maintaining stable results on general\-purpose semantic understanding tasks such as C\-MTEB, confirming the effectiveness of temporal unrolling for complex semantic reasoning\. RT treats semantic reasoning as a temporally unfolded computational process, enabling the model to acquire additional reasoning steps at test time and thereby extend its effective reasoning depth without modifying parameters\. Our results confirm that recursion and variable\-length computation underlie complex reasoning, and that decoder\-only architectures are inherently more effective for semantic reasoning\. Although RT method performs well across various tasks, its still relies on heuristic control without fine\-grained learning or validation\. Future directions include integrating lightweight retraining or reinforcement learning\[[2](https://arxiv.org/html/2511.13726#bib.bib2)\]for reward modeling\[[9](https://arxiv.org/html/2511.13726#bib.bib9)\]and developing more efficient temporal unrolling strategies under decoder\-only architectures\.

Prior research indicates that large reasoning models acquire their reasoning ability through exposure to reasoning trajectories during pre\-training\[[12](https://arxiv.org/html/2511.13726#bib.bib12),[5](https://arxiv.org/html/2511.13726#bib.bib5)\]\. This insight likely explains RT’s effectiveness on decoder\-only models such as Qwen3\-Embedding\-8B\. At present, our experiments focus mainly on semantic reasoning tasks, without extending to symbolic or arithmetic reasoning, and the range of evaluated models remains limited\. Therefore, further studies are required to disentangle whether the observed improvements arise from differences in architecture, task type, or training data, and to verify the general advantage of decoder\-only architectures under the RT framework\.

## References

- DeepSeek\-AI et al\. \[2025\]DeepSeek\-AI, D\. Guo, D\. Yang, H\. Zhang, J\. Song, R\. Zhang, R\. Xu, Q\. Zhu, S\. Ma, P\. Wang, et al\.Deepseek\-r1: Incentivizing reasoning capability in llms via reinforcement learning\.*arXiv preprint arXiv:2501\.12948*, 2025\.URL[https://arxiv\.org/abs/2501\.12948](https://arxiv.org/abs/2501.12948)\.
- Dong et al\. \[2025\]Q\. Dong, L\. Dong, Y\. Tang, T\. Ye, Y\. Sun, Z\. Sui, and F\. Wei\.Reinforcement pre\-training\.*arXiv preprint arXiv:2506\.08007*, 2025\.URL[https://arxiv\.org/abs/2506\.08007](https://arxiv.org/abs/2506.08007)\.
- Fan et al\. \[2024\]W\. Fan, Y\. Ding, L\. Ning, S\. Wang, H\. Li, D\. Yin, T\.\-S\. Chua, and Q\. Li\.A survey on rag meeting llms: Towards retrieval\-augmented large language models\.In*Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining \(KDD\)*\. Association for Computing Machinery, 2024\.
- Graves et al\. \[2014\]A\. Graves, G\. Wayne, and I\. Danihelka\.Neural turing machines\.*arXiv preprint arXiv:1410\.5401*, 2014\.URL[https://arxiv\.org/abs/1410\.5401](https://arxiv.org/abs/1410.5401)\.
- Hatamizadeh et al\. \[2025\]A\. Hatamizadeh, S\. N\. Akter, S\. Prabhumoye, J\. Kautz, M\. Patwary, M\. Shoeybi, B\. Catanzaro, Y\. Choi, et al\.Rlp: Reinforcement as a pretraining objective\.*arXiv preprint arXiv:2510\.01265*, 2025\.URL[https://arxiv\.org/abs/2510\.01265](https://arxiv.org/abs/2510.01265)\.
- Huang and Chang \[2023\]J\. Huang and K\. C\.\-C\. Chang\.Towards reasoning in large language models: A survey\.In*Findings of the Association for Computational Linguistics: ACL 2023*\. Association for Computational Linguistics, 2023\.
- LeCun et al\. \[2015\]Y\. LeCun, Y\. Bengio, and G\. Hinton\.Deep learning\.*Nature*, 521\(7553\):436–444, 2015\.URL[https://www\.nature\.com/articles/nature14539](https://www.nature.com/articles/nature14539)\.
- Li et al\. \[2024\]Z\. Li, H\. Liu, D\. Zhou, and T\. Ma\.Chain of thought empowers transformers to solve inherently serial problems\.In*Proceedings of the International Conference on Learning Representations \(ICLR\)*, 2024\.
- Lightman et al\. \[2024\]H\. Lightman, V\. Kosaraju, Y\. Burda, H\. Edwards, B\. Baker, T\. Lee, J\. Leike, J\. Schulman, I\. Sutskever, and K\. Cobbe\.Let’s verify step by step\.In*Proceedings of the International Conference on Learning Representations \(ICLR\)*, 2024\.
- Plaat et al\. \[2025\]A\. Plaat, M\. van Duijn, N\. van Stein, M\. Preuss, P\. van der Putten, and K\. J\. Batenburg\.Agentic large language models: A survey\.*arXiv preprint arXiv:2503\.23037*, 2025\.URL[https://arxiv\.org/abs/2503\.23037](https://arxiv.org/abs/2503.23037)\.
- Raffel et al\. \[2020\]C\. Raffel, N\. Shazeer, A\. Roberts, K\. Lee, S\. Narang, M\. Matena, Y\. Zhou, W\. Li, P\. J\. Liu, et al\.Exploring the limits of transfer learning with a unified text\-to\-text transformer\.*Journal of Machine Learning Research*, 21\(140\):1–67, 2020\.URL[https://jmlr\.org/papers/v21/20\-074\.html](https://jmlr.org/papers/v21/20-074.html)\.
- Ruis et al\. \[2025\]L\. Ruis, M\. Mozes, J\. Bae, S\. R\. Kamalakara, D\. Talupuru, A\. Locatelli, R\. Kirk, T\. Rocktäschel, E\. Grefenstette, M\. Bartolo, et al\.Procedural knowledge in pretraining drives reasoning in large language models\.In*Proceedings of the International Conference on Learning Representations \(ICLR\)*, 2025\.
- Sastre and Rosá \[2025\]I\. Sastre and A\. Rosá\.Memory tokens: Large language models can generate reversible sentence embeddings\.In*Proceedings of the First Workshop on Large Language Model Memorization \(L2M2\) at ACL 2025*, 2025\.
- Saunshi et al\. \[2025\]N\. Saunshi, N\. Dikkala, Z\. Li, S\. Kumar, and S\. J\. Reddi\.Reasoning with latent thoughts: On the power of looped transformers\.In*Proceedings of the International Conference on Learning Representations \(ICLR\)*, 2025\.
- Setlur et al\. \[2024\]A\. Setlur, C\. Nagpal, A\. Fisch, X\. Geng, J\. Eisenstein, R\. Agarwal, A\. Agarwal, J\. Berant, and A\. Kumar\.Rewarding progress: Scaling automated process verifiers for llm reasoning\.*arXiv preprint arXiv:2410\.08146*, 2024\.doi:10\.48550/arXiv\.2410\.08146\.URL[https://arxiv\.org/abs/2410\.08146](https://arxiv.org/abs/2410.08146)\.
- Snell et al\. \[2024\]C\. Snell, J\. Lee, K\. Xu, and A\. Kumar\.Scaling llm test\-time compute optimally can be more effective than scaling model parameters\.*arXiv preprint arXiv:2408\.03314*, 2024\.URL[https://arxiv\.org/abs/2408\.03314](https://arxiv.org/abs/2408.03314)\.
- Su et al\. \[2024\]H\. Su, H\. Yen, M\. Xia, W\. Shi, N\. Muennighoff, H\.\-Y\. Wang, H\. Liu, Q\. Shi, Z\. S\. Siegel, M\. Tang, R\. Sun, J\. Yoon, S\. O\. Arik, D\. Chen, and T\. Yu\.Bright: A realistic and challenging benchmark for reasoning\-intensive retrieval\.*arXiv preprint arXiv:2407\.12883*, 2024\.URL[https://arxiv\.org/abs/2407\.12883](https://arxiv.org/abs/2407.12883)\.
- Wei et al\. \[2022\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. Chi, Q\. Le, and D\. Zhou\.Chain\-of\-thought prompting elicits reasoning in large language models\.*arXiv preprint arXiv:2201\.11903*, 2022\.URL[https://arxiv\.org/abs/2201\.11903](https://arxiv.org/abs/2201.11903)\.
- Xiao et al\. \[2023\]S\. Xiao, Z\. Liu, P\. Zhang, N\. Muennighoff, D\. Lian, and J\.\-Y\. Nie\.C\-pack: Packed resources for general chinese embeddings\.*arXiv preprint arXiv:2309\.07597*, 2023\.URL[https://arxiv\.org/abs/2309\.07597](https://arxiv.org/abs/2309.07597)\.
- Zhang et al\. \[2025\]Y\. Zhang, M\. Li, D\. Long, X\. Zhang, H\. Lin, B\. Yang, P\. Xie, A\. Yang, D\. Liu, J\. Lin, F\. Huang, and J\. Zhou\.Qwen3 embedding: Advancing text embedding and reranking through foundation models\.*arXiv preprint arXiv:2506\.05176*, 2025\.URL[https://arxiv\.org/abs/2506\.05176](https://arxiv.org/abs/2506.05176)\.
- Zhu et al\. \[2025\]R\.\-J\. Zhu, T\. Peng, T\. Cheng, X\. Qu, J\. Huang, D\. Zhu, H\. Wang, K\. Xue, X\. Zhang, Y\. Shan, T\. Cai, T\. Kergan, A\. Kembay, A\. Smith, C\. Lin, B\. Nguyen, Y\. Pan, Y\. Chou, Z\. Cai, Z\. Wu, Y\. Zhao, T\. Liu, J\. Yang, W\. Zhou, C\. Zheng, C\. Li, Y\. Zhou, Z\. Li, Z\. Zhang, J\. Liu, G\. Zhang, W\. Huang, and J\. Eshraghian\.A survey on latent reasoning\.*arXiv preprint arXiv:2507\.06203*, 2025\.URL[https://arxiv\.org/abs/2507\.06203](https://arxiv.org/abs/2507.06203)\.