CIFQA: A Deterministic Tool-Grounded Multi-Agent LLM Framework for Financial Query Answering

# 1 Introduction
Source: [https://arxiv.org/html/2608.26114](https://arxiv.org/html/2608.26114)
CIFQA: A Deterministic Tool\-Grounded Multi\-Agent LLM Framework for Financial Query Answering

Kunjesh Parekh1∗Dr\. Anil Kumar Tiwari1Dr\. Divya Saxena1

1School of Artificial Intelligence and Data Science, Indian Institute of Technology Jodhpur, Rajasthan 342030, India

∗Corresponding author:[P23ai0003@iitj\.ac\.in](mailto:[email protected])Co\-author:[akt@iitj\.ac\.in](mailto:[email protected])Co\-author:[divyasaxena@iitj\.ac\.in](mailto:[email protected])

###### Abstract

Calculation\-intensive financial question answering requires not only language understanding, but also exact executable reasoning over structured rates, temporal conditions, numerical formulas, and rule\-based constraints\. Large Language Models \(LLMs\), despite strong performance on natural language tasks, often fail on such queries because probabilistic text generation does not consistently execute deterministic multi\-step computation\. As a result, LLMs may produce arithmetic hallucinations: numerically incorrect but plausible financial answers, even when relevant formulas and contextual information are available\. We introduce Calculation\-Intensive Financial Query Answering, CIFQA, a deterministic tool\-grounded multi\-agent LLM framework for financial query answering\. CIFQA separates linguistic interpretation from numerical execution by assigning LLM agents to query understanding, routing, parameter extraction, computation planning, and response formulation, while deterministic Python\-based tools perform financial operations such as rate lookup, calendar\-aware tenure computation, compounding, payout handling, and rule application\. In this way, CIFQA treats financial QA as an executable reasoning task rather than a purely text\-generation problem\. We instantiate CIFQA for fixed deposit query answering and evaluate it on a curated benchmark of fixed deposit queries\. CIFQA achieves 95\.54% accuracy on calculation\-intensive queries and 90\.87% overall accuracy, substantially outperforming direct LLM baselines even when provided with complete formulas, rate cards, and benchmark instructions — confirming the limitation is architectural rather than informational\. Category\-wise analysis shows strong performance on quarterly payout, monthly payout, cumulative FD, TDS\-related, and edge\-case queries\. Ablation results further demonstrate that deterministic components, particularly rolling\-year adjustment, exact rate lookup, tenure computation, and premature\-withdrawal logic, are key contributors to performance\. Notably, a 17B open\-source backbone operating within CIFQA outperforms substantially larger frontier models including GPT\-5\.3, Gemini 3, and Claude Sonnet 4\.6 evaluated with complete formulas and rate information, demonstrating that architectural design is a more critical determinant of numerical reliability than model scale\. While instantiated on fixed deposit queries, CIFQA provides a generalizable and modular design pattern for calculation\-intensive financial reasoning, with natural extensions toward multi\-instrument decision support, interest rate trend analysis, and cross\-asset liquidity optimisation\.

Keywords:Calculation\-Intensive Financial Query Answering \(CIFQA\); Financial Question Answering; Deterministic Financial Computation; Tool\-Grounded Multi\-Agent LLM Systems; Hybrid Multi\-Agent LLM Architecture; Arithmetic Hallucination; Financial Reasoning; Multi\-Agent Financial AI; Executable Financial Reasoning; Agentic AI

Large Language Models \(LLMs\) are increasingly being adopted across financial services for customer support, advisory systems, document analysis, and financial question answering\[[38](https://arxiv.org/html/2608.26114#bib.bib1),[35](https://arxiv.org/html/2608.26114#bib.bib2),[37](https://arxiv.org/html/2608.26114#bib.bib3)\]\. Recent benchmarks such as FinanceBench have further highlighted both the potential and limitations of LLMs for financial\-domain question answering\[[15](https://arxiv.org/html/2608.26114#bib.bib7)\]\. Their strong natural language understanding capabilities enable interaction with complex financial information through conversational interfaces\. However, financial applications differ fundamentally from conventional text\-generation tasks: correctness is often dependent on exact arithmetic execution, calendar\-aware reasoning, structured rate retrieval, and strict adherence to financial rules and regulatory conditions\. In such settings, even minor numerical deviations can produce financially incorrect outputs, making reliability a critical requirement for real\-world deployment\[[6](https://arxiv.org/html/2608.26114#bib.bib11),[33](https://arxiv.org/html/2608.26114#bib.bib25),[9](https://arxiv.org/html/2608.26114#bib.bib19),[22](https://arxiv.org/html/2608.26114#bib.bib12),[36](https://arxiv.org/html/2608.26114#bib.bib13),[13](https://arxiv.org/html/2608.26114#bib.bib45),[16](https://arxiv.org/html/2608.26114#bib.bib46)\]\.

Despite strong language\-generation capabilities, frontier LLMs exhibit systematic execution failures on calculation\-intensive financial tasks involving exact arithmetic, calendar\-aware reasoning, and rule\-based financial computation\. Figure[1](https://arxiv.org/html/2608.26114#S1.F1)presents representative examples across GPT\-5\.3, Claude Sonnet 4\.6, and Gemini 3 on fixed deposit \(FD\) queries involving interest\-rate selection, rolling\-year denominator handling, payout reconstruction, and premature\-withdrawal settlement computation\.

![Refer to caption](https://arxiv.org/html/2608.26114v1/LLM_failure.png)Figure 1:Representative execution failures across frontier LLMs on calculation\-intensive fixed deposit queries despite complete formulas, rate cards, and benchmark instructions\.The observed failures are not isolated numerical mistakes, but systematic execution errors arising from the inability of probabilistic language models to reliably perform deterministic financial computation\. Common failure modes include incorrect interest\-rate slab selection from structured rate tables, improper handling of rolling\-year denominators across leap\-year boundaries, payout schedule reconstruction failures, premature\-withdrawal settlement inconsistencies, and numerical drift in chained multi\-step calculations\. Importantly, these failures persist even when models are provided with complete financial formulas, rate cards, and detailed benchmark instructions, indicating that the limitation is architectural rather than informational\.

This paper focuses on a class of problems that we define asCalculation\-Intensive Financial Query Answering \(CIFQA\), where generating a correct response requires exact arithmetic computation over structured financial data under domain\-specific financial rules\. Representative examples include fixed deposit interest calculation, loan amortization, bond yield estimation, investment return analysis, and tax\-related settlement computation\[[38](https://arxiv.org/html/2608.26114#bib.bib1),[35](https://arxiv.org/html/2608.26114#bib.bib2),[37](https://arxiv.org/html/2608.26114#bib.bib3),[15](https://arxiv.org/html/2608.26114#bib.bib7)\]\. Unlike conventional financial question answering tasks that primarily involve retrieval, summarization, or textual reasoning, CIFQA requires deterministic execution of computation pipelines involving exact rate grounding, calendar\-aware date handling, payout scheduling, compounding, and conditional financial\-rule application\.

To address this limitation, we proposeCIFQA, a deterministic tool\-grounded multi\-agent LLM framework for executable financial reasoning that builds upon recent advances in tool\-augmented reasoning, program\-aided language models, and agentic AI systems\[[9](https://arxiv.org/html/2608.26114#bib.bib19),[25](https://arxiv.org/html/2608.26114#bib.bib20),[40](https://arxiv.org/html/2608.26114#bib.bib21),[34](https://arxiv.org/html/2608.26114#bib.bib29),[3](https://arxiv.org/html/2608.26114#bib.bib24),[24](https://arxiv.org/html/2608.26114#bib.bib22),[26](https://arxiv.org/html/2608.26114#bib.bib33),[28](https://arxiv.org/html/2608.26114#bib.bib40),[7](https://arxiv.org/html/2608.26114#bib.bib41)\]\. CIFQA enforces a strict separation between language understanding and numerical execution: LLM agents are responsible for query interpretation, routing, parameter extraction, computation planning, and response generation, while all arithmetic operations are delegated to deterministic execution engines for rate lookup, calendar\-aware computation, payout handling, and financial rule execution\. By explicitly preventing LLMs from performing arithmetic operations directly, CIFQA eliminates deterministic computation failures arising from probabilistic token prediction and ensures reproducible financial computation\.

We instantiate CIFQA in the domain of fixed deposit \(FD\) query answering and evaluate it on a curated benchmark of 126 FD queries spanning calculation\-intensive reasoning, rate lookup, premature\-withdrawal handling, policy interpretation, and edge\-case scenarios\. CIFQA achieves95\.54% accuracyon calculation\-intensive queries and90\.87% overall accuracy, substantially outperforming GPT\-5\.3, Gemini 3, and Claude Sonnet 4\.6\. Remarkably, a comparatively smaller 17B open\-source backbone operating within CIFQA outperforms substantially larger frontier LLMs evaluated with complete financial formulas, rate cards, and benchmark instructions, demonstrating that deterministic executable computation is more important than model scale for reliable financial reasoning\.

The key contributions of this work are as follows:

- •We identify calculation\-intensive financial question answering as a distinct class of financial QA problems where correct responses require exact executable reasoning over structured financial data, temporal conditions, numerical formulas, and domain\-specific rules\. Unlike conventional financial QA tasks that mainly involve retrieval or text generation, these queries require precise operations such as rate selection, calendar\-aware computation, compounding, payout handling, and rule application\.
- •We propose a deterministic tool\-grounded multi\-agent LLM framework for financial query answering, CIFQA\. The framework separates language\-centric tasks from numerical execution: LLM agents perform query interpretation, routing, parameter extraction, computation planning, and response formulation, while deterministic tools execute financial computations and rule\-based operations in a verifiable manner\.
- •We instantiate the framework for fixed deposit query answering and evaluate it on a curated benchmark of calculation\-intensive queries\. CIFQA achieves 95\.54% accuracy on calculation\-intensive queries and 90\.87% overall accuracy, outperforming direct LLM baselines provided with relevant formulas and rate information\. Category\-wise and ablation analyses show that deterministic modules such as rolling\-year adjustment, exact rate lookup, tenure computation, and premature\-withdrawal logic are important for improving calculation correctness and financial reasoning accuracy\.

The remainder of the paper is organised as follows\. Section 2 reviews related work on financial NLP, retrieval\-augmented systems, tool\-augmented reasoning, and multi\-agent LLM frameworks\. Section 3 formalises the CIFQA problem setting\. Section 4 presents the CIFQA framework architecture\. Section 5 describes implementation details\. Section 6 outlines the experimental setup\. Section 7 presents results and analysis\. Section 8 discusses generalisation and limitations, and Section 9 concludes the paper\.

## 2Related Work

Research on financial question answering has primarily focused on text understanding, information extraction, sentiment analysis, and financial\-domain adaptation of language models\[[38](https://arxiv.org/html/2608.26114#bib.bib1),[35](https://arxiv.org/html/2608.26114#bib.bib2),[37](https://arxiv.org/html/2608.26114#bib.bib3),[15](https://arxiv.org/html/2608.26114#bib.bib7),[18](https://arxiv.org/html/2608.26114#bib.bib5),[41](https://arxiv.org/html/2608.26114#bib.bib6)\]\. Models such as domain\-adapted transformers \(e\.g\., FinBERT\[[38](https://arxiv.org/html/2608.26114#bib.bib1)\]\), large\-scale financial foundation models such as BloombergGPT\[[35](https://arxiv.org/html/2608.26114#bib.bib2)\], and open financial LLMs such as FinGPT\[[37](https://arxiv.org/html/2608.26114#bib.bib3)\]have demonstrated strong performance in financial text understanding, classification, and retrieval tasks\. However, these approaches do not address scenarios where responses require precise numerical computation over structured financial data\. Financial QA benchmarks such as FinQA\[[4](https://arxiv.org/html/2608.26114#bib.bib10)\], TAT\-QA\[[43](https://arxiv.org/html/2608.26114#bib.bib8)\], and ConvFinQA\[[5](https://arxiv.org/html/2608.26114#bib.bib9)\]have demonstrated that numerical reasoning over financial documents remains a significant challenge for existing models, even when domain knowledge is available\.

Retrieval\-Augmented Generation \(RAG\) systems have been widely adopted to improve factual correctness by providing external knowledge to Large Language Models \(LLMs\)\[[19](https://arxiv.org/html/2608.26114#bib.bib16),[10](https://arxiv.org/html/2608.26114#bib.bib17),[1](https://arxiv.org/html/2608.26114#bib.bib18)\]\. While effective for knowledge\-intensive tasks, RAG frameworks remain insufficient for calculation\-intensive settings, as they retrieve information but do not guarantee correct execution of multi\-step arithmetic computations\.

Recent work has explored chain\-of\-thought reasoning, tool\-augmented inference, program\-aided language models, and reasoning\-action frameworks, where LLMs generate intermediate reasoning traces, executable programs, or external tool calls to improve numerical reasoning performance\[[33](https://arxiv.org/html/2608.26114#bib.bib25),[9](https://arxiv.org/html/2608.26114#bib.bib19),[25](https://arxiv.org/html/2608.26114#bib.bib20),[40](https://arxiv.org/html/2608.26114#bib.bib21),[3](https://arxiv.org/html/2608.26114#bib.bib24),[42](https://arxiv.org/html/2608.26114#bib.bib26),[31](https://arxiv.org/html/2608.26114#bib.bib27),[39](https://arxiv.org/html/2608.26114#bib.bib28),[24](https://arxiv.org/html/2608.26114#bib.bib22),[21](https://arxiv.org/html/2608.26114#bib.bib23)\]\. These approaches improve accuracy by augmenting LLMs with external computation capabilities; however, they still rely on the LLM to correctly determine the computation process, generate executable programs, or invoke appropriate tools\. Similarly, multi\-agent LLM frameworks decompose tasks into specialized roles such as planner, executor, and verifier, improving reasoning through iterative refinement and coordinated agent interactions\[[34](https://arxiv.org/html/2608.26114#bib.bib29),[20](https://arxiv.org/html/2608.26114#bib.bib30),[12](https://arxiv.org/html/2608.26114#bib.bib31),[30](https://arxiv.org/html/2608.26114#bib.bib32),[26](https://arxiv.org/html/2608.26114#bib.bib33),[29](https://arxiv.org/html/2608.26114#bib.bib34),[14](https://arxiv.org/html/2608.26114#bib.bib35)\]\. However, these systems typically retain LLMs within the computation loop, making exact arithmetic correctness dependent on the reliability of intermediate LLM decisions\. In contrast, CIFQA removes arithmetic execution entirely from the LLM reasoning loop and delegates all financial computation to deterministic engines, ensuring reproducible and verifiable financial calculations\.

Recent research has explored hybrid AI systems that combine language models with deterministic computation, external tools, and executable reasoning modules to improve reliability in structured reasoning tasks\[[25](https://arxiv.org/html/2608.26114#bib.bib20),[9](https://arxiv.org/html/2608.26114#bib.bib19),[40](https://arxiv.org/html/2608.26114#bib.bib21),[11](https://arxiv.org/html/2608.26114#bib.bib36),[23](https://arxiv.org/html/2608.26114#bib.bib37),[2](https://arxiv.org/html/2608.26114#bib.bib38)\]\. LLMs have also been shown to exhibit systematic fragility in mathematical reasoning — performance drops significantly when only numerical values in a problem are altered\[[22](https://arxiv.org/html/2608.26114#bib.bib12)\], and errors accumulate in multi\-step arithmetic due to fundamental limitations in numerical precision\[[36](https://arxiv.org/html/2608.26114#bib.bib13),[27](https://arxiv.org/html/2608.26114#bib.bib14),[8](https://arxiv.org/html/2608.26114#bib.bib15)\]\. While these approaches improve reasoning accuracy, they generally rely on LLMs to generate executable programs, select tools, or orchestrate computation steps\. In contrast, CIFQA enforces a stricter separation between language understanding and numerical execution by removing arithmetic computation entirely from the LLM reasoning loop and delegating all financial calculations to deterministic engines\. To the best of our knowledge, deterministic tool\-grounded architectures have not been systematically evaluated for calculation\-intensive financial query answering tasks involving calendar\-aware computation, structured rate lookup, payout scheduling, and financial\-rule execution\.

In contrast to existing approaches,CIFQAenforces a strict separation between language processing and numerical computation\. Instead of relying on LLM\-generated reasoning or code, all arithmetic operations are executed through deterministic computation engines\. This design eliminates arithmetic hallucinations at their source and ensures exact correctness in calculation\-intensive financial queries\.

## 3Problem Definition: CIFQA

We defineCalculation\-Intensive Financial Query Answering \(CIFQA\)as a class of question answering tasks in which generating a correct response requires exact numerical computation over structured financial data, along with rule\-based reasoning under domain\-specific constraints\.

Formally, a CIFQA queryqqconsists of a natural language input describing a financial task, which must be mapped to a structured representation containing parameters such as principal amount, interest rate, tenure, compounding frequency, and applicable rules \(e\.g\., taxation thresholds or payout conditions\)\. The goal is to compute an exact numerical outputyysuch that:

y=f​\(q,R,𝒞\),y=f\(q,R,\\mathcal\{C\}\),whereRRrepresents structured financial data \(e\.g\., rate tables\), and𝒞\\mathcal\{C\}denotes domain\-specific computational rules, including calendar\-aware calculations, compounding formulas, and regulatory constraints\.

Unlike conventional question answering tasks, CIFQA exhibits the following defining characteristics:

- •Exactness requirement:Even minor numerical deviations render outputs incorrect in real\-world financial applications\.
- •Multi\-step computation:Queries require chained arithmetic operations, often involving compounding and time\-based calculations\.
- •Structured data dependency:Accurate responses depend on correct retrieval of values from rate tables and policy documents\.
- •Rule\-based logic:Conditional rules such as tax deductions, penalty clauses, or payout frequencies must be applied correctly\.
- •Temporal sensitivity:Computations often depend on calendar\-specific factors such as leap years, day\-count conventions, and rolling periods\.

To evaluate CIFQA systems, we construct a curated evaluation set of 126 fixed deposit \(FD\) queries designed by domain experts\. The evaluation set spans three primary categories: \(i\) calculation\-intensive queries requiring precise interest computation, \(ii\) interest\-rate lookup queries involving retrieval of applicable FD rates under varying customer and tenure conditions, and \(iii\) policy\-related queries involving interpretation of financial rules, taxation conditions, and payout policies\. In addition, multiple edge\-case scenarios are included within these categories to evaluate robustness under unusual or boundary conditions\.

For calculation\-intensive queries, ground truth answers are computed manually and validated using spreadsheet\-based implementations to ensure exact numerical correctness\. Model outputs are evaluated using strict tolerance\-based evaluation criteria, requiring both numerical accuracy and correct formatting, reflecting the precision requirements of real\-world financial applications\.

CIFQA differs fundamentally from existing financial QA approaches\[[38](https://arxiv.org/html/2608.26114#bib.bib1),[35](https://arxiv.org/html/2608.26114#bib.bib2),[37](https://arxiv.org/html/2608.26114#bib.bib3),[15](https://arxiv.org/html/2608.26114#bib.bib7)\]in that it emphasizes deterministic computation rather than textual understanding\. While existing financial LLMs and benchmarks primarily focus on financial language understanding, information retrieval, sentiment analysis, or financial\-domain knowledge, CIFQA targets exact executable financial reasoning where numerical correctness is the primary evaluation criterion\. This distinction necessitates architectures that integrate deterministic computation with LLM\-based language understanding, motivating the design of the CIFQA framework\.

## 4CIFQA Framework

The CIFQA framework is designed to eliminate arithmetic hallucinations by enforcing a strict separation between language understanding and numerical computation\. The design is inspired by recent advances in tool\-augmented reasoning and agentic AI systems\[[25](https://arxiv.org/html/2608.26114#bib.bib20),[9](https://arxiv.org/html/2608.26114#bib.bib19),[40](https://arxiv.org/html/2608.26114#bib.bib21),[34](https://arxiv.org/html/2608.26114#bib.bib29),[20](https://arxiv.org/html/2608.26114#bib.bib30),[12](https://arxiv.org/html/2608.26114#bib.bib31)\], but differs in that arithmetic execution is entirely delegated to deterministic computation engines rather than being orchestrated through LLM\-generated reasoning steps\. Instead of relying on Large Language Models \(LLMs\) to perform arithmetic reasoning, CIFQA decomposes the problem into specialized components, where LLM\-based agents handle interpretation and planning, and deterministic computation engines execute all numerical operations\.

### 4\.1Framework Overview

Building upon the failure modes identified in Figure[1](https://arxiv.org/html/2608.26114#S1.F1), CIFQA addresses the limitations of direct LLM\-based financial query answering through a strict separation between language understanding and deterministic computation\. Instead of relying on LLMs to perform arithmetic reasoning internally, CIFQA delegates all numerical operations to deterministic execution engines while restricting LLM agents to query interpretation, planning, and response generation\.

Given a natural language financial query, CIFQA processes the input through a multi\-agent LLM pipeline consisting of five stages: \(i\) routing, \(ii\) parameter extraction, \(iii\) planning, \(iv\) deterministic execution, and \(v\) response generation\. Figure[2](https://arxiv.org/html/2608.26114#S4.F2)illustrates the overall architecture\.

![Refer to caption](https://arxiv.org/html/2608.26114v1/architecture2.png)Figure 2:CIFQA framework architecture showing separation between LLM\-based agents and deterministic computation engines\.At a high level, the framework operates as follows: the input query is first classified to determine whether it requires computation, policy reasoning, or a combination of both\. Relevant parameters are then extracted and structured, followed by the generation of a computation plan\. All numerical operations are subsequently executed by deterministic computation engines, and the final answer is synthesized into a natural language response\.

### 4\.2Multi\-Agent LLM Design

CIFQA employs a modular multi\-agent LLM architecture, where each agent is responsible for a specific stage of the pipeline\. The router, extractor, planner, and response generator collectively form a coordinated multi\-agent LLM system, while deterministic computation engines execute all numerical operations outside the LLM reasoning loop\.

#### Router Agent

The router agent classifies incoming queries into categories such as calculation\-intensive, policy\-based, or hybrid queries\. This step determines whether deterministic computation is required and selects the appropriate execution path\.

#### Extractor Agent

The extractor agent converts the natural language query into a structured representation by identifying key parameters such as principal amount, tenure, interest rate, payout frequency, and applicable conditions\. This step is critical for ensuring that downstream computation receives accurate inputs\.

#### Planner Agent

The planner agent generates a structured computation plan based on the extracted parameters and domain rules\. For calculation\-intensive queries, this includes selecting the appropriate formula, determining compounding intervals, handling partial periods, and incorporating calendar\-aware adjustments\.

#### Executor \(Deterministic Computation Engines\)

All numerical computation is performed by deterministic Python\-based computation engines\. These include components for rate lookup, interest calculation, calendar\-aware day\-count computation, and rule\-based logic such as tax deduction or payout handling\. By delegating all arithmetic operations to these engines, CIFQA ensures exact numerical correctness\.

#### Response Generator

The response generator converts the computed outputs into a coherent natural language response\. This includes formatting numerical values, incorporating explanatory steps if required, and ensuring clarity for end users\.

### 4\.3Separation of Language Understanding and Computation

A key design principle of CIFQA is the strict separation between language understanding and deterministic computation, building upon prior work in tool\-grounded and program\-aided reasoning while enforcing complete removal of arithmetic execution from the LLM loop\[[25](https://arxiv.org/html/2608.26114#bib.bib20),[9](https://arxiv.org/html/2608.26114#bib.bib19),[40](https://arxiv.org/html/2608.26114#bib.bib21)\]\. LLMs are never used to perform arithmetic operations; instead, they are limited to language understanding, planning, and response synthesis\. All computations are executed by deterministic engines, ensuring reproducibility and eliminating numerical drift\.

This separation addresses a fundamental limitation of LLMs: while they are effective at interpreting language, they are inherently unreliable for exact numerical computation due to their probabilistic nature\. By removing arithmetic reasoning from the LLM pipeline, CIFQA eliminates arithmetic hallucinations at their source\.

### 4\.4Execution Flow

The end\-to\-end execution flow of CIFQA is illustrated in Figure[3](https://arxiv.org/html/2608.26114#S4.F3)and can be summarized as follows:

1. 1\.A user submits a natural language financial query\.
2. 2\.The router agent determines the query type and execution path\.
3. 3\.The extractor agent converts the query into structured parameters\.
4. 4\.The planner agent generates a computation plan\.
5. 5\.Deterministic computation engines execute all numerical operations\.
6. 6\.The response generator produces the final answer\.

This pipeline ensures that all stages of computation are transparent, modular, and verifiable, making CIFQA suitable for deployment in high\-stakes financial applications where correctness is critical\.

![Refer to caption](https://arxiv.org/html/2608.26114v1/Query_flow.png)Figure 3:Example end\-to\-end execution flow of CIFQA for a calculation\-intensive fixed deposit query\. The figure illustrates query sanitization, validation, routing, parameter extraction, planning, deterministic computation, and final response generation\. All arithmetic operations are executed through deterministic computation engines, while LLM agents are restricted to language understanding, planning, and response synthesis\.

## 5Implementation

The CIFQA framework is implemented as a hybrid multi\-agent LLM system combining Large Language Model \(LLM\) inference with deterministic Python\-based computation engines\. The implementation is designed to ensure reproducibility, modularity, and exact numerical correctness\.

### 5\.1LLM Backend

All language understanding and planning tasks are performed using an LLM accessed via API\-based inference\. The model is responsible for query routing, parameter extraction, computation planning, and response generation\. Importantly, the LLM is not used for any arithmetic computation, ensuring that all numerical operations remain deterministic\.

### 5\.2Deterministic Computation Engines

Numerical computation is handled by a set of modular Python\-based engines, each responsible for a specific aspect of financial calculation\. These include:

- •Rate Lookup Engine:Retrieves applicable interest rates from structured rate tables based on tenure and customer category\.
- •Interest Computation Engine:Performs exact interest calculations using appropriate compounding formulas and payout schedules\.
- •Calendar Engine:Handles date\-related computations, including leap years, day\-count conventions, and rolling periods\.
- •Rule Engine:Applies conditional financial rules such as tax deduction thresholds and payout\-specific adjustments\.

All engines are implemented using deterministic logic to ensure consistency and reproducibility across executions\.

### 5\.3Knowledge Sources

The framework utilizes structured financial data sources, including rate tables and policy rules, which are incorporated into the system as deterministic lookup modules\. This eliminates ambiguity in parameter selection and ensures alignment with real\-world financial specifications\.

### 5\.4System Integration

The CIFQA pipeline is orchestrated through a sequential execution flow, where outputs from each agent are passed as structured inputs to subsequent components\. The system is implemented in Python, with clear separation between LLM\-driven components and deterministic computation modules\.

This modular design allows individual components to be independently updated or extended, enabling adaptation to other financial domains without modifying the overall architecture\.

## 6Experimental Setup

This section describes the evaluation protocol used to assess the performance of CIFQA on calculation\-intensive financial queries\.

### 6\.1Evaluation Dataset

We evaluate CIFQA on a curated set of 126 fixed deposit \(FD\) queries designed by domain experts\. The evaluation set covers three primary categories:

- •Calculation\-intensive queries:Require precise interest computation involving compounding, partial periods, and calendar\-aware calculations\.
- •Interest\-rate lookup queries:Require retrieval of applicable FD interest rates under varying tenure, customer category, deposit amount, and booking\-date conditions\.
- •Policy\-related queries:Involve interpretation of financial rules such as taxation thresholds, payout conditions, premature withdrawal policies, and TDS\-related scenarios\.

In addition, multiple edge\-case scenarios are included across these categories to evaluate robustness under unusual or boundary conditions, including irregular tenures, large principal amounts, ambiguous phrasing, and calendar\-sensitive calculations\.

For the subset of 101 calculation\-intensive queries, ground truth answers are computed manually and verified using spreadsheet\-based implementations to ensure exact numerical correctness\.

### 6\.2Evaluation Metrics

We adopt strict evaluation criteria reflecting the requirements of real\-world financial applications\. A response is considered correct only if:

- •The final numerical value matches the ground\-truth value within an absolute tolerance of±1\\pm 1INR\.
- •Intermediate computations \(where applicable\) follow correct financial logic\.
- •The output formatting is consistent with expected financial representations\.

Any response producing a numerical deviation beyond the accepted tolerance threshold is classified as anarithmetic hallucination\. This includes errors arising from incorrect arithmetic execution, improper day\-count handling, incorrect rate application, compounding mistakes, or failure to apply domain\-specific financial rules correctly\.

This strict evaluation protocol ensures that numerically meaningful deviations are treated as errors, distinguishing CIFQA from conventional QA benchmarks that allow approximate semantic correctness\.

### 6\.3Baseline Models

We compare CIFQA against several state\-of\-the\-art frontier Large Language Models \(LLMs\), including:

- •GPT\-5\.3 \(OpenAI\)
- •Gemini 3 \(Google\)
- •Claude Sonnet 4\.6 \(Anthropic\)

This evaluation setting is intentionally designed to provide frontier LLMs with complete financial knowledge and computational instructions, allowing the study to isolate failures arising from numerical execution rather than information retrieval or missing domain knowledge\.

To ensure a fair comparison against production\-grade systems, all frontier LLM baselines were evaluated through their native conversational interfaces rather than raw API inference\. This setup allows the models to utilize their full deployed capabilities, including tool access, enhanced reasoning orchestration, and interface\-level optimizations available to end users\. Preliminary experiments using direct API\-only inference resulted in substantially lower and less stable numerical accuracy across models\. Therefore, reported baseline results reflect the strongest practically deployable performance of each system\.

### 6\.4Evaluation Protocol

Each query is independently evaluated by generating responses from baseline models and CIFQA\. Model outputs are manually compared against ground truth answers to assess correctness\. For calculation\-intensive queries, evaluation focuses on exact numerical accuracy, while policy and edge\-case queries are evaluated based on logical correctness and adherence to domain rules\.

This evaluation protocol ensures a fair and rigorous comparison between CIFQA and baseline LLMs, particularly in scenarios requiring precise numerical reasoning\.

## 7Results and Analysis

This section evaluates CIFQA with a primary focus on its ability to eliminate arithmetic hallucinations in calculation\-intensive financial queries\. The evaluation explicitly distinguishes between direct LLM inference and structured multi\-agent LLM execution, highlighting the role of architectural design in achieving numerical correctness\.

### 7\.1Evaluation Setup and Comparison Protocol

Baseline models \(GPT\-5\.3, Gemini 3, and Claude Sonnet\) are evaluated using a single\-prompt setup, where each query is provided along with relevant contextual information such as formulas and rate tables\. The models are required to directly generate the final answer without intermediate structured reasoning or external computation\.

In contrast, CIFQA follows a multi\-agent LLM execution paradigm, where the underlying LLM operates within a structured pipeline consisting of routing, parameter extraction, planning, and deterministic execution\. The CIFQA framework is instantiated using Llama\-Scout\-17B as the primary backbone model\.

To isolate the effect of architectural design from model scale, the same CIFQA pipeline is evaluated with alternative backbone models, including Llama\-70B and Llama\-8B, while keeping all other components unchanged\. This enables a controlled comparison between model capacity and system design\.

The CIFQA system is tuned using a small set of 21 development queries to refine prompts and ensure stable agent behavior\. All reported results are evaluated on the remaining queries, ensuring that performance reflects generalization rather than memorization\.

### 7\.2Calculation\-Intensive Performance

Table[1](https://arxiv.org/html/2608.26114#S7.T1)presents performance on 101 calculation\-intensive queries\.

Table 1:Performance on calculation\-intensive queriesGPT\-5\.3Gemini 3Claude Sonnet 4\.6Llama\-8BLlama\-70BCIFQA0505010010045\.0545\.0570\.370\.383\.6683\.6668\.8168\.8189\.689\.695\.5495\.54Accuracy \(%\)Figure 4:Accuracy comparison on calculation\-intensive queriesCIFQA achieves95\.54% accuracy, significantly outperforming both proprietary frontier models and larger open\-source baselines\. This improvement is achieved despite baseline models operating in a direct single\-prompt setting, while CIFQA leverages a structured multi\-agent LLM execution framework\.

The results demonstrate that increasing model scale alone is insufficient to resolve arithmetic unreliability in financial reasoning\. While larger models such as Llama\-70B improve performance compared to smaller models, they remain inferior to CIFQA\. This confirms that the primary limitation of LLMs in financial reasoning lies in unreliable arithmetic execution rather than insufficient model capacity\.

### 7\.3Arithmetic Hallucination Rate

To directly evaluate numerical reliability, we additionally report theArithmetic Hallucination Rate \(AHR\), defined as the percentage of calculation\-intensive queries producing numerically incorrect outputs beyond the accepted evaluation tolerance\.

Formally,

AHR\(%\)=100−Accuracy\(%\)\\mathrm\{AHR\}\(\\%\)=100\-\\mathrm\{Accuracy\}\(\\%\)
where accuracy is measured under the strict financial correctness criteria defined in Section 6\.

Table[2](https://arxiv.org/html/2608.26114#S7.T2)presents the corresponding arithmetic hallucination rates for all evaluated models\.

Table 2:Arithmetic Hallucination Rate \(AHR\) on calculation\-intensive queriesThe results show that CIFQA substantially reduces arithmetic hallucinations compared to both proprietary and open\-source LLM baselines\. The low hallucination rate achieved by CIFQA demonstrates that deterministic computation is highly effective for calculation\-intensive financial reasoning tasks\.

### 7\.4Category\-wise Analysis

Table 3:Category\-wise accuracy comparison across CIFQA and frontier LLMsTable[3](https://arxiv.org/html/2608.26114#S7.T3)presents a category\-wise comparison between CIFQA and frontier LLMs\. CIFQA achieves near\-perfect performance across most calculation\-intensive categories, including quarterly payout, monthly payout, cumulative FD, TDS\-related queries, edge cases, and half\-yearly payout scenarios\. The largest performance gains relative to frontier LLMs are observed in premature\-withdrawal and payout\-based calculations, which require precise calendar\-aware execution, exact rate grounding, and rule\-based financial computation\.

The category\-wise results further demonstrate that deterministic execution is particularly beneficial for financial tasks involving multi\-step arithmetic reasoning\. Across the calculation\-intensive categories, CIFQA consistently outperforms all frontier LLM baselines, achieving perfect accuracy in several categories where even the strongest baseline models exhibit substantial performance degradation\. These findings indicate that the dominant source of error in financial reasoning systems is not language understanding, but unreliable numerical execution and rule application\.

The results further reveal that not all categories within the benchmark rely equally on deterministic arithmetic execution\. While CIFQA substantially outperforms all baseline models across traditional calculation\-intensive categories, its performance is comparatively lower on the*Rate Analytics Query*and*RAG Rules*categories\. Importantly, this does not reflect a limitation in rate retrieval, as CIFQA performs exact rate lookup deterministically\. Rather, Rate Analytics queries require higher\-level reasoning over structured rate tables to identify patterns, relationships, and trends across customer segments, deposit amounts, and tenure ranges\. Similarly, RAG Rules queries depend more heavily on policy interpretation and retrieval than numerical execution\. This behavior is consistent with the design objective of CIFQA, which prioritizes reliable financial computation through deterministic execution while providing limited optimization for analytical reasoning and retrieval\-intensive policy interpretation\.

### 7\.5Overall Performance

For completeness, CIFQA achieves90\.87% overall accuracy\. The gap between calculation accuracy and overall accuracy is primarily driven by policy\-heavy queries, which are outside the primary optimization scope of CIFQA\.

### 7\.6Error Analysis

Baseline LLMs exhibit consistent failure modes in calculation\-intensive queries, including incorrect compounding, misinterpretation of tenure, and numerical drift in multi\-step reasoning\.

We observe three dominant arithmetic failure categories across frontier LLMs:

- •F1: Structured parameter misselection— incorrect retrieval or application of interest\-rate slabs, payout conditions, or tenure mappings\.
- •F2: Calendar\-aware execution errors— incorrect handling of rolling\-year computations, leap years, quarterly boundaries, and date\-sensitive interest schedules\.
- •F3: Numerical precision drift— accumulation of rounding inconsistencies and arithmetic deviations during multi\-step financial calculations\.

Figure[1](https://arxiv.org/html/2608.26114#S1.F1)presents representative examples of these failure categories across GPT\-5\.3, Claude Sonnet 4\.6, and Gemini 3\. The observed errors demonstrate that arithmetic failures are not isolated numerical mistakes but systematic execution failures arising from incorrect parameter grounding, calendar\-aware computation errors, and numerical drift during multi\-step financial reasoning\.

The category\-level performance trends shown in Table[3](https://arxiv.org/html/2608.26114#S7.T3)are consistent with the representative failure cases illustrated in Figure[1](https://arxiv.org/html/2608.26114#S1.F1)\. Across frontier LLMs, the dominant sources of error arise from incorrect rate selection, calendar\-aware execution failures, payout reconstruction errors, and numerical drift in multi\-step financial calculations\. CIFQA avoids these failures by delegating all numerical operations to deterministic computation engines while restricting LLM agents to language understanding, planning, and response generation\.

CIFQA eliminates these errors by delegating all computation to deterministic computation engines\. The remaining errors are primarily associated with:

- •Ambiguities in policy interpretation for RAG\-based queries
- •Occasional extraction or routing errors

#### Routing Accuracy

The router component demonstrates near\-perfect performance for calculation\-intensive queries, with only a single observed misclassification\. This indicates that query\-type identification is not a major source of error for arithmetic tasks\. In contrast, most routing\-related errors occur in policy\-heavy \(RAG\-based\) queries, where boundaries between retrieval and computation are less clearly defined\.

These findings confirm that arithmetic hallucination is the dominant source of error in financial query answering, and that CIFQA effectively addresses this limitation\.

### 7\.7Ablation Study

Unlike end\-to\-end LLM systems, CIFQA enables component\-level attribution of numerical reliability through deterministic module isolation\. To isolate the contribution of individual deterministic components, we perform a targeted ablation study by selectively disabling specific computation modules within CIFQA and evaluating their impact on calculation\-intensive queries\.

Table 4:Component\-level ablation analysisEach ablation corresponds to disabling a specific deterministic computation capability within CIFQA while keeping all other components unchanged\.

- •Rolling\-year adjustment logichas the largest impact, with accuracy dropping to48\.00%\. This highlights the importance of correctly handling multi\-period interest calculations across year boundaries, particularly in scenarios involving leap years and partial\-year segmentation\.
- •Rate lookup moduleandpremature withdrawal override logicsignificantly affect performance, reducing accuracy to approximately76–77%\. These components ensure correct parameter grounding and conditional rule application, both of which are essential in real\-world financial computations\.
- •Tenure computation logic, which governs segmentation of total duration into compounding intervals, affects all queries and leads to a drop to84\.65%, indicating its role in maintaining structural correctness of calculations\.
- •Exact day\-count \(nn\) computationhas a comparatively localized impact, with accuracy remaining at88\.89%, suggesting that its contribution is limited to specific edge cases rather than the majority of queries\.

Overall, the ablation study demonstrates that CIFQA’s performance is driven by precise deterministic handling of financial computation primitives\. In particular, calendar\-aware logic and structured parameter grounding emerge as the most critical components, reinforcing the importance of deterministic execution in calculation\-intensive financial reasoning\.

## 8Discussion

The results demonstrate that CIFQA effectively eliminates arithmetic hallucinations in calculation\-intensive financial queries by enforcing a strict separation between language understanding and numerical computation\. This architectural design has broader implications beyond the specific fixed deposit \(FD\) use case evaluated in this work\.

### 8\.1Generalization to Other Financial Domains

Although CIFQA is instantiated on FD query answering, the framework is inherently domain\-agnostic\. The deterministic computation layer can be extended to other financial applications by replacing only domain\-specific modules while keeping all multi\-agent LLM components unchanged\. Figure[5](https://arxiv.org/html/2608.26114#S8.F5)illustrates this concretely for loan EMI calculation: the router, extractor, planner, and response generator agents require no modification, while only four deterministic modules are swapped — the compounding engine is replaced by an amortisation schedule engine, the rolling\-year engine by a calendar day\-count engine, the payout engine by a prepayment penalty engine, and rate tables are updated to loan rate slabs\. This modular boundary is precisely what makes CIFQA domain\-agnostic\.

![Refer to caption](https://arxiv.org/html/2608.26114v1/cifqa_emi_extension.png)Figure 5:CIFQA framework extension to loan EMI domain\. Multi\-agent LLM components \(router, extractor, planner, response generator\) remain unchanged across domains\. Only the deterministic computation engines are replaced with domain\-specific equivalents, confirming the modular and domain\-agnostic design of CIFQA\.Potential applications include:

- •Loan amortization and EMI calculations— replace compounding and payout engines with amortisation schedule and prepayment penalty engines\.
- •Bond pricing and yield computation— replace rate lookup with yield curve interpolation and day\-count convention engines\.
- •Portfolio return and investment analysis— replace interest engines with return computation and rebalancing rule engines\.
- •Tax computation and compliance systems— replace FD rule engine with jurisdiction\-specific tax slab and deduction engines\.

In each case, the core principle remains unchanged: LLMs handle interpretation and planning, while deterministic engines ensure exact numerical correctness\.

### 8\.2Architecture vs Model Scaling

A key finding of this work is that increasing model size alone does not resolve arithmetic reasoning errors\. Even large\-scale models exhibit significant performance gaps in calculation\-intensive tasks\. In contrast, CIFQA enables smaller models to outperform larger ones by removing the burden of arithmetic reasoning from the LLM\.

This suggests that architectural design, rather than model scale, is the primary determinant of performance in domains requiring exact computation\. Notably, the 17B CIFQA instantiation outperforms substantially larger proprietary models despite having significantly fewer parameters, indicating that deterministic execution can compensate for limitations in model scale when exact numerical correctness is required\. The findings are consistent with prior observations that LLMs often struggle with arithmetic and symbolic reasoning despite scaling improvements\[[33](https://arxiv.org/html/2608.26114#bib.bib25),[9](https://arxiv.org/html/2608.26114#bib.bib19),[25](https://arxiv.org/html/2608.26114#bib.bib20),[22](https://arxiv.org/html/2608.26114#bib.bib12),[2](https://arxiv.org/html/2608.26114#bib.bib38),[32](https://arxiv.org/html/2608.26114#bib.bib44),[17](https://arxiv.org/html/2608.26114#bib.bib43)\], and indicate that deterministic execution remains necessary for reliable financial computation\.

### 8\.3Separation of Concerns in LLM Systems

CIFQA exemplifies a broader design paradigm for LLM\-based systems: separating probabilistic language understanding from deterministic computation through external execution engines\[[25](https://arxiv.org/html/2608.26114#bib.bib20),[9](https://arxiv.org/html/2608.26114#bib.bib19),[40](https://arxiv.org/html/2608.26114#bib.bib21),[11](https://arxiv.org/html/2608.26114#bib.bib36),[23](https://arxiv.org/html/2608.26114#bib.bib37)\]\. This separation improves reliability, interpretability, and reproducibility, particularly in high\-stakes domains such as finance\.

The results indicate that LLMs are well\-suited for tasks involving ambiguity, interpretation, and planning, but should not be relied upon for exact numerical execution\. Integrating deterministic computation components provides a principled solution to this limitation\.

### 8\.4Limitations

While CIFQA achieves near\-perfect performance on calculation\-intensive queries, certain limitations remain:

- •Performance on policy\-heavy \(RAG\-based\) queries is comparatively lower, as these tasks involve complex rule interpretation beyond deterministic computation\.
- •The framework relies on accurate parameter extraction and routing, and errors in these stages can propagate to downstream components\.
- •The current evaluation focuses on fixed deposit scenarios, and broader validation across multiple financial domains remains future work\.

Addressing these limitations will require improved integration of retrieval and reasoning mechanisms alongside deterministic computation\.

### 8\.5Implications for Financial AI Systems

The findings of this work suggest that reliable financial AI systems should not rely solely on end\-to\-end LLM reasoning\. Instead, hybrid architectures that combine LLM\-based reasoning with deterministic computation offer a more robust and scalable approach\.

By ensuring exact numerical correctness, CIFQA provides a pathway for deploying LLM\-based systems in real\-world financial applications where precision is critical\.

## 9Conclusion

This paper introduced CIFQA, a deterministic tool\-grounded multi\-agent LLM framework for calculation\-intensive financial query answering\. CIFQA addresses a fundamental limitation of contemporary LLMs: their inability to reliably execute exact financial computations despite strong natural language understanding capabilities\. By enforcing a strict separation between language understanding and numerical execution, CIFQA transforms financial query answering from a text\-generation problem into an executable reasoning problem\.

Evaluated on a curated benchmark of fixed deposit queries, CIFQA achieves95\.54% accuracy on calculation\-intensive tasksand90\.87% overall accuracy, significantly outperforming both proprietary and open\-source LLMs including GPT\-5\.3, Gemini 3, and Claude Sonnet 4\.6\. Crucially, a 17B open\-source backbone operating within the CIFQA framework outperforms substantially larger frontier models evaluated with complete formulas and rate information, demonstrating that architectural design — specifically, the strict separation of language understanding from deterministic computation — is a more important determinant of numerical reliability than model scale\.

A detailed ablation study confirms that deterministic components, particularly calendar\-aware rolling\-year computation and exact rate lookup, are the primary drivers of this performance advantage\. These findings highlight that arithmetic hallucinations in financial reasoning are an architectural problem, not an information problem, and that hybrid systems combining LLM\-based interpretation with verifiable deterministic execution provide a principled solution\.

Beyond the fixed deposit domain, CIFQA establishes a generalizable design pattern for calculation\-intensive reasoning across financial applications\. The modular separation between LLM agents and deterministic engines means that extending CIFQA to loan amortization, bond yield computation, or tax settlement requires only replacing domain\-specific computation modules, with all language understanding components remaining unchanged\.

Looking ahead, the most significant opportunity for future work lies in extending CIFQA from reactive query answering toward proactive financial decision support\. Several high\-value scenarios remain unaddressed by the current system\. First,interest rate trend and pattern analysis: while CIFQA already performs exact rate lookup and identifies the tenure bracket yielding maximum interest for a given customer profile, a decision\-support extension could go further by analysing structural patterns across the rate card — for example, identifying that senior citizens consistently receive a 0\.50 percentage point premium over regular citizens across all tenure slabs, detecting tenure ranges where the rate curve flattens or spikes, or flagging booking\-period windows where rates have historically been most favourable\. Such pattern\-level reasoning requires moving beyond deterministic lookup toward analytical inference over structured rate data, representing a natural next frontier for the CIFQA framework\. Second,multi\-FD liquidation planning: when a customer requires urgent liquidity across a portfolio of fixed deposits with different tenures, rates, and penalty conditions, the system could deterministically compute the net cost of breaking each FD — accounting for premature withdrawal penalties, accrued interest forfeiture, and residual tenure — and recommend the optimal liquidation sequence that minimises financial loss\. Third,cross\-asset class optimisation: in scenarios where a customer holds an FD and requires short\-term liquidity, the system could compare the effective cost of breaking the FD against the cost of taking a short\-term loan against it\. For example, if a customer holds an FD earning 7\.5% annually and requires funds for 30 days, taking a loan at 9% for one month may be financially superior to premature FD closure, since the effective loan cost for 30 days is approximately 0\.74% while the penalty and interest forfeiture on breaking the FD may exceed 1–2%\. Such cross\-instrument comparisons require exactly the kind of deterministic multi\-step computation that CIFQA is architecturally designed to support, making this a natural and high\-impact direction for extension\.

These extensions would require augmenting the planner agent with multi\-objective reasoning capabilities — comparing outcomes across instruments, time horizons, and penalty structures — while retaining the deterministic execution layer that ensures correctness\. The CIFQA architecture is well\-positioned for this evolution: the same separation of concerns that eliminates arithmetic hallucinations in single\-query answering also provides the computational reliability necessary for trustworthy financial decision support in multi\-instrument, multi\-objective scenarios\.

## References

- \[1\]\(2023\)Self\-RAG: learning to retrieve, generate, and critique through self\-reflection\.arXiv preprint arXiv:2310\.11511\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p2.1)\.
- \[2\]F\. Chartonet al\.\(2025\)Architectural limits of LLMs in symbolic computation and structured reasoning\.arXiv preprint arXiv:2507\.10624\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p4.1),[§8\.2](https://arxiv.org/html/2608.26114#S8.SS2.p2.1)\.
- \[3\]W\. Chen, X\. Ma, X\. Wang, and W\. W\. Cohen\(2023\)Program of thoughts prompting: disentangling computation from reasoning for numerical reasoning tasks\.InTransactions on Machine Learning Research,Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p5.1),[§2](https://arxiv.org/html/2608.26114#S2.p3.1)\.
- \[4\]Z\. Chen, W\. Chen, C\. Smiley, S\. Shah, I\. Borova, D\. Langdon, R\. Moussa, M\. Beane, T\. Huang, B\. Routledge, and W\. Y\. Wang\(2021\)FinQA: a dataset of numerical reasoning over financial data\.arXiv preprint arXiv:2109\.00122\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p1.1)\.
- \[5\]Z\. Chen, S\. Li, J\. Smiley, Z\. Ma, M\. B\. Shah, C\. Shah, Z\. Yang, H\. Shi, and W\. Y\. Wang\(2022\)ConvFinQA: exploring the chain of numerical reasoning in conversational finance question answering\.arXiv preprint arXiv:2210\.03849\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p1.1)\.
- \[6\]K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman\(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p1.1)\.
- \[7\]A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur,et al\.\(2024\)The Llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p5.1)\.
- \[8\]N\. Dziri, X\. Lu, M\. Sclar, X\. L\. Li, L\. Jian, B\. Y\. Lin, P\. West, C\. Bhagavatula, R\. L\. Bras, J\. D\. Hwang, S\. Sanyal, S\. Welleck, G\. Bhatt, S\. Ruder, X\. Ren, A\. Ettinger, Z\. Harchaoui, and Y\. Choi\(2023\)Faith and fate: limits of transformers on compositionality\.arXiv preprint arXiv:2305\.18654\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p4.1)\.
- \[9\]L\. Gao, A\. Madaan, S\. Zhou, U\. Alon, P\. Yang, J\. Callan, and G\. Neubig\(2023\)Program\-aided language models\.InInternational Conference on Machine Learning \(ICML\),Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p1.1),[§1](https://arxiv.org/html/2608.26114#S1.p5.1),[§2](https://arxiv.org/html/2608.26114#S2.p3.1),[§2](https://arxiv.org/html/2608.26114#S2.p4.1),[§4\.3](https://arxiv.org/html/2608.26114#S4.SS3.p1.1),[§4](https://arxiv.org/html/2608.26114#S4.p1.1),[§8\.2](https://arxiv.org/html/2608.26114#S8.SS2.p2.1),[§8\.3](https://arxiv.org/html/2608.26114#S8.SS3.p1.1)\.
- \[10\]Y\. Gao, Y\. Xiong, X\. Gao, K\. Jia, J\. Pan, Y\. Bi, Y\. Dai, J\. Sun, and H\. Wang\(2023\)Retrieval\-augmented generation for large language models: a survey\.arXiv preprint arXiv:2312\.10997\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p2.1)\.
- \[11\]A\. d\. Garcez and L\. C\. Lamb\(2020\)Neuro\-symbolic AI: the third wave\.AI Magazine41\(2\),pp\. 21–36\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p4.1),[§8\.3](https://arxiv.org/html/2608.26114#S8.SS3.p1.1)\.
- \[12\]S\. Hong, M\. Zhuge, J\. Chen, X\. Zheng, Y\. Cheng, C\. Zhang, J\. Wang, Z\. Wang, S\. Yau, Z\. Lin,et al\.\(2023\)MetaGPT: meta programming for multi\-agent collaborative frameworks\.arXiv preprint arXiv:2308\.00352\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p3.1),[§4](https://arxiv.org/html/2608.26114#S4.p1.1)\.
- \[13\]L\. Huang, W\. Yu, W\. Ma, W\. Zhong, Z\. Feng, H\. Wang, Q\. Chen, W\. Peng, X\. Feng, B\. Qin, and T\. Liu\(2023\)A survey on hallucination in large language models: principles, taxonomy, challenges, and open questions\.arXiv preprint arXiv:2311\.05232\.Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p1.1)\.
- \[14\]X\. Huang, W\. Liu, X\. Chen, X\. Wang, H\. Wang, D\. Lian, Y\. Wang, R\. Tang, and E\. Chen\(2024\)Understanding the planning of LLM agents: a survey\.arXiv preprint arXiv:2402\.02716\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p3.1)\.
- \[15\]P\. Islam, A\. Kannappan, D\. Kiela, R\. Qian, N\. Scherrer, and B\. Vidgen\(2023\)FinanceBench: a new benchmark for financial question answering\.arXiv preprint arXiv:2311\.11944\.Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p1.1),[§1](https://arxiv.org/html/2608.26114#S1.p4.1),[§2](https://arxiv.org/html/2608.26114#S2.p1.1),[§3](https://arxiv.org/html/2608.26114#S3.p7.1)\.
- \[16\]Z\. Ji, N\. Lee, R\. Frieske, T\. Yu, D\. Su, Y\. Xu, A\. Zeng, Y\. S\. Fung, Y\. Hong, and P\. Fung\(2023\)Survey of hallucination in natural language generation\.ACM Computing Surveys55\(12\),pp\. 1–38\.Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p1.1)\.
- \[17\]J\. Kaplan, S\. McCandlish, T\. Henighan, T\. B\. Brown, B\. Chess, R\. Child, S\. Gray, A\. Radford, J\. Wu, and D\. Amodei\(2020\)Scaling laws for neural language models\.arXiv preprint arXiv:2001\.08361\.Cited by:[§8\.2](https://arxiv.org/html/2608.26114#S8.SS2.p2.1)\.
- \[18\]Y\. Lee, Y\. Chen, Z\. Huang, Y\. Chen, and X\. Liu\(2024\)A survey of large language models in finance \(finllms\)\.arXiv preprint arXiv:2402\.02315\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p1.1)\.
- \[19\]P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.\(2020\)Retrieval\-augmented generation for knowledge\-intensive NLP tasks\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p2.1)\.
- \[20\]G\. Li, H\. A\. A\. K\. Hammoud, H\. Itani, D\. Khizbullin, and B\. Ghanem\(2023\)CAMEL: communicative agents for mind exploration of large language model society\.arXiv preprint arXiv:2303\.17760\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p3.1),[§4](https://arxiv.org/html/2608.26114#S4.p1.1)\.
- \[21\]M\. Li, Y\. Zhao, B\. Yu, F\. Song, H\. Li, H\. Yu, Z\. Li, F\. Huang, and Y\. Li\(2023\)API\-Bank: a comprehensive benchmark for tool\-augmented LLMs\.arXiv preprint arXiv:2304\.08244\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p3.1)\.
- \[22\]I\. Mirzadeh, K\. Alizadeh\-Vahid, H\. Shahrokhi, O\. Tuzel, S\. Bengio, and M\. Farajtabar\(2024\)GSM\-Symbolic: understanding the limitations of mathematical reasoning in large language models\.arXiv preprint arXiv:2410\.05229\.Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p1.1),[§2](https://arxiv.org/html/2608.26114#S2.p4.1),[§8\.2](https://arxiv.org/html/2608.26114#S8.SS2.p2.1)\.
- \[23\]U\. Nawaz, M\. Anees\-ur\-Rahaman, and Z\. Saeed\(2025\)A review of neuro\-symbolic AI integrating reasoning and learning for advanced cognitive systems\.Intelligent Systems with Applications26,pp\. 200541\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p4.1),[§8\.3](https://arxiv.org/html/2608.26114#S8.SS3.p1.1)\.
- \[24\]S\. G\. Patil, T\. Zhang, X\. Wang, and J\. E\. Gonzalez\(2023\)Gorilla: large language model connected with massive APIs\.arXiv preprint arXiv:2305\.15334\.Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p5.1),[§2](https://arxiv.org/html/2608.26114#S2.p3.1)\.
- \[25\]T\. Schick, J\. Dwivedi\-Yu, R\. Dessì, R\. Raileanu, M\. Lomeli, E\. Hambro, L\. Zettlemoyer, N\. Cancedda, and T\. Scialom\(2023\)Toolformer: language models can teach themselves to use tools\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p5.1),[§2](https://arxiv.org/html/2608.26114#S2.p3.1),[§2](https://arxiv.org/html/2608.26114#S2.p4.1),[§4\.3](https://arxiv.org/html/2608.26114#S4.SS3.p1.1),[§4](https://arxiv.org/html/2608.26114#S4.p1.1),[§8\.2](https://arxiv.org/html/2608.26114#S8.SS2.p2.1),[§8\.3](https://arxiv.org/html/2608.26114#S8.SS3.p1.1)\.
- \[26\]N\. Shinn, F\. Cassano, E\. Berman, A\. Gopinath, K\. Narasimhan, and S\. Yao\(2023\)Reflexion: language agents with verbal reinforcement learning\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p5.1),[§2](https://arxiv.org/html/2608.26114#S2.p3.1)\.
- \[27\]A\. Singh, S\. Bhattamishra, and P\. Bhattacharyya\(2025\)Mathematical reasoning in large language models: assessing logical and arithmetic errors across wide numerical ranges\.arXiv preprint arXiv:2502\.08680\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p4.1)\.
- \[28\]H\. Touvron, L\. Martin, K\. Stone, P\. Albert, A\. Almahairi, Y\. Babaei, N\. Bashlykov, S\. Batra, P\. Bhargava, S\. Bhosale,et al\.\(2023\)Llama 2: open foundation and fine\-tuned chat models\.arXiv preprint arXiv:2307\.09288\.Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p5.1)\.
- \[29\]L\. Wang, C\. Ma, X\. Feng, Z\. Zhang, H\. Yang, J\. Zhang, Z\. Chen, J\. Tang, X\. Chen, Y\. Lin, W\. X\. Zhao, Z\. Wei, and J\. Wen\(2024\)A survey on large language model based autonomous agents\.Frontiers of Computer Science18\(6\),pp\. 186345\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p3.1)\.
- \[30\]X\. Wanget al\.\(2023\)Multi\-agent collaboration with large language models\.arXiv preprint arXiv:2304\.03442\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p3.1)\.
- \[31\]X\. Wang, J\. Wei, D\. Schuurmans, Q\. Le, E\. Chi, S\. Narang, A\. Chowdhery, and D\. Zhou\(2022\)Self\-consistency improves chain of thought reasoning in language models\.arXiv preprint arXiv:2203\.11171\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p3.1)\.
- \[32\]J\. Wei, Y\. Tay, R\. Bommasani, C\. Raffel, B\. Zoph, S\. Borgeaud, D\. Yogatama, M\. Bosma, D\. Zhou, D\. Metzler, E\. H\. Chi, T\. Hashimoto, O\. Vinyals, P\. Liang, J\. Dean, and W\. Fedus\(2022\)Emergent abilities of large language models\.Transactions on Machine Learning Research\.Cited by:[§8\.2](https://arxiv.org/html/2608.26114#S8.SS2.p2.1)\.
- \[33\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, E\. Chi, S\. Narang, A\. Chowdhery, Q\. Le, and D\. Zhou\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p1.1),[§2](https://arxiv.org/html/2608.26114#S2.p3.1),[§8\.2](https://arxiv.org/html/2608.26114#S8.SS2.p2.1)\.
- \[34\]Q\. Wu, G\. Bansal, J\. Zhang, Y\. Wu, B\. Li, E\. Zhu, L\. Jiang, X\. Zhang, C\. Liu, A\. Awadallah,et al\.\(2023\)AutoGen: enabling next\-gen LLM applications via multi\-agent conversation\.arXiv preprint arXiv:2308\.08155\.Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p5.1),[§2](https://arxiv.org/html/2608.26114#S2.p3.1),[§4](https://arxiv.org/html/2608.26114#S4.p1.1)\.
- \[35\]S\. Wu, O\. Irsoy, S\. Lu, V\. Dabravolski, M\. Dredze, S\. Gehrmann, P\. Kambadur, D\. Rosenberg, and G\. Mann\(2023\)BloombergGPT: a large language model for finance\.arXiv preprint arXiv:2303\.17564\.Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p1.1),[§1](https://arxiv.org/html/2608.26114#S1.p4.1),[§2](https://arxiv.org/html/2608.26114#S2.p1.1),[§3](https://arxiv.org/html/2608.26114#S3.p7.1)\.
- \[36\]G\. Yang, Z\. Wen, J\. Chen, J\. Hu, S\. S\. Du, and L\. Wang\(2024\)How numerical precision affects mathematical reasoning capabilities of LLMs\.arXiv preprint arXiv:2410\.13857\.Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p1.1),[§2](https://arxiv.org/html/2608.26114#S2.p4.1)\.
- \[37\]H\. Yang, X\. Liu, and C\. D\. Wang\(2023\)FinGPT: open\-source financial large language models\.arXiv preprint arXiv:2306\.06031\.Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p1.1),[§1](https://arxiv.org/html/2608.26114#S1.p4.1),[§2](https://arxiv.org/html/2608.26114#S2.p1.1),[§3](https://arxiv.org/html/2608.26114#S3.p7.1)\.
- \[38\]Y\. Yanget al\.\(2020\)FinBERT: financial sentiment analysis with pre\-trained language models\.arXiv preprint arXiv:2006\.08097\.Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p1.1),[§1](https://arxiv.org/html/2608.26114#S1.p4.1),[§2](https://arxiv.org/html/2608.26114#S2.p1.1),[§3](https://arxiv.org/html/2608.26114#S3.p7.1)\.
- \[39\]S\. Yao, D\. Yu, J\. Zhao, I\. Shafran, T\. L\. Griffiths, Y\. Cao, and K\. Narasimhan\(2023\)Tree of thoughts: deliberate problem solving with large language models\.InAdvances in Neural Information Processing Systems \(NeurIPS\),Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p3.1)\.
- \[40\]S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao\(2023\)ReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§1](https://arxiv.org/html/2608.26114#S1.p5.1),[§2](https://arxiv.org/html/2608.26114#S2.p3.1),[§2](https://arxiv.org/html/2608.26114#S2.p4.1),[§4\.3](https://arxiv.org/html/2608.26114#S4.SS3.p1.1),[§4](https://arxiv.org/html/2608.26114#S4.p1.1),[§8\.3](https://arxiv.org/html/2608.26114#S8.SS3.p1.1)\.
- \[41\]H\. Zhao, Z\. Liu, Z\. Wu, Y\. Li, T\. Yang, P\. Shu, S\. Xu, H\. Dai, L\. Zhao, G\. Mai, N\. Liu, and T\. Liu\(2024\)Revolutionizing finance with llms: an overview of applications and insights\.arXiv preprint arXiv:2401\.11641\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p1.1)\.
- \[42\]D\. Zhou, N\. Schärli, L\. Hou, J\. Wei, N\. Scales, X\. Wang, D\. Schuurmans, C\. Cui, O\. Bousquet, Q\. Le, and E\. Chi\(2022\)Least\-to\-most prompting enables complex reasoning in large language models\.arXiv preprint arXiv:2205\.10625\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p3.1)\.
- \[43\]F\. Zhu, W\. Lei, Y\. Huang, C\. Wang, S\. Zhang, J\. Lv, F\. Feng, and T\. Chua\(2021\)TAT\-QA: a question answering benchmark on a hybrid of tabular and textual content in finance\.InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics \(ACL\-IJCNLP\),pp\. 3277–3287\.Cited by:[§2](https://arxiv.org/html/2608.26114#S2.p1.1)\.