Tag
This paper introduces a benchmark to study how large language models arbitrate conflicting evidence from text and numerical sources, finding that models use heuristic strategies with biases towards recency and external tools.
This paper uses mechanistic interpretability to analyze how LLaMA 3.1 8B models numerical sequences, revealing that it internally computes and stores first differences to extrapolate patterns.
Introduces CreditCardQA, a financial literacy benchmark for numerical reasoning derived from real credit card agreements. Evaluates LLMs under Chain-of-Thought and Program-of-Thought prompting, finding Program-of-Thought yields consistent gains and errors stem from misapplied financial rules rather than arithmetic.
This paper investigates how irrelevant numbers in prompts cause anchoring effects in language models and localizes the internal pathways carrying this signal using attribution-based circuit methods on Qwen and Llama models.
This paper introduces FinAgent-RAG, an agentic framework for financial document question answering that combines iterative retrieval, Program-of-Thought reasoning, and adaptive resource allocation to improve accuracy and reduce costs.
WildTableBench introduces the first question-answering benchmark for real-world table images, revealing that existing multimodal foundation models struggle significantly with structural perception and numerical reasoning, with only one model exceeding 50% accuracy.
This paper introduces Triadic Suffix Tokenization (TST), a deterministic tokenization scheme that partitions digits into three-digit triads with explicit magnitude markers to improve numerical reasoning in large language models. The method addresses inconsistent number fragmentation in standard tokenizers by providing transparent order-of-magnitude relationships at the token level, with two implementation variants offering scalable vocabulary expansion.