Benchmarking Large Language Models on Floating-Point Error Classification
Summary
This paper introduces InterFLOPBench, a benchmark for evaluating LLMs on detecting floating-point errors in C code, finding that recent models achieve high F1-scores, though performance varies by error type.
View Cached Full Text
Cached at: 07/01/26, 05:37 AM
# Benchmarking Large Language Models on Floating-Point Error Classification Source: [https://arxiv.org/abs/2606.31308](https://arxiv.org/abs/2606.31308) [View PDF](https://arxiv.org/pdf/2606.31308) > Abstract:This paper investigates the capability of Large Language Models \(LLMs\) to detect and classify floating\-point errors statically in software code\. We introduce InterFLOPBench, a benchmark of 90 C kernels with 1 130 test samples designed to evaluate LLMs across six categories of floating\-point error: cancellation, comparison, division by zero, overflow, underflow and NaN, compared across 14 LLMs\. The evaluation framework treats floating\-point error detection as a multi\-label classification problem and employs the F1\-score metric to measure performance\. Results demonstrate that latest models \(Qwen 3 32b, Gemini 2\.5 Flash, Phi 4 Reasoning, DeepSeek R1T2, and gpt\-oss 20b and 120b\) achieve a performance greater than 0\.88 overall F1\-score\. Performance varies between error categories, between explicit operations such as division by zero \(Average F1\-score: 0\.8479\) and more subtle numerical phenomena such as underflow \(Average F1\-score: 0\.6059\) and cancellation \(Average F1\-score: 0\.6164\)\. ## Submission history From: Lisa Taldir \[[view email](https://arxiv.org/show-email/021d3e8e/2606.31308)\] \[via CCSD proxy\] **\[v1\]**Tue, 30 Jun 2026 08:18:45 UTC \(258 KB\)
Similar Articles
Benchmarking LLMs
A study or report on benchmarking large language models, likely comparing performance across various tasks.
Are Large Language Models Reliable Reviewers? A Benchmark for Error Detection in Financial Documents
This paper introduces FinED-Bench, the first public benchmark for financial error detection in documents, covering nine real-world scenarios with over 900 documents from 2025. It evaluates current LLMs like GPT-4o and Qwen3-14B, showing they still struggle with high-complexity cases, while fine-tuning can improve weaker models.
LinAlg-Bench: A Forensic Benchmark Revealing Structural Failure Modes in LLM Mathematical Reasoning
Introduces LinAlg-Bench, a diagnostic benchmark evaluating 10 frontier LLMs on structured linear algebra computation across matrix dimensions, revealing that LLM mathematical failure is structurally constrained and transitions from execution errors to computational abandonment at 4x4 scale.
ParBench: A Benchmark for Reliable Evaluation of LLM Parallel Code Translation
ParBench is a benchmark framework for evaluating LLM-based parallel API translation across CUDA, OpenMP, OpenCL, and OpenMP target offload, focusing on executable, reproducible conditions and robustness testing.
Benchmarking Large Language Models on Multi-Sensor Physical Hazard Assessment
The paper benchmarks five large language models on multi-sensor physical hazard assessment, revealing that all tested models fail to produce precautionary warnings when multiple sensors are simultaneously elevated below their individual safety limits, while achieving near-perfect accuracy on single-sensor violations.