floating-point

Tag

Cards List
#floating-point

Benchmarking Large Language Models on Floating-Point Error Classification

arXiv cs.AI · yesterday Cached

This paper introduces InterFLOPBench, a benchmark for evaluating LLMs on detecting floating-point errors in C code, finding that recent models achieve high F1-scores, though performance varies by error type.

0 favorites 0 likes
#floating-point

Compiler-Assisted Floating-Point Error Analysis and Profiling with FPChecker

Hacker News Top · 3d ago Cached

A half-day tutorial at ISC High Performance 2026 on using compiler-assisted tools (FPChecker/LLVM) for floating-point error analysis and profiling in C/C++ scientific codes.

0 favorites 0 likes
#floating-point

Die analysis of the 8087 math coprocessor's fast bit shifter (2020)

Hacker News Top · 2026-06-22 Cached

Die analysis of the Intel 8087 math coprocessor's fast bit shifter, exploring its architecture and role in floating-point operations.

0 favorites 0 likes
#floating-point

When float division beats integer division

Lobsters Hottest · 2026-06-16 Cached

A blog post explaining a counterintuitive optimization where using float division (DIVSD) instead of integer division (IDIVQ) yields faster performance on modern CPUs, with benchmarks and assembly analysis.

0 favorites 0 likes
#floating-point

Unity vs. Floating Point

Hacker News Top · 2026-06-15 Cached

This article examines the performance and precision differences between Unity's Mathf and System.MathF in C#, revealing that Unity's Mono runtime often performs double-precision math even for float operations, with implications for performance optimization.

0 favorites 0 likes
#floating-point

Comparison and Benchmarking of Rust Decimal Crates

Lobsters Hottest · 2026-06-15 Cached

A detailed technical article comparing and benchmarking various Rust decimal crates, covering fixed-point vs floating-point and fixed vs arbitrary precision designs.

0 favorites 0 likes
#floating-point

Intermediate Floating-Point Precision

Lobsters Hottest · 2026-06-14 Cached

This article explores how intermediate floating-point precision in C++ code depends on compiler settings, CPU flags, and architecture, particularly on x87 FPU, and how this affects performance and calculation results.

0 favorites 0 likes
#floating-point

The adder at the heart of Intel's 8087 floating-point chip

Lobsters Hottest · 2026-06-13 Cached

A detailed reverse-engineering analysis of the 69-bit adder in Intel's 8087 floating-point coprocessor from 1980, explaining its fast carry techniques and role in computing transcendental functions.

0 favorites 0 likes
#floating-point

The adder at the heart of Intel's 8087 floating-point chip

Hacker News Top · 2026-06-13 Cached

This article reverse-engineers the 69-bit adder at the core of Intel's 8087 floating-point coprocessor from 1980, explaining its architecture and carry-chain techniques.

0 favorites 0 likes
#floating-point

dMX: Differentiable Mixed-Precision Assignment for Low-Precision Floating-Point Formats

arXiv cs.LG · 2026-06-04 Cached

dMX is a differentiable mixed-precision quantization framework that learns optimal floating-point bit-width assignments per layer for LLMs, targeting the MXFP family of formats defined by the OCP standard. It uses continuous optimization with temperature-based annealing and a budget-aware regularization term, consistently outperforming KL-divergence heuristics on Llama, Qwen3, and SmolLM2 models.

0 favorites 0 likes
#floating-point

Novel Aspects of IEEE SA P3109 Arithmetic Formats for Machine Learning

arXiv cs.LG · 2026-06-04 Cached

The IEEE P3109 draft standard defines a parameterized family of binary floating-point formats and operations tailored for machine learning, supporting configurable width, precision, signedness, and infinities, along with extensive rounding modes including stochastic rounding and a novel scale-invariant approximation measure called kappa-approximation.

0 favorites 0 likes
#floating-point

Why does your agent still give different answers at temperature 0?

Reddit r/AI_Agents · 2026-06-04

Setting temperature to 0 does not guarantee deterministic tool calls in agents due to batched inference causing floating-point reduction order shifts, leading to token flips and different actions under load.

0 favorites 0 likes
#floating-point

Should you normalize RGB values by 255 or 256?

Lobsters Hottest · 2026-06-01 Cached

The article compares two methods for normalizing RGB values (dividing by 255 vs 256) and explains the consequences for floating-point conversion and rounding, including uneven bin widths at the extremes.

0 favorites 0 likes
#floating-point

Microcode inside the Intel 8087 floating-point chip: register exchange

Hacker News Top · 2026-05-30 Cached

A detailed reverse-engineering analysis of the microcode inside the Intel 8087 floating-point coprocessor, focusing on the FXCH register exchange instruction and the chip's internal architecture.

0 favorites 0 likes
#floating-point

Floor and Ceil versus Denormals on CPU and GPU

Hacker News Top · 2026-05-26 Cached

Explores the behavior of floor and ceil functions when applied to denormalized floating-point numbers, highlighting differences between CPU and GPU implementations and potential pitfalls.

0 favorites 0 likes
#floating-point

DEL: Digit Entropy Loss for Numerical Learning of Large Language Models

arXiv cs.CL · 2026-05-21 Cached

This paper introduces Digit Entropy Loss (DEL), a novel loss function for numerical learning in large language models that reformulates entropy optimization to improve digit-level prediction accuracy and handle floating-point numbers, consistently outperforming existing methods on mathematical reasoning benchmarks.

0 favorites 0 likes
#floating-point

RISC-V and Floating Point

Hacker News Top · 2026-05-18

A report on the RISC-V architecture's floating point capabilities and updates.

0 favorites 0 likes
#floating-point

Schanuel's Conjecture and the Semantics of Triton's FPSan

Hacker News Top · 2026-05-16 Cached

FPSan is a Triton compiler pass that enables verification of algebraic equivalence of floating-point programs by replacing floating-point operations with integer operations, relying on Schanuel's conjecture for correctness.

0 favorites 0 likes
#floating-point

25+ years of pathfinding problems with C++

Lobsters Hottest · 2026-05-11 Cached

The Engineering Director of Age of Empires provides an in-depth analysis of the technical debt in the series' pathfinding system over the past 25 years, pointing out that legacy code, dynamic map mechanics, and floating-point errors caused by SIMD instruction sets replacing x87 extended precision are the root causes of classic bugs such as units clipping through walls.

0 favorites 0 likes
#floating-point

Fc, a lossless compressor for floating-point streams

Hacker News Top · 2026-05-10 Cached

fc is an open-source lossless compressor for IEEE-754 64-bit double streams, offering superior compression ratios for structured data compared to zstd and fpzip, though with slower encoding speeds.

0 favorites 0 likes
Next →
← Back to home

Submit Feedback