CurveFP: Rational-Radix Logarithmic Datatypes with Closed Products for Language Models

# Rational-Radix Logarithmic Datatypes with Closed Products for Language Models
Source: [https://arxiv.org/html/2608.10010](https://arxiv.org/html/2608.10010)
Ye Qiao Department of Electrical Engineering and Computer Science University of California, Irvine Irvine, CA, USA

###### Abstract

Low\-precision datatypes reduce language\-model cost, but most formats optimize scalar fidelity while leaving the arithmetic induced by their products unchanged\. We introduceCurveFP, a closed\-product codebook family that distributes quantized magnitudes across interleaved logarithmic curves under compact block scales\. A rational radix tunes dynamic range against local resolution, while uniform curve indices make every nonzero product algebraically closed\. Product formation becomes an exact sign XOR and integer\-index update, and a derived finite phase count determines the accumulation schedule\. We instantiate this algebra asCurveFP8E4C3/E5C2 for training andCurveFP7E3C3 for compact deployment\. In evaluation,CurveFP7beats tensorwise FP8 perplexity on four 7B–9B models with one fewer element bit and stays within 1\.32% of native quality\.CurveFP8lowers operand NMSE in all 36 paired forward and backward GEMM comparisons\. Across three matched 128\.3M\-parameter triplets, every mode completes 3B\-token pretraining per seed;CurveFP8reaches mean BF16\-inference perplexity 22\.5366 versus 22\.5407 for FP8 and incurs a lower format\-induced penalty in all three seeds\. A 36\-cell downstream matrix finds lower WikiText\-103 perplexity for theCurveFP8\-trained checkpoints in all 12 seed\-format comparisons, with mixed PG\-19 and task deltas\. Together, these results establishCurveFPas an arithmetic co\-design that combines FP8\-class numerical behavior, seven\-bit inference, and a substantially simpler product path\.

## 1Introduction

Low\-precision arithmetic has become a primary lever for scaling language\-model training and inference, yet prevailing datatypes optimize how individual values are approximated while largely accepting the cost of their induced products\. FP8 established that role\-specific exponent and mantissa layouts can match 16\-bit training quality\(Micikeviciuset al\.,[2022](https://arxiv.org/html/2608.10010#bib.bib1)\); microscaling formats then amortized range through per\-block scales\(Rouhaniet al\.,[2023](https://arxiv.org/html/2608.10010#bib.bib8)\)\. Post\-training methods attack the same numerical bottleneck from the data side, moving activation outliers into weights\(Xiaoet al\.,[2023](https://arxiv.org/html/2608.10010#bib.bib10)\)or rotating them away\(Ashkbooset al\.,[2024](https://arxiv.org/html/2608.10010#bib.bib11); Liuet al\.,[2025](https://arxiv.org/html/2608.10010#bib.bib12)\)\. These advances make quantization substantially more accurate, but their element products still require a conventional integer or floating\-point multiplier\.

Product structure is an underused source of efficiency in low\-precision design\. Learned product/subspace codebooks offer flexible compression points\(Wanget al\.,[2026](https://arxiv.org/html/2608.10010#bib.bib16); Qiaoet al\.,[2026b](https://arxiv.org/html/2608.10010#bib.bib17)\), but do not by themselves impose a scalar closed\-product algebra\. Uniform integers simplify accumulation but depend increasingly on fine\-grained scales as activation range widens\. Logarithmic representations make multiplication additive, but coarse powers of two sacrifice local resolution and still require an accumulation strategy\(Miyashitaet al\.,[2016](https://arxiv.org/html/2608.10010#bib.bib2)\)\. Additive powers\-of\-two codes recover more levels through several shift\-add terms\(Liet al\.,[2020](https://arxiv.org/html/2608.10010#bib.bib13)\)\. These trade\-offs motivate a stronger objective than scalar error alone: jointly design the level geometry and product algebra so that lower precision improves both representation density and arithmetic regularity\.

We realize this objective withCurveFP, a block\-scaled closed\-product codebook family\. As illustrated in Figure[1](https://arxiv.org/html/2608.10010#S1.F1),CurveFPspreads quantized magnitudes acrossKKinterleaved logarithmic curves\. Each nonzero element stores a sign, a two’s\-complement exponent, and a curve index under a shared power\-of\-two scale\. Uniform curve spacing makes multiplication exact in the code domain: sign XOR, curve\-index addition, and exponent addition with a carry\. A reduced rational radixr=2p/qr=2^\{p/q\}tunes dynamic range against local precision, while the derived phase countq​K/gcd⁡\(p,q​K\)qK/\\gcd\(p,qK\)exposes the number of fractional exponent classes that accumulation must combine\. Thus every range–resolution choice has an explicit accumulation contract, and every representable product lands on a known phase without lookup, projection, or a general product\-forming multiplier\.

![Refer to caption](https://arxiv.org/html/2608.10010v1/x1.png)Figure 1:CurveFPco\-designs representation and multiplication\.Power\-of\-two scales supply operand\-block range, while sign, exponent, and curve fields encode each element\. Uniform curve indices make products exactly closed under XOR and integer addition\. Rational radix producesHHfixed phase classes; products route to signed integer bins that are combined using fixed phase weights\. This mapping defines the inference and training formats evaluated below\.One algebra supports two practical operating points\.CurveFP8uses E4C3 for forward weights and activations and E5C2 for backward gradients, directly mirroring the range allocation of FP8 E4M3/E5M2 while replacing mantissa products with closed curve products\.CurveFP7uses E3C3 to reduce the element width by 12\.5% for deployment while retaining eight product phases\. Block scaling supplies the local range that seven\-bit activations need; compact binary scale exponents preserve the shift\-and\-count accumulation structure\. A single numeric family therefore spans training and inference with the same closure rule\.

The empirical evidence shows that this arithmetic co\-design delivers FP8\-class model quality\. Across Llama\-3\-8B, Qwen3\-8B, Qwen3\.5\-9B, and Falcon\-H1\-7B, block\-scaledCurveFP7beats tensorwise FP8 perplexity on all four models with one fewer element bit and remains within 1\.32% of the native reference\. For training,CurveFP8yields lower operand NMSE than FP8 in all 36 paired forward\-, activation\-gradient\-, and weight\-gradient windows\. Most importantly, three matched 128\.3M\-parameter experiments train BF16, FP8, andCurveFP8for 3\.00B tokens per mode and seed\. All nine lanes complete without divergence\. Under BF16 inference,CurveFP8attains mean perplexity 22\.5366 versus 22\.5407 for FP8, wins two of three paired seeds, and has the lower format\-induced penalty in all three\. The seed variation supports FP8\-class training parity rather than statistical superiority\.

Our contributions are:

- •We introduce a block\-scaled codebook family distributed across rational\-radix logarithmic curves, make its curve indices exactly closed under multiplication, and derive the finite phase count connecting representation choice to accumulation\.
- •We instantiate one algebra asCurveFP8for FP8\-class training andCurveFP7for seven\-bit deployment, with compact power\-of\-two scale metadata and no general multiplier in product formation\.
- •We provide end\-to\-end evidence spanning four\-model inference, rational\-radix and scale ablations, all three training GEMMs, three matched 3B\-token pretraining runs, and a 36\-cell OOD/downstream format\-transfer study\.

## 2Related Work

#### Low\-precision datatypes\.

FP8 combines E4M3 forward operands with E5M2 gradients to retain training range and has become the principal eight\-bit floating\-point baseline\(Micikeviciuset al\.,[2022](https://arxiv.org/html/2608.10010#bib.bib1)\)\. Microscaling extends this approach by sharing scale factors across blocks of narrow floating\-point or integer elements\(Rouhaniet al\.,[2023](https://arxiv.org/html/2608.10010#bib.bib8)\); hybrid block floating point likewise shares exponents during DNN training but retains linear significands\(Drumondet al\.,[2018](https://arxiv.org/html/2608.10010#bib.bib6)\)\. At the accelerator level, TeLLMe couples ternary weights with table\-lookup matrix multiplication for end\-to\-end LLM prefill and decode on edge FPGAs\(Qiaoet al\.,[2026a](https://arxiv.org/html/2608.10010#bib.bib18)\)\.

#### Logarithmic neural\-network arithmetic\.

Classical LNS encodes scalar magnitudes in the log domain, making multiplication additive; neural\-network work has applied it to quantized CNNs/LSTMs, accelerator arithmetic, and low\-precision training with multiplicative updates\(Miyashitaet al\.,[2016](https://arxiv.org/html/2608.10010#bib.bib2); Kouretas and Paliouras,[2018](https://arxiv.org/html/2608.10010#bib.bib3); Christet al\.,[2022](https://arxiv.org/html/2608.10010#bib.bib4); Zhaoet al\.,[2022](https://arxiv.org/html/2608.10010#bib.bib5)\)\. Flattening CurveFP’s exponent and curve fields inton=K​e\+kn=Ke\+kgives a finite block\-scaled LNS lattice, which explains the shared additive product algebra\. CurveFP uses that algebra through a different datatype interface: an explicit multi\-curve codebook, compact scale contracts, theH=q​K/gcd⁡\(p,q​K\)H=qK/\\gcd\(p,qK\)phase law, and role\-specific LLM formats trained with conventional AdamW\. APoT instead improves resolution through multiple shift\-add terms\(Liet al\.,[2020](https://arxiv.org/html/2608.10010#bib.bib13)\)\.

#### LLM quantization and structured codebooks\.

Modern LLM post\-training quantization often reshapes tensors before applying a conventional datatype\. SmoothQuant migrates activation outliers into weights\(Xiaoet al\.,[2023](https://arxiv.org/html/2608.10010#bib.bib10)\); QuaRot and SpinQuant use fixed or learned rotations to suppress outliers\(Ashkbooset al\.,[2024](https://arxiv.org/html/2608.10010#bib.bib11); Liuet al\.,[2025](https://arxiv.org/html/2608.10010#bib.bib12)\)\. Learned representations offer a complementary route: LO\-BCQ clusters blocks and assigns each cluster an optimized scalar codebook\(Elangovanet al\.,[2025](https://arxiv.org/html/2608.10010#bib.bib14)\), while NestQuant uses self\-similar nested lattices for low\-error matrix products\(Savkinet al\.,[2025](https://arxiv.org/html/2608.10010#bib.bib15)\)\. These methods target aggressive W4A4 inference and can exceed simple scalar formats in accuracy\. Product/subspace quantization follows another path: APEX\-Q permits arbitrary subvector dimensions, while FASQ offers calibration\-free flexible compression points and custom GPU kernels\(Wanget al\.,[2026](https://arxiv.org/html/2608.10010#bib.bib16); Qiaoet al\.,[2026b](https://arxiv.org/html/2608.10010#bib.bib17)\)\. Our objective is different:CurveFPparameterizes its levels with a rational radix and uniform curve indices, obtaining a compact closed\-product algebra shared by inference and training\. This structure reaches the FP8 quality regime without calibration\-time transformations\.

## 3CurveFP

#### Overview\.

CurveFPstarts from a closed\-product codebook view: distribute values across structured curves so representation quality and product composition improve together\. We first define the rational\-radix element code, then derive closed product formation and phase\-structured accumulation, and finally instantiate role\-specific formats for training and compact inference\.

### 3\.1A rational\-radix datatype family

CurveFPrepresents a tensor block with one power\-of\-two scale and represents each element by a sign, an exponent, and a curve index\. LetEEandCCdenote the exponent and curve\-index widths, letK=2CK=2^\{C\}, and choose a reduced rational radixr=2p/qr=2^\{p/q\}\. A nonzero quantized value in blockbbis

x^=\(−1\)s​2ab​re\+k/K,e∈\{−2E−1,…,2E−1−1\},k∈\{0,…,K−1\},\\widehat\{x\}=\(\-1\)^\{s\}2^\{a\_\{b\}\}r^\{e\+k/K\},\\qquad e\\in\\\{\-2^\{E\-1\},\\ldots,2^\{E\-1\}\-1\\\},\\quad k\\in\\\{0,\\ldots,K\-1\\\},\(1\)whereaba\_\{b\}is a signed shared\-scale exponent\. The element therefore occupies1\+E\+C1\+E\+Cbits\. We reserve magnitude index zero as a sentinel and omit the smallest nonzero exponent/curve combination, leaving2E\+C−12^\{E\+C\}\-1positive magnitudes; values outside the scaled range map to an endpoint\. Unlike a learned codebook, the levels require no table of arbitrary constants: adjacent curve indices differ by the fixed ratior1/Kr^\{1/K\}, and adjacent exponents differ byrr\.

Givenaba\_\{b\}, quantization preserves the source sign and selects the magnitude whose reconstructed value is nearest in absolute error, with saturation at the endpoints\. Static operands choose a representable power\-of\-two scale by reconstruction\-MSE search; dynamic training operands use ceil\-absmax scaling\. We write element\-wise tensor quantization asQ​\(𝑿\)Q\(\\bm\{X\}\), soQ​\(𝑿\)i​j=x^i​jQ\(\\bm\{X\}\)\_\{ij\}=\\widehat\{x\}\_\{ij\}\.

The rational radix parameterizes the spacing of this uniform logarithmic lattice\. Settingp=q=1p=q=1gives levels2e\+k/K2^\{e\+k/K\}\. Choosingp/q<1p/q<1contracts the range and increases local resolution without changing the element width, whereasp/q\>1p/q\>1expands the range\.CurveFPassociates every spacing with the phase count derived below, exposing a direct quality–complexity control\.

### 3\.2Closed products and phase\-structured accumulation

Uniform curve indices make the product codebook exactly closed\. For two nonzero operands with curve indicesiiandjj, define

ℓ=\(i\+j\)modK,c=⌊i\+jK⌋\.\\ell=\(i\+j\)\\bmod K,\\qquad c=\\left\\lfloor\\frac\{i\+j\}\{K\}\\right\\rfloor\.\(2\)Their product has signsx⊕sws\_\{x\}\\oplus s\_\{w\}, shared\-scale exponentax\+awa\_\{x\}\+a\_\{w\}, element exponentex\+ew\+ce\_\{x\}\+e\_\{w\}\+c, and curve indexℓ\\ell\. Product formation thus uses an XOR and small integer additions, with no general variable\-by\-variable multiplier and no nearest\-code projection\. A zero sentinel short\-circuits the product to zero without applying the index update\. This closed product path feeds a fixed phase reduction with binary scale transport; Section[5](https://arxiv.org/html/2608.10010#S5)summarizes its deployment implications\.

Closure holds on widened product coordinates\. Requantizing to a finiteEE\-bit destination may saturate; no codebook projection is needed before that explicit output requantization\.

Rational radices preserve this closure while changing the number of binary phases needed by a dot product\. From Equation[1](https://arxiv.org/html/2608.10010#S3.E1), every unscaled magnitude is

re\+k/K=2p​\(e​K\+k\)/\(q​K\)\.r^\{e\+k/K\}=2^\{p\(eK\+k\)/\(qK\)\}\.\(3\)LetN=q​KN=qK\. As the integern=e​K\+kn=eK\+kadvances, the fractional exponent is the residuep​nmodNpn\\bmod Ndivided byNN\. The additive subgroup generated byppinℤN\\mathbb\{Z\}\_\{N\}has order

H=q​Kgcd⁡\(p,q​K\)H=\\frac\{qK\}\{\\gcd\(p,qK\)\}\(4\)and therefore definesHHphase classes on the underlying lattice\. A finite format may leave some classes empty, but every dot product can be scheduled over theseHHphases and written as

𝒙^𝖳​𝒘^=∑h=0H−12h/H​∑tnh,t​2t,\\widehat\{\\bm\{x\}\}^\{\\mathsf\{T\}\}\\widehat\{\\bm\{w\}\}=\\sum\_\{h=0\}^\{H\-1\}2^\{h/H\}\\sum\_\{t\}n\_\{h,t\}2^\{t\},\(5\)where eachnh,tn\_\{h,t\}is a signed integer count\. Products are routed exactly to integer counts indexed by binary shift; only theHHfixed phase weights remain after reduction\.

#### Constructive arithmetic realization\.

Letg=gcd⁡\(p,q​K\)g=\\gcd\(p,qK\),nx=ex​K\+kxn\_\{x\}=e\_\{x\}K\+k\_\{x\},nw=ew​K\+kwn\_\{w\}=e\_\{w\}K\+k\_\{w\}, andu=p​\(nx\+nw\)u=p\(n\_\{x\}\+n\_\{w\}\)\. Each nonzero lane incrementsAh,tA\_\{h,t\}byδ=\(−1\)sx⊕sw\\delta=\(\-1\)^\{s\_\{x\}\\oplus s\_\{w\}\}ath=\(umodq​K\)/gh=\(u\\bmod qK\)/gandt=ax\+aw\+⌊u/\(q​K\)⌋t=a\_\{x\}\+a\_\{w\}\+\\lfloor u/\(qK\)\\rfloor; zero suppresses the update\. Forp=q=1p=q=1,h=\(kx\+kw\)modKh=\(k\_\{x\}\+k\_\{w\}\)\\bmod Kandt=ax\+aw\+ex\+ew\+⌊\(kx\+kw\)/K⌋t=a\_\{x\}\+a\_\{w\}\+e\_\{x\}\+e\_\{w\}\+\\lfloor\(k\_\{x\}\+k\_\{w\}\)/K\\rfloor, requiring only zero/XOR, one\(C\+1\)\(C\+1\)\-bit curve adder, and signed adders\. A length\-DDdot needswc=1\+⌈log2⁡\(D\+1\)⌉w\_\{c\}=1\+\\lceil\\log\_\{2\}\(D\+1\)\\rceil\-bit signed counters, orH​B​wcHBw\_\{c\}bits per output forBBreachable bins\. Equal addresses may be signed\-popcount\-compressed; a shift\-add scan andHHhard\-wired coefficients combine phases with at mostH−1H\-1additions before requantization\. The codebook algebra therefore specifies the required operations and accumulator state without committing to one bank layout or processor architecture\.

### 3\.3Shared scales, operating points, and training protocol

Shared scales let the element code focus on within\-block variation while amortizing range metadata\. For a block ofGGvalues and anSS\-bit scale exponent, the effective storage rate is

R=1\+E\+C\+SGbits per value\.R=1\+E\+C\+\\frac\{S\}\{G\}\\quad\\text\{bits per value\}\.\(6\)Because2ab2^\{a\_\{b\}\}is itself binary, it shifts the exponent\-bin index without introducing another product phase\. We evaluate tensorwise, rowwise, and fixed\-size block scaling, but use block scaling as the compact deployment contract because activations require local range adaptation\.

Table[1](https://arxiv.org/html/2608.10010#S3.T1)gives the principal operating points\. The eight\-bit training pair follows the familiar split between a precision\-oriented forward format and a range\-oriented gradient format\.CurveFP8E4C3 quantizes weights and activations in the forward linear layers, while E5C2 quantizes the output gradients used by both activation\- and weight\-gradient matrix products\. Moving one bit fromCCtoEEpreserves 127 positive levels, expands gradient range, and reduces the phase count from eight to four; this mirrors the precision\-oriented E4M3/range\-oriented E5M2 FP8 split\. Parameters and optimizer states remain full precision, and nonlinear operations execute in bfloat16\.CurveFP7E3C3 keeps eight curve phases in a seven\-bit element for compact post\-training deployment\.

Table 1:PrincipalCurveFPoperating points\. Positive levels exclude the reserved zero code\. All use binary radixp/q=1p/q=1;HHis the phase count from Equation[4](https://arxiv.org/html/2608.10010#S3.E4)\.The training operator quantizes the operands of every non\-head linear layer\. In the forward pass it computesQA​\(𝑿\)​QW​\(𝑾\)𝖳Q\_\{A\}\(\\bm\{X\}\)Q\_\{W\}\(\\bm\{W\}\)^\{\\mathsf\{T\}\}\. In the backward pass it reuses the quantized forward operands and quantizes the output gradientQG​\(∇𝒀\)Q\_\{G\}\(\\nabla\\bm\{Y\}\), yieldingQG​\(∇𝒀\)​QW​\(𝑾\)Q\_\{G\}\(\\nabla\\bm\{Y\}\)Q\_\{W\}\(\\bm\{W\}\)for the activation gradient andQG​\(∇𝒀\)𝖳​QA​\(𝑿\)Q\_\{G\}\(\\nabla\\bm\{Y\}\)^\{\\mathsf\{T\}\}Q\_\{A\}\(\\bm\{X\}\)for the weight gradient\. This placement matches the three matrix multiplications whose operand formats a native low\-precision training kernel must support\.

## 4Experiments

### 4\.1Experimental design

Our experiments evaluateCurveFPas a complete datatype family across inference and training\. We organize the evidence along three axes\. First, post\-training quantization on four open language\-model architectures tests the inference quality frontier and scale\-amortization behavior\. Second, backward\-GEMM diagnostics and controlled training runs test all three matrix products used by low\-precision training\. Third, matched from\-scratch pretraining followed by out\-of\-distribution and downstream evaluation tests whether any local numerical advantage survives optimization and format transfer\. We compare against BF16/FP16, the standard FP8 E4M3\-forward/E5M2\-backward pair\(Micikeviciuset al\.,[2022](https://arxiv.org/html/2608.10010#bib.bib1)\), microscaling formats, and signed integers with practical real\-valued scales\.

All formats share identical GEMM kernels and evaluation code, isolating datatype and optimization effects\. Unless stated otherwise, we quantize every linear layer except the tied output vocabulary head and retain BF16/FP16 for normalization and nonlinear operations\. Every comparison uses the same data, tokenizer, model weights or initialization, and evaluation code; all reported numbers are derived from saved JSON artifacts\.

### 4\.2CurveFP7 provides an INT8\-class inference alternative at seven bits

We first evaluate post\-training W/A quantization on four architectures spanning dense attention, grouped\-query attention, a recurrent hybrid, and a DeltaNet\-attention hybrid: Llama\-3\-8B, Qwen3\-8B, Falcon\-H1\-7B, and Qwen3\.5\-9B\(Grattafiori and others,[2024](https://arxiv.org/html/2608.10010#bib.bib27); Yang and others,[2025](https://arxiv.org/html/2608.10010#bib.bib28); Zuo and others,[2025](https://arxiv.org/html/2608.10010#bib.bib29); Qwen Team,[2026](https://arxiv.org/html/2608.10010#bib.bib30)\)\. Each run scores the complete WikiText\-2 test split with the same 2,048\-token rolling\-loglikelihood protocol\(Merityet al\.,[2016](https://arxiv.org/html/2608.10010#bib.bib19)\)\. Within each model, all methods share the same weights, output\-head exclusion, and K/V precision\. Llama and Qwen3 hold dense K/V at common tensorwise FP8 to isolate W/A arithmetic\. The principal comparison uses tensorwise and rowwise FP8 E4M3 againstCurveFP7E3C3 with one compact binary scale per 64 or 512 values\. The G64 format costs 7\.062 weight and 7\.078 activation bits per value including scales; G512 reduces these rates to 7\.008 and 7\.010\.

![Refer to caption](https://arxiv.org/html/2608.10010v1/x2.png)Figure 2:CurveFP7enters the FP8 quality regime and remains stable as scale groups widen\.\(a\) Perplexity increase over each native model\. G64CurveFP7beats tensorwise FP8 on all four architectures and wins against rowwise FP8 on Llama and Qwen3\.5 while trailing it on Qwen3 and Falcon\-H1\. \(b\) A representative complete granularity sweep on Falcon\-H1, reporting actual WikiText\-2 perplexity from G128 through rowwise and tensorwise scaling\. INT8 leads narrowly at G128/G256;CurveFP7crosses at G512 and remains better as scales are shared more broadly, including 6\.71 versus 10\.10 PPL at tensorwise scaling\.CurveFP7establishes a compelling seven\-bit alternative to INT8 inference\. At G64 it improves over tensorwise FP8 on every model and stays within 1\.32% perplexity of native\. Against rowwise FP8, it wins on Llama and Qwen3\.5 and trails by only 0\.0064 PPL on Qwen3 and 0\.0158 on Falcon\-H1\. More importantly, Figure[2](https://arxiv.org/html/2608.10010#S4.F2)b shows the practical granularity frontier on Falcon\-H1\. INT8 leads by only 0\.0095 and 0\.0103 PPL at G128 and G256;CurveFP7crosses at G512 \(5\.6702 versus 5\.6739\), then leads rowwise \(5\.7051 versus 5\.7881\) and tensorwise \(6\.7139 versus 10\.1033\)\. Thus the seven\-bit format remains competitive at block granularity and becomes more robust as scale sharing increases, while retaining lower data/metadata rate and closed\-product arithmetic\.

### 4\.3Rational radix validates the quality–phase control

All principal training and deployment formats use binary radix\. To test the rational generalization itself, Table[2](https://arxiv.org/html/2608.10010#S4.T2)varies only the radix while holding the E4C3 element width, G512 block size, and scale count fixed\. The near\-binaryr=28/9r=2^\{8/9\}adds one predicted phase and improves all four models, reducing perplexity by 0\.0003 to 0\.0111\. The denserr=23/4r=2^\{3/4\}reaches the best perplexity on every model but requires the predicted 32 phases\. This controlled ablation validates the design law: denser radices improve representation, Equation[4](https://arxiv.org/html/2608.10010#S3.E4)exposes their accumulation cost, and the eight\-phase binary point remains the main operating point\. Relative to binary radix, 32 phases improve PPL by only 0\.0055–0\.0259 across the four models while quadrupling the number of phase classes\.

Table 2:E4C3 perplexity at G512 as rational radix trades phases for local resolution\. Lower is better\.
### 4\.4CurveFP8 reduces error in all three training GEMMs

Before full pretraining, we measure operand reconstruction error at real forward and backward matrix products from Pythia\-410M\(Bidermanet al\.,[2023](https://arxiv.org/html/2608.10010#bib.bib9)\)and Llama\-3\.2\-3B\(Grattafiori and others,[2024](https://arxiv.org/html/2608.10010#bib.bib27)\)\. The diagnostic captures forwardX​W𝖳XW^\{\\mathsf\{T\}\}, activation\-gradient∇Y​W\\nabla YW, and weight\-gradient∇Y𝖳​X\\nabla Y^\{\\mathsf\{T\}\}Xwindows, then comparesCurveFP8E4C3/E5C2 with FP8 E4M3/E5M2 under the same tensors\. Each window is one disjoint 64\-token WikiText\-2 training segment and one real backward pass\. For each linear role, we select three layers spaced through model depth, giving 12 sampled Pythia linears and 21 sampled Llama linears per window\. As Table[3](https://arxiv.org/html/2608.10010#S4.T3)shows,CurveFP8lowers normalized MSE in every one of the 36 paired comparisons\. Its aggregate NMSE is 8\.2–11\.1% lower depending on model and GEMM, so the closed code’s benefit is not confined to forward inference\.

Table 3:CurveFP8NMSE divided by FP8 NMSE on paired training\-GEMM windows\. Values below one favorCurveFP8; all 36 pairs are wins\.
### 4\.5CurveFP8 completes three matched 3B\-token runs

The decisive training test repeats a matched triplet at three independent random seeds, yielding nine 128\.3M\-parameter Llama\-style decoders\. Within each triplet, BF16, FP8, andCurveFP8start from identical random parameters and train on exactly 2,999,943,168 FineWeb\-Edu tokens\(Penedoet al\.,[2024](https://arxiv.org/html/2608.10010#bib.bib7)\)\. The model has 12 layers, width 768, intermediate width 2560, 12 attention heads, and four key/value heads\. We train for 30,517 updates at sequence length 1024 and global batch size 96 using AdamW, a 1,000\-step warmup, and cosine decay from6×10−46\\times 10^\{\-4\}to6×10−56\\times 10^\{\-5\}\. Within each triplet, all modes share the seed, token order, optimizer, and schedule\. FP8 uses E4M3 forward operands and E5M2 output gradients with dynamic FP32 scales;CurveFP8uses E4C3 forward operands and E5C2 gradients with compact power\-of\-two scales\. Parameters and Adam states remain FP32 in both quantized lanes\. Training disables K/V caching; the K/V projection linears use the lane’s forward operand format\. Final evaluation covers all 4,193,280 held\-out target tokens per lane\.

We evaluate every checkpoint in two ways\. First, each model runs in the arithmetic used during training: BF16, FP8, orCurveFP8\. This is the deployment result\. Second, we run the same learned weights in BF16 for all three models\. This common\-arithmetic comparison reveals which training mode learned the better checkpoint\. Within each row of Table[4](https://arxiv.org/html/2608.10010#S4.T4), the gap between these two evaluations is the inference quantization loss\.

![Refer to caption](https://arxiv.org/html/2608.10010v1/x3.png)Figure 3:CurveFP8and FP8 track BF16 across three seeds\.Both panels run all checkpoints in BF16, removing inference quantization to compare learned weights\. Lines show three\-seed means at 30 matched checkpoints; bands span the seed\-wise minimum and maximum\. Right: each quantized mode after subtracting its seed\-matched BF16 checkpoint, in milli\-NLL\. Markers are staggered only to reveal nearly coincident means\.Table 4:Full held\-out evaluation after matched 3B\-token pretraining\. Values are mean±\\pmsample standard deviation across three seeds\.The low\-precision endpoints are statistically unresolved at this scale\.CurveFP8\-minus\-FP8 perplexity is−0\.0110±0\.0217\-0\.0110\\pm 0\.0217in training arithmetic and−0\.0040±0\.0206\-0\.0040\\pm 0\.0206under BF16 inference;CurveFP8wins two of three seeds in both views\. The format\-induced penalty is more consistent: FP8 loses0\.0758±0\.00290\.0758\\pm 0\.0029PPL andCurveFP8loses0\.0688±0\.00110\.0688\\pm 0\.0011, soCurveFP8reduces the penalty in all three seeds by0\.0070±0\.00290\.0070\\pm 0\.0029PPL\. Figure[3](https://arxiv.org/html/2608.10010#S4.F3)likewise shows overlapping trajectories; the mean paired step\-loss difference is−0\.00030±0\.00153\-0\.00030\\pm 0\.00153across seeds\. All nine lanes complete without divergence or a non\-finite update\. These results establish full\-run FP8\-class training parity, not statistical superiority\. This is the intended criterion: preserve FP8 learned quality while replacing its product algebra with exact phase/bin routing; the consistently lower format\-induced penalty is an additional gain\.

### 4\.6CurveFP8 transfers across seeds and runtime formats

We next test whether the pretraining outcome transfers beyond the held\-out FineWeb\-Edu distribution\. OOD language modeling uses the complete WikiText\-103 and PG\-19 test sets\(Merityet al\.,[2016](https://arxiv.org/html/2608.10010#bib.bib19); Raeet al\.,[2019](https://arxiv.org/html/2608.10010#bib.bib20)\)\. The zero\-shot suite covers HellaSwag, PIQA, ARC\-Easy/Challenge, WinoGrande, LAMBADA, and BLiMP\(Zellerset al\.,[2019](https://arxiv.org/html/2608.10010#bib.bib21); Bisket al\.,[2020](https://arxiv.org/html/2608.10010#bib.bib22); Clarket al\.,[2018](https://arxiv.org/html/2608.10010#bib.bib23); Sakaguchiet al\.,[2019](https://arxiv.org/html/2608.10010#bib.bib24); Papernoet al\.,[2016](https://arxiv.org/html/2608.10010#bib.bib25); Warstadtet al\.,[2020](https://arxiv.org/html/2608.10010#bib.bib26)\); the five\-shot suite repeats all but ARC\-Challenge and BLiMP\. We report normalized accuracy for HellaSwag, PIQA, and ARC, standard accuracy for WinoGrande and LAMBADA, and the aggregate BLiMP minimal\-pair accuracy\. Every cell uses lm\-evaluation\-harness 0\.4\.11, the same tokenizer and 1,024\-token context, and 10,000 bootstrap iterations\. Together the suite probes commonsense and physical reasoning, science knowledge, discourse prediction, and grammatical competence\.

Table 5:Final checkpoints under BF16 inference\. Values are absolute mean±\\pmsample standard deviation across three seeds\. Zero\-shot is the six\-task core mean; five\-shot is the five\-task mean\. Task values are percentages\. Bold is best\.Table[5](https://arxiv.org/html/2608.10010#S4.T5)reports the absolute checkpoint quality:CurveFP8has the best mean perplexity on both OOD corpora, BF16 leads the aggregate zero\- and five\-shot means, andCurveFP8leads BLiMP\. The paired result is strongest on WikiText\-103:CurveFP8lowers its perplexity in all 12 seed\-format comparisons\. PG\-19 favorsCurveFP8in 8/12 comparisons, but its seed variance is larger than the mean gain\. Zero\- and five\-shot means favorCurveFP8in 7/12 and 6/12 comparisons, respectively, and their per\-format means remain within0\.240\.24points of zero\. The largest single\-seed conservative combined\-error score is\|z\|=1\.43\|z\|=1\.43\. We therefore conclude consistent WikiText improvement and robust transfer parity, not broad downstream superiority\.

![Refer to caption](https://arxiv.org/html/2608.10010v1/x4.png)Figure 4:Matching training and runtime arithmetic reduces mean format\-transfer penalty\.Cells are three\-seed means relative to BF16 inference on the same checkpoint\. OOD values average WikiText\-103 and PG\-19 log\-PPL changes in milli\-NLL; task values are accuracy\-point changes\. Matched FP8 andCurveFP8incur 3\.48 and 4\.01 milli\-NLL, versus 6\.94 and 7\.02 when applied after BF16 training\. TheCurveFP8\-trained checkpoint transfers toCurveFP7\-G256 at \+4\.27 milli\-NLL and \+0\.01 five\-shot points\.The full matrix separates checkpoint quality from evaluation\-format sensitivity\. Switching a BF16\-trained checkpoint to FP8,CurveFP8, orCurveFP7costs 6\.94–7\.40 milli\-NLL on OOD text\. Matching training and runtime arithmetic cuts the FP8 andCurveFP8penalties by about 50% and 43%, respectively\. Compact deployment remains stable:CurveFP7\-G256 changes theCurveFP8\-trained zero\- and five\-shot means by−0\.16\-0\.16and\+0\.01\+0\.01points\. Across trained\-checkpoint comparisons, the persistent WikiText\-103 gain coexists with seed\-dependent PG\-19 and task outcomes\. This is the expected signature of transfer parity with one consistent OOD advantage, rather than an unsupported universal gain\.

## 5Algorithmic Contribution and Arithmetic Realization

The contribution is complete at the algorithmic level: a closed\-product datatype/arithmetic family and end\-to\-end ML validation\. Its phase/bin equations constructively specify per\-lane index adders, grouping, signed counters, and eight\-bank \(E3C3/E4C3\) or four\-bank \(E5C2\) reduction; histogram and Kulisch layouts realize the same algorithm\. This establishes data\-dependent multiplier elimination and an architecture\-neutral interface; the algorithmic result is independent of process\-specific area, latency, and energy\.

Three properties close the algorithmic loop\. First, the element fields compose exactly: sign XOR, curve\-index addition, carry, exponent addition, and binary scale transport determine every nonzero product without a lookup or projection\. Second, Equation[4](https://arxiv.org/html/2608.10010#S3.E4)converts radix choice into an explicit accumulation contract\. A product is routed to one ofHHphases and one binary bin, exact counter width follows from dot length, and only the fixed phase combination and final requantization round\. Third, the role\-specific formats cover the complete linear\-layer lifecycle: E4C3 for forward weights and activations, E5C2 for output gradients, and E3C3 for compact inference\. The same mapping therefore serves forward, activation\-gradient, and weight\-gradient GEMMs rather than a single inference operator\.

This contract admits multiple implementations without changing numerical semantics\. A banked histogram merges equal\(h,t\)\(h,t\)addresses through signed popcounts; a per\-phase Kulisch design instead shifts each unit update into a wide accumulator\. Both produce the same phase sums before fixed\-coefficient reduction\. Bank organization, lane count, and pipeline depth are architecture choices, but the operations, state bounds, and output semantics are fixed by the datatype\. The contribution is thus not a proposal contingent on a particular accelerator: it is an executable arithmetic specification whose model\-level behavior is validated across inference, training, and format transfer\.

## 6Conclusion

CurveFPdemonstrates that low\-precision values and their product algebra can be designed together\. Its interleaved logarithmic curves give exact index\-domain products, the lawH=q​K/gcd⁡\(p,q​K\)H=qK/\\gcd\(p,qK\)exposes accumulation complexity, and binary block scales extend range without adding phases\. The resulting E3C3/E4C3 and E5C2 formats use eight and four phases, respectively, and map constructively to index adders, signed accumulation, and fixed phase reduction\. This structure delivers competitive quality across the model lifecycle\.CurveFP7reaches 7\.008/7\.010 W/A bits per value at G512, beats tensorwise FP8 on four 7B–9B models, and crosses INT8 on Falcon\-H1 at G512 before widening its advantage under rowwise and tensorwise sharing\.CurveFP8wins all 36 paired forward/backward GEMM\-error comparisons and completes nine 128\.3M\-parameter, 3B\-token training lanes\. Its three\-seed training\-format PPL is 22\.6054 versus 22\.6164 for FP8, and its format penalty is lower in all three seeds \(0\.0688 versus 0\.0758 PPL\)\. The learned checkpoints improve WikiText\-103 in all 12 seed\-format comparisons; mixed PG\-19 and task means appropriately support transfer parity rather than universal superiority\. Together, the exact product law, finite phase schedule, architecture\-neutral realization, and end\-to\-end evidence complete the algorithmic case forCurveFP\. The family provides one coherent FP8\-class training path and a seven\-bit INT8 alternative while making product structure, scale metadata, and accumulation cost explicit design parameters\.

## Reproducibility Statement

Section[3](https://arxiv.org/html/2608.10010#S3)defines the formats, quantization rule, product mapping, and phase count\. Section[4](https://arxiv.org/html/2608.10010#S4)reports the model, data, optimizer, task, and precision contracts used in each principal experiment\. The supplementary code fixes the seed and data contracts, implements resumable checkpointing and atomic artifact generation, validates every matrix cell, and regenerates result summaries and figures from produced artifacts\. The constructive arithmetic realization is summarized in Section[5](https://arxiv.org/html/2608.10010#S5)\.

## AI Use Statement

We used generative AI tools to assist with experiment scripts, derivation checking, figure preparation, literature review, and language editing\.

## References

- S\. Ashkboos, A\. Mohtashami, M\. L\. Croci, B\. Li, P\. Cameron, M\. Jaggi, D\. Alistarh, and T\. Hoefler \(2024\)QuaRot: outlier\-free 4\-bit inference in rotated LLMs\.arXiv preprint arXiv:2404\.00456\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2404.00456),[Link](https://arxiv.org/abs/2404.00456)Cited by:[§1](https://arxiv.org/html/2608.10010#S1.p1.1),[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px3.p1.1)\.
- S\. Biderman, H\. Schoelkopf, Q\. Anthony, H\. Bradley, K\. O’Brien, E\. Hallahan, M\. A\. Khan, S\. Purohit, U\. S\. Prashanth, E\. Raff, A\. Skowron, L\. Sutawika, and O\. van der Wal \(2023\)Pythia: a suite for analyzing large language models across training and scaling\.arXiv preprint arXiv:2304\.01373\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2304.01373),[Link](https://arxiv.org/abs/2304.01373)Cited by:[§4\.4](https://arxiv.org/html/2608.10010#S4.SS4.p1.3)\.
- PIQA: reasoning about physical commonsense in natural language\.InProceedings of the AAAI Conference on Artificial Intelligence,External Links:[Link](https://arxiv.org/abs/1911.11641)Cited by:[§4\.6](https://arxiv.org/html/2608.10010#S4.SS6.p1.1)\.
- M\. Christ, F\. de Dinechin, and F\. Pétrot \(2022\)Low\-precision logarithmic arithmetic for neural network accelerators\.In2022 IEEE 33rd International Conference on Application\-Specific Systems, Architectures and Processors,pp\. 72–79\.External Links:[Document](https://dx.doi.org/10.1109/ASAP54787.2022.00021),[Link](https://doi.org/10.1109/ASAP54787.2022.00021)Cited by:[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px2.p1.2)\.
- P\. Clark, I\. Cowhey, O\. Etzioni, T\. Khot, A\. Sabharwal, C\. Schoenick, and O\. Tafjord \(2018\)Think you have solved question answering? try ARC, the AI2 reasoning challenge\.arXiv preprint arXiv:1803\.05457\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1803.05457),[Link](https://arxiv.org/abs/1803.05457)Cited by:[§4\.6](https://arxiv.org/html/2608.10010#S4.SS6.p1.1)\.
- M\. Drumond, T\. Lin, M\. Jaggi, and B\. Falsafi \(2018\)Training DNNs with hybrid block floating point\.InAdvances in Neural Information Processing Systems,Vol\.31\.External Links:[Link](https://arxiv.org/abs/1804.01526)Cited by:[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px1.p1.1)\.
- R\. Elangovan, C\. Sakr, A\. Raghunathan, and B\. Khailany \(2025\)LO\-BCQ: block clustered quantization for 4\-bit \(W4A4\) LLM inference\.Transactions on Machine Learning Research\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2502.05376),[Link](https://arxiv.org/abs/2502.05376)Cited by:[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px3.p1.1)\.
- A\. Grattafioriet al\.\(2024\)The Llama 3 herd of models\.arXiv preprint arXiv:2407\.21783\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2407.21783),[Link](https://arxiv.org/abs/2407.21783)Cited by:[§4\.2](https://arxiv.org/html/2608.10010#S4.SS2.p1.1),[§4\.4](https://arxiv.org/html/2608.10010#S4.SS4.p1.3)\.
- I\. Kouretas and V\. Paliouras \(2018\)Logarithmic number system for deep learning\.In2018 7th International Conference on Modern Circuits and Systems Technologies,pp\. 1–4\.External Links:[Document](https://dx.doi.org/10.1109/MOCAST.2018.8376572),[Link](https://doi.org/10.1109/MOCAST.2018.8376572)Cited by:[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px2.p1.2)\.
- Y\. Li, X\. Dong, and W\. Wang \(2020\)Additive powers\-of\-two quantization: an efficient non\-uniform discretization for neural networks\.InInternational Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/1909.13144)Cited by:[§1](https://arxiv.org/html/2608.10010#S1.p2.1),[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px2.p1.2)\.
- Z\. Liu, C\. Zhao, I\. Fedorov, B\. Soran, D\. Choudhary, R\. Krishnamoorthi, V\. Chandra, Y\. Tian, and T\. Blankevoort \(2025\)SpinQuant: LLM quantization with learned rotations\.InInternational Conference on Learning Representations,External Links:[Link](https://arxiv.org/abs/2405.16406)Cited by:[§1](https://arxiv.org/html/2608.10010#S1.p1.1),[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px3.p1.1)\.
- S\. Merity, C\. Xiong, J\. Bradbury, and R\. Socher \(2016\)Pointer sentinel mixture models\.arXiv preprint arXiv:1609\.07843\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1609.07843),[Link](https://arxiv.org/abs/1609.07843)Cited by:[§4\.2](https://arxiv.org/html/2608.10010#S4.SS2.p1.1),[§4\.6](https://arxiv.org/html/2608.10010#S4.SS6.p1.1)\.
- P\. Micikevicius, D\. Stosic, N\. Burgess, M\. Cornea, P\. Dubey, R\. Grisenthwaite, S\. Ha, A\. Heinecke, P\. Judd, J\. Kamalu, N\. Mellempudi, S\. Oberman, M\. Shoeybi, M\. Siu, and H\. Wu \(2022\)FP8 formats for deep learning\.arXiv preprint arXiv:2209\.05433\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2209.05433),[Link](https://arxiv.org/abs/2209.05433)Cited by:[§1](https://arxiv.org/html/2608.10010#S1.p1.1),[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2608.10010#S4.SS1.p1.1)\.
- D\. Miyashita, E\. H\. Lee, and B\. Murmann \(2016\)Convolutional neural networks using logarithmic data representation\.arXiv preprint arXiv:1603\.01025\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1603.01025),[Link](https://arxiv.org/abs/1603.01025)Cited by:[§1](https://arxiv.org/html/2608.10010#S1.p2.1),[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px2.p1.2)\.
- D\. Paperno, G\. Kruszewski, A\. Lazaridou, N\. Pham, R\. Bernardi, S\. Pezzelle, M\. Baroni, G\. Boleda, and R\. Fernández \(2016\)The LAMBADA dataset: word prediction requiring a broad discourse context\.InProceedings of the 54th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1525–1534\.External Links:[Document](https://dx.doi.org/10.18653/v1/P16-1144),[Link](https://aclanthology.org/P16-1144/)Cited by:[§4\.6](https://arxiv.org/html/2608.10010#S4.SS6.p1.1)\.
- G\. Penedo, H\. Kydlíček, L\. Ben Allal, A\. Lozhkov, M\. Mitchell, C\. Raffel, L\. von Werra, and T\. Wolf \(2024\)The FineWeb datasets: decanting the web for the finest text data at scale\.arXiv preprint arXiv:2406\.17557\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2406.17557),[Link](https://arxiv.org/abs/2406.17557)Cited by:[§4\.5](https://arxiv.org/html/2608.10010#S4.SS5.p1.2)\.
- Y\. Qiao, Z\. Chen, Y\. Zhang, Y\. Wang, and S\. Huang \(2026a\)TeLLMe: an efficient end\-to\-end ternary LLM prefill and decode accelerator with table\-lookup matmul on edge FPGAs\.InProceedings of the 2026 ACM/SIGDA International Symposium on Field\-Programmable Gate Arrays,pp\. 247–257\.External Links:[Document](https://dx.doi.org/10.1145/3748173.3779191),[Link](https://doi.org/10.1145/3748173.3779191)Cited by:[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px1.p1.1)\.
- Y\. Qiao, Y\. Wang, Z\. Chen, H\. Kwon, and S\. Huang \(2026b\)FASQ: flexible accelerated subspace quantization for calibration\-free LLM compression\.arXiv preprint arXiv:2605\.04084\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2605.04084),[Link](https://arxiv.org/abs/2605.04084)Cited by:[§1](https://arxiv.org/html/2608.10010#S1.p2.1),[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px3.p1.1)\.
- Qwen Team \(2026\)Qwen3\.5: towards native multimodal agents\.External Links:[Link](https://qwen.ai/blog?id=qwen3.5)Cited by:[§4\.2](https://arxiv.org/html/2608.10010#S4.SS2.p1.1)\.
- J\. W\. Rae, A\. Potapenko, S\. M\. Jayakumar, and T\. P\. Lillicrap \(2019\)Compressive transformers for long\-range sequence modelling\.arXiv preprint arXiv:1911\.05507\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1911.05507),[Link](https://arxiv.org/abs/1911.05507)Cited by:[§4\.6](https://arxiv.org/html/2608.10010#S4.SS6.p1.1)\.
- B\. D\. Rouhani, R\. Zhao, A\. More,et al\.\(2023\)Microscaling data formats for deep learning\.arXiv preprint arXiv:2310\.10537\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2310.10537),[Link](https://arxiv.org/abs/2310.10537)Cited by:[§1](https://arxiv.org/html/2608.10010#S1.p1.1),[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px1.p1.1)\.
- K\. Sakaguchi, R\. Le Bras, C\. Bhagavatula, and Y\. Choi \(2019\)WinoGrande: an adversarial winograd schema challenge at scale\.arXiv preprint arXiv:1907\.10641\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1907.10641),[Link](https://arxiv.org/abs/1907.10641)Cited by:[§4\.6](https://arxiv.org/html/2608.10010#S4.SS6.p1.1)\.
- S\. Savkin, E\. Porat, O\. Ordentlich, and Y\. Polyanskiy \(2025\)NestQuant: nested lattice quantization for matrix products and LLMs\.InProceedings of the 42nd International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.267,pp\. 53042–53062\.External Links:[Link](https://proceedings.mlr.press/v267/savkin25a.html)Cited by:[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px3.p1.1)\.
- Y\. Wang, Y\. Qiao, S\. Huang, and H\. Kwon \(2026\)APEX\-Q: arbitrary\-dimension product\-extension quantization for accelerated LLM deployment \(student abstract\)\.Proceedings of the AAAI Conference on Artificial Intelligence40\(48\),pp\. 41424–41426\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v40i48.42293),[Link](https://doi.org/10.1609/aaai.v40i48.42293)Cited by:[§1](https://arxiv.org/html/2608.10010#S1.p2.1),[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px3.p1.1)\.
- A\. Warstadt, A\. Parrish, H\. Liu, A\. Mohananey, W\. Peng, S\. Wang, and S\. R\. Bowman \(2020\)BLiMP: the benchmark of linguistic minimal pairs for english\.Transactions of the Association for Computational Linguistics8,pp\. 377–392\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00321),[Link](https://doi.org/10.1162/tacl_a_00321)Cited by:[§4\.6](https://arxiv.org/html/2608.10010#S4.SS6.p1.1)\.
- G\. Xiao, J\. Lin, M\. Seznec, H\. Wu, J\. Demouth, and S\. Han \(2023\)SmoothQuant: accurate and efficient post\-training quantization for large language models\.InProceedings of the 40th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.202,pp\. 38087–38099\.External Links:[Link](https://proceedings.mlr.press/v202/xiao23c.html)Cited by:[§1](https://arxiv.org/html/2608.10010#S1.p1.1),[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px3.p1.1)\.
- A\. Yanget al\.\(2025\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2505.09388),[Link](https://arxiv.org/abs/2505.09388)Cited by:[§4\.2](https://arxiv.org/html/2608.10010#S4.SS2.p1.1)\.
- R\. Zellers, A\. Holtzman, Y\. Bisk, A\. Farhadi, and Y\. Choi \(2019\)HellaSwag: can a machine really finish your sentence?\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,pp\. 4791–4800\.External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1472),[Link](https://aclanthology.org/P19-1472/)Cited by:[§4\.6](https://arxiv.org/html/2608.10010#S4.SS6.p1.1)\.
- J\. Zhao, S\. Dai, R\. Venkatesan, B\. Zimmer, M\. Ali, M\. Liu, B\. Khailany, W\. J\. Dally, and A\. Anandkumar \(2022\)LNS\-Madam: low\-precision training in logarithmic number system using multiplicative weight update\.IEEE Transactions on Computers71\(12\),pp\. 3179–3190\.External Links:[Document](https://dx.doi.org/10.1109/TC.2022.3202747),[Link](https://doi.org/10.1109/TC.2022.3202747)Cited by:[§2](https://arxiv.org/html/2608.10010#S2.SS0.SSS0.Px2.p1.2)\.
- J\. Zuoet al\.\(2025\)Falcon\-H1: a family of hybrid\-head language models redefining efficiency and performance\.arXiv preprint arXiv:2507\.22448\.External Links:[Link](https://arxiv.org/abs/2507.22448)Cited by:[§4\.2](https://arxiv.org/html/2608.10010#S4.SS2.p1.1)\.