Tag
该论文系统研究了文本分类器在不同服务上下文（批量大小、数值精度、硬件等）下的非不变性，发现标签稳定并不意味着分数稳定，且在 bf16 下仅改变批量形状就能导致高达 56.7 个百分点的预测概率质量偏移，并为每种机制推导了标签稳定性的充分条件和缓解方法。
This paper investigates the losslessness of Orthrus, a hybrid autoregressive-diffusion model for inference acceleration, finding that it requires high numerical precision (FP32) for exact trajectory matching, while BF16 divergence does not impair downstream performance.
This paper identifies a numerical underflow failure mode in ALiBi positional encodings that can zero out attention weights, analyzes its impact, and evaluates mitigation strategies for training models with ALiBi.
This paper formalizes the numerical divergence between training and inference kernels in modern AI post-training pipelines, proposing a kernel contract specification and a chain of Lipschitz-style bounds to mitigate off-policy bias, slice-level regressions, and reproducibility issues.
The IEEE P3109 draft standard defines a parameterized family of binary floating-point formats and operations tailored for machine learning, supporting configurable width, precision, signedness, and infinities, along with extensive rounding modes including stochastic rounding and a novel scale-invariant approximation measure called kappa-approximation.