Predicting integers from continuous parameters
Summary
Study proposes Bitwise and discrete Laplace distributions to predict integer labels directly from continuous neural-network parameters, outperforming standard regression across tabular, sequential and image tasks.
View Cached Full Text
Cached at: 04/22/26, 06:17 AM
Paper page - Predicting integers from continuous parameters
Source: https://huggingface.co/papers/2602.10751
Abstract
Research examines direct modeling of integer-labeled data using discrete probability distributions with continuous parameters suitable for neural network training, evaluating Bitwise and discrete Laplace distributions against traditional regression approaches.
We study the problem of predicting numeric labels that are constrained to the integers or to a subrange of the integers. For example, the number of up-votes on social media posts, or the number of bicycles available at a public rental station. While it is possible to model these as continuous values, and to apply traditional regression, this approach changes the underlying distribution on the labels from discrete to continuous.Discrete distributionshave certain benefits, which leads us to the question whether suchinteger labelscan be modeled directly by a discrete distribution, whose parameters are predicted from the features of a given instance. Moreover, we focus on the use case of output distributions ofneural networks, which adds the requirement that the parameters of the distribution be continuous so thatbackpropagationandgradient descentmay be used to learn the weights of the network. We investigate several options for such distributions, some existing and some novel, and test them on a range of tasks, includingtabular learning,sequential predictionandimage generation. We find that overall the best performance comes from two distributions: Bitwise, which represents the target integer in bits and places a Bernoulli distribution on each, and a discrete analogue of the Laplace distribution, which uses a distribution with exponentially decaying tails around a continuous mean.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2602\.10751
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2602.10751 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2602.10751 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2602.10751 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Error Aware Distribution Prediction for Lightweight Implicit Neural Representations
The paper proposes a lightweight method that reformulates regression-based INR training as a classification task by discretizing continuous targets into bins, enabling flexible distribution modeling for error-aware uncertainty estimation in scientific data compression.
Towards Closing the Autoregressive Gap in Language Modeling via Entropy-Gated Continuous Bitstream Diffusion
This paper introduces a diffusion language model that treats text as a continuous process over binary bitstreams, using entropy-gated stochastic sampling to close the performance gap with autoregressive models. It achieves state-of-the-art results on LM1B and OWT benchmarks while reducing memory footprint.
Text-to-Distribution Prediction with Quantile Tokens and Neighbor Context
Amazon and Stanford researchers propose Quantile Token Regression, inserting dedicated quantile tokens into LLM inputs to predict full probability distributions, achieving ~4 point MAPE reduction and 2× narrower intervals on Airbnb and Stack Overflow benchmarks.
BitLM: Unlocking Multi-Token Language Generation with Bitwise Continuous Diffusion
This paper introduces BitLM, a language model that uses bitwise continuous diffusion to generate multiple tokens in parallel, aiming to overcome the sequential bottleneck of traditional autoregressive generation while preserving causal structure.
Variable Bit-width Quantization: Learning Per-Group Precision for "Bigger-but-Smaller" Language Models
Introduces Variable Bit-width Quantization (VBQ), a training-time method where each group of 64 weights learns its own bit-width (1,2,4,8) via Gumbel-Softmax relaxation. VBQ discovers a heterogeneous allocation that yields a 'bigger-but-smaller' regime, e.g., a 131M parameter model at 1.82 mean bits beats a 55M FP16 model while using less storage, and a 1.46B model matches a 593M FP16 with ~3.7x less storage.