HyperZip: Efficient Data Compression through Personalized Diffusion LLMs with Hypernetworks

# HyperZip: Efficient Data Compression through Personalized Diffusion LLMs with Hypernetworks.
Source: [https://arxiv.org/html/2609.36357](https://arxiv.org/html/2609.36357)
Khang TranNhatHai PhanAffiliation:New Jersey Institute of TechnologyEmail:[\{tqn,kt36,phan\}@njit\.edu](mailto:)

###### Abstract

Large language models \(LLMs\) have shown strong potential for lossless data compression, but existing approaches are constrained by the high computational cost and low throughput of autoregressive decoding\. We proposeHyperZip, an efficient and scalable LLM\-based compression framework that leverages diffusion\-based LLMs \(dLLMs\) with Multi\-Token Prediction \(MTP\) to accelerate LLM\-based data compression processes\. We identify a trade\-off in diffusion\-based compression, where increasing decoding throughput degrades the compression rate\. To mitigate this trade\-off,HyperZipemploys a hypernetwork to generate data\-specific updates from a context representation, adapting the dLLM to the target data without costly fine\-tuning, resulting in a low compression rate and high throughput\. Extensive experiments show thatHyperZipachieves a superior trade\-off between compression rate and speed compared with state\-of\-the\-art baselines\.

## 1Introduction

Motivation\.The advances in LLMs have been adopted in many real\-world applications across multiple domains[Raiaan et al\. \(2024\)](https://arxiv.org/html/2609.36357#bib.bib2);[Hagos et al\. \(2024\)](https://arxiv.org/html/2609.36357#bib.bib4);[Chkirbene et al\. \(2024\)](https://arxiv.org/html/2609.36357#bib.bib5);[Naveed et al\. \(2025\)](https://arxiv.org/html/2609.36357#bib.bib3), including data management and storage[Miao et al\. \(2024\)](https://arxiv.org/html/2609.36357#bib.bib6);[Zhang et al\. \(2024b\)](https://arxiv.org/html/2609.36357#bib.bib7);[Bagui et al\. \(2026\)](https://arxiv.org/html/2609.36357#bib.bib8);[Tummuri \(2025\)](https://arxiv.org/html/2609.36357#bib.bib9)\. Recent works show that LLM\-based data compressors can achieve better compression rates than traditional methods such as gzip, resulting in more efficient data storage[Delétang et al\. \(2024\)](https://arxiv.org/html/2609.36357#bib.bib24);[Li et al\. \(2025c\)](https://arxiv.org/html/2609.36357#bib.bib27);[Chen et al\. \(2026b\)](https://arxiv.org/html/2609.36357#bib.bib10);[Pan et al\. \(2025\)](https://arxiv.org/html/2609.36357#bib.bib33)\. However, existing LLM\-based data compressors are limited by the resources required to deploy an LLM and its autoregressive decoding generative mechanism, which requires high\-performance servers to execute the compressing and decompressing tasks[Kipf et al\. \(2026\)](https://arxiv.org/html/2609.36357#bib.bib11);[Haddadi and Teahan \(2026\)](https://arxiv.org/html/2609.36357#bib.bib12)\. This degrades compression and decompression speed and limits adoption in real\-world use, where users only have limited computing resources\. Therefore, it’s critical to develop a scalable, efficient LLM\-based data compressor that compresses quickly while maintaining a low compression rate\.

Challenges\.Designing such a mechanism is very challenging\. Because existing LLM\-based data compressors use LLMs as the probabilistic model for lossless arithmetic encoding, they face the following challenges\.\(1\)The typical generation process of LLMs follows the autoregressive generation pipeline, which only generates the logit for a single token in one forward pass, limiting the compression speed of large files with millions of tokens[Kipf et al\. \(2026\)](https://arxiv.org/html/2609.36357#bib.bib11)\.\(2\)To achieve a lower compression rate, the LLMs used for compression must exhibit low perplexity on the data being compressed, thereby assigning higher confidence to the predicted logits of subsequent tokens\. Achieving such low perplexity typically requires either complex models with billions of parameters or an additional fine\-tuning process tailored to the target data[Zhang et al\. \(2025a\)](https://arxiv.org/html/2609.36357#bib.bib14)\.

Our solution\.In this work, we proposeHyperZip, a novel, efficient, and scalable framework of LLM\-based data compressors\.HyperZipaddresses the first challenge by replacing autoregressive LLMs with diffusion\-based LLMs \(dLLMs\) that can perform Multi\-Token Prediction \(MTP\), effectively increasing compression speed\. Our preliminary results \(details in Appendix[B](https://arxiv.org/html/2609.36357#A2)\) show that diffusion\-based MTP can significantly improve compression speed while maintaining a low compression rate, outperforming other existing MTP mechanisms[Gloeckle et al\. \(2024\)](https://arxiv.org/html/2609.36357#bib.bib15);[Hu et al\. \(2025\)](https://arxiv.org/html/2609.36357#bib.bib16)\. However, our results also reveal a trade\-off between compression ratio and speed when using diffusion\-based MTP for data compression\. Specifically, we can achieve faster compression by lowering the confidence threshold for diffusion decoding, which increases perplexity on the compressed data and yields a higher compression rate\. This behavior reflects the throughput\-performance trade\-off inherent in diffusion\-based LLMs[Fu et al\. \(2025\)](https://arxiv.org/html/2609.36357#bib.bib17)\. As with the second challenge, more complex, generalized models or an additional fine\-tuning step can address this, but they require more computational resources\.

To address this issue, the core ofHyperZipemploys a hypernetwork pipeline[Chauhan et al\. \(2024\)](https://arxiv.org/html/2609.36357#bib.bib18);[Tan et al\. \(2026\)](https://arxiv.org/html/2609.36357#bib.bib19)to generate a LoRA update[Hu et al\. \(2021\)](https://arxiv.org/html/2609.36357#bib.bib20)for the dLLMs given a context vector representing the data to be compressed, adapting the model’s weights to the data\. This adaptation improves the model’s predictive confidence and reduces the compression rate without requiring costly additional fine\-tuning\. At inference time, users can compress the context vector along with the target data, which the model then uses to update during decompression\. This pipeline letsHyperZippersonalize dLLMs to the data without expensive fine\-tuning while maintaining low compression rates\. As a result,HyperZipachieves the best trade\-off between compression rate and speed, facilitating practical adoption\.

Contribution\.Our contribution is as follows:\(1\)We proposeHyperZip, an efficient and scalable framework for LLM\-based data compression\.HyperZipreplaces conventional LLMs with dLLMs that can perform multi\-token prediction, enabling faster compression for real\-world adoption\.\(2\)To address the trade\-off between compression rate and speed of dLLMs, we design a Hypernetwork structure to personalize diffusion models, aligning them to the data, desired compression speed, and latency\. This makes LLM\-based data compression practical for real\-world use\.\(3\)We conduct extensive experiments to evaluateHyperZip, and the results show thatHyperZipachieves the best trade\-off between compression rate and speed compared with state\-of\-the\-art baselines\.

## 2Background & Preliminaries

Arithmetic Coding\.Given a data file which contain a sequenceX=\[x1,x2,…,xn\]∈VnX=\[x\_\{1\},x\_\{2\},\.\.\.,x\_\{n\}\]\\in V^\{n\}of lengthnnfrom a finite set of vocabularyVV, arithmetic coding[Pasco \(1977\)](https://arxiv.org/html/2609.36357#bib.bib22);[Rissanen \(1976\)](https://arxiv.org/html/2609.36357#bib.bib23)encodes a sequencex1:nx\_\{1:n\}into a floating number using a probability modelρ\\rho\. It iteratively narrows an intervalIk,k∈\[n\]I\_\{k\},k\\in\[n\], starting fromI0=\[0,1\)I\_\{0\}=\[0,1\), by partitioning the interval into sub\-intervals proportional toρ\(xk\|x1:k−1\)\\rho\(x\_\{k\}\|x\_\{1:k\-1\}\), and keepingIIas the one corresponding to the interval associated withxkx\_\{k\}\. Afternnsteps, the final intervalInI\_\{n\}uniquely identifiesx1:nx\_\{1:n\}, and a floating number picked fromInI\_\{n\}is the compressed file\.

Neural compression\.Neural compression[Yang et al\. \(2023\)](https://arxiv.org/html/2609.36357#bib.bib35)employs an autoregressive language model as an approximation for the probability modelρ\\rhoin arithmetic coding[Delétang et al\. \(2024\)](https://arxiv.org/html/2609.36357#bib.bib24);[Li et al\. \(2025c\)](https://arxiv.org/html/2609.36357#bib.bib27);[Mittu et al\. \(2024\)](https://arxiv.org/html/2609.36357#bib.bib29);[Heurtel\-Depeiges et al\. \(2024\)](https://arxiv.org/html/2609.36357#bib.bib30);[Zhang et al\. \(2025c\)](https://arxiv.org/html/2609.36357#bib.bib32);[Pan et al\. \(2025\)](https://arxiv.org/html/2609.36357#bib.bib33);[Bellard \(2021\)](https://arxiv.org/html/2609.36357#bib.bib58)\. At each encoding iterationk∈\[n\]k\\in\[n\], the arithmetic encoder replaces the probability modelρ\(xk\|x1:k−1\)\\rho\(x\_\{k\}\|x\_\{1:k\-1\}\)with an LLM’s logitqθ\(xk\|x1:k−1\)q\_\{\\theta\}\(x\_\{k\}\|x\_\{1:k\-1\}\), whereθ\\thetais the LLM’s parameters\. These approaches have outperformed strong classical compressors, such as Gzip[Deutsch \(1996\)](https://arxiv.org/html/2609.36357#bib.bib34), because LLMs generalize well and can accurately approximate a probability modelρ\\rhoof the target file\. In addition, these methods are also applied efficiently in other domains, such as image and audio[Delétang et al\. \(2024\)](https://arxiv.org/html/2609.36357#bib.bib24);[Zhao et al\. \(2026\)](https://arxiv.org/html/2609.36357#bib.bib26);[Chen et al\. \(2025\)](https://arxiv.org/html/2609.36357#bib.bib25);[Li et al\. \(2025c\)](https://arxiv.org/html/2609.36357#bib.bib27)\.

Multi\-token prediction with dLLMs[Austin et al\. \(2021\)](https://arxiv.org/html/2609.36357#bib.bib36);[Hoogeboom et al\. \(2021\)](https://arxiv.org/html/2609.36357#bib.bib37);[Shi et al\. \(2024\)](https://arxiv.org/html/2609.36357#bib.bib60);[Ren et al\. \(2026\)](https://arxiv.org/html/2609.36357#bib.bib59);[Nie et al\. \(2025\)](https://arxiv.org/html/2609.36357#bib.bib61)\. To generate a sequenceX=\[x1,…,xn\]X=\[x\_\{1\},\\dots,x\_\{n\}\]of lengthnn, a dLLMfθf\_\{\\theta\}starts by a masked sequenceX0=\[mask,…,mask\]X\_\{0\}=\[\\texttt\{mask\},\\dots,\\texttt\{mask\}\]\. Then, it iteratively unmasks the tokens in the sequence until all are unmasked\. Specifically, at each iterationkk,XkX\_\{k\}is updated by forwardingXk−1X\_\{k\-1\}throughfθf\_\{\\theta\}, and replacing the masked tokens with the tokens that achieve logits higher than a threshold confidence hyperparameterγ\\gamma, which can be multiple tokens at a time\. If the logits at the masked tokens are not higher thanγ\\gamma, they remain for the next iteration\. This inference pipeline achieves faster decoding speed than autoregressive LLM decoding[Cai and Li \(2026\)](https://arxiv.org/html/2609.36357#bib.bib66);[Fu et al\. \(2026b\)](https://arxiv.org/html/2609.36357#bib.bib62);[Liu et al\. \(2025b\)](https://arxiv.org/html/2609.36357#bib.bib63);[Li et al\. \(2025a\)](https://arxiv.org/html/2609.36357#bib.bib64);[Mohamed et al\. \(2026\)](https://arxiv.org/html/2609.36357#bib.bib65)\.

Hypernetwork\.A hypernetworkhϕh\_\{\\phi\}is a neural network trained to generate the parametersθ\\thetaof a separate target networkfθf\_\{\\theta\}[Ha et al\. \(2017\)](https://arxiv.org/html/2609.36357#bib.bib21)\. Formally, given an input or contextzz\(e\.g\., a task descriptor, layer index, or conditioning signal\), the hypernetwork producesθ=hϕ​\(z\)\\theta=h\_\{\\phi\}\(z\), and onlyϕ\\phiis updated during training, whileθ\\thetais instantiated on the fly for eachzz[Ivison and Peters \(2022\)](https://arxiv.org/html/2609.36357#bib.bib56)\. Recent work attempt to shrink the hypernetwork’s output to a low\-dimensional parameter subspace rather than the full weight tensor[Charakorn et al\. \(2025a\)](https://arxiv.org/html/2609.36357#bib.bib40);[Charakorn et al\. \(2026\)](https://arxiv.org/html/2609.36357#bib.bib41)\. In particular, hypernetworks have been used to generate low\-rank adaptation \(LoRA[Hu et al\. \(2022\)](https://arxiv.org/html/2609.36357#bib.bib39)\) updatesΔ​θ=hϕ​\(z\)=Az​Bz⊤\\Delta\\theta=h\_\{\\phi\}\(z\)=A\_\{z\}B\_\{z\}^\{\\top\}, withAz∈ℝd×rA\_\{z\}\\in\\mathbb\{R\}^\{d\\times r\},Bz∈ℝk×rB\_\{z\}\\in\\mathbb\{R\}^\{k\\times r\},r≪min⁡\(d,k\)r\\ll\\min\(d,k\), reducing the hypernetwork’s output dimensionality from𝒪⁡\(d​k\)\\mathcal\{O\}\(dk\)to𝒪⁡\(r⁡\(d\+k\)\)\\mathcal\{O\}\(r\(d\+k\)\)per layer\. More background can be found in the Appendix[A](https://arxiv.org/html/2609.36357#A1)

## 3HyperZip’s Framework

In this section, we introduceHyperZip, a novel, efficient, and scalable framework for personalized LLM\-based data compression\.

![Refer to caption](https://arxiv.org/html/2609.36357v1/images/prelim_result.png)Figure 1:Compression–throughput trade\-off across MTP mechanisms\.Trade\-off of compression rate and throughput\.We first present our preliminary results to characterize the trade\-off between compression rate and speed incurred by MTP and dLLMs\. We consider two main categories following[Zhang et al\. \(2025b\)](https://arxiv.org/html/2609.36357#bib.bib54):\(1\)AR\-Based which includesEAGLE\-3[Li et al\. \(2025b\)](https://arxiv.org/html/2609.36357#bib.bib42),L\-MTP[Liu et al\. \(2025a\)](https://arxiv.org/html/2609.36357#bib.bib44), andself\-distillation[Kirchenbauer et al\. \(2026\)](https://arxiv.org/html/2609.36357#bib.bib45); and\(2\)Non\-AR\-based which includeDFlash[Chen et al\. \(2026a\)](https://arxiv.org/html/2609.36357#bib.bib43),Nemotron\-Labs\-Diffusion[Fu et al\. \(2026a\)](https://arxiv.org/html/2609.36357#bib.bib46), andFast\-dLLM\-v2[Wu et al\. \(2025\)](https://arxiv.org/html/2609.36357#bib.bib47)\. Figure[1](https://arxiv.org/html/2609.36357#S3.F1)illustrates the compression ratio and throughput of the considered mechanisms\. In general, MTP mechanisms achieve higher throughput than autoregresisve generation\. Specifically, by decreasing the confidence thresholdγ\\gamma, diffusion\-based methods achieve the highest inference speed, but they incur a higher compression ratio\. The key reason is sampler method choose to unmask sequence with lower confidence about the current context, so it requires a longer bitstring to compress the target data file\. This behavior of dLLMs highlights a trade\-off between compression rate and throughput, which can be addressed by aligning the dLLMs with the data, increasing its confidence for each token at the decoding process\. Complete settings and results are provided in Appendix[B](https://arxiv.org/html/2609.36357#A2)\.

![Refer to caption](https://arxiv.org/html/2609.36357v1/images/Hyperzip-pipeline.png)Figure 2:HyperZip’s pipeline\.Main pipeline\.To address the trade\-off between compression ratio and throughput,HyperZipleverages a hypernetwork to generate updates for dLLMs at inference time to align them with the target data\. Figure[2](https://arxiv.org/html/2609.36357#S3.F2)illustrates this pipeline\. To compress a target file containing a sequenceX∈VnX\\in V^\{n\},HyperZipuses an embedding model to extract a context vectore=emb​\(X\)e=\\texttt\{emb\}\(X\)\. Then, the hypernetworkhϕh\_\{\\phi\}takeseeas input to generate a LoRA updateΔθ\\Delta\_\{\\theta\}for a dLLMfθf\_\{\\theta\}, personalizing it toward the target file\. Using the updated dLLM,HyperZipincurs a multi\-token prediction decoding process to get the logits of each tokenxi∈X,i∈\[n\]x\_\{i\}\\in X,i\\in\[n\], and encode it with arithmetic encoding into a bitstring representation\. This bitstring, along with the context vectoree, forms the compressed data\. Unlike previous works[Zhang et al\. \(2025a\)](https://arxiv.org/html/2609.36357#bib.bib14),HyperZipconcatenates only the context vector with the bitstring, which is much smaller than an LLM’s LoRA update, making the compressed data smaller and more efficient to transmit\. To decompress,eepasses through the hypernetworkhϕh\_\{\\phi\}to update the dLLMfθf\_\{\\theta\}\. Then,fθf\_\{\\theta\}runs an arithmetic decoding process on a masked version of the target file, revealing each token if its logit exceedsγ\\gamma\. In addition, because dLLMs are personalized to the data, multiple tokens often have high confidence, so many tokens are revealed at each decoding step\. This process can be parallelized, making decompression faster than conventional LLMs\.

Hypernetwork\.The core ofHyperZipis a hypernetwork structure\. Motivated by Text\-to\-LoRA[Charakorn et al\. \(2025b\)](https://arxiv.org/html/2609.36357#bib.bib53),HyperZipemploys an embedding modelemb​\(⋅\)\\texttt\{emb\}\(\\cdot\)to extract a context vectoreefromXX, which serves as the main input for the hypernetworkhϕh\_\{\\phi\}\. In the training process,ϕ\\phiis learned given a training datasetDDby optimizing the following objective:arg⁡min⁡∑i=1\|D\|ϕ⁡ℓ⁡\(xi,θ0\+hϕ​\(emb​\(xi\)\)\)\\arg\\min\_\{\\phi\}\\sum\_\{i=1\}^\{\|D\|\}\\ell\\Big\(x\_\{i\};\\theta\_\{0\}\+h\_\{\\phi\}\(\\texttt\{emb\}\(x\_\{i\}\)\)\\Big\), whereℓ⁡\(⋅,⋅\)\\ell\(\\cdot;\\cdot\)is the loss function given the downstream task ofDD, andθ0\\theta\_\{0\}is the pretrained parameter of the dLLM\. In addition, at inference time, the hypernetworkϕ\\phiis fixed and can be deployed on a service provider server to generate updates to the dLLM for the compression and decompression processes\.

## 4Experiments

Experimental Setting\.We conduct extensive experiments on theFineWeb\-Edudataset[Penedo et al\. \(2024\)](https://arxiv.org/html/2609.36357#bib.bib52)andenwik9dataset[Hutter \(2006\)](https://arxiv.org/html/2609.36357#bib.bib49)to analyze the performance ofHyperZip\. We compareHyperZipagainst extensive baselines including theAuto Regressive model[Yang and others \(2024\)](https://arxiv.org/html/2609.36357#bib.bib57),L3TC[Zhang et al\. \(2024a\)](https://arxiv.org/html/2609.36357#bib.bib31),Fast\-dLLM\-v2[Wu et al\. \(2025\)](https://arxiv.org/html/2609.36357#bib.bib47)andstandard supervised fine\-tuning\. Appendix[C](https://arxiv.org/html/2609.36357#A3)provides details on the baselines, evaluation metrics, and implementation\.

![Refer to caption](https://arxiv.org/html/2609.36357v1/images/main_results_legend.png)

![Refer to caption](https://arxiv.org/html/2609.36357v1/images/main_results.png)\(a\)FineWeb\-Edu
![Refer to caption](https://arxiv.org/html/2609.36357v1/images/main_results_enwiki9.png)\(b\)enwik9

Figure 3:Compression rate and throughput ofHyperZipvs considered baselines\.Compression Ratio and Throughput\.In Figure[3](https://arxiv.org/html/2609.36357#S4.F3), we compareHyperZipwith considered baselines in terms of compression ratio and decompression throughput\. Lower compression ratio and higher throughput are better, so points closer to the bottom right indicate a better trade\-off\. We evaluate compression backbones ranging from 150M to 1\.5B parameters and set the confidence thresholdγ=0\.5\\gamma=0\.5for diffusion\-based models\. Overall,HyperZipachieves a better trade\-off across the considered baselines by adapting the compression model to each\. These results demonstrateHyperZip’s efficiency and its potential for real\-world deployment\.

Table 1:Compression ratio and throughput across document topics\.ModelCR incl\. update vector\(↓\\downarrow\)Throughput\(tok/s\)\(↑\\uparrow\)ScientificLiteratureProgrammingGeneral WebFast\-dLLM16\.115\.718\.016\.3141\.0Fine\-tuning15\.915\.417\.716\.1140\.2HyperZip \(ours\)15\.615\.316\.315\.8138\.9

![Refer to caption](https://arxiv.org/html/2609.36357v1/images/generation_multiple_topics_legend.png)

![Refer to caption](https://arxiv.org/html/2609.36357v1/images/generation_multiple_topics_perplexity_5_context_embedding.png)\(a\)Context embedding
![Refer to caption](https://arxiv.org/html/2609.36357v1/images/generation_multiple_topics_perplexity_5_hyperzip_activation.png)\(b\)HyperZip output

Figure 4:t\-SNE visualization\.Generalizability\.We evaluate the generalization ofHyperZipacross different document topics, including law, scientific text, news, literature, programming, and general web text\. Table[1](https://arxiv.org/html/2609.36357#S4.T1)shows thatHyperZipconsistently improves compression over the vanilla and fine\-tuned dLLMs while maintaining similar throughput\. For example, on programming text, it achieves a compression ratio of 16\.3% compared with 18\.0% for vanilla Fast\-dLLM and 17\.7% for its fine\-tuned version\. We observe similar behavior for other topics\. Its throughput remains comparable at 138\.9 tok/s versus 141\.0 and 140\.2 tok/s\. These results indicateHyperZip’s strong generalizability, highlighting its advantage in real\-world scenarios where users have different data topics\. In addition, Figure[4](https://arxiv.org/html/2609.36357#S4.F4)shows that the context vectors form clear topic\-specific clusters, whileHyperZipoutputs exhibit greater overlap across topics, highlighting thatHyperZippreserves topic\-dependent information while mapping documents to a more shared representation\. Therefore,HyperZipcan be generalize for different topic as indicated by the results in Table[1](https://arxiv.org/html/2609.36357#S4.T1)\.

## 5Conclusion

In this work, we introducedHyperZip, an efficient and scalable framework that optimizes LLM\-based lossless compression through diffusion models while preserving compression effectiveness through document\-specific hypernetwork adaptation\. Extensive experiments show thatHyperZipachieves a better compression throughput trade\-off than state\-of\-the\-art baselines across datasets and model scales, making LLM\-based compression more practical for real\-world adoption\.

## References

- Arriolaet al\.\(2025\)M\. Arriola, A\. Gokaslan, J\. T\. Chiu, Z\. Yang, Z\. Qi, J\. Han, S\. S\. Sahoo, and V\. KuleshovBlock diffusion: interpolating between autoregressive and diffusion language models\.InInternational Conference on Learning Representations,Cited by:[§A\.2](https://arxiv.org/html/2609.36357#A1.SS2.p1.1),[§C\.1](https://arxiv.org/html/2609.36357#A3.SS1.SSS0.Px1.p1.1)\.
- Austinet al\.\(2021\)J\. Austin, D\. D\. Johnson, J\. Ho, D\. Tarlow, and R\. van den BergStructured denoising diffusion models in discrete state\-spaces\.InAdvances in Neural Information Processing Systems,Vol\.34\.Cited by:[§A\.2](https://arxiv.org/html/2609.36357#A1.SS2.p1.1),[§2](https://arxiv.org/html/2609.36357#S2.p3.1)\.
- Baguiet al\.\(2026\)S\. Bagui, M\. Malagutti, R\. Morelli, and M\. TamascelliLarge language models in database management system optimization: a survey\.ACM Transactions on Intelligent Systems and Technology\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p1.1)\.
- Bellard \(2021\)F\. BellardNNCP v2: lossless data compression with transformer\.Technical reportAmarisoft\.External Links:[Link](https://bellard.org/nncp/nncp_v2.pdf)Cited by:[§2](https://arxiv.org/html/2609.36357#S2.p2.1)\.
- Ben Allalet al\.\(2025\)L\. Ben Allal, A\. Lozhkov, E\. Bakouch, G\. Martín Blázquez, G\. Penedo, L\. Tunstall, A\. Marafioti, H\. Kydlíček, A\. Piqueres Lajarín, V\. Srivastav, J\. Lochner, C\. Fahlgren, X\. Nguyen, C\. Fourrier, B\. Burtenshaw, H\. Larcher, H\. Zhao, C\. Zakka, M\. Morlon, C\. Raffel, L\. von Werra, and T\. WolfSmolLM2: when smol goes big—data\-centric training of a small language model\.arXiv preprint arXiv:2502\.02737\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2502.02737),[Link](https://arxiv.org/abs/2502.02737)Cited by:[§C\.1](https://arxiv.org/html/2609.36357#A3.SS1.SSS0.Px1.p1.1)\.
- Cai and Li \(2026\)C\. Cai and G\. LiConfidence\-based decoding is provably efficient for diffusion language models\.arXiv preprint arXiv:2603\.22248\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2603.22248)Cited by:[§2](https://arxiv.org/html/2609.36357#S2.p3.1)\.
- Charakornet al\.\(2025a\)R\. Charakorn, E\. Cetin, Y\. Tang, and R\. T\. LangeText\-to\-lora: instant transformer adaptation\.InProceedings of the 42nd International Conference on Machine Learning,Cited by:[§A\.3](https://arxiv.org/html/2609.36357#A1.SS3.p2.1),[§2](https://arxiv.org/html/2609.36357#S2.p4.1)\.
- Charakornet al\.\(2025b\)R\. Charakorn, E\. Cetin, Y\. Tang, and R\. T\. LangeText\-to\-lora: instant transformer adaption\.arXiv preprint arXiv:2506\.06105\.Cited by:[§C\.1](https://arxiv.org/html/2609.36357#A3.SS1.SSS0.Px2.p1.1),[§3](https://arxiv.org/html/2609.36357#S3.p4.1)\.
- Charakornet al\.\(2026\)R\. Charakorn, E\. Cetin, S\. Uesaka, and R\. T\. LangeDoc\-to\-lora: learning to instantly internalize contexts\.arXiv preprint arXiv:2602\.15902\.Cited by:[§A\.3](https://arxiv.org/html/2609.36357#A1.SS3.p2.1),[§2](https://arxiv.org/html/2609.36357#S2.p4.1)\.
- Chauhanet al\.\(2024\)V\. K\. Chauhan, J\. Zhou, P\. Lu, S\. Molaei, and D\. A\. CliftonA brief review of hypernetworks in deep learning: vk chauhan et al\.\.Artificial Intelligence Review57\(9\),pp\. 250\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p4.1)\.
- Chenet al\.\(2023\)C\. Chen, S\. Borgeaud, G\. Irving, J\. Lespiau, L\. Sifre, and J\. JumperAccelerating large language model decoding with speculative sampling\.arXiv preprint arXiv:2302\.01318\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2302.01318),[Link](https://arxiv.org/abs/2302.01318)Cited by:[§B\.1](https://arxiv.org/html/2609.36357#A2.SS1.p1.1)\.
- Chenet al\.\(2026a\)J\. Chen, Y\. Liang, and Z\. LiuDFlash: block diffusion for flash speculative decoding\.arXiv preprint arXiv:2602\.06036\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2602.06036)Cited by:[§B\.2](https://arxiv.org/html/2609.36357#A2.SS2.p4.1),[§3](https://arxiv.org/html/2609.36357#S3.p2.1)\.
- Chenet al\.\(2025\)K\. Chen, P\. Zhang, H\. Liu, J\. Liu, Y\. Liu, J\. Huang, S\. Wang, H\. Yan, and H\. LiLarge language models for lossless image compression: next\-pixel prediction in language space is all you need\.InAdvances in Neural Information Processing Systems,Vol\.38\.External Links:[Document](https://dx.doi.org/10.52202/085713-5264),[Link](https://proceedings.neurips.cc/paper_files/paper/2025/hash/e7349e785900b93d8b4971a3f2c1cefe-Abstract-Conference.html)Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p4.1),[§2](https://arxiv.org/html/2609.36357#S2.p2.1)\.
- Chenet al\.\(2026b\)K\. Chen, P\. Zhang, H\. Liu, J\. Liu, Y\. Liu, J\. Huang, S\. Wang, H\. Yan, and H\. LiLarge language models for lossless image compression: next\-pixel prediction in language space is all you need\.Advances in Neural Information Processing Systems38,pp\. 157548–157569\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p1.1)\.
- Chkirbeneet al\.\(2024\)Z\. Chkirbene, R\. Hamila, A\. Gouissem, and U\. DevrimLarge language models \(llm\) in industry: a survey of applications, challenges, and trends\.In2024 IEEE 21st International Conference on Smart Communities: Improving Quality of Life using AI, Robotics and IoT \(HONET\),pp\. 229–234\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p1.1)\.
- Delétanget al\.\(2024\)G\. Delétang, A\. Ruoss, P\. Duquenne, E\. Catt, T\. Genewein, C\. Mattern, J\. Grau\-Moya, L\. K\. Wenliang, M\. Aitchison, L\. Orseau, M\. Hutter, and J\. VenessLanguage modeling is compression\.InInternational Conference on Learning Representations,External Links:[Link](https://proceedings.iclr.cc/paper_files/paper/2024/hash/3cbf627fa24fb6cb576e04e689b9428b-Abstract-Conference.html)Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p3.1),[§1](https://arxiv.org/html/2609.36357#S1.p1.1),[§2](https://arxiv.org/html/2609.36357#S2.p2.1)\.
- Deutsch \(1996\)L\. P\. DeutschGZIP file format specification version 4\.3\.RFCTechnical Report1952,Internet Engineering Task Force\.External Links:[Document](https://dx.doi.org/10.17487/RFC1952)Cited by:[§2](https://arxiv.org/html/2609.36357#S2.p2.1)\.
- Dhankharet al\.\(2026\)N\. Dhankhar, D\. Baha, and A\. SaparovScaling laws for hypernetwork\-based knowledge injection in large language models\.External Links:2607\.19604,[Link](https://arxiv.org/abs/2607.19604)Cited by:[§A\.3](https://arxiv.org/html/2609.36357#A1.SS3.p2.1)\.
- Fuet al\.\(2025\)H\. Fu, B\. Huang, V\. Adams, C\. Wang, V\. Srinivasan, and J\. JiaoFrom bits to rounds: parallel decoding with exploration for diffusion language models\.arXiv preprint arXiv:2511\.21103\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p3.1)\.
- Fuet al\.\(2026a\)Y\. Fu, L\. Whalen, A\. Garg, C\. Wu, M\. Khadkevich, N\. Oswald, E\. Xie, D\. Egert, S\. T\. Sreenivas, S\. Diao, C\. Yu, Y\. Yu, W\. Chen, S\. Norouzi, J\. Liu, S\. Lan, L\. Zhu, J\. Wang, J\. Jiang, M\. Mardani, M\. Maghoumi, S\. Han, A\. Jukić, N\. Tajbakhsh, J\. Kautz, and P\. MolchanovNemotron\-Labs\-Diffusion: a tri\-mode language model unifying autoregressive, diffusion, and self\-speculation decoding\.arXiv preprint arXiv:2607\.05722\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2607.05722)Cited by:[§A\.2](https://arxiv.org/html/2609.36357#A1.SS2.p1.1),[§B\.2](https://arxiv.org/html/2609.36357#A2.SS2.p7.1),[§B\.3](https://arxiv.org/html/2609.36357#A2.SS3.p1.1),[§3](https://arxiv.org/html/2609.36357#S3.p2.1)\.
- Fuet al\.\(2026b\)Y\. Fu, L\. Whalen, Z\. Ye, X\. Dong, S\. Diao, J\. Liu, C\. Wu, H\. Zhang, E\. Xie, S\. Han, M\. Khadkevich, J\. Kautz, Y\. C\. Lin, and P\. MolchanovEfficient\-DLM: from autoregressive to diffusion language models, and beyond in speed\.InProceedings of the 43rd International Conference on Machine Learning,Cited by:[§2](https://arxiv.org/html/2609.36357#S2.p3.1)\.
- Gloeckleet al\.\(2024\)F\. Gloeckle, B\. Y\. Idrissi, B\. Rozière, D\. Lopez\-Paz, and G\. SynnaeveBetter & faster large language models via multi\-token prediction\.arXiv preprint arXiv:2404\.19737\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p3.1)\.
- Haet al\.\(2017\)D\. Ha, A\. Dai, and Q\. V\. LeHyperNetworks\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[§A\.3](https://arxiv.org/html/2609.36357#A1.SS3.p1.1),[§2](https://arxiv.org/html/2609.36357#S2.p4.1)\.
- Haddadi and Teahan \(2026\)M\. Haddadi and W\. J\. TeahanComparing text compression capabilities of large language models with traditional compression algorithms\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 4: Student Research Workshop\),pp\. 219–232\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p1.1)\.
- Hagoset al\.\(2024\)D\. H\. Hagos, R\. Battle, and D\. B\. RawatRecent advances in generative ai and large language models: current status, challenges, and perspectives\.IEEE transactions on artificial intelligence5\(12\),pp\. 5873–5893\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p1.1)\.
- Heurtel\-Depeigeset al\.\(2024\)D\. Heurtel\-Depeiges, A\. Ruoss, J\. Veness, and T\. GeneweinCompression via pre\-trained transformers: a study on byte\-level multimodal data\.External Links:2410\.05078,[Document](https://dx.doi.org/10.48550/arXiv.2410.05078),[Link](https://arxiv.org/abs/2410.05078)Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p3.1),[§2](https://arxiv.org/html/2609.36357#S2.p2.1)\.
- Hoogeboomet al\.\(2021\)E\. Hoogeboom, D\. Nielsen, P\. Jaini, P\. Forré, and M\. WellingArgmax flows and multinomial diffusion: learning categorical distributions\.InAdvances in Neural Information Processing Systems,Vol\.34\.Cited by:[§A\.2](https://arxiv.org/html/2609.36357#A1.SS2.p1.1),[§2](https://arxiv.org/html/2609.36357#S2.p3.1)\.
- Huet al\.\(2021\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLora: low\-rank adaptation of large language models\.arXiv preprint arXiv:2106\.09685\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p4.1)\.
- Huet al\.\(2022\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.InInternational Conference on Learning Representations,Cited by:[§A\.3](https://arxiv.org/html/2609.36357#A1.SS3.p2.1),[§2](https://arxiv.org/html/2609.36357#S2.p4.1)\.
- Huet al\.\(2025\)Y\. Hu, Z\. Liu, Z\. Dong, T\. Peng, B\. McDanel, and S\. Q\. ZhangSpeculative decoding and beyond: an in\-depth survey of techniques\.arXiv preprint arXiv:2502\.19732\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p3.1)\.
- Hutter \(2006\)M\. HutterThe human knowledge compression contest\.Note:[https://prize\.hutter1\.net/](https://prize.hutter1.net/)Source of the enwik8 dataset; accessed August 24, 2026Cited by:[§B\.1](https://arxiv.org/html/2609.36357#A2.SS1.p1.1),[§C\.2](https://arxiv.org/html/2609.36357#A3.SS2.SSS0.Px1.p2.1),[§4](https://arxiv.org/html/2609.36357#S4.p1.1)\.
- Ivison and Peters \(2022\)H\. Ivison and M\. PetersHyperDecoders: instance\-specific decoders for multi\-task NLP\.InFindings of the Association for Computational Linguistics: EMNLP 2022,Abu Dhabi, United Arab Emirates,pp\. 1715–1730\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.findings-emnlp.124),[Link](https://aclanthology.org/2022.findings-emnlp.124/)Cited by:[§A\.3](https://arxiv.org/html/2609.36357#A1.SS3.p1.1),[§2](https://arxiv.org/html/2609.36357#S2.p4.1)\.
- Kipfet al\.\(2026\)A\. Kipf, T\. Schmidt, P\. Kuo, S\. Krid, M\. Rengert, L\. Heller, A\. Zimmerer, M\. Stoian, V\. Pandey, and A\. van RenenWaiting to decompress: the economics of llm\-based compression\.\.InCIDR,Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p1.1),[§1](https://arxiv.org/html/2609.36357#S1.p2.1)\.
- Kirchenbaueret al\.\(2026\)J\. Kirchenbauer, A\. Hans, B\. Bartoldson, M\. Goldblum, A\. Panda, and T\. GoldsteinMulti\-token prediction via self\-distillation\.arXiv preprint arXiv:2602\.06019\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2602.06019)Cited by:[§B\.2](https://arxiv.org/html/2609.36357#A2.SS2.p2.1),[§3](https://arxiv.org/html/2609.36357#S3.p2.1)\.
- Leviathanet al\.\(2023\)Y\. Leviathan, M\. Kalman, and Y\. MatiasFast inference from transformers via speculative decoding\.InProceedings of the 40th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.202,pp\. 19274–19286\.External Links:[Link](https://proceedings.mlr.press/v202/leviathan23a.html)Cited by:[§B\.1](https://arxiv.org/html/2609.36357#A2.SS1.p1.1)\.
- Liet al\.\(2025a\)P\. Li, Y\. Zhou, D\. Muhtar, L\. Yin, S\. Yan, L\. Shen, Y\. Liang, S\. Vosoughi, and S\. LiuDiffusion language models know the answer before decoding\.arXiv preprint arXiv:2508\.19982\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2508.19982)Cited by:[§2](https://arxiv.org/html/2609.36357#S2.p3.1)\.
- Liet al\.\(2025b\)Y\. Li, F\. Wei, C\. Zhang, and H\. ZhangEAGLE\-3: scaling up inference acceleration of large language models via training\-time test\.arXiv preprint arXiv:2503\.01840\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2503.01840)Cited by:[§B\.2](https://arxiv.org/html/2609.36357#A2.SS2.p3.1),[§3](https://arxiv.org/html/2609.36357#S3.p2.1)\.
- Liet al\.\(2025c\)Z\. Li, C\. Huang, X\. Wang, H\. Hu, C\. Wyeth, D\. Bu, Q\. Yu, W\. Gao, X\. Liu, and M\. LiLossless data compression by large models\.Nature Machine Intelligence\.External Links:[Document](https://dx.doi.org/10.1038/s42256-025-01033-7),[Link](https://doi.org/10.1038/s42256-025-01033-7)Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p3.1),[§1](https://arxiv.org/html/2609.36357#S1.p1.1),[§2](https://arxiv.org/html/2609.36357#S2.p2.1)\.
- Liuet al\.\(2025a\)X\. Liu, X\. Xia, W\. Zhao, M\. Zhang, X\. Yu, X\. Su, S\. Yang, S\. Ng, and T\. ChuaL\-MTP: leap multi\-token prediction beyond adjacent context for large language models\.InAdvances in Neural Information Processing Systems,Cited by:[§B\.2](https://arxiv.org/html/2609.36357#A2.SS2.p5.1),[§3](https://arxiv.org/html/2609.36357#S3.p2.1)\.
- Liuet al\.\(2025b\)Y\. Liu, Y\. Cao, H\. Li, G\. Luo, Z\. Chen, W\. Wang, X\. Liang, B\. Qi, L\. Wu, C\. Tian, Y\. Zhang, Y\. Li, T\. Lu, Y\. Qiao, J\. Dai, and W\. WangSequential diffusion language models\.arXiv preprint arXiv:2509\.24007\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2509.24007)Cited by:[§2](https://arxiv.org/html/2609.36357#S2.p3.1)\.
- Miaoet al\.\(2024\)X\. Miao, Z\. Jia, and B\. CuiDemystifying data management for large language models\.InCompanion of the 2024 International Conference on Management of Data,pp\. 547–555\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p1.1)\.
- Mittuet al\.\(2024\)F\. Mittu, Y\. Bu, A\. Gupta, A\. Devireddy, A\. E\. Ozdarendeli, A\. Singh, and G\. AnumanchipalliFineZip: pushing the limits of large language models for practical lossless text compression\.External Links:2409\.17141,[Document](https://dx.doi.org/10.48550/arXiv.2409.17141),[Link](https://arxiv.org/abs/2409.17141)Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p3.1),[§2](https://arxiv.org/html/2609.36357#S2.p2.1)\.
- Mohamedet al\.\(2026\)A\. Mohamed, Y\. Zhang, M\. Vazirgiannis, and G\. ShangFast\-decoding diffusion language models via progress\-aware confidence schedules\.InFindings of the Association for Computational Linguistics: ACL 2026,pp\. 35793–35807\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.1782)Cited by:[§2](https://arxiv.org/html/2609.36357#S2.p3.1)\.
- Naveedet al\.\(2025\)H\. Naveed, A\. U\. Khan, S\. Qiu, M\. Saqib, S\. Anwar, M\. Usman, N\. Akhtar, N\. Barnes, and A\. MianA comprehensive overview of large language models\.ACM Transactions on Intelligent Systems and Technology16\(5\),pp\. 1–72\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p1.1)\.
- Nieet al\.\(2025\)S\. Nie, F\. Zhu, Z\. You, X\. Zhang, J\. Ou, J\. Hu, J\. Zhou, Y\. Lin, J\. Wen, and C\. LiLarge language diffusion models\.arXiv preprint arXiv:2502\.09992\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2502.09992)Cited by:[§A\.2](https://arxiv.org/html/2609.36357#A1.SS2.p1.1),[§2](https://arxiv.org/html/2609.36357#S2.p3.1)\.
- Panet al\.\(2025\)Z\. Pan, S\. Wang, P\. Liao, and J\. LiUnderstanding LLM behaviors via compression: data generation, knowledge acquisition and scaling laws\.InAdvances in Neural Information Processing Systems,Vol\.38\.External Links:[Document](https://dx.doi.org/10.52202/085713-5613),[Link](https://proceedings.neurips.cc/paper_files/paper/2025/file/f61d7778e89b9221d1ea0ce8428b7014-Paper-Conference.pdf)Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p3.1),[§1](https://arxiv.org/html/2609.36357#S1.p1.1),[§2](https://arxiv.org/html/2609.36357#S2.p2.1)\.
- Pasco \(1977\)R\. C\. PascoSource coding algorithms for fast data compression\.Ph\.D\. Thesis,Stanford University\.Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p2.1),[§2](https://arxiv.org/html/2609.36357#S2.p1.1)\.
- Penedoet al\.\(2024\)G\. Penedo, H\. Kydlíček, L\. Ben Allal, A\. Lozhkov, M\. Mitchell, C\. Raffel, L\. von Werra, and T\. WolfThe fineweb datasets: decanting the web for the finest text data at scale\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 30811–30849\.Cited by:[§C\.2](https://arxiv.org/html/2609.36357#A3.SS2.SSS0.Px1.p1.1),[§4](https://arxiv.org/html/2609.36357#S4.p1.1)\.
- Raiaanet al\.\(2024\)M\. A\. K\. Raiaan, M\. S\. H\. Mukta, K\. Fatema, N\. M\. Fahad, S\. Sakib, M\. M\. J\. Mim, J\. Ahmad, M\. E\. Ali, and S\. AzamA review on large language models: architectures, applications, taxonomies, open issues and challenges\.IEEE access12,pp\. 26839–26874\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p1.1)\.
- Renet al\.\(2026\)T\. Ren, R\. Li, X\. Chen, Y\. Li, and Z\. ZhaoAdapting diffusion language models for lossless pixel\-level image transmission\.arXiv preprint arXiv:2606\.06273\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2606.06273)Cited by:[§A\.2](https://arxiv.org/html/2609.36357#A1.SS2.p1.1),[§2](https://arxiv.org/html/2609.36357#S2.p3.1)\.
- Rissanen \(1976\)J\. J\. RissanenGeneralized Kraft inequality and arithmetic coding\.IBM Journal of Research and Development20\(3\),pp\. 198–203\.Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p2.1),[§2](https://arxiv.org/html/2609.36357#S2.p1.1)\.
- Shannon \(1948\)C\. E\. ShannonA mathematical theory of communication\.The Bell System Technical Journal27\(3\),pp\. 379–423\.Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p2.1)\.
- Shiet al\.\(2024\)J\. Shi, K\. Han, Z\. Wang, A\. Doucet, and M\. K\. TitsiasSimplified and generalized masked diffusion for discrete data\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:[Document](https://dx.doi.org/10.52202/079017-3277)Cited by:[§A\.2](https://arxiv.org/html/2609.36357#A1.SS2.p1.1),[§2](https://arxiv.org/html/2609.36357#S2.p3.1)\.
- Tanet al\.\(2026\)Z\. Tan, Z\. Zhang, H\. Wen, Z\. Li, R\. Zhang, P\. Chen, F\. Mo, Z\. Liu, Q\. Zeng, Q\. Yin,et al\.Instant personalized large language model adaptation via hypernetwork\.InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 23557–23580\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p4.1)\.
- Tummuri \(2025\)S\. S\. R\. TummuriEnhancing data pipelines with foundation models: a new approach to automated schema mapping and sql generation\.InInternational Conference on Advances in Computer Engineering and Communication Systems,pp\. 21–30\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p1.1)\.
- Valmeekamet al\.\(2023\)C\. S\. K\. Valmeekam, K\. Narayanan, D\. Kalathil, J\. Chamberland, and S\. ShakkottaiLLMZip: lossless text compression using large language models\.External Links:2306\.04050,[Document](https://dx.doi.org/10.48550/arXiv.2306.04050),[Link](https://arxiv.org/abs/2306.04050)Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p4.1)\.
- Wittenet al\.\(1987\)I\. H\. Witten, R\. M\. Neal, and J\. G\. ClearyArithmetic coding for data compression\.Communications of the ACM30\(6\),pp\. 520–540\.Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p2.1)\.
- Wuet al\.\(2025\)C\. Wu, H\. Zhang, S\. Xue, S\. Diao, Y\. Fu, Z\. Liu, P\. Molchanov, P\. Luo, S\. Han, and E\. XieFast\-dLLM v2: efficient block\-diffusion llm\.arXiv preprint arXiv:2509\.26328\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2509.26328)Cited by:[§A\.2](https://arxiv.org/html/2609.36357#A1.SS2.p1.1),[§B\.2](https://arxiv.org/html/2609.36357#A2.SS2.p8.1),[§B\.3](https://arxiv.org/html/2609.36357#A2.SS3.p1.1),[§C\.1](https://arxiv.org/html/2609.36357#A3.SS1.SSS0.Px1.p1.1),[§3](https://arxiv.org/html/2609.36357#S3.p2.1),[§4](https://arxiv.org/html/2609.36357#S4.p1.1)\.
- Xianget al\.\(2026\)L\. Xiang, C\. Zhai, S\. Wei, J\. Wu, Q\. Xie, X\. Tan, K\. Chen, and W\. HeP\-MTP: efficient document parsing via multi\-token prediction with progressive depth scaling\.arXiv preprint arXiv:2606\.24447\.External Links:[Link](https://arxiv.org/abs/2606.24447)Cited by:[§B\.3](https://arxiv.org/html/2609.36357#A2.SS3.p4.1)\.
- Yanget al\.\(2025\)A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. QiuQwen3 technical report\.arXiv preprint arXiv:2505\.09388\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2505.09388)Cited by:[§B\.1](https://arxiv.org/html/2609.36357#A2.SS1.p1.1)\.
- Yanget al\.\(2024\)A\. Yanget al\.Qwen2\.5 technical report\.arXiv preprint arXiv:2412\.15115\.Cited by:[§4](https://arxiv.org/html/2609.36357#S4.p1.1)\.
- Yanget al\.\(2023\)Y\. Yang, S\. Mandt, and L\. TheisAn introduction to neural data compression\.Foundations and Trends in Computer Graphics and Vision15\(2\),pp\. 113–200\.External Links:[Document](https://dx.doi.org/10.1561/0600000107)Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p3.1),[§2](https://arxiv.org/html/2609.36357#S2.p2.1)\.
- Zhanget al\.\(2024a\)J\. Zhang, Z\. Cheng, Y\. Zhao, S\. Wang, D\. Zhou, G\. Lu, and L\. SongL3TC: leveraging RWKV for learned lossless low\-complexity text compression\.External Links:2412\.16642,[Document](https://dx.doi.org/10.48550/arXiv.2412.16642),[Link](https://arxiv.org/abs/2412.16642)Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p4.1),[§4](https://arxiv.org/html/2609.36357#S4.p1.1)\.
- Zhanget al\.\(2025a\)J\. Zhang, Z\. Cheng, Y\. Zhao, S\. Wang, D\. Zhou, G\. Lu, and L\. SongL3TC: leveraging rwkv for learned lossless low\-complexity text compression\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.39,pp\. 13251–13259\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p2.1),[§3](https://arxiv.org/html/2609.36357#S3.p3.1)\.
- Zhanget al\.\(2025b\)L\. Zhang, L\. Fang, C\. Duan, M\. He, L\. Pan, P\. Xiao, S\. Huang, Y\. Zhai, X\. Hu, P\. S\. Yu, and A\. LiuA survey on parallel text generation: from parallel decoding to diffusion language models\.arXiv preprint arXiv:2508\.08712\.Cited by:[§3](https://arxiv.org/html/2609.36357#S3.p2.1)\.
- Zhanget al\.\(2024b\)M\. Zhang, Z\. Ji, Z\. Luo, Y\. Wu, and C\. ChaiApplications and challenges for large language models: from data management perspective\.In2024 IEEE 40th International Conference on Data Engineering \(ICDE\),pp\. 5530–5541\.Cited by:[§1](https://arxiv.org/html/2609.36357#S1.p1.1)\.
- Zhanget al\.\(2025c\)Q\. Zhang, M\. Li, Z\. Wang, R\. Liao, and L\. WangTest\-time steering for lossless text compression via weighted product of experts\.InFindings of the Association for Computational Linguistics: EMNLP 2025,Suzhou, China,pp\. 2076–2088\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.110),[Link](https://aclanthology.org/2025.findings-emnlp.110/)Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p3.1),[§2](https://arxiv.org/html/2609.36357#S2.p2.1)\.
- Zhaoet al\.\(2026\)Y\. Zhao, Z\. Cheng, J\. Zhang, D\. Zhou, Q\. Gu, Q\. Wang, and L\. SongOmniZip: learning a unified and lightweight lossless compressor for multi\-modal data\.External Links:2602\.22286,[Document](https://dx.doi.org/10.48550/arXiv.2602.22286),[Link](https://arxiv.org/abs/2602.22286)Cited by:[§A\.1](https://arxiv.org/html/2609.36357#A1.SS1.p4.1),[§2](https://arxiv.org/html/2609.36357#S2.p2.1)\.

## Appendix ADetailed Background and Preliminaries

We consider streams of dataX=\[x1,x2,…,xn\]∈VnX=\[x\_\{1\},x\_\{2\},\\ldots,x\_\{n\}\]\\in V^\{n\}of lengthnnfrom a finite vocabularyVV\. The goal is compressing this data into the smallest reprentation\.

### A\.1LLMs in Data Compression

Coding Distribution\.Many lossless compression compressed based on coding distributionρ\\rho\. It is a sequence of probability mass functions withρt:Vt↦\(0,1\]\\rho\_\{t\}:V^\{t\}\\mapsto\(0,1\],t∈ℕt\\in\\mathbb\{N\}, satisfying the chain ruleρt\(x1:t\)=∑y∈Vρt\+1\(x1:ty\)\\rho\_\{t\}\(x\_\{1:t\}\)=\\sum\_\{y\\in V\}\\rho\_\{t\+1\}\(x\_\{1:t\}y\)for allx1:t∈Vtx\_\{1:t\}\\in V^\{t\},t≤nt\\leq n, and y is next token of the sequence\. The conditional probability of a symbolxtx\_\{t\}given past contextx1:t−1x\_\{1:t\-1\}isρ\(xt∣x1:t−1\):=ρ\(x1:t\)/ρ\(x1:t−1\)\.\\rho\(x\_\{t\}\\mid x\_\{1:t\-1\}\):=\\rho\(x\_\{1:t\}\)/\\rho\(x\_\{1:t\-1\}\)\.

Arithmetic CodingArithmetic Coding\[[47](https://arxiv.org/html/2609.36357#bib.bib22),[51](https://arxiv.org/html/2609.36357#bib.bib23),[57](https://arxiv.org/html/2609.36357#bib.bib13)\]encodes a sequencex1:nx\_\{1:n\}into a floating number using coding distributionρ\\rho\. The encoder narrows a nested sequence of sub\-intervals of\[0,1\)\[0,1\), starting fromI0=\[0,1\)I\_\{0\}=\[0,1\)\. In stepkk, the current interval is partitioned into sub\-intervals proportional toρ\(⋅∣x1:k−1\)\\rho\(\\cdot\\mid x\_\{1:k\-1\}\), and the one corresponding to the true symbolxkx\_\{k\}is retained asIkI\_\{k\}with correspondρ\(xk\|x1:k−1\)\\rho\(x\_\{k\}\|x\_\{1:k\-1\}\)\. Afternnsteps, final intervalInI\_\{n\}uniquely identifiesx1:nx\_\{1:n\}, and its truncated binary expansion forms the encoded bitstreamc\(x1:n\)c\(x\_\{1:n\}\)\. Its expected code lengthLρL\_\{\\rho\}approaches the optimal Shannon’s bound:Lρ≥H⁡\(ρ\):=Ex∼ρ​\[−l​o​g2​ρ​\(x\)\]L\_\{\\rho\}\\geq H\(\\rho\):=E\_\{x\\sim\\rho\}\[\-log\_\{2\}\\rho\(x\)\]\.\[[52](https://arxiv.org/html/2609.36357#bib.bib1)\]\.

LLM\-Based Arithmetic Coding\.Since the true source distributionρ\\rhois unknown, neural compression\[[62](https://arxiv.org/html/2609.36357#bib.bib35)\]approximates it with an autoregressive language model,qθ\(x1:n\)=∏i=1nqθ\(xi∣x1:i−1\)q\_\{\\theta\}\(x\_\{1:n\}\)=\\prod\_\{i=1\}^\{n\}q\_\{\\theta\}\(x\_\{i\}\\mid x\_\{1:i\-1\}\)\[[38](https://arxiv.org/html/2609.36357#bib.bib27),[42](https://arxiv.org/html/2609.36357#bib.bib29),[26](https://arxiv.org/html/2609.36357#bib.bib30),[67](https://arxiv.org/html/2609.36357#bib.bib32),[46](https://arxiv.org/html/2609.36357#bib.bib33)\]\. The arithmetic encoder usesqθ\(⋅∣x1:i−1\)q\_\{\\theta\}\(\\cdot\\mid x\_\{1:i\-1\}\), whose log\-loss corresponds to the expected code length\[[16](https://arxiv.org/html/2609.36357#bib.bib24)\]:

minθLqθ=minθEx1:n∼ρ\[−log2qθ\(x1:n\)\]=minθEx1:n∼ρ\[∑i=1n−log2qθ\(xi∣x1:i−1\)\]\.\\min\_\{\\theta\}L\_\{q\_\{\\theta\}\}=\\min\_\{\\theta\}E\_\{x\_\{1:n\}\\sim\\rho\}\\left\[\-\\log\_\{2\}q\_\{\\theta\}\(x\_\{1:n\}\)\\right\]=\\min\_\{\\theta\}E\_\{x\_\{1:n\}\\sim\\rho\}\\left\[\\sum\_\{i=1\}^\{n\}\-\\log\_\{2\}q\_\{\\theta\}\(x\_\{i\}\\mid x\_\{1:i\-1\}\)\\right\]\.
These methods outperform classical text compressors, including Gzip, LZMA, and Bzip2\[[63](https://arxiv.org/html/2609.36357#bib.bib31),[56](https://arxiv.org/html/2609.36357#bib.bib28)\], and can outperform PNG and FLAC on image and audio bytes\[[68](https://arxiv.org/html/2609.36357#bib.bib26),[13](https://arxiv.org/html/2609.36357#bib.bib25)\]\.

### A\.2Multi\-Token Prediction with Diffusion Language Models

Basics of Diffusion Language Model\.Language Diffusion Models\[[2](https://arxiv.org/html/2609.36357#bib.bib36),[27](https://arxiv.org/html/2609.36357#bib.bib37),[53](https://arxiv.org/html/2609.36357#bib.bib60),[50](https://arxiv.org/html/2609.36357#bib.bib59),[45](https://arxiv.org/html/2609.36357#bib.bib61)\]are generative models produce text in parallel through an iterative denoising steps\. Thus, compared to Auto Regressive generative model, DLMs shows efficientcy and advantage in reducing inference latency and capture bidirectional context\. This field shows promising speed but burden with the generation capability\. Recent advances specificially block diffusion model\[[1](https://arxiv.org/html/2609.36357#bib.bib38),[20](https://arxiv.org/html/2609.36357#bib.bib46),[58](https://arxiv.org/html/2609.36357#bib.bib47)\]set a new state\-of\-the\-art performance among diffusion models on language modeling while preserve the speed up\. People also introduce confidence\-based sampler method\. These new methods offer diffusion model a more stable generation while excel at latency optimization

Inference process of DLMs and its MTP process\.Divide the token sequence intoKKordered blocks,

X=\(X\(1\),…,X\(K\)\)\.X=\(X^\{\(1\)\},\\ldots,X^\{\(K\)\}\)\.This mechanism processes blocks autoregressively while reconstructing multiple tokens in parallel within each block\. When processing blockkkwith sequenceX\(k\)=\(x1\(k\),x2\(k\),…,x\|X\(k\)\|\(k\)\)X^\{\(k\)\}=\(x\_\{1\}^\{\(k\)\},x\_\{2\}^\{\(k\)\},\\ldots,x\_\{\|X^\{\(k\)\}\|\}^\{\(k\)\}\), the goal is to reconstruct the whole block using the minimum number of steps\. The preceding blocksX\(1:k−1\)X^\{\(1:k\-1\)\}are fixed, and the active block is initialized as

X0\(k\)=\[mask\]⊗\|X\(k\)\|\.X\_\{0\}^\{\(k\)\}=\[\\texttt\{mask\}\]^\{\\otimes\|X^\{\(k\)\}\|\}\.
At reconstruction stepss,Xs\(k\)X\_\{s\}^\{\(k\)\}denotes the state of blockkkat stepss\. Define the remaining masked positions as

ℳk,s=\{i:xs,i\(k\)=\[mask\]\}\.\\mathcal\{M\}\_\{k,s\}=\\big\\\{\\,i:x\_\{s,i\}^\{\(k\)\}=\[\\texttt\{mask\}\]\\,\\big\\\}\.For eachi∈ℳk,si\\in\\mathcal\{M\}\_\{k,s\}, the model produces a marginal distribution over the vocabulary for that position, conditioned on the fixed prefix and the current partially\-denoised block:

qθ\(v\)=qθ\(xs,i\(k\)=v\|X\(1:k−1\),Xs\(k\)\),v∈V\.q\_\{\\theta\}\(v\)=q\_\{\\theta\}\\\!\\left\(x\_\{s,i\}^\{\(k\)\}=v\\;\\middle\|\\;X^\{\(1:k\-1\)\},X\_\{s\}^\{\(k\)\}\\right\),\\qquad v\\in V\.A shared deterministic rule then selects, among the still\-masked positions, which ones to commit at this step – typically those where the model’s top prediction is confident enough:

Uk,s=\{i∈ℳk,s:maxv∈V⁡qθ​\(v\)\>γ\}\.U\_\{k,s\}=\\big\\\{\\,i\\in\\mathcal\{M\}\_\{k,s\}:\\max\_\{v\\in V\}q\_\{\\theta\}\(v\)\>\\gamma\\,\\big\\\}\.Positions inUk,sU\_\{k,s\}are unmasked \(fixed to their argmax value\), the state is updated toXs\+1\(k\)X\_\{s\+1\}^\{\(k\)\}, and the process repeats untilℳk,s=∅\\mathcal\{M\}\_\{k,s\}=\\emptyset\.

### A\.3Hypernetwork in LLMs\.

A hypernetworkhϕh\_\{\\phi\}is a neural network trained to generate the parameters of a separate target networkfθf\_\{\\theta\}, rather than learning those parameters directly via gradient descent on the target’s own loss\[[23](https://arxiv.org/html/2609.36357#bib.bib21)\]\. Formally, given an input or contextzz\(e\.g\., a task descriptor, layer index, or conditioning signal\), the hypernetwork producesΔ​θ=hϕ​\(z\)\\Delta\\theta=h\_\{\\phi\}\(z\), and onlyϕ\\phiis updated during training, whileΔ​θ\\Delta\\thetais instantiated on the fly for eachzz\[[32](https://arxiv.org/html/2609.36357#bib.bib56)\]\. This decouples the number of trainable parameters from the size of the target network, and allows the target’s effective parameters to vary dynamically withzzwithout storing or fine\-tuning a separate copy offθf\_\{\\theta\}for each context\.

Applied naively to LLM\-scale target networks, however, this formulation requireshϕh\_\{\\phi\}to output on the order of10910^\{9\}–101110^\{11\}parameters, making direct weight generation computationally prohibitive and empirically unstable at scale\[[18](https://arxiv.org/html/2609.36357#bib.bib68)\]\. Recent work addresses this by restricting the hypernetwork’s output to a low\-dimensional parameter subspace rather than the full weight tensor\[[7](https://arxiv.org/html/2609.36357#bib.bib40),[9](https://arxiv.org/html/2609.36357#bib.bib41)\]\. In particular, hypernetworks have been used to generate low\-rank adaptation \(LoRA\[[29](https://arxiv.org/html/2609.36357#bib.bib39)\]\) updatesΔ​θ=hϕ​\(z\)=Az​Bz⊤\\Delta\\theta=h\_\{\\phi\}\(z\)=A\_\{z\}B\_\{z\}^\{\\top\}, withAz∈ℝd×rA\_\{z\}\\in\\mathbb\{R\}^\{d\\times r\},Bz∈ℝk×rB\_\{z\}\\in\\mathbb\{R\}^\{k\\times r\},r≪min⁡\(d,k\)r\\ll\\min\(d,k\), reducing the hypernetwork’s output dimensionality from𝒪⁡\(d​k\)\\mathcal\{O\}\(dk\)to𝒪⁡\(r⁡\(d\+k\)\)\\mathcal\{O\}\(r\(d\+k\)\)per layer\. This enables input\- or task\-conditioned weight generation at LLM scale, allowingΔ​θ\\Delta\\thetato be produced in a single forward pass ofhϕh\_\{\\phi\}rather than requiring gradient\-based fine\-tuning for each new contextzz\.

## Appendix BDetailed Multi\-Token Prediction Study

### B\.1Experimental Setup\.

Whenever supported, we use Qwen3\-4B\[[60](https://arxiv.org/html/2609.36357#bib.bib48)\]as the common backbone and evaluate all methods onenwiki8\[[31](https://arxiv.org/html/2609.36357#bib.bib49)\]using the same settings with 1 GPU A100, precision, batch size, and sequence length\. We report compression ratio, compression throughput in tokens per second, and the average number of tokens processed per forward pass\. Unlike conventional speculative generation, where drafted tokens must be verified by a target model\[[35](https://arxiv.org/html/2609.36357#bib.bib50),[11](https://arxiv.org/html/2609.36357#bib.bib51)\], lossless compression directly provides the ground\-truth sequence\. Drafted tokens can therefore be compared with the observed tokens without a separate verification\-model invocation\.

![Refer to caption](https://arxiv.org/html/2609.36357v1/images/prelim_result.png)Figure 5:Compression–throughput trade\-off across inference methods and configurations\. Lower compressed size and higher throughput are preferred\. Each curve connects configurations from the corresponding hyperparameter sweep\. The throughput axis uses a base\-2 logarithmic scale\. Detail of how these methods custom speed present in Baseline
### B\.2Baselines\.

We briefly describe each baseline and the hyperparameter that controls its decoding speed\.

MTP via Self\-Distillation\[[34](https://arxiv.org/html/2609.36357#bib.bib45)\]predicts a block of future tokens in parallel\. Increasing the block size allows more tokens to be generated per forward pass, potentially improving throughput at the cost of prediction accuracy\. We evaluate block sizes of\{1,10,15,20\}\\\{1,10,15,20\\\}\.

EAGLE\-3\[[37](https://arxiv.org/html/2609.36357#bib.bib42)\]uses a lightweight draft model to propose multiple future tokens, which are then verified in parallel by the target model\. We control its decoding speed by varying the number of drafted tokens per iteration; longer drafts provide more opportunities for acceleration but may reduce the acceptance rate\.

DFlash\[[12](https://arxiv.org/html/2609.36357#bib.bib43)\]combines an autoregressive target model with a diffusion\-based drafter that proposes a block of future tokens in parallel\. We use a five\-layer draft model and vary the block size over\{5,10,15,20\}\\\{5,10,15,20\\\}to obtain different throughput operating points\.

L\-MTP\[[39](https://arxiv.org/html/2609.36357#bib.bib44)\]augments an autoregressive model with multiple MLP prediction heads, where the number of heads determines how many future tokens can be predicted in parallel\. Because different head configurations require separate training, L\-MTP cannot flexibly adjust its decoding speed at inference time\. We therefore evaluate a single configuration with four prediction heads, whose outputs are verified in parallel\.

Autoregressive Modelgenerates one token at a time, conditioning each prediction on all preceding tokens\. Unlike parallel decoding methods, it has no hyperparameter for varying the number of tokens generated per forward pass and therefore serves as our single\-token decoding baseline\.

Nemotron\-Labs\-Diffusion\[[20](https://arxiv.org/html/2609.36357#bib.bib46)\]generates tokens within each block in parallel through iterative denoising\. We vary the confidence threshold over0\.8,0\.5,0\.4,0\.3,0\.2\{0\.8,0\.5,0\.4,0\.3,0\.2\}to obtain different decoding speeds\. Lower thresholds allow more tokens to be accepted at each denoising step, increasing throughput but potentially reducing prediction quality and compression effectiveness\.

Fast\-dLLM\-v2\[[58](https://arxiv.org/html/2609.36357#bib.bib47)\]also uses confidence\-based parallel decoding\. We vary its confidence threshold over0\.7,0\.5,0\.45,0\.4,0\.35\{0\.7,0\.5,0\.45,0\.4,0\.35\}to evaluate different throughput operating points\. As with Nemotron\-Labs\-Diffusion, lower thresholds increase decoding parallelism but may worsen compression effectiveness\.

### B\.3Results and Analysis\.

Figure[5](https://arxiv.org/html/2609.36357#A2.F5)reveals a clear trade\-off between decompression throughput and compression effectiveness\. All parallel decoding methods improve throughput over the autoregressive baseline, but generally produce larger compressed representations\. Overall, diffusion based methods, including Nemotron\-Labs\-Diffusion\[[20](https://arxiv.org/html/2609.36357#bib.bib46)\]and Fast\-dLLM\-v2\[[58](https://arxiv.org/html/2609.36357#bib.bib47)\], provide a more favorable trade\-off than AR\-based methods, particularly at high\-throughput operating points\.

Among the AR\-based methods, MTP via self\-distillation increases throughput from approximately 25 to 180 tokens/s, while its compressed size rises from 12% to 42%\. EAGLE\-3 reaches approximately 50 tokens/s, but its compressed size increases sharply with draft length\. These results suggest that aggressively predicting more future tokens with auxiliary prediction heads can substantially degrade the probability estimates used by the arithmetic coder\. Although DFlash employs a diffusion\-based drafter, it follows a similar trend: throughput increases from approximately 50 to 170 tokens/s, while compressed size grows from 20% to 42%\. This suggests that adding a diffusion drafter can accelerate decoding but remains constrained by the underlying autoregressive generation process\.

In contrast, diffusion\-based methods maintain better compression effectiveness at comparable throughput\. Nemotron\-Labs\-Diffusion reaches approximately 140 tokens/s with a compressed size below 20%, whereas Fast\-dLLM\-v2 achieves approximately 128 tokens/s with a compressed size of 17\.3%\. At more aggressive settings, Fast\-dLLM\-v2 further increases throughput to approximately 320 tokens/s, although its compressed size rises to about 28%\. Thus, Fast\-dLLM\-v2 provides the strongest overall trade\-off among the evaluated methods rather than uniformly achieving the best compression at every operating point\.

We hypothesize that this advantage arises from the flexibility of confidence\-based diffusion decoding\. Its throughput can be adjusted at inference time by changing the confidence threshold, whereas AR\-based methods are more strongly constrained by the prediction horizon and drafting configuration used during training\[[59](https://arxiv.org/html/2609.36357#bib.bib55)\]\.

## Appendix CExperiment Details

### C\.1Model Architecture

#### Diffusion language model\.

Our compression backbones follow the architectures and pretrained weights of BD3LM and Fast\-dLLM\-v2 where they will distill pretrained autoregressive model to get the diffusion language model\[[1](https://arxiv.org/html/2609.36357#bib.bib38),[58](https://arxiv.org/html/2609.36357#bib.bib47)\]\. We evaluate three model scales: 150M, 0\.5B, and 1\.5B parameters distill from SmolLM2\-135M\[[5](https://arxiv.org/html/2609.36357#bib.bib67)\]\. The backbone parameters remain frozen during hypernetwork training\.

Each backbone operates on blocks of tokens and iteratively unmasks tokens according to the confidence of its predictions\. Its decoding throughput is controlled primarily by two hyperparameters: the block sizeBBand the confidence thresholdγ\\gamma\. A largerBBprovides more opportunities for parallel prediction, while a lowerγ\\gammapermits more tokens to be unmasked during each model evaluation\. Both changes can improve throughput, but may produce less confident probability estimates and therefore increase the resulting arithmetic\-coded bitstream\.

The original backbones are trained with a block size of3232\. We adapt them to a block size of256256using the training procedure described in Appendix[C\.2](https://arxiv.org/html/2609.36357#A3.SS2), thereby enabling greater decoding parallelism\. During training, we setγ=1\\gamma=1, which disables confidence\-based parallel unmasking and ensures that the training objective is not affected by the inference\-time decoding policy\. Unless otherwise stated, we useγ=0\.5\\gamma=0\.5at inference time, as it provides the best empirical trade\-off between compression effectiveness and throughput on the validation set\.

Table[2](https://arxiv.org/html/2609.36357#A3.T2)summarizes the backbone configurations\.

Table 2:Diffusion\-language\-model configurations used in our experiments\. Replace the placeholders with the exact checkpoint information\.BackboneParametersLayersHidden sizeVocabularyBD3LM\-135M135M3057649152BD3LM\-0\.5B0\.5B24896151936Fast\-dLLM\-V2\-1\.5B1\.5B281536151936
#### Hypernetwork\.

Our hypernetwork is inspired by Text\-to\-LoRA\[[8](https://arxiv.org/html/2609.36357#bib.bib53)\]and contains two components: \(i\) a document encoder that produces a semantic representation of the input document and \(ii\) a multilayer perceptron \(MLP\) that generates document\-specific LoRA parameters for the frozen diffusion backbone\.

For each documentdd, we obtain a document representation𝐞d∈ℝ1536\\mathbf\{e\}\_\{d\}\\in\\mathbb\{R\}^\{1536\}using GTE\-Qwen2\-1\.5B\-Instruct111[https://huggingface\.co/Alibaba\-NLP/gte\-Qwen2\-1\.5B\-instruct](https://huggingface.co/Alibaba-NLP/gte-Qwen2-1.5B-instruct)\. For documents longer than the encoder’s maximum input length, we truncate the document to the first 32,000 tokens / divide it into chunks\.

To distinguish the LoRA parameters generated for different locations in the backbone, we associate each adapted modulemmand layerℓ\\ellwith learned embeddings𝐞m∈ℝ32\\mathbf\{e\}\_\{m\}\\in\\mathbb\{R\}^\{32\}and𝐞ℓ∈ℝ32\\mathbf\{e\}\_\{\\ell\}\\in\\mathbb\{R\}^\{32\}, respectively\. These embeddings are concatenated with the document representation:

z=\[𝐞d;𝐞m;𝐞ℓ\]\.z=\[\\mathbf\{e\}\_\{d\};\\mathbf\{e\}\_\{m\};\\mathbf\{e\}\_\{\\ell\}\]\.The resulting conditioning vector is passed through anLL\-layer MLP:

Δ​θ=hϕ​\(𝐳\),\\Delta\\theta=h\_\{\\phi\}\(\\mathbf\{z\}\),wherehϕh\_\{\\phi\}denotes the shared hypernetwork\.

For a target weight matrix𝐖∈ℝdout×din\\mathbf\{W\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{out\}\}\\times d\_\{\\mathrm\{in\}\}\}, the hypernetwork produces the two LoRA factors

𝐀∈ℝr×din,𝐁∈ℝdout×r\.\\mathbf\{A\}\\in\\mathbb\{R\}^\{r\\times d\_\{\\mathrm\{in\}\}\},\\qquad\\mathbf\{B\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{out\}\}\\times r\}\.The personalized weight is then

𝐖′=𝐖\+αr​𝐁𝐀,\\mathbf\{W\}^\{\\prime\}=\\mathbf\{W\}\+\\frac\{\\alpha\}\{r\}\\mathbf\{B\}\\mathbf\{A\},whererris the LoRA rank andα\\alphais the LoRA scaling factor\. We generate adaption of each transformer layer\. All backbone weights remain frozen; only the hypernetwork, layer embeddings, and module embeddings are optimized\.

Table[3](https://arxiv.org/html/2609.36357#A3.T3)reports the complete hypernetwork configuration\.

Table 3:Hypernetwork configuration\.ComponentConfigurationDocument encoderGTE\-Qwen2\-1\.5B\-InstructDocument\-embedding dimension1536Encoder input length32,000tokensModule\-embedding dimensiondmd\_\{m\}32Layer\-embedding dimensiondℓd\_\{\\ell\}32LoRA rankrr8Target modulesqq,vvprojectionsHypernetwork parameter55M

### C\.2Training Details

#### Meta\-training data\.

We construct the meta\-training corpus from a subset of FineWeb\-Edu\[[48](https://arxiv.org/html/2609.36357#bib.bib52)\], treating each sample as an individual documentdd\. The resulting training corpus contains approximately 900 million tokens after tokenization with the backbone tokenizer\. We construct a disjoint FineWeb\-Edu evaluation split containing 100,000 documents and approximately 120 million tokens\. We remove any document appearing in the training split and additionally perform deduplication to reduce potential overlap between the training and evaluation sets\.

We also evaluate on enwik9\[[31](https://arxiv.org/html/2609.36357#bib.bib49)\]with no enwik9 text is used for hypernetwork training, hyperparameter selection, or early stopping\.

#### Training objective\.

For every documentcc, the document encoder first produces𝐞c\\mathbf\{e\}\_\{c\}\. The hypernetwork then generates the LoRA factors used to personalize the frozen diffusion backbone\. The personalized model is trained with the same masked diffusion objective as the underlying backbone:

ℒ\(θ\)=𝔼d,t,𝐱~t\[−∑i∈ℳtlogqθ\(xi∣𝐱~t,𝐞d\)\],\\mathcal\{L\}\(\\theta\)=\\mathbb\{E\}\_\{d,t,\\widetilde\{\\mathbf\{x\}\}\_\{t\}\}\\left\[\-\\sum\_\{i\\in\\mathcal\{M\}\_\{t\}\}\\log q\_\{\\theta\}\\left\(x\_\{i\}\\mid\\widetilde\{\\mathbf\{x\}\}\_\{t\},\\mathbf\{e\}\_\{d\}\\right\)\\right\],whereℳt\\mathcal\{M\}\_\{t\}is the set of masked positions at diffusion steptt,𝐱~t\\widetilde\{\\mathbf\{x\}\}\_\{t\}is the corrupted sequence, and the backbone parameters are fixed\. Gradients are propagated only through the generated LoRA parameters and the trainable hypernetwork components\.

#### Optimization\.

We train for 3,500 optimization steps with a global batch size of 128 and a learning rate of2×10−52\\times 10^\{\-5\}\. Training is performed on four NVIDIA A100 GPUs using bfloat16 precision\. The learning rate follows a cosine schedule with warm\-up fraction of 0\.1\.

We use sequences of length 8192, a block size of256256, and a sub\-block size of88\. The confidence threshold is fixed atγ=1\\gamma=1during training\.

### C\.3Compression and Evaluation Protocol

#### Lossless compression procedure\.

We tokenize each document using the tokenizer associated with its compression backbone\. The personalized diffusion model produces the conditional probabilities used by the arithmetic coder\. Encoding and decoding use identical model parameters, tokenization, adapter\-generation procedures, and probability quantization to guarantee deterministic reconstruction\. We verify losslessness by checking that the decoded byte sequence exactly matches the original input\.

### C\.4Evaluation Metrics

We evaluate the trade\-off between compression effectiveness and processing speed\. Lower compression ratios and higher throughput are preferred\.

#### Compression ratio\.

LetSrawS\_\{\\mathrm\{raw\}\}denote the size of the original document in bytes andSpayloadS\_\{\\mathrm\{payload\}\}the size of its arithmetic\-coded representation\. We report the payload compression ratio as

CRpayload=100×SpayloadSraw\.\\mathrm\{CR\}\_\{\\mathrm\{payload\}\}=100\\times\\frac\{S\_\{\\mathrm\{payload\}\}\}\{S\_\{\\mathrm\{raw\}\}\}\.\(1\)This metric is expressed as a percentage; a lower value indicates better compression\. For example,CR=12%\\mathrm\{CR\}=12\\%means that the compressed payload occupies 12% of the original file size\.

#### Compression and decompression throughput\.

Throughput is measured in tokens per second:

Throughput=NtokensT,\\mathrm\{Throughput\}=\\frac\{N\_\{\\mathrm\{tokens\}\}\}\{T\},\(2\)
Table 4:Compression ratio across document topics and compression throughput\. Lower compression ratio and higher throughput are preferred\.Model𝜸\\gammaCR incl\. update vector\(↓\\downarrow\)Throughput\(tok/s\)\(↑\\uparrow\)ScientificLiteratureProgrammingGeneral WebFast\-dLLM0\.516\.115\.718\.016\.3141\.0FineZip–9\.238\.469\.248\.2212\.6Fine\-tuning0\.515\.915\.417\.716\.1140\.2HyperZip \(ours\)0\.515\.615\.316\.315\.8138\.9

## NeurIPS Paper Checklist

The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact\. Do not remove the checklist:The papers not including the checklist will be desk rejected\.The checklist should follow the references and follow the \(optional\) supplemental material\. The checklist does NOT count towards the page limit\.

Please read the checklist guidelines carefully for information on how to answer these questions\. For each question in the checklist:

- •You should answer\[Yes\],\[No\], or\[N/A\]\.
- •\[N/A\]means either that the question is Not Applicable for that particular paper or the relevant information is Not Available\.
- •Please provide a short \(1–2 sentence\) justification right after your answer \(even for\[N/A\]\)\.

The checklist answers are an integral part of your paper submission\.They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers\. You will also be asked to include it \(after eventual revisions\) with the final version of your paper, and its final version will be published with the paper\.

The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation\. While\[Yes\]is generally preferable to\[No\], it is perfectly acceptable to answer\[No\]provided a proper justification is given \(e\.g\., error bars are not reported because it would be too computationally expensive” or “we were unable to find the license for the dataset we used”\)\. In general, answering\[No\]or\[N/A\]is not grounds for rejection\. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate\. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix\. If you answer\[Yes\]to a question, in the justification please point to the section\(s\) where related material for the question can be found\.

IMPORTANT, please:

- •Delete this instruction block, but keep the section heading “NeurIPS Paper Checklist",
- •Keep the checklist subsection headings, questions/answers and guidelines below\.
- •Do not modify the questions and only use the provided macros for your answers\.

1. 1\.Claims
2. Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
3. Answer:\[Yes\]
4. Guidelines: - •The answer\[N/A\]means that the abstract and introduction do not include the claims made in the paper\. - •The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations\. A\[No\]or\[N/A\]answer to this question will not be perceived well by the reviewers\. - •The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings\. - •It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper\.
5. 2\.Limitations
6. Question: Does the paper discuss the limitations of the work performed by the authors?
7. Answer:\[No\]
8. Guidelines: - •The answer\[N/A\]means that the paper has no limitation while the answer\[No\]means that the paper has limitations, but those are not discussed in the paper\. - •The authors are encouraged to create a separate “Limitations” section in their paper\. - •The paper should point out any strong assumptions and how robust the results are to violations of these assumptions \(e\.g\., independence assumptions, noiseless settings, model well\-specification, asymptotic approximations only holding locally\)\. The authors should reflect on how these assumptions might be violated in practice and what the implications would be\. - •The authors should reflect on the scope of the claims made, e\.g\., if the approach was only tested on a few datasets or with a few runs\. In general, empirical results often depend on implicit assumptions, which should be articulated\. - •The authors should reflect on the factors that influence the performance of the approach\. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting\. Or a speech\-to\-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon\. - •The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size\. - •If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness\. - •While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper\. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community\. Reviewers will be specifically instructed to not penalize honesty concerning limitations\.
9. 3\.Theory assumptions and proofs
10. Question: For each theoretical result, does the paper provide the full set of assumptions and a complete \(and correct\) proof?
11. Answer:\[N/A\]
12. Justification: We do not do theoretical analysis\.
13. Guidelines: - •The answer\[N/A\]means that the paper does not include theoretical results\. - •All the theorems, formulas, and proofs in the paper should be numbered and cross\-referenced\. - •All assumptions should be clearly stated or referenced in the statement of any theorems\. - •The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition\. - •Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material\. - •Theorems and Lemmas that the proof relies upon should be properly referenced\.
14. 4\.Experimental result reproducibility
15. Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper \(regardless of whether the code and data are provided or not\)?
16. Answer:\[Yes\]
17. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •If the paper includes experiments, a\[No\]answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not\. - •If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable\. - •Depending on the contribution, reproducibility can be accomplished in various ways\. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model\. In general\. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model \(e\.g\., in the case of a large language model\), releasing of a model checkpoint, or other means that are appropriate to the research performed\. - •While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution\. For example 1. \(a\)If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm\. 2. \(b\)If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully\. 3. \(c\)If the contribution is a new model \(e\.g\., a large language model\), then there should either be a way to access this model for reproducing the results or a way to reproduce the model \(e\.g\., with an open\-source dataset or instructions for how to construct the dataset\)\. 4. \(d\)We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility\. In the case of closed\-source models, it may be that access to the model is limited in some way \(e\.g\., to registered users\), but it should be possible for other researchers to have some path to reproducing or verifying the results\.
18. 5\.Open access to data and code
19. Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
20. Answer:\[TODO\]
21. Justification:\\justificationTODO
22. Guidelines: - •The answer\[N/A\]means that paper does not include experiments requiring code\. - • - •While we encourage the release of code and data, we understand that this might not be possible, so\[No\]is an acceptable answer\. Papers cannot be rejected simply for not including code, unless this is central to the contribution \(e\.g\., for a new open\-source benchmark\)\. - •The instructions should contain the exact command and environment needed to run to reproduce the results\. See the NeurIPS code and data submission guidelines \([https://neurips\.cc/public/guides/CodeSubmissionPolicy](https://neurips.cc/public/guides/CodeSubmissionPolicy)\) for more details\. - •The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc\. - •The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines\. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why\. - •At submission time, to preserve anonymity, the authors should release anonymized versions \(if applicable\)\. - •Providing as much information as possible in supplemental material \(appended to the paper\) is recommended, but including URLs to data and code is permitted\.
23. 6\.Experimental setting/details
24. Question: Does the paper specify all the training and test details \(e\.g\., data splits, hyperparameters, how they were chosen, type of optimizer\) necessary to understand the results?
25. Answer:\[Yes\]
26. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them\. - •The full details can be provided either with the code, in appendix, or as supplemental material\.
27. 7\.Experiment statistical significance
28. Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
29. Answer:\[No\]
30. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The authors should answer\[Yes\]if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper\. - •The factors of variability that the error bars are capturing should be clearly stated \(for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions\)\. - •The method for calculating the error bars should be explained \(closed form formula, call to a library function, bootstrap, etc\.\) - •The assumptions made should be given \(e\.g\., Normally distributed errors\)\. - •It should be clear whether the error bar is the standard deviation or the standard error of the mean\. - •It is OK to report 1\-sigma error bars, but one should state it\. The authors should preferably report a 2\-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified\. - •For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range \(e\.g\., negative error rates\)\. - •If error bars are reported in tables or plots, the authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text\.
31. 8\.Experiments compute resources
32. Question: For each experiment, does the paper provide sufficient information on the computer resources \(type of compute workers, memory, time of execution\) needed to reproduce the experiments?
33. Answer:\[Yes\]
34. Guidelines: - •The answer\[N/A\]means that the paper does not include experiments\. - •The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage\. - •The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute\. - •The paper should disclose whether the full research project required more compute than the experiments reported in the paper \(e\.g\., preliminary or failed experiments that didn’t make it into the paper\)\.
35. 9\.Code of ethics
37. Answer:\[Yes\]
38. Guidelines: - •The answer\[N/A\]means that the authors have not reviewed the NeurIPS Code of Ethics\. - •If the authors answer\[No\], they should explain the special circumstances that require a deviation from the Code of Ethics\. - •The authors should make sure to preserve anonymity \(e\.g\., if there is a special consideration due to laws or regulations in their jurisdiction\)\.
39. 10\.Broader impacts
40. Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
41. Answer:\[Yes\]
42. Guidelines: - •The answer\[N/A\]means that there is no societal impact of the work performed\. - •If the authors answer\[N/A\]or\[No\], they should explain why their work has no societal impact or why the paper does not address societal impact\. - •Examples of negative societal impacts include potential malicious or unintended uses \(e\.g\., disinformation, generating fake profiles, surveillance\), fairness considerations \(e\.g\., deployment of technologies that could make decisions that unfairly impact specific groups\), privacy considerations, and security considerations\. - •The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments\. However, if there is a direct path to any negative applications, the authors should point it out\. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate Deepfakes for disinformation\. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster\. - •The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from \(intentional or unintentional\) misuse of the technology\. - •If there are negative societal impacts, the authors could also discuss possible mitigation strategies \(e\.g\., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML\)\.
43. 11\.Safeguards
44. Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse \(e\.g\., pre\-trained language models, image generators, or scraped datasets\)?
45. Answer:\[N/A\]
46. Guidelines: - •The answer\[N/A\]means that the paper poses no such risks\. - •Released models that have a high risk for misuse or dual\-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters\. - •Datasets that have been scraped from the Internet could pose safety risks\. The authors should describe how they avoided releasing unsafe images\. - •We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort\.
47. 12\.Licenses for existing assets
48. Question: Are the creators or original owners of assets \(e\.g\., code, data, models\), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
49. Answer:\[TODO\]
50. Justification:\\justificationTODO
51. Guidelines: - •The answer\[N/A\]means that the paper does not use existing assets\. - •The authors should cite the original paper that produced the code package or dataset\. - •The authors should state which version of the asset is used and, if possible, include a URL\. - •The name of the license \(e\.g\., CC\-BY 4\.0\) should be included for each asset\. - •For scraped data from a particular source \(e\.g\., website\), the copyright and terms of service of that source should be provided\. - •If assets are released, the license, copyright information, and terms of use in the package should be provided\. For popular datasets,[paperswithcode\.com/datasets](https://paperswithcode.com/datasets)has curated licenses for some datasets\. Their licensing guide can help determine the license of a dataset\. - •For existing datasets that are re\-packaged, both the original license and the license of the derived asset \(if it has changed\) should be provided\. - •If this information is not available online, the authors are encouraged to reach out to the asset’s creators\.
52. 13\.New assets
53. Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
54. Answer:\[Yes\]
55. Justification: we will release the code when paper get accepted\.
56. Guidelines: - •The answer\[N/A\]means that the paper does not release new assets\. - •Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates\. This includes details about training, license, limitations, etc\. - •The paper should discuss whether and how consent was obtained from people whose asset is used\. - •At submission time, remember to anonymize your assets \(if applicable\)\. You can either create an anonymized URL or include an anonymized zip file\.
57. 14\.Crowdsourcing and research with human subjects
58. Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation \(if any\)?
59. Answer:\[N/A\]
60. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper\. - •According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector\.
61. 15\.Institutional review board \(IRB\) approvals or equivalent for research with human subjects
62. Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board \(IRB\) approvals \(or an equivalent approval/review based on the requirements of your country or institution\) were obtained?
63. Answer:\[N/A\]
64. Guidelines: - •The answer\[N/A\]means that the paper does not involve crowdsourcing nor research with human subjects\. - •Depending on the country in which research is conducted, IRB approval \(or equivalent\) may be required for any human subjects research\. If you obtained IRB approval, you should clearly state this in the paper\. - •We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution\. - •For initial submissions, do not include any information that would break anonymity \(if applicable\), such as the institution conducting the review\.
65. 16\.Declaration of LLM usage
66. Question: Does the paper describe the usage of LLMs if it is an important, original, or non\-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does*not*impact the core methodology, scientific rigor, or originality of the research, declaration is not required\.
67. Answer:\[No\]
68. Guidelines: - •The answer\[N/A\]means that the core method development in this research does not involve LLMs as any important, original, or non\-standard components\. - •Please refer to our LLM policy in the NeurIPS handbook for what should or should not be described\.