LaCache: Exact Caching and Precision-Adaptive Inference for Diffusion Large Language Models

# LaCache: Exact Caching and Precision-Adaptive Inference for Diffusion Large Language Models
Source: [https://arxiv.org/html/2607.16339](https://arxiv.org/html/2607.16339)
Xingru Chen1Zelang Liang111footnotemark:1Yongjia Ma111footnotemark:1Jiqing Zhan1Shuling Yang1Lian Wen1Kun Zhan1 1Li Auto Inc\. Corresponding author\.

###### Abstract

Diffusion\-based Large Language Models \(DLLMs\) enable parallel generation via Semi\-Autoregressive \(SAR\) decoding in text generation\. However, current methods suffer from severe operator\-level redundancy: they recompute the entire sequence during denoising steps, ignoring that the prefix and masked suffix remain invariant within a block\. We proposeLaCache, a training\-free acceleration framework that alleviates this redundancy through lossless caching and mixed precision\. Specifically, LaCache employsLossless State Memoization \(LSM\)by caching three types of intermediate results: \(i\)EmbedCachefor embedding outputs, \(ii\)RoPECachefor token\-wise pre\-attention states, and \(iii\)FACachefor the online softmax statistics within FlashAttention\. These caches allow the model to skip redundant computation on unchanged tokens without altering the output\. To further alleviate memory\-bandwidth bottlenecks, LaCache integrates a per\-group FP8 quantization strategy for FFN layers, tailored to step\-dependent activation distributions across the diffusion process\. Experiments demonstrate that LaCache alone achieves approximately 1\.3×\\timesend\-to\-end speedup over vanilla DLLM\. When combined with existing acceleration methods, LaCache reaches up to 40\.2×\\timesend\-to\-end speedup while maintaining comparable task accuracy\.

LaCache: Exact Caching and Precision\-Adaptive Inference for Diffusion Large Language Models

Xingru Chen1††thanks:Equal contribution\.Zelang Liang111footnotemark:1Yongjia Ma111footnotemark:1Jiqing Zhan1

Shuling Yang1Lian Wen1Kun Zhan1††thanks:Corresponding author\.1Li Auto Inc\.

††Primary contactchenxr@buaa\.edu\.cn## 1Introduction

Large language models \(LLMs\)Yanget al\.\([2025a](https://arxiv.org/html/2607.16339#bib.bib36)\); Dubeyet al\.\([2024](https://arxiv.org/html/2607.16339#bib.bib37)\); Brownet al\.\([2020](https://arxiv.org/html/2607.16339#bib.bib19)\); Maet al\.\([2026](https://arxiv.org/html/2607.16339#bib.bib46)\)are widely used in chatMaoet al\.\([2024](https://arxiv.org/html/2607.16339#bib.bib40)\), code generationJianget al\.\([2024](https://arxiv.org/html/2607.16339#bib.bib38)\), and complex reasoningWanet al\.\([2024](https://arxiv.org/html/2607.16339#bib.bib39)\); Liuet al\.\([2025a](https://arxiv.org/html/2607.16339#bib.bib26)\); Fenget al\.\([2026](https://arxiv.org/html/2607.16339#bib.bib52)\)\. In deployment, inference latency and compute cost are often the main bottlenecksLouet al\.\([2023](https://arxiv.org/html/2607.16339#bib.bib32)\)\. Autoregressive decoding generates tokens sequentially, which limits efficiency and hardware parallelismDonget al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib29)\); Fenget al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib44)\); Yanget al\.\([2025c](https://arxiv.org/html/2607.16339#bib.bib45)\)\. This motivates parallel decoding and non autoregressive generation\.

![Refer to caption](https://arxiv.org/html/2607.16339v1/x1.png)Figure 1:Left:DLLMs split total generation tokens into sequential blocks and do parallel decoding across multiple steps inside each block\. During one block’s generation, the tokens outside of the block stay unchanged, which results in repetition calculation\.Right: the latency of the modules in LLaDA\-Instruct is mainly present at the transformer layers and the final sampler\. With our method LaCache, the latency of the modules is significantly reduced\.Diffusion large language models \(DLLMs\)Nieet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib1)\); Yeet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib3)\); Lyuet al\.\([2019](https://arxiv.org/html/2607.16339#bib.bib33)\); Yanget al\.\([2025b](https://arxiv.org/html/2607.16339#bib.bib51)\); Maet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib43)\); Khannaet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib21)\)formulate text generation as iterative denoising, enabling multiple token positions to be updated in one step and employ bidirectional attention to leverage full context\. Despite this potential, open\-source DLLMs suffer from significant computation overheadNieet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib1)\); Luoet al\.\([2024](https://arxiv.org/html/2607.16339#bib.bib49)\)\. We attribute this inefficiency to two main constraints: First, bidirectional attention prevents direct reuse of standard KV cache mechanisms\. Second, parallel decoding can degrade generation quality due to violations of token dependenciesWuet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib4)\); Xieet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib50)\); Maet al\.\([2024](https://arxiv.org/html/2607.16339#bib.bib48)\)\.

![Refer to caption](https://arxiv.org/html/2607.16339v1/fig/main_structure.png)Figure 2:The whole procedure of one inference step\. After updating the cache in the first step, in the subsequent steps, the input only needs to contain the tokens of the current block, which saves most of the computational work of the embedding layer, norm, qkv linear, rotary embedding and Flash attention in the first layer of transformers\. In addition, the method of mixed\-precision acceleration is applied to all layers except the first: gateup linear, ffout linear\. The FP8 quantization operations are fused with the ff Norm and Swiglu, which further saves overhead\. Moreover, The output of the Transformer layers will be truncated again, and only the tokens of the current block will be retained for subsequent sampling operations, which decreased the latency of the float Softmax operation\.Most open\-source DLLMs adopt a semi\-autoregressive \(SAR\)Nieet al\.\([2024](https://arxiv.org/html/2607.16339#bib.bib41)\), block\-wise inference scheme\. The model initializes future positions with a fixed number of\[MASK\]tokens and partitions them into blocks\. It then performs multiple denoising steps within each block\. Across these steps, only the current block tokens change while the prompt and other blocks maintain the same token IDs\. Despite this, the model executes a full forward pass at each step, creating substantial redundant computation\. As shown in Fig\.[1](https://arxiv.org/html/2607.16339#S1.F1)\(Left\), the high input overlap within a block leads to repeated computationWeiet al\.\([2025b](https://arxiv.org/html/2607.16339#bib.bib28)\)\. It appears in token wise operators, such as embedding, QKV projections and RoPE\. It also appears inside attention computation, especially in the tiled calculation of first layer FlashAttention\. The profiling result in Fig\.[1](https://arxiv.org/html/2607.16339#S1.F1)\(right\) shows that latency is dominated by Transformer layers and the final sampler\. This observation suggests that eliminating repeated operator\-level computation on unchanged tokens can yield stable and composable end\-to\-end speedup\. Existing inference acceleration techniques primarily reduce the number of denoising steps or the effective context length\. Fast DLLMWuet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib4)\)employs confidence threshold decoding and approximate caching to increase throughput\. DPadChenet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib5)\)applies suffix dropout, including a sliding window and distance decay, to reduce the number of tokens used in attention\. Other works explore guidance tokens or adaptive decoding schedulesWeiet al\.\([2025a](https://arxiv.org/html/2607.16339#bib.bib7)\); Sunet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib6)\)\. While these methods are effective and complementary to our approach, they do not directly eliminate the operator\-level redundancy caused by unchanged tokens within a SAR block, leaving room for further acceleration\.

We proposeLaCache, a training\-free acceleration framework for SAR inference in DLLMs\. LaCache reduces operator\-level redundancy by reusing intermediate states of unchanged tokens across denoising steps within each SAR block\. It combinesLossless State Memoization \(LSM\), which reuses computation with identical outputs, and a mixed\-precision strategy that improves hardware efficiency\. Firstly,EmbedCachestores embedding outputs of unchanged tokens\. Second,RoPECachestores first\-layer token\-wise pre\-attention results, including RoPE\. Third,FACachestores the online softmax states in first\-layer FlashAttention, including the running maximum, the normalizer, and the unnormalized output accumulator, which allow us to skip attention tiles that do not involve the current block\. Motivated by the diverse activation distributions across layers and denoising steps, LaCache further applies a fine\-grained per\-group FP8 quantization on FFN layers, which adapts scales to local activation patterns and runs on FP8 Tensor Core kernels fused with surrounding operators to alleviate memory\-bandwidth bottlenecks while preserving model accuracy\. Experiments show that LaCache achieves approximately 1\.3×\\timesend\-to\-end inference speedup over vanilla DLLM\. When combined with existing acceleration schemes, it reaches up to 40\.2×\\timesend\-to\-end speedup with comparable quality\. Our main contributions are as follows\.

- •We identify operator\-level redundancy in SAR inference of DLLMs and introduce three lossless cache components that cover token\-wise operators and the online softmax state of first\-layer FlashAttention, alleviating the redundant computation within each SAR block\.
- •We integrate a per\-group FP8 quantization for FFN layers, adapting to diverse activation distributions and FP8 Tensor Core kernels to achieve speedup while preserving accuracy\.
- •We demonstrate consistent speedups across multiple models and benchmarks, showing that LaCache composes well with existing acceleration methods\.

## 2Related Work

Accelerating Diffusion Language Models\.Recent research optimizes dLLM inference efficiency from multiple perspectives\. Several worksJianget al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib27)\); Liuet al\.\([2025b](https://arxiv.org/html/2607.16339#bib.bib42)\); Wuet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib4)\)introduce approximate KV cache mechanisms to reuse hidden states across denoising steps\. Specifically, Fast\-dLLMWuet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib4)\)enhances throughput via confidence\-aware parallel decoding, which replaces top\-k sampling with a threshold\-based strategy\. Dllm\-cacheLiuet al\.\([2025b](https://arxiv.org/html/2607.16339#bib.bib42)\)stored and retrieved key–value \(KV\) states generated during transformer inference\. From a complementary angle, DPadChenet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib5)\)reduces the computational cost of attention by pruning the context length, employing suffix dropout with a sliding window\. Other works focus on adaptive decoding schedules, such as inserting guidance tokensWeiet al\.\([2025a](https://arxiv.org/html/2607.16339#bib.bib7)\)or monitoring token stability to dynamically adjust generation thresholdsSunet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib6)\)\. While these methods effectively optimize the decoding schedule or context graph, they do not directly eliminate the operator\-level redundancy on invariant tokens within a block, a gap that our framework is intended to bridge\.

Low\-Precision Inference\.Quantization has been widely adopted to accelerate LLM inference\. SmoothQuantXiaoet al\.\([2023](https://arxiv.org/html/2607.16339#bib.bib10)\)proposes per\-channel quantization that smooths activation outliers across channels to maintain accuracy\. COATXiet al\.\([2024](https://arxiv.org/html/2607.16339#bib.bib11)\)introduces mixed\-granularity quantization, combining per\-tensor and per\-group schemes to balance efficiency and precision\. Similarly, DeepSeek\-V3Liuet al\.\([2024](https://arxiv.org/html/2607.16339#bib.bib9)\)adopts per\-group quantization for linear layers\. MOSSZhanget al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib12)\)refines this with a two\-level scaling strategy to reduce precision loss during training\. Building on these insights, LaCache applies per\-group FP8 quantization to FFN linears in dLLMs, exploiting the activation distribution characteristics observed in SAR inference\.

## 3Method

![Refer to caption](https://arxiv.org/html/2607.16339v1/x2.png)Figure 3:The update and resuse scheme of our FACache\.Left: Update the cache in the first Flash\-attention in the first step\. When the qkv tile has no overlap with current generation Block, the cache follows same calculation procedure to collect the intermidia results\.Right: Reuse the cache in the first Flash\-attention in the following steps\. When the q tile has no overlap with current generation Block, the registers would be initialized with the data in cache\. And if the kv tile has no overlap with current generation Block as well, the calculation could be skipped directly\.In this section, we start by analyzing the redundant computation in DLLMs, which is raised by the repetition in the input, and the steady activation dynamics suitable for quantization during inference\. We then introduce the lossless cache strategy, which contains three parts of the intermediate results during inference\. Finally, we propose a fine\-grained mixed precision strategy to accelerate the inference with barely precision loss\.

### 3\.1Overview

LaCache proposes a Lossless State Memoization \(LSM\) strategy through the integration of three specialized components:EmbedCache,RoPECache, andFACache\. This mechanism ensures mathematically equivalent output while strictly skipping redundant computations for invariant tokens\. To further enhance throughput, we incorporate a fine\-grained FP8 mixed\-precision strategy for compute\-intensive linear layers\. The overall architecture is presented in Figure[2](https://arxiv.org/html/2607.16339#S1.F2)\.

### 3\.2The Acceleration space in DLLMs

#### 3\.2\.1Redundant calculation in DLLM inference

Within the multiple generation steps of a SAR block, only the tokens in the current block are updated, while the token IDs outside the block remain unchanged\. Therefore, redundant computation appears in token\-wise operators \(embedding, attention norm, QKV projection, RoPE\) and in parts of the first\-layer FlashAttention tiling procedure\. With a proper cache strategy, these redundant costs can be reduced without changing the model output\.

The structure of DLLMs is still Transformer\-based, and linear layers dominate the runtime\. A common way to accelerate linear layers is low\-precision computationKalamkaret al\.\([2019](https://arxiv.org/html/2607.16339#bib.bib23)\); Leeet al\.\([2009](https://arxiv.org/html/2607.16339#bib.bib25)\); Micikeviciuset al\.\([2017](https://arxiv.org/html/2607.16339#bib.bib24)\); Liuet al\.\([2024](https://arxiv.org/html/2607.16339#bib.bib9)\)\. As shown in Fig\.[4](https://arxiv.org/html/2607.16339#S3.F4), the peak of activation in the first layer among multiple steps presents a concentration in several channels, and the peak density decreases as future tokens get unmasked\. In middle layers, the activation value is relatively flat with only a few channels having sporadic spikes\. In later layers, the values of most channels of the same token are basically the same\. This suggest that a fine\-grained scheme can better balance efficiency and accuracy, especially under SAR inputs that include many masked tokens\.

### 3\.3Lossless State Memoization

To eliminate operator\-level redundancy, we introduce Lossless State Memoization \(LSM\)\. It strictly memoizes the intermediate results of token\-wise operators and the internal online softmax statistics of the first\-layer FlashAttention for the invariant region\. The memoized states are populated at the first denoising step of each SAR block and reused across subsequent steps within the same block, ensuring mathematically equivalent outputs with reduced computational cost\.

![Refer to caption](https://arxiv.org/html/2607.16339v1/x3.png)Figure 4:The visualization of activations before the ffout linear in LLaDA\-Instruct among multiple steps and blocks\.#### 3\.3\.1Notation

Let the full sequence have lengthNN\. For a SAR block, let𝒞\\mathcal\{C\}denote the index set of tokens in the current block \(which may change across denoising steps\), and let𝒰=\{1,…,N\}∖𝒞\\mathcal\{U\}=\\\{1,\\dots,N\\\}\\setminus\\mathcal\{C\}denote the remaining tokens \(prompt/prefix and masked suffix outside the current block\) whose*token IDs*stay unchanged within this block\.

#### 3\.3\.2EmbedCache and RoPECache

Embedding, \(RMS\)Norm, linear projections, and RoPE are applied independently to each token\. Therefore, for any token\-wise operatorff, we havef​\(x𝒰\)f\(x\_\{\\mathcal\{U\}\}\)unchanged across steps inside the block\. We cache: \(i\)EmbedCache: the embedding outputs for tokens in𝒰\\mathcal\{U\}; and \(ii\)RoPECache: the first\-layer token\-wise pre\-attention results \(after attention norm, QKV projections, and RoPE\) for tokens in𝒰\\mathcal\{U\}\. In subsequent steps, we only compute these token\-wise operators for tokens in𝒞\\mathcal\{C\}and reuse the cached tensors for𝒰\\mathcal\{U\}, avoiding redundant work in embedding, attention norm, QKV linear, and RoPE\.

Formally, letH=Embed​\(x\)H=\\mathrm\{Embed\}\(x\)be the embedding output and letH^=Norm​\(H\)\\hat\{H\}=\\mathrm\{Norm\}\(H\)be the normalized hidden states at the first layer\. The first\-layer projections are:

Q,K=RoPE​\(H^​WQ,K\),V=H^​WV\.Q,K=\\mathrm\{RoPE\}\(\\hat\{H\}W\_\{Q,K\}\),\\quad V=\\hat\{H\}W\_\{V\}\.\(1\)Within a SAR block,x𝒰x\_\{\\mathcal\{U\}\}is unchanged, soH𝒰H\_\{\\mathcal\{U\}\}and\(Q𝒰,K𝒰,V𝒰\)\(Q\_\{\\mathcal\{U\}\},K\_\{\\mathcal\{U\}\},V\_\{\\mathcal\{U\}\}\)can be computed once and reused\.

![Refer to caption](https://arxiv.org/html/2607.16339v1/fig/full_main_latency.png)Figure 5:The average inference latency of LaCache and previous methods on LLaDA\-instruct model among multiple benchmarks\. In all tasks, whether combined with parallel or DPad, LaCache can further reduce the latency on the basis of the original method\.
#### 3\.3\.3FACache

The remaining redundancy in the first layer comes from FlashAttention’s streaming over key–value blocks\. As shown in Fig\.[3](https://arxiv.org/html/2607.16339#S3.F3), for query blocks that belong to𝒰\\mathcal\{U\}, both the query vectors and the key–value blocks from𝒰\\mathcal\{U\}remain unchanged within the SAR block\. We therefore cache the*online\-softmax state*of FlashAttention for tiles that do not overlap the current block\.

Concretely, for a query blockQiQ\_\{i\}and a streamed KV block\(Kj,Vj\)\(K\_\{j\},V\_\{j\}\), defineSi\(j\)=Qi​Kj⊤S\_\{i\}^\{\(j\)\}=Q\_\{i\}K\_\{j\}^\{\\top\}\. FlashAttention maintains per\-row state\(m,ℓ,O~\)\(m,\\ell,\\tilde\{O\}\)and updates it as:

m\(j\)\\displaystyle m^\{\(j\)\}=max⁡\(m\(j−1\),RM​\(S\(j\)\)\),\\displaystyle=\\max\\\!\\left\(m^\{\(j\-1\)\},\\ \\mathrm\{RM\}\(S^\{\(j\)\}\)\\right\),\(2\)α\(j\)\\displaystyle\\alpha^\{\(j\)\}=exp⁡\(m\(j−1\)−m\(j\)\),\\displaystyle=\\exp\\\!\\left\(m^\{\(j\-1\)\}\-m^\{\(j\)\}\\right\),\(3\)P~\(j\)\\displaystyle\\tilde\{P\}^\{\(j\)\}=exp⁡\(S\(j\)−m\(j\)\),\\displaystyle=\\exp\\\!\\left\(S^\{\(j\)\}\-m^\{\(j\)\}\\right\),\(4\)ℓ\(j\)\\displaystyle\\ell^\{\(j\)\}=α\(j\)⊙ℓ\(j−1\)\+RS​\(P~\(j\)\),\\displaystyle=\\alpha^\{\(j\)\}\\odot\\ell^\{\(j\-1\)\}\+\\mathrm\{RS\}\(\\tilde\{P\}^\{\(j\)\}\),\(5\)O~\(j\)\\displaystyle\\tilde\{O\}^\{\(j\)\}=α\(j\)⊙O~\(j−1\)\+P~\(j\)​Vj,\\displaystyle=\\alpha^\{\(j\)\}\\odot\\tilde\{O\}^\{\(j\-1\)\}\+\\tilde\{P\}^\{\(j\)\}V\_\{j\},\(6\)whereRM​\(⋅\)\\mathrm\{RM\}\(\\cdot\)andRS​\(⋅\)\\mathrm\{RS\}\(\\cdot\)denote the row\-wise max and sum respectively\. All of the scalings are applied row\-wise \(with broadcasting over the feature dimension\)\. The final attention output is obtained by row\-wise normalizationO=diag\(ℓ\(Tc\)\)−1O~\(Tc\)O=\\operatorname\{diag\}\(\\ell^\{\(T\_\{c\}\)\}\)^\{\-1\}\\tilde\{O\}^\{\(T\_\{c\}\)\}\.

OurFACachestores these state variables*restricted to the unchanged region*: for query blocksQi⊆𝒰Q\_\{i\}\\subseteq\\mathcal\{U\}, we cache and update\(mcache,ℓcache,O~cache\)\(m\_\{\\text\{cache\}\},\\ell\_\{\\text\{cache\}\},\\tilde\{O\}\_\{\\text\{cache\}\}\)to be updated like\(m,ℓ,O~\)\(\{m\},\\ell,\\tilde\{O\}\)after streaming over KV blocks\(Kj,Vj\)\(K\_\{j\},V\_\{j\}\)that are fully inside𝒰\\mathcal\{U\}\. In implementation, these correspond to the HBM buffersRowMaxcache\(mcachem\_\{\\text\{cache\}\}\),RowSumcache\(ℓcache\\ell\_\{\\text\{cache\}\}\), andOcache\(O~cache\\tilde\{O\}\_\{\\text\{cache\}\}\)\.

In subsequent steps within the same SAR block, FlashAttention initializes\(m,ℓ,O~\)\(m,\\ell,\\tilde\{O\}\)with the cached state and only processes KV blocks that overlap𝒞\\mathcal\{C\}, while tiles where both query and KV blocks are in𝒰\\mathcal\{U\}are skipped entirely\.

### 3\.4Mixed precision strategy

As shown in Fig\.[4](https://arxiv.org/html/2607.16339#S3.F4), the peaks of activations are literally not higher than the valleys, and densely exist in masked tokens, which mainly provide position informationLuxembourget al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib30)\)instead of semantic information\. Moreover, previous methods such as DuQuantLinet al\.\([2024](https://arxiv.org/html/2607.16339#bib.bib47)\)focus on memory savings without fully exploiting hardware capabilities for acceleration\. Therefore we applied the mix\-precision strategy in all layers except the first to access acceleration while maintaining precision\. The reason to exclude the first layer is that the precision would decrease significantly if applied the quantization in the first layer\. Specifically,, We leveraged per\-group fp8 quantization before the gate\-up linear and ffout linear and applied DeepGEMM in those linears, where activations are less\-sensitive, and maintained the original fp16 settings for the left modules\. The quantization group on activation is 1x128 while on model weights it’s 128x128\. The equation process could be expressed as:

xi′=clip​\(xis​c​a​l​eg,Qmin,Qmax\),i∈g,x^\{\\prime\}\_\{i\}=\\mathrm\{clip\}\\\!\\left\(\\frac\{x\_\{i\}\}\{scale\_\{g\}\},Q\_\{\\min\},Q\_\{\\max\}\\right\),\\quad i\\in g,\(7\)scaleg=maxi∈g⁡\|xi\|Qmax\.\\text\{scale\}\_\{g\}=\\frac\{\\max\_\{i\\in g\}\|x\_\{i\}\|\}\{Q\_\{\\max\}\}\.\(8\)Wherexxis the high precision tensor splitted intoGGgroups,x′x^\{\\prime\}is the quantized counterpart,QminQ\_\{\\min\},QmaxQ\_\{\\max\}are the min and max value of the low\-precision format respectively, ands​c​a​l​egscale\_\{g\}is quantization scale of groupgg\.

Meanwhile, to reduce the quantization and dequantization computation overhead, we fused the operation with the AddNorm operation before the gate\-up linear, and with the Swiglu operation before ffout linear, which reduced the memory access timeGuoet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib22)\)\.

Moreover, at the sampling stage, the output would be cut to include only the tokens of current generation block, and therefore accelerate the sampling procedure\.

## 4Experiments

### 4\.1Experimental setup

![Refer to caption](https://arxiv.org/html/2607.16339v1/fig/longseq-speed-LLaDA.png)Figure 6:Comparison of token generation speed in long text inference scenario of Humaneval benchmark\.Models and Baselines\.All experiments are conducted on an NVIDIA H200 141GB GPU\. We evaluate LaCache on three variants of diffusion\-based large language models: LLaDA\(base\)Nieet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib1)\), LLaDA\-instruct and LLaDA\-1\.5Zhuet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib2)\)\. Following are the compared baselines:

Vanilla: the original LLaDANieet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib1)\)backbone\.

\+Parallel\(Fast\-dllm\): a sampling strategy that replaces the top\-k strategy in vanilla with a threshold strategyWuet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib4)\)and allows more tokens to be decoded in parallel\.

\+DPad: a suffix dropout strategyChenet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib5)\)that drops a number of suffix tokens at each step with a sliding\-window or distance\-decay strategy\.

Benchmarks and metrics\.We conduct experiments across a range of task categories\. Reasoning performance is evaluated on GSM8KCobbeet al\.\([2021](https://arxiv.org/html/2607.16339#bib.bib13)\)and MATHHendryckset al\.\([2021](https://arxiv.org/html/2607.16339#bib.bib15)\), while code generation is assessed using HumanEvalChen \([2021](https://arxiv.org/html/2607.16339#bib.bib14)\)and MBPPAustinet al\.\([2021](https://arxiv.org/html/2607.16339#bib.bib16)\)\. We examine performance from two complementary perspectives:

Accuracy:Measured using task\-dependent evaluation criteria\. For code generation, we report pass@1, and for reasoning tasks, we use standard accuracy metrics such as flexible\-extract and strict\-match accuracy on GSM8K\. All accuracy results are presented as percentages\.

Efficiency:Assessed in terms of mean inference latency per example and throughput, reported as Tokens Per Second \(TPS\), which is calculated over the whole output<eos\>token is emitted\.

### 4\.2Main Result

Table 1:Accuracy of quantization granularities\.Accuracy\(%\)BenchmarkMethodFlexibleStrictLaCache76\.3549\.43\\cellcolor\[HTML\]FFFFFFPer Tensor\\cellcolor\[HTML\]FFFFFF75\.82\\cellcolor\[HTML\]FFFFFF49\.66GSM8K\(4\-shot\)\\cellcolor\[HTML\]FFFFFFPer Token\\cellcolor\[HTML\]FFFFFF76\.27\\cellcolor\[HTML\]FFFFFF49\.96LaCache73\.772\.47\\cellcolor\[HTML\]FFFFFFPer Tensor\\cellcolor\[HTML\]FFFFFF73\.4\\cellcolor\[HTML\]FFFFFF72\.22Arc\_easy\\cellcolor\[HTML\]FFFFFFPer Token\\cellcolor\[HTML\]FFFFFF73\.48\\cellcolor\[HTML\]FFFFFF71\.97LaCache43\.3444\.97\\cellcolor\[HTML\]FFFFFFPer Tensor\\cellcolor\[HTML\]FFFFFF42\.83\\cellcolor\[HTML\]FFFFFF46\.33Arc\_challenge\\cellcolor\[HTML\]FFFFFFPer Token\\cellcolor\[HTML\]FFFFFF42\.66\\cellcolor\[HTML\]FFFFFF46\.08Table 2:Efficiency of quantization methods#### 4\.2\.1Latency

Overall, LaCache achieved the fastest speed on all benchmarks and model variants \(Tab\.[4](https://arxiv.org/html/2607.16339#S4.T4)\) with 10\.3%\-42\.5% faster than the Vanilla Backbone and the accuracy largely lossless, even 4 points increased in the HumalEval\. When combined with Parallel, it accelerates the backbone by 11\.3%\-42\.9%, reaching Vanilla’s3\.7X\-12\.3Xoverall\. When combined with DPad, it achieved a 10\.5%\-30\.8% acceleration on benchmarks other than humanEval, and a 19% acceleration on gsm8k when combined with both Parallel and DPad\.

In addition, the reason why there is slight acceleration on Humaneval is that the length of prompt is short which results in short length of total input\. As shown in Tab\.[8](https://arxiv.org/html/2607.16339#A10.T8), we further evaluate with longer generation length 3\.7% speedup over the results a 3\.7% speedup over the vanilla backbone, 11\.5% speedup over the parallel backbone, and 8\.3% speedup over the DPad\.

Overall, combined with Parallel\(fast\-DLLM\), LaCache can achieve the fastest speed and accuracy without loss, while the speed is slightly inferior when further combined with DPad, probably because the DPad removes more suffix tokens, resulting in shorter gen tokens and less pronounced acceleration\.

![Refer to caption](https://arxiv.org/html/2607.16339v1/x4.png)Figure 7:The response examples of the backbone and LaCache\(Ours\) generated on different datasets\. A\. LaCache’s answers are basically the same as backbone in most scenarios, except that the low\-precision strategy makes the model slightly tend to generate the end symbol later, so that the correct format of the answer can be output more completely\. B\. In the scenario of long text generation, when answering difficult code questions using the enumeration method, LaCache tends to generate more enum content, so that the answer is closer to the correct answer\.
#### 4\.2\.2Throughput

LaCache achieved the best speed in this metric with 10\.1%\-42\.5% faster than Vanilla; When combined with parallel, it accelerates by 22\.3%\-43\.0%, reaching3\.7X\-12\.2Xof Vanilla\. When combined with DPad, it accelerates by 12\.0%\-31\.7%; When combined with DPad\+parallel, it is accelerated by 4\.1%\-20\.3%\.

#### 4\.2\.3Accuracy

LaCache has basically no loss of accuracy on all datasets, and there is a slight improvement in the strict\-match score\. This shows that the low\-precision quantization of the per\-group does not bring negative losses to the model\.

Table 3:Ablation results of the caching strategy alone\.EfficiencyBenchmarkMethodLatency\(s\)TPSVanilla10\.5919\.78GSM8K\(4\-shot\)LaCache\(w/o fp8\)10\.5419\.86LaCache9\.621\.78Vanilla9\.9321\.11MATH\(4\-shot\)LaCache\(w/o fp8\)9\.0923\.04LaCache6\.9730\.08Vanilla13\.47\.93HumanEval\(0\-shot\)LaCache\(w/o fp8\)138LaCache12\.188\.69Vanilla24\.072\.68MBPP\(3\-shot\)LaCache\(w/o fp8\)23\.92\.7LaCache21\.512\.95Table 4:Comprehensive benchmark results of LLaDA\-Instruct with LaCache over four tasks\.
#### 4\.2\.4Performance on other dllms

On Llada\-base, LaCache is 20%\-30% faster than the corresponding backbone on GSM8K, MATH, and MBPP datasets, and the final speed reaches2\.3X\-5\.9Xof vanilla\. On LLaDA\-1\.5, it is 10%\-20% faster than the corresponding backbone on GSM8K, MATH, and MBPP datasets, and the final speed reaches3\.3X\-12\.3Xof vanilla\.

Meanwhile, as shown in Fig\.[7](https://arxiv.org/html/2607.16339#S4.F7), LaCache is able to generate longer enum answers when faced with difficult questions, thus getting closer to the correct answer than backbones\.

#### 4\.2\.5Comparison with different quantization methods

As shown in Tab\.[1](https://arxiv.org/html/2607.16339#S4.T1)and Tab\.[2](https://arxiv.org/html/2607.16339#S4.T2), the per\-tile fp8 quantization\(LaCache\) achieves higher accuracy compared with other granularities and higher efficiency than prior methods such as DuQuantLinet al\.\([2024](https://arxiv.org/html/2607.16339#bib.bib47)\)\.

#### 4\.2\.6Ablation Study

We first evaluate the effect of applying caching alone\. As shown in Tab\.[3](https://arxiv.org/html/2607.16339#S4.T3.fig1), caching achieves approximately 3% speedup over the vanilla backbone across all benchmarks, compensating for the inability to apply mixed\-precision acceleration to the first layer\.

We further compare the accuracy of different quantization granularities\. As shown in Tab\.[1](https://arxiv.org/html/2607.16339#S4.T1), the accuracy loss grows as the granularity becomes coarser, indicating that per\-tile quantization is more suitable for DLLMs than the other two methods\.

## 5Conclusion

We presentLaCache, an inference acceleration framework designed to eliminate operator\-level redundancy in DLLMs\. LaCache introducesLossless State Memoization \(LSM\)to reuse invariant intermediate states in token\-wise operators and first\-layer FlashAttention, complemented by a fine\-grained FP8 mixed\-precision strategy for compute\-bound linear layers\. Experiments demonstrate that LaCache consistently achieves lossless acceleration across multiple DLLM architectures and benchmarks\. Furthermore, it is fully composable with existing schedule\-level optimization methods, delivering 3–12×\\timesspeedups on standard sequences and up to 40\.2×\\timeson long contexts\. LaCache provides a general, training\-free, and complementary solution for efficient DLLM deployment without compromising generation quality\.

## Limitations

Although the proposed LaCache achieves significant acceleration, certain limitations remain\. First, lossless caching is theoretically restricted to the first layer of the model\. Due to the bidirectional propagation of hidden states in subsequent layers, implementing caching mechanisms deeper in the network necessitates approximate methods, which may introduce slight degradations in generation quality\. Second, from a hardware perspective, the mixed\-precision strategy depends heavily on modern accelerators equipped with native support for low\-precision formats, such as FP8\. This reliance restricts the direct deployment of the framework on older hardware architectures that lack optimized matrix calculation engines\.

## Acknowledgments

## References

- J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le,et al\.\(2021\)Program synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.Cited by:[§4\.1](https://arxiv.org/html/2607.16339#S4.SS1.p5.1)\.
- T\. Brown, B\. Mann, N\. Ryder, M\. Subbiah, J\. D\. Kaplan, P\. Dhariwal, A\. Neelakantan, P\. Shyam, G\. Sastry, A\. Askell,et al\.\(2020\)Language models are few\-shot learners\.Advances in neural information processing systems33,pp\. 1877–1901\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p1.1)\.
- M\. Chen \(2021\)Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[§4\.1](https://arxiv.org/html/2607.16339#S4.SS1.p5.1)\.
- X\. Chen, S\. Huang, C\. Guo, C\. Wei, Y\. He, J\. Zhang, H\. Li, Y\. Chen,et al\.\(2025\)DPad: efficient diffusion language models with suffix dropout\.arXiv preprint arXiv:2508\.14148\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p3.1),[§2](https://arxiv.org/html/2607.16339#S2.p1.1),[§4\.1](https://arxiv.org/html/2607.16339#S4.SS1.p4.1)\.
- K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano,et al\.\(2021\)Training verifiers to solve math word problems\.arXiv preprint arXiv:2110\.14168\.Cited by:[§4\.1](https://arxiv.org/html/2607.16339#S4.SS1.p5.1)\.
- Y\. Dong, C\. F\. Ruan, Y\. Cai, Z\. Xu, Y\. Zhao, R\. Lai, and T\. Chen \(2025\)Xgrammar: flexible and efficient structured generation engine for large language models\.Proceedings of Machine Learning and Systems7\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p1.1)\.
- A\. Dubey, A\. Jauhri, A\. Pandey, A\. Kadian, A\. Al\-Dahle, A\. Letman, A\. Mathur, A\. Schelten, A\. Yang, A\. Fan,et al\.\(2024\)The llama 3 herd of models\.arXiv e\-prints,pp\. arXiv–2407\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p1.1)\.
- H\. Feng, Y\. Ma, D\. Di, L\. Fan, T\. Su, and X\. Wu \(2025\)DiTalker: a unified dit\-based framework for high\-quality and speaking styles controllable portrait animation\.arXiv preprint arXiv:2508\.06511\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p1.1)\.
- H\. Feng, Y\. Ma, D\. Di, L\. Fan, and T\. Su \(2026\)CogPortrait: fine\-grained eye\-region control in portrait animation via hierarchical agent planning\.External Links:2605\.28056,[Link](https://arxiv.org/abs/2605.28056)Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p1.1)\.
- W\. Guo, M\. Mishra, X\. Cheng, I\. Stoica, and T\. Dao \(2025\)SonicMoE: accelerating moe with io and tile\-aware optimizations\.arXiv preprint arXiv:2512\.14080\.Cited by:[§3\.4](https://arxiv.org/html/2607.16339#S3.SS4.p2.1)\.
- D\. Hendrycks, C\. Burns, S\. Kadavath, A\. Arora, S\. Basart, E\. Tang, D\. Song, and J\. Steinhardt \(2021\)Measuring mathematical problem solving with the math dataset\.arXiv preprint arXiv:2103\.03874\.Cited by:[§4\.1](https://arxiv.org/html/2607.16339#S4.SS1.p5.1)\.
- J\. Jiang, F\. Wang, J\. Shen, S\. Kim, and S\. Kim \(2024\)A survey on large language models for code generation\.ACM Transactions on Software Engineering and Methodology\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p1.1)\.
- Y\. Jiang, Y\. Cai, X\. Luo, J\. Fu, J\. Wang, C\. Liu, and X\. Yang \(2025\)D2cache: accelerating diffusion\-based llms via dual adaptive caching\.arXiv preprint arXiv:2509\.23094\.Cited by:[§2](https://arxiv.org/html/2607.16339#S2.p1.1)\.
- D\. Kalamkar, D\. Mudigere, N\. Mellempudi, D\. Das, K\. Banerjee, S\. Avancha, D\. T\. Vooturi, N\. Jammalamadaka, J\. Huang, H\. Yuen,et al\.\(2019\)A study of bfloat16 for deep learning training\.arXiv preprint arXiv:1905\.12322\.Cited by:[§3\.2\.1](https://arxiv.org/html/2607.16339#S3.SS2.SSS1.p2.1)\.
- S\. Khanna, S\. Kharbanda, S\. Li, H\. Varma, E\. Wang, S\. Birnbaum, Z\. Luo, Y\. Miraoui, A\. Palrecha, S\. Ermon,et al\.\(2025\)Mercury: ultra\-fast language models based on diffusion\.arXiv preprint arXiv:2506\.172981\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p2.1)\.
- D\. D\. Lee, P\. Pham, Y\. Largman, and A\. Ng \(2009\)Advances in neural information processing systems 22\.Tech Rep\.Cited by:[§3\.2\.1](https://arxiv.org/html/2607.16339#S3.SS2.SSS1.p2.1)\.
- H\. Lin, H\. Xu, Y\. Wu, J\. Cui, Y\. Zhang, L\. Mou, L\. Song, Z\. Sun, and Y\. Wei \(2024\)Duquant: distributing outliers via dual transformation makes stronger quantized llms\.Advances in Neural Information Processing Systems37,pp\. 87766–87800\.Cited by:[§3\.4](https://arxiv.org/html/2607.16339#S3.SS4.p1.8),[§4\.2\.5](https://arxiv.org/html/2607.16339#S4.SS2.SSS5.p1.1)\.
- A\. Liu, B\. Feng, B\. Xue, B\. Wang, B\. Wu, C\. Lu, C\. Zhao, C\. Deng, C\. Zhang, C\. Ruan,et al\.\(2024\)Deepseek\-v3 technical report\.arXiv preprint arXiv:2412\.19437\.Cited by:[§2](https://arxiv.org/html/2607.16339#S2.p2.1),[§3\.2\.1](https://arxiv.org/html/2607.16339#S3.SS2.SSS1.p2.1)\.
- Y\. Liu, W\. Chen, Y\. Bai, X\. Liang, G\. Li, W\. Gao, and L\. Lin \(2025a\)Aligning cyber space with physical world: a comprehensive survey on embodied ai\.IEEE/ASME Transactions on Mechatronics\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p1.1)\.
- Z\. Liu, Y\. Yang, Y\. Zhang, J\. Chen, C\. Zou, Q\. Wei, S\. Wang, and L\. Zhang \(2025b\)Dllm\-cache: accelerating diffusion large language models with adaptive caching\.arXiv preprint arXiv:2506\.06295\.Cited by:[§2](https://arxiv.org/html/2607.16339#S2.p1.1)\.
- A\. Lou, C\. Meng, and S\. Ermon \(2023\)Discrete diffusion language modeling by estimating the ratios of the data distribution\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p1.1)\.
- C\. Luo, D\. Di, X\. Yang, Y\. Ma, Z\. Xue, C\. Wei, and Y\. Liu \(2024\)TrAME: trajectory\-anchored multi\-view editing for text\-guided 3d gaussian splatting manipulation\.External Links:2407\.02034,[Link](https://arxiv.org/abs/2407.02034)Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p2.1)\.
- O\. Luxembourg, H\. Permuter, and E\. Nachmani \(2025\)Plan for speed–dilated scheduling for masked diffusion language models\.arXiv preprint arXiv:2506\.19037\.Cited by:[§3\.4](https://arxiv.org/html/2607.16339#S3.SS4.p1.8)\.
- H\. Lyu, N\. Sha, S\. Qin, M\. Yan, Y\. Xie, and R\. Wang \(2019\)Advances in neural information processing systems\.Advances in neural information processing systems32\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p2.1)\.
- Y\. Ma, J\. Chen, D\. Di, Q\. Xie, L\. Fan, W\. Chen, N\. Zhao, and X\. Yang \(2026\)Tuning\-free long video generation via global\-local collaborative diffusion\.ACM Trans\. Multimedia Comput\. Commun\. Appl\.22\(7\)\.External Links:ISSN 1551\-6857,[Link](https://doi.org/10.1145/3794855),[Document](https://dx.doi.org/10.1145/3794855)Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p1.1)\.
- Y\. Ma, D\. Di, X\. Liu, X\. Chen, L\. Fan, W\. Chen, and T\. Su \(2025\)Adams bashforth moulton solver for inversion and editing in rectified flow\.arXiv preprint arXiv:2503\.16522\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p2.1)\.
- Y\. Ma, B\. Dou, T\. Zhang, and Z\. Yuan \(2024\)RD\-nerf: neural robust distilled feature fields for sparse\-view scene segmentation\.InICASSP 2024 \- 2024 IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\),Vol\.,pp\. 3470–3474\.External Links:[Document](https://dx.doi.org/10.1109/ICASSP48485.2024.10447068)Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p2.1)\.
- M\. Mao, P\. Ting, Y\. Xiang, M\. Xu, J\. Chen, and J\. Lin \(2024\)Multi\-user chat assistant \(muca\): a framework using llms to facilitate group conversations\.arXiv preprint arXiv:2401\.04883\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p1.1)\.
- P\. Micikevicius, S\. Narang, J\. Alben, G\. Diamos, E\. Elsen, D\. Garcia, B\. Ginsburg, M\. Houston, O\. Kuchaiev, G\. Venkatesh,et al\.\(2017\)Mixed precision training\.arXiv preprint arXiv:1710\.03740\.Cited by:[§3\.2\.1](https://arxiv.org/html/2607.16339#S3.SS2.SSS1.p2.1)\.
- S\. Nie, F\. Zhu, C\. Du, T\. Pang, Q\. Liu, G\. Zeng, M\. Lin, and C\. Li \(2024\)Scaling up masked diffusion models on text\.arXiv preprint arXiv:2410\.18514\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p3.1)\.
- S\. Nie, F\. Zhu, Z\. You, X\. Zhang, J\. Ou, J\. Hu, J\. Zhou, Y\. Lin, J\. Wen, and C\. Li \(2025\)Large language diffusion models\.arXiv preprint arXiv:2502\.09992\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p2.1),[§4\.1](https://arxiv.org/html/2607.16339#S4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2607.16339#S4.SS1.p2.1)\.
- X\. Sun, M\. Li, Z\. Zheng, J\. Chen, H\. Xu, Y\. Liang, and X\. Chen \(2025\)STDD: spatio\-temporal dynamics\-driven token refinement in diffusion language models\.arXiv preprint arXiv:2601\.04205\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p3.1),[§2](https://arxiv.org/html/2607.16339#S2.p1.1)\.
- A\. Vaswani, N\. Shazeer, N\. Parmar, J\. Uszkoreit, L\. Jones, A\. N\. Gomez, Ł\. Kaiser, and I\. Polosukhin \(2017\)Attention is all you need\.Advances in neural information processing systems30\.Cited by:[§B\.2](https://arxiv.org/html/2607.16339#A2.SS2.p1.8)\.
- Y\. Wan, W\. Wang, Y\. Yang, Y\. Yuan, J\. Huang, P\. He, W\. Jiao, and M\. Lyu \(2024\)LogicAsker: evaluating and improving the logical reasoning ability of large language models\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 2124–2155\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p1.1)\.
- L\. Wei, W\. Chen, P\. Tang, X\. Guo, L\. Ye, R\. Wang, and M\. Li \(2025a\)Orchestrating dual\-boundaries: an arithmetic intensity inspired acceleration framework for diffusion language models\.arXiv preprint arXiv:2511\.21759\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p3.1),[§2](https://arxiv.org/html/2607.16339#S2.p1.1)\.
- Q\. Wei, Y\. Zhang, Z\. Liu, D\. Liu, and L\. Zhang \(2025b\)Accelerating diffusion large language models with slowfast: the three golden principles\.arXiv preprint arXiv:2506\.10848\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p3.1)\.
- C\. Wu, H\. Zhang, S\. Xue, Z\. Liu, S\. Diao, L\. Zhu, P\. Luo, S\. Han, and E\. Xie \(2025\)Fast\-dllm: training\-free acceleration of diffusion llm by enabling kv cache and parallel decoding\.arXiv preprint arXiv:2505\.22618\.Cited by:[Appendix F](https://arxiv.org/html/2607.16339#A6.p1.1),[§1](https://arxiv.org/html/2607.16339#S1.p2.1),[§1](https://arxiv.org/html/2607.16339#S1.p3.1),[§2](https://arxiv.org/html/2607.16339#S2.p1.1),[§4\.1](https://arxiv.org/html/2607.16339#S4.SS1.p3.1)\.
- H\. Xi, H\. Cai, L\. Zhu, Y\. Lu, K\. Keutzer, J\. Chen, and S\. Han \(2024\)Coat: compressing optimizer states and activation for memory\-efficient fp8 training\.arXiv preprint arXiv:2410\.19313\.Cited by:[§2](https://arxiv.org/html/2607.16339#S2.p2.1)\.
- G\. Xiao, J\. Lin, M\. Seznec, H\. Wu, J\. Demouth, and S\. Han \(2023\)Smoothquant: accurate and efficient post\-training quantization for large language models\.InInternational conference on machine learning,pp\. 38087–38099\.Cited by:[§2](https://arxiv.org/html/2607.16339#S2.p2.1)\.
- Q\. Xie, Y\. Ma, D\. Di, X\. Gao, and X\. Yang \(2025\)MoCA: identity\-preserving text\-to\-video generation via mixture of cross attention\.InProceedings of the 7th ACM International Conference on Multimedia in Asia,MMAsia ’25,New York, NY, USA\.External Links:ISBN 9798400720055,[Link](https://doi.org/10.1145/3743093.3770948),[Document](https://dx.doi.org/10.1145/3743093.3770948)Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p2.1)\.
- A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv,et al\.\(2025a\)Qwen3 technical report\.arXiv preprint arXiv:2505\.09388\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p1.1)\.
- J\. Yang, D\. Di, B\. Ma, J\. Cui, X\. Yang, Y\. Ma, W\. Sun, W\. Chen, Z\. Xue, M\. Wang, and Y\. Liu \(2025b\)TV\-3dg: mastering text\-to\-3d customized generation with visual prompt\.IEEE Transactions on Pattern Analysis and Machine Intelligence\(\),pp\. 1–18\.External Links:[Document](https://dx.doi.org/10.1109/TPAMI.2025.3587105)Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p2.1)\.
- J\. Yang, Y\. Ma, D\. Di, J\. Cui, H\. Li, W\. Chen, Y\. Xie, X\. Yang, and W\. Zuo \(2025c\)Qr\-lora: efficient and disentangled fine\-tuning via qr decomposition for customized generation\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 17587–17597\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p1.1)\.
- J\. Ye, Z\. Xie, L\. Zheng, J\. Gao, Z\. Wu, X\. Jiang, Z\. Li, and L\. Kong \(2025\)Dream 7b: diffusion large language models\.arXiv preprint arXiv:2508\.15487\.Cited by:[§1](https://arxiv.org/html/2607.16339#S1.p2.1)\.
- Y\. Zhang, H\. Zhen, M\. Yuan, and B\. Yu \(2025\)MOSS: efficient and accurate fp8 llm training with microscaling and automatic scaling\.arXiv preprint arXiv:2511\.05811\.Cited by:[§2](https://arxiv.org/html/2607.16339#S2.p2.1)\.
- F\. Zhu, R\. Wang, S\. Nie, X\. Zhang, C\. Wu, J\. Hu, J\. Zhou, J\. Chen, Y\. Lin, J\. Wen,et al\.\(2025\)LLaDA 1\.5: variance\-reduced preference optimization for large language diffusion models\.arXiv preprint arXiv:2505\.19223\.Cited by:[§4\.1](https://arxiv.org/html/2607.16339#S4.SS1.p1.1)\.

## Appendix

## Appendix AImpact Statement

This work proposes a method to improve the efficiency of diffusion large language model \(DLLM\) inference\. By reducing computational costs, our approach can lower the barrier to deploying and experimenting with large\-scale models, potentially benefiting research accessibility and real\-world applications such as dialogue systems, code generation, and reasoning tasks\.

However, like other advances in DLLM efficiency, this work may also contribute to broader adoption of powerful language models, which raises concerns regarding misuse, biased outputs, and the amplification of harmful or misleading content\. These risks are not specific to our method but are inherent to the deployment of diffusion large\-scale language models in general\.

Our work focuses on improving the computational efficiency of existing models and does not introduce new model capabilities or training data\. As such, it does not directly increase the risk of misuse beyond what is already present in current DLLM deployments\. Responsible use, proper evaluation, and alignment techniques remain essential for mitigating potential negative impacts\.

Overall, we believe that the benefits of improved efficiency—such as reduced computational cost and energy consumption—outweigh the potential risks, provided that the technology is applied responsibly\.

## Appendix BPreliminary

### B\.1the Inference and Sampling in DLLM

DLLM adopts a semi\-autoregressive \(SAR\) inference paradigm that balances the latency advantages of parallel decoding with the quality and stability of autoregressive generation\. Instead of generating one token at a time or decoding the entire sequence in a fully parallel manner, SAR inference produces tokens in blocks, allowing partial parallelism while preserving causal dependencies across blocks\. Given an input promptx=\(x1,…,xt0\)x=\(x\_\{1\},…,x\_\{t\_\{0\}\}\), DLLM generates the output sequence by partitioning the decoding process into blocks of size B\. At inference time, the model predicts future tokens simultaneously within each decoding step, yielding candidate tokens\(y\(t\+1\),…,y\(t\+B\)\)\(y\_\{\(t\+1\)\},…,y\_\{\(t\+B\)\}\)\. This block\-wise generation reduces the total number of decoding iterations by a factor of B compared to fully autoregressive decoding\. Importantly, tokens within the same block do not condition on one another; instead, the model relies on a shared contextual representation to predict all tokens jointly\. This structure preserves strict causality across blocks while relaxing token\-level dependencies within each block, leading to a semi\-autoregressive dependency pattern\.

### B\.2the Foundational Principles of Flash\-attention

Self\-attentionVaswaniet al\.\([2017](https://arxiv.org/html/2607.16339#bib.bib17)\)has*quadratic*compute and memory cost in the sequence length\. FlashAttention reduces the memory overhead by computing attention in tiles and avoiding materializing the full attention score matrix in HBM\. Concretely, the sequence dimensionNNis divided into row blocks of sizeBrB\_\{r\}and column blocks of sizeBcB\_\{c\}\. Queries are partitioned intoTr=⌈N/Br⌉T\_\{r\}=\\lceil N/B\_\{r\}\\rceilblocksQ1,…,QTrQ\_\{1\},\\ldots,Q\_\{T\_\{r\}\}, while keys and values are partitioned intoTc=⌈N/Bc⌉T\_\{c\}=\\lceil N/B\_\{c\}\\rceilblocksK1,…,KTcK\_\{1\},\\ldots,K\_\{T\_\{c\}\}andV1,…,VTcV\_\{1\},\\ldots,V\_\{T\_\{c\}\}\.

For a query blockQiQ\_\{i\}, FlashAttention streams over all key–value blocks\(Kj,Vj\)\(K\_\{j\},V\_\{j\}\), computing block scores on\-chip:

Si\(j\)=Qi​Kj⊤∈ℝBr×Bc\.S\_\{i\}^\{\(j\)\}=Q\_\{i\}K\_\{j\}^\{\\top\}\\in\\mathbb\{R\}^\{B\_\{r\}\\times B\_\{c\}\}\.\(9\)To compute the softmax without storing allSi\(j\)S\_\{i\}^\{\(j\)\}, FlashAttention uses an online \(log\-sum\-exp\) formulation that maintains per\-row*state variables*: a running maximummm, a normalizerℓ\\ell, and an ab\-normalized output accumulatorO~\\tilde\{O\}\. After processing all key–value blocks, the output is obtained by row\-wise normalizationO=O~/ℓO=\\tilde\{O\}/\\ell\. We will reuse\(m,ℓ,O~\)\(m,\\ell,\\tilde\{O\}\)as the cached attention state in Sec\.[3\.3\.3](https://arxiv.org/html/2607.16339#S3.SS3.SSS3)\.

## Appendix CThe Speed\-Accuracy Trade\-off of only Caching Strategy on more layers

As shown in Fig\.[8](https://arxiv.org/html/2607.16339#A4.F8), when applying the caching strategy across multiple layers \(layers 0 and 3–17 of LLaDA\-Instruct\), a speed\-accuracy trade\-off emerges, governed by the cache update interval\. As the interval increases, inference accelerates at the cost of greater accuracy degradation\. We empirically identify a configuration that effectively balances efficiency and accuracy\.

## Appendix DThe acceleration of LaCache on LLaDA\-base on multiple benchmarks

The effect of LaCache on the LLaDA\-base model among multiple datasets is presented in Tab\.[6](https://arxiv.org/html/2607.16339#A10.T6), achieving 10%\-30% acceleration on datasets other than HumanEval, and the effect is basically lossless\. In addition, the reason for the lack of acceleration on the HumanEval dataset is that the overall prompt length of the dataset is short, resulting in insignificant acceleration\.

![Refer to caption](https://arxiv.org/html/2607.16339v1/fig/pareto_curve_tps.png)Figure 8:The accuracy\-speedup curve of caching strategy applied to multiple layers\. As shown in the figure, as the cache update interval increases, inference speed improves while accuracy degrades more significantly\.
## Appendix EThe acceleration of LaCache on LLaDA\-1\.5 on multiple benchmarks

The acceleration performance of LaCache on the LLaDA\-1\.5 model among multiple datasets is presented in Tab\.[7](https://arxiv.org/html/2607.16339#A10.T7), achieving 10%\-30% acceleration on datasets other than HumanEval, and the effect is basically lossless\. In addition, the reason for the lack of acceleration on the HumanEval dataset is that the overall prompt length of the dataset is short, resulting in insignificant acceleration\.

## Appendix FThe acceleration of LaCache on fast\-dllm\-v2 on multiple benchmarks

As shown in Tab\.[9](https://arxiv.org/html/2607.16339#A10.T9), FACache along could contribute an additional 6% speedup to Fast\-dLLM\-V2Wuet al\.\([2025](https://arxiv.org/html/2607.16339#bib.bib4)\), demonstrating the complementary acceleration potential of LaCache\.

## Appendix GThe acceleration of LaCache on LLaDA\-Instruct on more benchmarks and more type of GPU

As shown in Tab\.[5](https://arxiv.org/html/2607.16339#A7.T5)and Tab\.[10](https://arxiv.org/html/2607.16339#A10.T10), LaCache maintains competitive accuracy across a broader range of benchmarks and consistent performance on H100 GPU\.

Table 5:Additional benchmark results of LLaDA\-Instruct with LaCache\.Accuracy\(%\)BenchmarkMethodFlexibleStrictVanilla73\.673\.3piqa\(4\-shot\)\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D173\.5\\cellcolor\[HTML\]FAF1D173\.3Vanilla78\.776\.6arc\_easy\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D179\\cellcolor\[HTML\]FAF1D176\.2Vanilla5054\.5arc\_chanllenge\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D149\.53\\cellcolor\[HTML\]FAF1D154\.6Vanilla50\.7967\.46hellaswag\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D150\.77\\cellcolor\[HTML\]FAF1D167\.43Vanilla68\.75\-winogrande\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D169\.06\\cellcolor\[HTML\]FAF1D1\-Vanilla27\.2327\.23gpqa\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D130\.8\\cellcolor\[HTML\]FAF1D130\.8
## Appendix HAblation of sequence length

As shown in Tab\.[8](https://arxiv.org/html/2607.16339#A10.T8), the acceleration performance of LaCache on the HumanEval dataset in the scenario of long text generation is presented in Tab\.[Table˜7](https://arxiv.org/html/2607.16339#A10.T7), and shows barely loss\.

## Appendix IThe answer example of LaCache compared with DPad

As shown in Fig\.[9](https://arxiv.org/html/2607.16339#A10.F9), although DPad makes the answer shorter, LaCache will make the answer longer, so as to ensure that the comments of the code are fully generated\.

## Appendix JThe pseudocode of FACache

As shown below, the FACache is updated in the first generation step and reused in later steps to reduce the redundancy in FlashAttention\.

Table 6:Comprehensive benchmark results of LLaDA\-base with LaCache over four tasks\.LLaDA\-baseEfficiencyAccuracy\(%\)BenchmarkMethodLatency\(s\)TPSGen\.LengthFlexibleStrictVanilla10\.5914\.7415662\.3269\.67\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D19\.66\\cellcolor\[HTML\]FAF1D115\.82\\cellcolor\[HTML\]FAF1D1153\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D164\.75\\cellcolor\[HTML\]FAF1D169\.75\+Parallel\(Fast\-dLLM\)5\.9426\.4215762\.9371\.04\\cellcolor\[HTML\]FAF1D1\+Parallel\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D14\.97\\cellcolor\[HTML\]FAF1D130\.72\\cellcolor\[HTML\]FAF1D1153\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D165\.81\\cellcolor\[HTML\]FAF1D170\.89\+DPad7\.4717\.2512969\.8371\.57\\cellcolor\[HTML\]FAF1D1\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D16\.7\\cellcolor\[HTML\]FAF1D119\.26\\cellcolor\[HTML\]FAF1D1129\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D170\.66\\cellcolor\[HTML\]FAF1D171\.04\+Parallel\+DPad3\.8533\.5712968\.9271\.04GSM8K\(4\-shot\)\\cellcolor\[HTML\]FAF1D1\+Parallel\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D13\.11\\cellcolor\[HTML\]FAF1D141\.54\\cellcolor\[HTML\]FAF1D1129\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D169\.8370\.51Vanilla9\.1416\.6915330\.127\.54\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D18\.17\\cellcolor\[HTML\]FAF1D1 18\.77\\cellcolor\[HTML\]FAF1D1153\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D130\.48\\cellcolor\[HTML\]FAF1D1 27\.66\+Parallel\(Fast\-dLLM\)4\.9830\.6715330\.227\.66\\cellcolor\[HTML\]FAF1D1\+Parallel\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D14\.1\\cellcolor\[HTML\]FAF1D137\.47\\cellcolor\[HTML\]FAF1D1153\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D130\.56\\cellcolor\[HTML\]FAF1D127\.78\+DPad8\.8417\.0515130\.0627\.38\\cellcolor\[HTML\]FAF1D1\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D17\.92\\cellcolor\[HTML\]FAF1D119\.14\\cellcolor\[HTML\]FAF1D1152\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D130\\cellcolor\[HTML\]FAF1D127\.52\+Parallel\+DPad4\.7431\.815130\.127\.42MATH\(4\-shot\)\\cellcolor\[HTML\]FAF1D1\+Parallel\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D13\.91\\cellcolor\[HTML\]FAF1D138\.82\\cellcolor\[HTML\]FAF1D1152\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D129\.94\\cellcolor\[HTML\]FAF1D127\.54Vanilla13\.558\.8912033\.54\-\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D114\.45\\cellcolor\[HTML\]FAF1D17\.44\\cellcolor\[HTML\]FAF1D1108\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D132\.93\\cellcolor\[HTML\]FAF1D1 \-\+Parallel\(Fast\-dLLM\)4\.1728\.8712033\.54\-\\cellcolor\[HTML\]FAF1D1\+Parallel\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D13\.84\\cellcolor\[HTML\]FAF1D128\.02\\cellcolor\[HTML\]FAF1D1108\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D132\.93\\cellcolor\[HTML\]FAF1D1\-\+DPad9\.969\.759732\.32\-\\cellcolor\[HTML\]FAF1D1\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D111\.92\\cellcolor\[HTML\]FAF1D18\.6\\cellcolor\[HTML\]FAF1D1102\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D134\.76\\cellcolor\[HTML\]FAF1D1 \-\+Parallel\+DPad2\.3641\.129732\.32\-HumanEval\(0\-shot\)\\cellcolor\[HTML\]FAF1D1\+Parallel\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D12\.8\\cellcolor\[HTML\]FAF1D136\.73\\cellcolor\[HTML\]FAF1D1103\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D135\.67\\cellcolor\[HTML\]FAF1D1\-Vanilla23\.992\.646340\-\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D121\.51\\cellcolor\[HTML\]FAF1D12\.91\\cellcolor\[HTML\]FAF1D163\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D139\\cellcolor\[HTML\]FAF1D1 \-\+Parallel\(Fast\-dLLM\)7\.328\.636340\.4\-\\cellcolor\[HTML\]FAF1D1\+Parallel\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D15\.96\\cellcolor\[HTML\]FAF1D110\.5\\cellcolor\[HTML\]FAF1D163\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D138\.6\\cellcolor\[HTML\]FAF1D1\-\+DPad21\.112\.856040\.6\-\\cellcolor\[HTML\]FAF1D1\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D118\.76\\cellcolor\[HTML\]FAF1D13\.23\\cellcolor\[HTML\]FAF1D161\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D140\\cellcolor\[HTML\]FAF1D1 \-\+Parallel\+DPad4\.5613\.176040\.6\-MBPP\(3\-shot\)\\cellcolor\[HTML\]FAF1D1\+Parallel\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D14\.09\\cellcolor\[HTML\]FAF1D114\.82\\cellcolor\[HTML\]FAF1D161\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D140\\cellcolor\[HTML\]FAF1D1\-Table 7:Comprehensive benchmark results of LLaDA\-1\.5 with LaCache over four tasks\.LLaDA\-1\.5EfficiencyAccuracy\(%\)BenchmarkMethodLatency\(s\)TPSGen\.LengthFlexibleStrictVanilla10\.620\.0621380\.4463\.38\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D19\.65\\cellcolor\[HTML\]FAF1D122\.22\\cellcolor\[HTML\]FAF1D1215\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D179\.83\\cellcolor\[HTML\]FAF1D162\.47\+Parallel\(Fast\-dLLM\)3\.1369\.7321581\.563\.53\\cellcolor\[HTML\]FAF1D1\+Parallel\+LaCache\\cellcolor\[HTML\]FAF1D12\.89\\cellcolor\[HTML\]FAF1D175\.09\\cellcolor\[HTML\]FAF1D1217\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D180\.21\\cellcolor\[HTML\]FAF1D160\.8\+DPad6\.4522\.6314680\.6780\.14\\cellcolor\[HTML\]FAF1D1\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D15\.76\\cellcolor\[HTML\]FAF1D125\.35\\cellcolor\[HTML\]FAF1D1146\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D181\.05\\cellcolor\[HTML\]FAF1D179\.98\+Parallel\+DPad2\.2664\.5514681\.881\.05GSM8K\(4\-shot\)\\cellcolor\[HTML\]FAF1D1\+Parallel\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D12\.05\\cellcolor\[HTML\]FAF1D171\.37\\cellcolor\[HTML\]FAF1D1146\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D182\.26\\cellcolor\[HTML\]FAF1D181\.2Vanilla9\.0924\.0121832\.933\.66\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D18\.18\\cellcolor\[HTML\]FAF1D126\.7\\cellcolor\[HTML\]FAF1D1218\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D133\.66\\cellcolor\[HTML\]FAF1D134\.04\+Parallel\(Fast\-dLLM\)3\.4862\.6321832\.7833\.62\\cellcolor\[HTML\]FAF1D1\+Parallel\+LaCache\\cellcolor\[HTML\]FAF1D13\.15\\cellcolor\[HTML\]FAF1D169\.4\\cellcolor\[HTML\]FAF1D1218\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D133\.58\\cellcolor\[HTML\]FAF1D133\.82\+DPad6\.9627\.3319036\.6633\.64\\cellcolor\[HTML\]FAF1D1\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D16\.27\\cellcolor\[HTML\]FAF1D130\.33\\cellcolor\[HTML\]FAF1D1190\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D136\.38\\cellcolor\[HTML\]FAF1D133\.56\+Parallel\+DPad3\.0462\.3819036\.0433\.24MATH\(4\-shot\)\\cellcolor\[HTML\]FAF1D1\+Parallel\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D12\.79\\cellcolor\[HTML\]FAF1D168\.37\\cellcolor\[HTML\]FAF1D1191\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D136\.44\\cellcolor\[HTML\]FAF1D133\.6Vanilla13\.55810841\.46\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D114\.44\\cellcolor\[HTML\]FAF1D17\.31\\cellcolor\[HTML\]FAF1D1106\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D1 42\.07\\cellcolor\[HTML\]FAF1D1\+Parallel\(Fast\-dLLM\)4\.4124\.4810839\.63\\cellcolor\[HTML\]FAF1D1\+Parallel\+LaCache\\cellcolor\[HTML\]FAF1D14\.58\\cellcolor\[HTML\]FAF1D123\.47\\cellcolor\[HTML\]FAF1D1107\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D141\.46\\cellcolor\[HTML\]FAF1D1\+DPad10\.079\.669739\.63\\cellcolor\[HTML\]FAF1D1\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D111\.84\\cellcolor\[HTML\]FAF1D17\.79\\cellcolor\[HTML\]FAF1D192\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D139\.94\\cellcolor\[HTML\]FAF1D1\+Parallel\+DPad3\.924\.289541\.46HumanEval\(0\-shot\)\\cellcolor\[HTML\]FAF1D1\+Parallel\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D14\.6\\cellcolor\[HTML\]FAF1D119\.81\\cellcolor\[HTML\]FAF1D191\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D140\.55\\cellcolor\[HTML\]FAF1D1Vanilla24\.072\.636338\.8\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D121\.48\\cellcolor\[HTML\]FAF1D13\.11\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D167\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D138\.2\\cellcolor\[HTML\]FAF1D1\+Parallel\(Fast\-dLLM\)2\.1529\.436339\.4\\cellcolor\[HTML\]FAF1D1\+Parallel\+LaCache\\cellcolor\[HTML\]FAF1D1 1\.96\\cellcolor\[HTML\]FAF1D1 34\.03\\cellcolor\[HTML\]FAF1D1 67\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D1 38\\cellcolor\[HTML\]FAF1D1\+DPad7\.4412\.079041\.2\\cellcolor\[HTML\]FAF1D1\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D17\.52\\cellcolor\[HTML\]FAF1D112\.23\\cellcolor\[HTML\]FAF1D192\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D138\.4\\cellcolor\[HTML\]FAF1D1\+Parallel\+DPad2\.2140\.559040\.2MBPP\(3\-shot\)\\cellcolor\[HTML\]FAF1D1\+Parallel\+DPad\+LaCache\\cellcolor\[HTML\]FAF1D12\.26\\cellcolor\[HTML\]FAF1D140\.01\\cellcolor\[HTML\]FAF1D191\\cellcolor\[HTML\]FAF1D1\\cellcolor\[HTML\]FAF1D140\.2\\cellcolor\[HTML\]FAF1D1Table 8:Ablation study on LLaDA\-instruct under different context lengths\.Table 9:Comprehensive benchmark results of Fast\-dllm\-V2 with FACache only\.BenchmarkMethodEfficiencyAccuracy\(%\)Latency\(s\)TPSGen\.LengthFlexibleGSM8K\(4\-shot\),bsz=256Vanilla1\.06310\.1932882\.79\\cellcolor\[HTML\]FAF1D1\+LaCache\(FACache\)\\cellcolor\[HTML\]FAF1D11\\cellcolor\[HTML\]FAF1D1326\.99\\cellcolor\[HTML\]FAF1D1328\\cellcolor\[HTML\]FAF1D182\.79MATH\(4\-shot\),bsz=256Vanilla1\.31491\.5264359\.94\\cellcolor\[HTML\]FAF1D1\+LaCache\(FACache\)\\cellcolor\[HTML\]FAF1D11\.23\\cellcolor\[HTML\]FAF1D1523\.48\\cellcolor\[HTML\]FAF1D1643\\cellcolor\[HTML\]FAF1D159\.94MBPP\(3\-shot\),bsz=256Vanilla0\.9315\.5327031\.2\\cellcolor\[HTML\]FAF1D1\+LaCache\(FACache\)\\cellcolor\[HTML\]FAF1D10\.84\\cellcolor\[HTML\]FAF1D1336\.92\\cellcolor\[HTML\]FAF1D1270\\cellcolor\[HTML\]FAF1D131\.2HumanEval\(0\-shot\),bsz=256Vanilla1\.57269\.6836965\.24\\cellcolor\[HTML\]FAF1D1\+LaCache\(FACache\)\\cellcolor\[HTML\]FAF1D11\.45\\cellcolor\[HTML\]FAF1D1289\.59\\cellcolor\[HTML\]FAF1D1362\\cellcolor\[HTML\]FAF1D165\.24Table 10:Comprehensive results On H100 of LlaDA\-Instruct with LaCache\.EfficiencyAccuracy\(%\)BenchmarkMethodLatency\(s\)TPSGen\.LengthFlexibleStrict/acc\_normVanilla11\.0720\.8223076\.0435\.41GSM8K\(4\-shot\)\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D18\.41\\cellcolor\[HTML\]FAF1D127\.5\\cellcolor\[HTML\]FAF1D1231\\cellcolor\[HTML\]FAF1D175\.66\\cellcolor\[HTML\]FAF1D135\.56Vanilla\\cellcolor\[HTML\]FFFFFF9\.58\\cellcolor\[HTML\]FFFFFF25\.96\\cellcolor\[HTML\]FFFFFF2498\.538\.5MATH\(4\-shot\)\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D17\.93\\cellcolor\[HTML\]FAF1D131\.32\\cellcolor\[HTML\]FAF1D1248\\cellcolor\[HTML\]FAF1D17\.88\\cellcolor\[HTML\]FAF1D138\.82\\rowcolor\[HTML\]FFFFFF\\cellcolor\[HTML\]FFFFFFVanilla25\.111\.9129915\.58\\rowcolor\[HTML\]EEF6C6\\cellcolor\[HTML\]FFFFFFMBPP\(3\-shot\)\\cellcolor\[HTML\]FAF1D1\+LaCache\\cellcolor\[HTML\]FAF1D118\.01\\cellcolor\[HTML\]FAF1D116\.64\\cellcolor\[HTML\]FAF1D1300\\cellcolor\[HTML\]FAF1D114\.8\\cellcolor\[HTML\]FAF1D1Vanilla\-\-\-78\.776\.6Arc\_Easy\\cellcolor\[HTML\]FAF1D1\+LaCache\-\-\-\\cellcolor\[HTML\]FAF1D179\.46\\cellcolor\[HTML\]FAF1D176\.18Vanilla\-\-\-5054\.5Arc\_Chanllenge\\cellcolor\[HTML\]FAF1D1\+LaCache\-\-\-\\cellcolor\[HTML\]FAF1D151\.28\\cellcolor\[HTML\]FAF1D152\.99Vanilla\-\-\-69\.53\-Winogrande\\cellcolor\[HTML\]FAF1D1\+LaCache\-\-\-\\cellcolor\[HTML\]FAF1D169\.06\\cellcolor\[HTML\]FAF1D1\-Vanilla\-\-\-\\cellcolor\[HTML\]FAF1D128\.35\\cellcolor\[HTML\]FAF1D128\.35GPQA\\cellcolor\[HTML\]FAF1D1\+LaCache\-\-\-\\cellcolor\[HTML\]FAF1D130\.13\\cellcolor\[HTML\]FAF1D130\.13Algorithm 1the first layer’s Flash\-Attention1:Matrices

Q,K,V,O​c​a​c​h​e∈ℝN×d,ℓ​c​a​c​h​e,R​o​w​M​a​x​c​a​c​h​e∈ℝN×4​dQ,K,V,Ocache\\in\\mathbb\{R\}^\{N\\times d\},\\ell\{cache\},RowMaxcache\\in\\mathbb\{R\}^\{N\\times 4d\}in HBM, block sizes

Bc,BrB\_\{c\},B\_\{r\},

G​e​n​S​t​e​p∈\{T​r​u​e,F​a​l​s​e\}GenStep\\in\\\{True,False\\\}, where

G​e​n​S​t​e​p=T​r​u​eGenStep=Trueif it’s in the first generation step\.

2:Divide

QQinto

Tr=⌈N/Br⌉T\_\{r\}=\\lceil N/B\_\{r\}\\rceilblocks

Q1,…,QTrQ\_\{1\},\\ldots,Q\_\{T\_\{r\}\}of size

Br×dB\_\{r\}\\times d, divide

K,VK,Vinto

Tc=⌈N/Bc⌉T\_\{c\}=\\lceil N/B\_\{c\}\\rceilblocks

K1,…,KTcK\_\{1\},\\ldots,K\_\{T\_\{c\}\}and

V1,…,VTcV\_\{1\},\\ldots,V\_\{T\_\{c\}\}of size

Bc×dB\_\{c\}\\times d, divide the cache

O​c​a​c​h​e∈ℝN×dOcache\\in\\mathbb\{R\}^\{N\\times d\}into

TrT\_\{r\}blocks

O​c​a​c​h​e1,…,O​c​a​c​h​eTrOcache\_\{1\},\\ldots,Ocache\_\{T\_\{r\}\}of size

Br×dB\_\{r\}\\times d, divide the cache

ℓ​c​a​c​h​e,R​o​w​M​a​x​c​a​c​h​e∈ℝN×4​d\\ell\{cache\},RowMaxcache\\in\\mathbb\{R\}^\{N\\times 4d\}into

TrT\_\{r\}blocks

ℓ​c​a​c​h​e1,…,ℓ​c​a​c​h​eTr\\ell\{cache\}\_\{1\},\\ldots,\\ell\{cache\}\_\{T\_\{r\}\}and

R​o​w​M​a​x​c​a​c​h​e1,…,R​o​w​M​a​x​c​a​c​h​eTrRowMaxcache\_\{1\},\\ldots,RowMaxcache\_\{T\_\{r\}\}of size

Br×4​dB\_\{r\}\\times 4d\.

3:Divide the output

O∈ℝN×dO\\in\\mathbb\{R\}^\{N\\times d\}into

TrT\_\{r\}blocks

O1,…,OTrO\_\{1\},\\ldots,O\_\{T\_\{r\}\}of size

Br×dB\_\{r\}\\times d, and divide the logsumexp

LLinto

TrT\_\{r\}blocks

L1,…,LTrL\_\{1\},\\ldots,L\_\{T\_\{r\}\}of size

BrB\_\{r\}\.

4:Define overlap indicator

Qo​v​e​r​l​a​p=\{T​r​u​e,if​Qi​overlaps with current BlockF​a​l​s​e,otherwise\.Q\_\{overlap\}=\\begin\{cases\}True,&\\text\{if \}Q\_\{i\}\\text\{ overlaps with current Block \}\\\\ False,&\\text\{otherwise\}\.\\end\{cases\}
5:

Ko​v​e​r​l​a​p=\{T​r​u​e,if​Kj​overlaps with current BlockF​a​l​s​e,otherwise\.K\_\{overlap\}=\\begin\{cases\}True,&\\text\{if \}K\_\{j\}\\text\{ overlaps with current Block \}\\\\ False,&\\text\{otherwise\}\.\\end\{cases\}
6:for

1≤i≤Tr1\\leq i\\leq T\_\{r\}do

7:Load

QiQ\_\{i\}from HBM to on\-chip SRAM\.

8:On chip, initialize

Oi\(0\)=𝟎Br×d,ℓi\(0\)=𝟎Br,mi\(0\)=\(−∞\)BrO\_\{i\}^\{\(0\)\}=\\mathbf\{0\}\_\{B\_\{r\}\\times d\},\\quad\\ell\_\{i\}^\{\(0\)\}=\\mathbf\{0\}\_\{B\_\{r\}\},\\quad m\_\{i\}^\{\(0\)\}=\(\-\\infty\)\_\{B\_\{r\}\}\.

9:if

G​e​n​S​t​e​p=T​r​u​eGenStep=Trueand

Qo​v​e​r​l​a​p=F​a​l​s​eQ\_\{overlap\}=Falsethen

10:

O​c​a​c​h​ei\(0\)=𝟎Br×d,ℓ​c​a​c​h​ei\(0\)=𝟎Br×4​d,R​o​w​M​a​x​c​a​c​h​ei\(0\)=𝟎Br×4​dOcache\_\{i\}^\{\(0\)\}=\\mathbf\{0\}\_\{B\_\{r\}\\times d\},\\quad\\ell\{cache\}\_\{i\}^\{\(0\)\}=\\mathbf\{0\}\_\{B\_\{r\}\\times 4d\},\\quad RowMaxcache\_\{i\}^\{\(0\)\}=\\mathbf\{0\}\_\{B\_\{r\}\\times 4d\}\.

11:elseif

G​e​n​S​t​e​p=F​a​l​s​eGenStep=Falseand

Qo​v​e​r​l​a​p=F​a​l​s​eQ\_\{overlap\}=Falsethen

12:

Oi\(0\)=O​c​a​c​h​ei\(0\),ℓi\(0\)=ℓ​c​a​c​h​ei\(0\),mi\(0\)=R​o​w​M​a​x​c​a​c​h​ei\(0\)O\_\{i\}^\{\(0\)\}=Ocache\_\{i\}^\{\(0\)\},\\quad\\ell\_\{i\}^\{\(0\)\}=\\ell\{cache\}\_\{i\}^\{\(0\)\},\\quad m\_\{i\}^\{\(0\)\}=RowMaxcache\_\{i\}^\{\(0\)\}\.⊳\\trianglerightInitialize with cache

13:endif

14:for

1≤j≤Tc1\\leq j\\leq T\_\{c\}do

15:if

G​e​n​S​t​e​p=F​l​a​s​eGenStep=Flaseand

Qo​v​e​r​l​a​p=F​a​l​s​eQ\_\{overlap\}=Falseand

Ko​v​e​r​l​a​p=F​a​l​s​eK\_\{overlap\}=Falsethen

16:continue\.⊳\\trianglerightAlready calculated, skip

17:endif

18:Load

Kj,VjK\_\{j\},V\_\{j\}from HBM to on\-chip SRAM\.

19:On chip, compute

Si\(j\)=Qi​Kj⊤∈ℝBr×Bc\.S\_\{i\}^\{\(j\)\}=Q\_\{i\}K\_\{j\}^\{\\top\}\\in\\mathbb\{R\}^\{B\_\{r\}\\times B\_\{c\}\}\.
20:On chip, compute

mi\(j\)=max⁡\(mi\(j−1\),rowmax⁡\(Si\(j\)\)\),P~i\(j\)=exp⁡\(Si\(j\)−mi\(j\)\)∈ℝBr×Bc,m\_\{i\}^\{\(j\)\}=\\max\\\!\\left\(m\_\{i\}^\{\(j\-1\)\},\\operatorname\{rowmax\}\(S\_\{i\}^\{\(j\)\}\)\\right\),\\tilde\{P\}\_\{i\}^\{\(j\)\}=\\exp\\\!\\left\(S\_\{i\}^\{\(j\)\}\-m\_\{i\}^\{\(j\)\}\\right\)\\in\\mathbb\{R\}^\{B\_\{r\}\\times B\_\{c\}\},
21:

ℓi\(j\)=emi\(j−1\)−mi\(j\)​ℓi\(j−1\)\+rowsum⁡\(P~i\(j\)\)\.\\ell\_\{i\}^\{\(j\)\}=e^\{m\_\{i\}^\{\(j\-1\)\}\-m\_\{i\}^\{\(j\)\}\}\\ell\_\{i\}^\{\(j\-1\)\}\+\\operatorname\{rowsum\}\(\\tilde\{P\}\_\{i\}^\{\(j\)\}\)\.
22:On chip, compute

Oi\(j\)=diag⁡\(emi\(j−1\)−mi\(j\)\)​Oi\(j−1\)\+P~i\(j\)​Vj\.O\_\{i\}^\{\(j\)\}=\\operatorname\{diag\}\\\!\\left\(e^\{m\_\{i\}^\{\(j\-1\)\}\-m\_\{i\}^\{\(j\)\}\}\\right\)O\_\{i\}^\{\(j\-1\)\}\+\\tilde\{P\}\_\{i\}^\{\(j\)\}V\_\{j\}\.
23:if

G​e​n​S​t​e​p=T​r​u​eGenStep=Trueand

Qo​v​e​r​l​a​p=F​a​l​s​eQ\_\{overlap\}=Falseand

Ko​v​e​r​l​a​p=F​a​l​s​eK\_\{overlap\}=Falsethen

24:

O​c​a​c​h​ei\(j\)=diag⁡\(emi\(j−1\)−mi\(j\)\)​O​c​a​c​h​ei\(j−1\)\+P~i\(j\)​Vj,Ocache\_\{i\}^\{\(j\)\}=\\operatorname\{diag\}\\\!\\left\(e^\{m\_\{i\}^\{\(j\-1\)\}\-m\_\{i\}^\{\(j\)\}\}\\right\)Ocache\_\{i\}^\{\(j\-1\)\}\+\\tilde\{P\}\_\{i\}^\{\(j\)\}V\_\{j\},
25:

ℓ​c​a​c​h​ei=emi\(j−1\)−mi\(j\)​ℓ​c​a​c​h​ei\+rowsum⁡\(P~i\(j\)\)\.\\ell\{cache\}\_\{i\}=e^\{m\_\{i\}^\{\(j\-1\)\}\-m\_\{i\}^\{\(j\)\}\}\\ell\{cache\}\_\{i\}\+\\operatorname\{rowsum\}\(\\tilde\{P\}\_\{i\}^\{\(j\)\}\)\.⊳\\trianglerightUpdate cache

26:endif

27:endfor

28:On chip, compute

Oi=diag\(ℓi\(Tc\)\)−1Oi\(Tc\)\.O\_\{i\}=\\operatorname\{diag\}\\\!\\left\(\\ell\_\{i\}^\{\(T\_\{c\}\)\}\\right\)^\{\-1\}O\_\{i\}^\{\(T\_\{c\}\)\}\.
29:On chip, compute

Li=mi\(Tc\)\+log⁡\(ℓi\(Tc\)\)\.L\_\{i\}=m\_\{i\}^\{\(T\_\{c\}\)\}\+\\log\\\!\\left\(\\ell\_\{i\}^\{\(T\_\{c\}\)\}\\right\)\.
30:Write

OiO\_\{i\}to HBM as the

ii\-th block of

OO\.

31:Write

LiL\_\{i\}to HBM as the

ii\-th block of

LL\.

32:if

G​e​n​S​t​e​p=T​r​u​eGenStep=Trueand

Qo​v​e​r​l​a​p=F​a​l​s​eQ\_\{overlap\}=Falsethen

33:Write

O​c​a​c​h​eiOcache\_\{i\}to HBM as the

ii\-th block of

O​c​a​c​h​eOcache\.

34:Write

ℓ​c​a​c​h​ei\\ell\{cache\}\_\{i\}to HBM as the

ii\-th block of

ℓ​c​a​c​h​e\\ell\{cache\}\.

35:Write

R​o​w​M​a​x​c​a​c​h​eiRowMaxcache\_\{i\}to HBM as the

ii\-th block of

R​o​w​M​a​x​c​a​c​h​eRowMaxcache\.⊳\\trianglerightStore cache

36:endif

37:endfor

![Refer to caption](https://arxiv.org/html/2607.16339v1/x5.png)Figure 9:The response examples of the DPad backbone and LaCache\(Ours\) generated with the prompt in MBPP benchmark\.