Faster but Different: Diagnosing and Controlling Content Drift in Accelerated Multimodal Diffusion Language Models

# Faster but Different: Diagnosing and Controlling Content Drift in Accelerated Multimodal Diffusion Language Models
Source: [https://arxiv.org/html/2607.29079](https://arxiv.org/html/2607.29079)
Yaoxuan DouSchool of Mathematics and Statistics, Beijing Institute of TechnologyBeijingChina[1120231803@bit\.edu\.cn](https://arxiv.org/html/2607.29079v1/mailto:[email protected])Yang ShuZhejiang UniversityHangzhouChina[shuyang@zju\.edu\.cn](https://arxiv.org/html/2607.29079v1/mailto:[email protected])

###### Abstract\.

Training\-free acceleration makes diffusion\-based multimodal large language models \(dMLLMs\) more deployable, but it may silently change generated content\. We study this serving\-time consistency problem on 300 real images, comparing Fast\-dLLM outputs with the same model’s unaccelerated outputs\. Across the mild parallelism induced in our long\-form setting \(1\.05–1\.25 committed tokens per step\), confidence\-threshold tuning changes decoding behavior but not baseline agreement\. State\-refresh ablations and an image\-swap intervention instead identify stale visual and generated\-text states as contributors to drift\. For the tested Fast\-dLLM implementation, shortening the KV\-cache refresh interval yields a monotonic speed–agreement frontier and near\-exact agreement at a measured1\.3×1\.3\\timesspeedup\. The initial diagnosis also appears with dLLM\-Cache and LaViDa, although dLLM\-Cache recovers agreement only after both caches are tightened, which removes its speed advantage\. Independent prompts and images reproduce the threshold\-insensitivity and refresh recovery\. A targeted audit finds genuine content substitution in half of 50 low\-agreement pairs\. In a separate blinded two\-annotator evaluation, the pooled accelerated\-minus\-baseline factual\-error difference is0\.000\.00\(95% CI\[−0\.17,\+0\.17\]\[\-0\.17,\+0\.17\]\); this sample detects no difference but does not establish factual equivalence\. Finally, none of the tested adaptive or smoothed\-refresh variants beats the fixed interval at matched compute\. Our contribution is a paired diagnostic and an implementation\-scoped consistency control, not an accuracy or safety guarantee\.

diffusion language models, multimodal Web assistants, information integrity, inference acceleration, KV cache, content drift

††copyright:none††ccs:Computing methodologies Natural language generation††ccs:Computing methodologies Computer vision††ccs:Computing methodologies Neural networks## 1\.Introduction

Multimodal assistants increasingly sit between users and visually rich information: screenshots, product pages, advertisements, news images, and social\-media content\. Realistic Web\-agent benchmarks already require models to combine visual page understanding with downstream actions\(Koh et al\.,[2024](https://arxiv.org/html/2607.29079#bib.bib6); Zheng et al\.,[2024](https://arxiv.org/html/2607.29079#bib.bib15)\)\. In these latency\-sensitive settings, a generated description or structured extraction may become evidence for a downstream search, content\-analysis, or agentic decision\. Acceleration is therefore not only a throughput concern\. If the serving configuration silently changes what the model claims is visible, it can make the same content yield inconsistent intermediate evidence even when aggregate task accuracy appears stable\. We study this inference\-time information\-integrity problem through diffusion\-based multimodal large language models \(dMLLMs\), a new non\-autoregressive family whose block\-parallel generation makes aggressive acceleration especially attractive\.

Unlike conventional left\-to\-right generation, dLLMs iteratively denoise a fully masked sequence through confidence\-guided unmasking\(Nie et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib11)\); LLaDA\-V\(You et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib13)\)extends this paradigm with a vision encoder\. The practical appeal is undercut by slow inference and dependency violations under parallel decoding\(Kang et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib4); Israel et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib3)\)\. Fast\-dLLM\(Wu et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib12)\)addresses both through approximate block\-wise KV caching and confidence\-aware parallel decoding\. Reported speedups reach an order of magnitude or more, and the confidence threshold is presented – and used in follow\-up work such as VRCD\(Yuan et al\.,[2026](https://arxiv.org/html/2607.29079#bib.bib14)\)– as the natural dial for trading inference speed against output quality\.

![Refer to caption](https://arxiv.org/html/2607.29079v1/x1.png)Figure 1\.Why acceleration changes what the model says, and the dial that controls it in the settings we study\. \(a\) Fast\-dLLM’s approximate KV cache re\-attends to the image only once everyprefix\_refresh\_intervalsteps \(default 32\): 124 of 128 denoising steps use cached visual states\. \(b\) The threshold sweep uses 300 MME images; refresh and variant sweeps use a paired 50\-image subset \(means with 95% bootstrap CIs\)\. Threshold \(red\) leaves agreement unchanged; shrinking the refresh interval \(blue,32→132\\to 1\) trades speed for agreement continuously; no adaptive or smoothed\-refresh variant \(grey×\\times\) beats this frontier\. Agreement is against the unaccelerated output, not ground truth\.Panel a contrasts full refresh steps, which attend to image and text tokens, with incremental steps that reuse cached states\. Panel b plots generation time against agreement: threshold settings form a low\-agreement cluster, refresh settings trace an increasing frontier, and adaptive variants lie below it\.This threshold\-as\-dial assumption has, to our knowledge, never been directly tested for multimodal, open\-ended generation, where the failure mode of interest is not a drop in task accuracy but a change in*what the model claims is in the image*\. We test it directly\. Using LLaDA\-V and Fast\-dLLM on 300 real images spanning diverse content categories, we generate detailed image descriptions under six configurations – an unaccelerated baseline and five confidence thresholds spanning the method’s operating range – and measure*baseline agreement*: how closely the accelerated output matches, in content, the unaccelerated output for the same image\. Two clarifications frame everything that follows\. First, baseline agreement is a*consistency*construct, not an accuracy construct: it quantifies whether acceleration changes what the model says, not whether either version is correct about the image; Section[9](https://arxiv.org/html/2607.29079#S9)separately evaluates image\-grounded factuality and detects no accelerated–baseline difference in a limited 50\-image study, without establishing equivalence\. Second, in the released implementation the threshold’s effective direction is the reverse of its intuitive reading – larger values commit*more*tokens per step \(Section[5](https://arxiv.org/html/2607.29079#S5)\)\. We find that the threshold\-as\-dial assumption does not hold in this tested range: once acceleration is enabled, content diverges from the unaccelerated output by a large, nearly constant amount regardless of threshold\. We call this an*off\-switch effect*: turning acceleration on moves the model onto a different generation trajectory whose distance from the unaccelerated trajectory is not controlled by the tested thresholds\.

We do not stop at the diagnosis\. Inspecting Fast\-dLLM, we trace the effect to its KV\-cache schedule: full recomputation occurs only at a fixed refresh interval, with intermediate steps reusing cached visual and generated\-text states \(Figure[1](https://arxiv.org/html/2607.29079#S1.F1)a\)\. Orthogonal refresh ablations show that both state groups contribute, with visual\-state refresh recovering more agreement; a direct image\-swap intervention shows that incremental steps cannot incorporate a changed image until a full refresh\. This is a scheduling\-induced analogue of the gradual visual\-information loss reported for autoregressive VLMs\(Li et al\.,[2025b](https://arxiv.org/html/2607.29079#bib.bib9)\)\. Guided by this mechanism, we show that the refresh interval – not the threshold – is the dominant observed consistency control for the tested Fast\-dLLM implementation: sweeping it produces a continuous, monotonic agreement curve, and its most conservative setting nearly eliminates drift while retaining a real speedup\.

To assess how much of this story is specific to one acceleration implementation, we repeat the full refresh\-interval sweep using an independent, differently\-engineered caching method, dLLM\-Cache\(Liu et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib10)\)\. The initial diagnosis recurs: the two implementations produce similar drift under their default configurations\. The single\-parameter remedy does not transfer: dLLM\-Cache’s drift stays nearly flat when only its prompt cache is refreshed more often, pointing to its independently scheduled generated\-token cache as a second candidate source\. A 15\-image joint pilot supports this interpretation, but recovering agreement by tightening both caches removes dLLM\-Cache’s speed advantage\. We report this as an implementation boundary condition\. Finally, because our core evidence relies on lexical similarity, we manually audit 50 low\-similarity generations and find that half reflect genuine content substitution, including invented content, rather than harmless paraphrase\.

For Web\-facing multimodal systems, the central contribution is a serving\-time diagnostic that detects when an efficiency mechanism changes the information extracted from visual content, separately from whether either output is factually correct\. Concretely, we contribute: \(1\) systematic evidence that confidence\-threshold tuning does not control content drift across the induced parallelism range, using 300 images, two additional output formats on an independent sample, and step\-level instrumentation; \(2\) causal evidence, in the tested Fast\-dLLM implementation, that KV\-cache staleness contributes to drift, from orthogonal state\-refresh ablations and an image\-swap intervention; \(3\) a continuously tunable refresh control with a measured efficiency–agreement frontier; \(4\) cross\-implementation, cross\-model, and matched\-latency checks that scope the diagnosis and remedy; \(5\) manual and blinded studies separating trajectory consistency from image\-grounded factuality; and \(6\) a negative\-results study showing that no tested adaptive or smoothed\-refresh variant beats the fixed interval at matched compute\.

## 2\.Related Work

Accelerating diffusion language model inference\.Fast\-dLLM\(Wu et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib12)\)is the primary object of our study, combining block\-wise approximate KV caching with confidence\-aware parallel decoding to accelerate LLaDA\-family models by an order of magnitude or more\. Fast\-dLLM\+\+\(Kasa et al\.,[2026](https://arxiv.org/html/2607.29079#bib.bib5)\)subsequently replaces weakest\-token selection with confidence\-profile decoding while leaving the cache implementation unchanged; it improves the text\-only accuracy–throughput frontier but does not study multimodal cache\-refresh consistency\. dLLM\-Cache\(Liu et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib10)\)pursues a related but independently engineered goal, combining long\-interval prompt caching with partial, similarity\-guided updates to the response cache\. Adaptive Parallel Decoding \(APD\)\(Israel et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib3)\)instead uses a small auxiliary autoregressive model to decide how many tokens to commit\. These works evaluate task accuracy or throughput rather than sensitivity of open\-ended multimodal content to exposed serving parameters\.

Quality degradation under parallel decoding\.ParallelBench\(Kang et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib4)\)is closest in spirit to our diagnosis: it introduces a benchmark to quantify quality degradation under parallel decoding in dLLMs, and attributes the degradation to the conditional\-independence assumption violating token dependencies\. However, ParallelBench evaluates purely text\-only dLLMs and does not study the multimodal setting, nor does it examine cache\-refresh scheduling as a distinct variable from decoding parallelism\. Our work extends this line of inquiry to multimodal generation and isolates a mechanism \(cache staleness\) that is orthogonal to the dependency\-violation account\.

Visual token handling in diffusion MLLMs\.A concurrent line of work studies how diffusion MLLMs allocate attention across visual tokens during parallel decoding\. VRCD\(Yuan et al\.,[2026](https://arxiv.org/html/2607.29079#bib.bib14)\)proposes redundancy\-aware reranking of co\-selected high\-confidence tokens within a single decoding step; a broader study of visual token redundancy\(Li et al\.,[2026](https://arxiv.org/html/2607.29079#bib.bib7)\)targets efficiency\-accuracy trade\-offs on standard VQA benchmarks\. Neither analyzes cache\-refresh scheduling, nor content divergence from a paired unaccelerated reference on open\-ended multimodal generation – the specific analysis we present\.

Hallucination and visual grounding in VLMs\.Visual Information Steering\(Li et al\.,[2025b](https://arxiv.org/html/2607.29079#bib.bib9)\)reports gradual loss of visually grounded token preference during autoregressive generation\. Our image\-swap intervention identifies a different, deterministic mechanism: Fast\-dLLM’s incremental steps reuse cached visual states and cannot incorporate a changed image until refresh\. We therefore use the autoregressive finding only as a structural analogy, not as evidence that the two mechanisms are identical\.

Multimodal agents on the Web\.VisualWebArena\(Koh et al\.,[2024](https://arxiv.org/html/2607.29079#bib.bib6)\)evaluates agents on realistic visually grounded Web tasks, while SeeAct\(Zheng et al\.,[2024](https://arxiv.org/html/2607.29079#bib.bib15)\)combines visual understanding and action grounding on websites\. These works motivate why stable visual evidence can matter downstream\. Our experiments do not evaluate Web navigation or task success; they isolate a lower\-level serving consistency question that such systems may encounter\.

Positioning summary\.Table[1](https://arxiv.org/html/2607.29079#S2.T1)summarizes how our work differs from the six most closely related papers\. VRCD and ParallelBench are recent arXiv preprints, so we treat them as concurrent work and read novelty claims with that caveat\. To the best of our knowledge, no prior work combines a multimodal setting with a controlled sweep of both confidence threshold and cache refresh interval against a paired unaccelerated reference, or tests the proposed control across two independently engineered caching implementations\.

Table 1\.How this paper differs from the closest related work\.
## 3\.Background

### 3\.1\.Diffusion Language Model Generation

Unlike autoregressive models, LLaDA\-family models\(Nie et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib11); You et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib13)\)generate a sequence ofgen\_lengthtokens by initializing them all to a mask token and iteratively replacing masked positions with predicted tokens overstepsdenoising steps, optionally divided intoblock\_length\-sized blocks for semi\-autoregressive block\-wise generation\. At each step, the model computes a full forward pass, and a remasking policy \(e\.g\., low\-confidence remasking\) decides which currently\-masked positions to commit based on the model’s predicted confidence\.

### 3\.2\.Fast\-dLLM Acceleration

Fast\-dLLM\(Wu et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib12)\)accelerates this process with two mechanisms\. First, an approximate block\-wise KV cache avoids recomputing attention for the prompt and already\-completed blocks at every step: a full forward pass \(recomputing keys and values for the entire sequence, including image tokens\) is performed only when the current step index is a multiple ofprefix\_refresh\_interval; all other steps perform an incremental forward pass restricted to the active generation block, reusing cached keys and values for everything before it\. Second, confidence\-aware parallel decoding replaces the fixed per\-step token budget with a threshold rule: any masked position whose predicted confidence exceedsthresholdis committed in the current step, allowing a variable, potentially large number of tokens to be unmasked per step when the model is confident\.

### 3\.3\.dLLM\-Cache

dLLM\-Cache\(Liu et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib10)\)is an independently engineered caching method for the same class of models\. It separately controls the refresh interval for the prompt cache \(prompt\_interval\_steps\) and for the response \(already\-generated\-token\) cache \(gen\_interval\_steps\), combined with a similarity\-based partial update rule \(transfer\_ratio\) for deciding which cached features to refresh between full recomputes\. We use it in Section[7](https://arxiv.org/html/2607.29079#S7)as an independent implementation to test the generality of our diagnosis\.

## 4\.Experimental Setup

Model and data\.We use LLaDA\-V \(GSAI\-ML/LLaDA\-V\), an 8B\-parameter dMLLM built on a LLaDA\-8B language backbone with a SigLIP2\-SO400M vision encoder\. We draw 300 images with a fixed random seed from the MME benchmark\(Fu et al\.,[2023](https://arxiv.org/html/2607.29079#bib.bib2)\); RQ1 uses all 300, while the more computationally intensive mechanism, refresh, and cross\-implementation studies use the same fixed 50\-image subset\. The primary prompt \(“Please describe the image in detail\.”\) usessteps=gen\_length=block\_length=128\. A prompt/sample\-generalization check draws 50 nonoverlapping images with seed 20260727 and uses a one\-sentence prompt \(length/steps 32\) and fixed\-schema JSON prompt \(length/steps 96\)\. An auxiliary cross\-model check uses the official LaViDa checkpoint and code\(Li et al\.,[2025a](https://arxiv.org/html/2607.29079#bib.bib8)\)on the primary 50 rows and prompt, sweeping its exposed prefix\-cache switch and denoising NFE ratio\.

Metrics and terminology\.For each accelerated configuration, we compare the generated text against the unaccelerated \(use\_fast\_dllm=False\) generation for the same image, computed once per image and reused as a common reference across all configurations\. We call the resulting quantity*baseline agreement*and its complement*content drift*\. Two distinct constructs must not be conflated here, and we keep them separate throughout\.*Baseline agreement*\(this section’s metrics\) measures whether acceleration changes what the model says relative to its own unaccelerated behavior – a consistency and reproducibility property\.*Image\-grounded factuality*measures whether a description is correct about the image – an accuracy property that requires looking at the image, which no text\-to\-text metric can assess, and which we probe separately via a manual audit and a ground\-truth spot\-check in Section[9](https://arxiv.org/html/2607.29079#S9)\. High agreement does not imply factual correctness \(the unaccelerated output can itself hallucinate\), and low agreement does not per se imply error \(it may be benign paraphrase\)\. We report \(i\) wall\-clock generation time; \(ii\) word\-level Jaccard similarity between the accelerated and unaccelerated text \(computed over lowercased alphabetic word sets\); \(iii\) distinct\-2 ratio; and \(iv\) repeat\-3 rate\. Unless stated otherwise, 95% CIs are paired percentile\-bootstrap intervals over images \(10,000 resamples, seed 0\)\. Section[9](https://arxiv.org/html/2607.29079#S9)corroborates Jaccard with BERTScore and manual evaluation\.

Reproducibility\.Primary experiments use one NVIDIA RTX 6000 Ada GPU \(48GB\), PyTorch 2\.1\.2/CUDA 12\.1, and the released LLaDA\-V code with Fast\-dLLM and dLLM\-Cache hooks\. Images are selected with recorded seeds; comparisons are paired by image\. Decoding is deterministic \(no temperature or sampling\), so drift is attributable to configuration, not run\-to\-run randomness\. Times exclude loading and preprocessing\. The LaViDa check uses its official repository \(commit24220c0\) and checkpoint in a separate environment on the same GPU\. We additionally repeat eight timing configurations 30 times each on an A800\-SXM4\-80GB, with warm\-up, synchronized timing, and globally randomized order: the no\-acceleration/default times are 71\.5/6\.3s \(11\.3×11\.3\\times\), matching 71\.9/6\.4s \(11\.3×11\.3\\times\) on the Ada GPU, and every qualitative speed claim transfers\. Table[6](https://arxiv.org/html/2607.29079#S10.T6)itemizes all 4,930 generations\. We will release scripts, raw logs, image identifiers, and analysis code upon publication\.

## 5\.RQ1: Is Content Drift Threshold\-Sensitive?

We generate all 300 images under six configurations: an unaccelerated baseline, and Fast\-dLLM withthreshold∈\{0\.9,0\.7,0\.5,0\.3,0\.1\}\\in\\\{0\.9,0\.7,0\.5,0\.3,0\.1\\\}\. In the released implementation, a masked position is committed early when confidence exceeds1−threshold/\(n\+1\)1\-\\texttt\{threshold\}/\(n\{\+\}1\); larger values therefore lower the bar and commit more tokens\. Instrumentation confirms thatthreshold=0\.9is most aggressive \(1\.2541\.254tokens/step\) andthreshold=0\.1most conservative \(1\.0551\.055\) – the reverse of an intuitive reading\. This sweep yields 1,800 real generations\.

![Refer to caption](https://arxiv.org/html/2607.29079v1/x2.png)Figure 2\.Threshold sweep \(300 images×\\times6 configurations, 1,800 generations; 95% paired\-bootstrap CIs\)\. \(a\) Acceleration yields a 10–12×\\timesspeedup\. \(b\) Agreement remains flat at≈\\approx0\.41–0\.420\.42\. \(c\) distinct\-2 remains stable\.Three plots compare the unaccelerated baseline with five threshold settings\. Generation time drops sharply for every accelerated setting, while Jaccard agreement and distinct\-2 remain nearly flat across thresholds\.Figure[2](https://arxiv.org/html/2607.29079#S5.F2)summarizes the result\. Acceleration reduces mean generation time from 72\.9s to 6\.1–6\.9s \(a 10–12×\\timesspeedup\)\. Mean Jaccard lies in the narrow range0\.4110\.411–0\.4170\.417\. A paired Friedman test finds no threshold effect \(χ2​\(4\)=1\.36\\chi^\{2\}\(4\)=1\.36,p=0\.85p=0\.85\); all 10 pairwise contrasts remain nonsignificant after Holm correction\. The extreme comparison \(threshold0\.1−0\.90\.1\-0\.9\) is only\+0\.0056\+0\.0056\(95% paired bootstrap CI\[−0\.0025,\+0\.0139\]\[\-0\.0025,\+0\.0139\], Wilcoxonp=0\.75p=0\.75\)\. Using a review\-stage equivalence region of±0\.02\\pm 0\.02Jaccard \(declared after, not before, the original experiment\), its 90% CI\[−0\.0012,\+0\.0125\]\[\-0\.0012,\+0\.0125\]lies wholly inside the region and paired TOST rejects nonequivalence \(p=0\.00034p=0\.00034\); paireddz=0\.078d\_\{z\}=0\.078and Cliff’sδ=0\.017\\delta=0\.017are negligible\. Thus the data bound the effect rather than merely failing to reject a large one\. distinct\-2 remains0\.8730\.873–0\.8770\.877and repeat\-3 at0–0\.3%0\.3\\%: outputs are fluent but describe different content\.

Is the threshold actually operative?A skeptical reading of this flatness is that the threshold simply was not doing anything in our pipeline – a plumbing failure rather than a finding\. We rule this out by re\-running the full sweep \(250 generations\) through an instrumented decoding path that logs per\-step commit counts and confidences while leaving the computation untouched\. Three facts result\. First, the instrumented runs reproduce the original outputs byte\-for\-byte in 250/250 cases, so the instrumentation observes the real code path with zero perturbation\. Second, the threshold demonstrably and monotonically changes decoding behavior: mean steps per generation fall from121\.4121\.4\(threshold0\.10\.1\) to102\.8102\.8\(threshold0\.90\.9\), tokens per step rise from1\.0551\.055to1\.2541\.254, and the fraction of multi\-token steps rises from4\.5%4\.5\\%to17\.3%17\.3\\%\(Wilcoxon on paired step counts,p<10−4p<10^\{\-4\}\); outputs at different thresholds differ on∼95%\{\\sim\}95\\%of image pairs\. Third, despite this verified behavioral change, agreement stays flat at0\.410\.41–0\.420\.42\. The dial turns; agreement does not move\. We accordingly scope the claim: the threshold induces only a mild parallelism range here \(1\.051\.05–1\.251\.25tokens/step\), and our insensitivity result is a statement about that range – which is precisely the method’s operating range on this workload at default settings\. A representative case: a painting the baseline describes as “a Rembrandt\-style street scene … a man on a horse” becomes, in every accelerated configuration, “an ancient Roman courtyard … ruins of a grand building” – a coherent but different scene, not a corrupted one\.

We term this the*off\-switch effect*: enabling Fast\-dLLM acceleration is better modeled as switching to a qualitatively different generation trajectory than as smoothly interpolating along a single quality\-speed curve controlled by threshold\. This directly contradicts the implicit deployment assumption – shared by Fast\-dLLM’s own presentation and by follow\-up work such as VRCD\(Yuan et al\.,[2026](https://arxiv.org/html/2607.29079#bib.bib14)\)– that the confidence threshold is the knob controlling how much acceleration changes the output\. The claim holds at default and practical cache\-refresh settings, within the parallelism range induced here; Section[6\.3](https://arxiv.org/html/2607.29079#S6.SS3)maps the one boundary regime \(near per\-step refresh\) where a small threshold interaction emerges\.

Prompt and sample generalization\.On 50 new MME images \(zero overlap with the primary 300\), sentence/JSON prompts give default\-cache agreement 0\.592/0\.492\. Threshold extremes remain indistinguishable \(differences−0\.0077/\+0\.0064\-0\.0077/\+0\.0064;p=0\.18/0\.86p=0\.18/0\.86\); the sentence contrast meets the review\-stage±0\.02\\pm 0\.02TOST criterion, while JSON does not establish equivalence\. Per\-step refresh recovers agreement to 0\.974/0\.921 at1\.31×/1\.82×1\.31\\times/1\.82\\timesspeedups; sentence/valid\-JSON compliance remains 96–98%/90–96%\. Thus both prompts replicate the diagnosis and remedy, with the structured result limited to no detected threshold effect\.

### 5\.1\.Does Drift Vary by Visual Content Type?

MME organizes its images into content categories\. Our 50\-image sample spans 11 of them, unevenly \(1–10 images per category, Table[2](https://arxiv.org/html/2607.29079#S5.T2)\), so we treat this breakdown as exploratory; rows with≤2\\leq 2images are included for completeness only\. A pattern nevertheless emerges:*artwork*has the lowest average agreement \(0\.317\), consistent with the Roman\-street\-painting example, where the model can produce different but plausible compositional details\.*Scene*and*posters*, with a few salient objects, score higher \(0\.450–0\.453\)\. This direction is consistent with, but does not by itself establish, our mechanistic account\.

Table 2\.Content drift by MME category \(exploratory\)\. “Images” is the number of distinct images per category; “Gen\.” is the number of accelerated generations \(=5×=5\\timesImages\)\. Rows with Images≤2\\leq 2are not statistically reliable\.

## 6\.RQ2: What Causes the Drift, and Can It Be Fixed?

### 6\.1\.Mechanism

Inspecting the Fast\-dLLM implementation, we find that its approximate KV cache is refreshed on a fixed schedule: a full forward pass over the entire sequence – including image tokens – occurs only when the current denoising step index is a multiple ofprefix\_refresh\_interval\(default 32\); all other steps perform an incremental forward pass restricted to the generation block, reusing cached key/value states for the prompt and image from the last refresh\. With the default interval, only 4 of 128 steps involve genuinely re\-attending to the image; the remaining 124 steps generate conditioned on an increasingly stale visual representation\.

This is structurally analogous to gradual visual\-information loss reported for autoregressive VLMs\(Li et al\.,[2025b](https://arxiv.org/html/2607.29079#bib.bib9)\)\. Here, however, the effect is a deterministic cache window in which incremental steps reuse image states and therefore cannot incorporate newly supplied visual information until the next full refresh\.

Causal isolation\.At interval 4, refreshing only image\-token states yields Jaccard0\.4980\.498, while refreshing only generated\-text states yields0\.4540\.454\(paired difference\+0\.044\+0\.044,p=0\.028p=0\.028\); both improve over the default interval\-32 reference \(0\.4240\.424\), but neither reaches full interval\-4 refresh \(0\.5720\.572\)\. Visual staleness is therefore the larger contributor, not the sole one\. In a separate intervention, we replace image A with image B after step 4\. With no subsequent full refresh \(interval 128\), swapped and non\-swapped outputs are byte\-identical in 10/10 pairs; with per\-step refresh, outputs instead move strongly toward B \(Jaccard to A/B baselines0\.228/0\.5890\.228/0\.589\)\. This directly verifies that refresh scheduling gates access to changed visual evidence\.

### 6\.2\.The Refresh Interval Is the Dominant Dial

![Refer to caption](https://arxiv.org/html/2607.29079v1/x3.png)Figure 3\.Speed and baseline agreement as a function ofprefix\_refresh\_interval\(50 images×\\times6 configurations, 300 generations; error bars are 95% bootstrap CIs\)\. Unlike threshold, the refresh interval produces a continuous, monotonic trade\-off between speed and agreement with the unaccelerated output, converging to near\-exact agreement at its most conservative setting while retaining a real speedup over the unaccelerated baseline \(dashed line\)\.A two\-axis plot shows that shorter refresh intervals increase both generation time and agreement\. Agreement rises monotonically from about 0\.42 at interval 32 to about 0\.99 at interval 1, which remains faster than the unaccelerated baseline\.We fixthreshold=0\.5and sweepprefix\_refresh\_interval∈\{32,16,8,4,2,1\}\\in\\\{32,16,8,4,2,1\\\}on the same 50 images\. Figure[3](https://arxiv.org/html/2607.29079#S6.F3)and Table[3](https://arxiv.org/html/2607.29079#S6.T3)show the result: unlike threshold, the refresh interval produces a clean, continuous, monotonic trade\-off\. Generation time increases from 6\.4s to 55\.3s as the interval shrinks from 32 to 1, while Jaccard similarity to the unaccelerated baseline increases monotonically from 0\.420 to 0\.987\. This trend is highly significant \(Page’sLLtest for the ordered alternative,z=11\.6z=11\.6,p<10−10p<10^\{\-10\}\)\. Four of five adjacent improvements survive Holm correction \(adjustedp=0\.00031p=0\.00031,0\.00630\.0063,0\.01350\.0135, and<10−7<10^\{\-7\}for32→1632\{\\to\}16,8→48\{\\to\}4,4→24\{\\to\}2, and2→12\{\\to\}1\); only16→816\{\\to\}8is nonsignificant \(pHolm=0\.247p\_\{\\mathrm\{Holm\}\}=0\.247\)\. Atprefix\_refresh\_interval=1– full recomputation, including image attention, at every step – 80% of the 50 generations are word\-for\-word identical to the unaccelerated baseline\. The configuration still runs1\.3×1\.3\\timesfaster than no acceleration at all \(55\.3s vs\. 71\.9s\), because confidence\-aware parallel decoding continues to commit multiple tokens per step whenever the model is confident, independent of cache freshness\. Cache refresh frequency and confidence\-driven parallelism are therefore two independent, separately controllable sources of speedup\. Contrary to common practice, it is the former – not the latter – that determines agreement with the unaccelerated output in the regimes we study\.

Table 3\.Speed–agreement trade\-off across refresh intervals \(50 images each\)\.
### 6\.3\.Does Threshold Interact with Refresh Interval?

Our threshold\-insensitivity result \(Section[5](https://arxiv.org/html/2607.29079#S5)\) was established at the defaultprefix\_refresh\_interval=32, and our refresh\-interval result above was established at the defaultthreshold=0\.5; neither directly tests whether the two parameters interact away from these reference points, a concern a reviewer of an earlier draft of this work correctly raised\. We test this with a3×33\\times 3grid,threshold∈\{0\.9,0\.5,0\.1\}×\\in\\\{0\.9,0\.5,0\.1\\\}\\timesprefix\_refresh\_interval∈\{32,4,1\}\\in\\\{32,4,1\\\}, filling in the four off\-diagonal cells not covered by our existing sweeps, on the same 50 images\.

At the default refresh interval \(32\), threshold remains flat \(mean Jaccard0\.4190\.419,0\.4200\.420,0\.4060\.406for threshold0\.90\.9,0\.50\.5,0\.10\.1\), replicating Section[5](https://arxiv.org/html/2607.29079#S5)\. At refresh=1=1, a targeted exploratory contrast gives agreement0\.9550\.955at threshold=0\.9=0\.9versus0\.9950\.995at threshold=0\.1=0\.1, a difference\-in\-differences of−0\.054\-0\.054relative to refresh=32=32\(paired Wilcoxonp=0\.0085p=0\.0085; 95% bootstrap CI\[−0\.092,−0\.021\]\[\-0\.092,\-0\.021\]\); the analogous refresh=4=4contrast is nonsignificant \(p=0\.37p=0\.37\)\. However, a mixed\-effects model on all 450 paired observations,agreement∼threshold×refresh\+\(1∣image\)\\text\{agreement\}\\sim\\text\{threshold\}\\times\\text\{refresh\}\+\(1\\mid\\text\{image\}\), finds no omnibus interaction \(likelihood\-ratioχ2​\(4\)=7\.42\\chi^\{2\}\(4\)=7\.42,p=0\.115p=0\.115\)\. We therefore do not claim a general interaction: the local near\-ceiling contrast is hypothesis\-generating, whereas the model\-level result supports refresh interval as the dominant agreement lever\.

## 7\.RQ3: Does This Generalize Across Implementations?

A natural concern is that both the diagnosis and the remedy are artifacts of Fast\-dLLM’s specific implementation\. We test this by repeating the refresh\-interval sweep with dLLM\-Cache\(Liu et al\.,[2025](https://arxiv.org/html/2607.29079#bib.bib10)\), an independently engineered caching method also integrated into the LLaDA\-V codebase\. dLLM\-Cache exposesprompt\_interval\_stepsas the direct analogue ofprefix\_refresh\_interval, controlling how often the prompt/image cache is fully refreshed\. We fix its other parameters at their released defaults \(gen\_interval\_steps=7andtransfer\_ratio=0\.25\) and sweepprompt\_interval\_stepsover\{25,16,8,4,2,1\}\\\{25,16,8,4,2,1\\\}on the same 50 images\.

![Refer to caption](https://arxiv.org/html/2607.29079v1/x4.png)Figure 4\.Measured latency–agreement frontiers for Fast\-dLLM and dLLM\-Cache \(50 paired images per point; error bars are 95% bootstrap CIs\)\. At every shared latency budget, the best feasible Fast\-dLLM setting has higher agreement; the unaccelerated baseline is shown separately\.Agreement is plotted against generation time for Fast\-dLLM and dLLM\-Cache\. The Fast\-dLLM curve rises toward the unaccelerated reference, whereas the dLLM\-Cache prompt\-refresh curve remains near 0\.4 agreement as latency increases\.Figure[4](https://arxiv.org/html/2607.29079#S7.F4)shows a split result\. The*diagnosis recurs*: dLLM\-Cache’s default configuration produces Jaccard 0\.394, close to Fast\-dLLM’s 0\.420, supporting a shared failure mode in these two tested approximate\-caching implementations\. The*single\-parameter remedy does not transfer*: asprompt\_interval\_stepsshrinks from 25 to 1, Jaccard moves only from 0\.394 to 0\.437, with zero exact matches at any setting \(Table[4](https://arxiv.org/html/2607.29079#S7.T4)\)\. A paired Friedman test detects a small effect \(χ2​\(5\)=22\.4\\chi^\{2\}\(5\)=22\.4,p<0\.001p<0\.001\), and the extreme comparison gives\+0\.043\+0\.043\(95% CI\[0\.015,0\.070\]\[0\.015,0\.070\], Wilcoxonp=0\.001p=0\.001\), over an order of magnitude smaller than Fast\-dLLM’s0\.5670\.567range\. More importantly, the actual\-latency frontier removes any setting\-index ambiguity: Fast\-dLLM has higher agreement at all six shared dLLM\-Cache latency budgets, by0\.1530\.153–0\.5500\.550\. Even full per\-step prompt\-cache recomputation therefore falls far short of the unaccelerated reference\.

We interpret this as an open boundary condition\. dLLM\-Cache exposes a second, independently scheduled cache –gen\_interval\_steps, refreshing the cache for*already\-generated*tokens – held at its default \(7\) above; we hypothesized it is a distinct drift source that prompt\-cache refresh alone cannot address\. In a 15\-image joint\-sweep pilot \(prompt\_interval\_steps=1;gen\_interval\_steps∈\{7,4,2,1\}\\in\\\{7,4,2,1\\\}\), Jaccard rises modestly from 7 to 2 \(0\.450→0\.459→0\.5060\.450\\to 0\.459\\to 0\.506\), then reaches0\.9450\.945at 1, with 13/15 outputs word\-for\-word identical to the unaccelerated baseline\. This pilot supports the generated\-token cache as a second drift source, but its small size limits generalization\. At both intervals equal to 1, average generation time is109\.0109\.0s, slower than the71\.971\.9s unaccelerated baseline\. Thus agreement is recoverable in the tested pilot, but only after eliminating the acceleration benefit\.

Table 4\.dLLM\-Cache: initial drift replicates, remedy does not \(50 images each\)\.#### Auxiliary cross\-model check: LaViDa\.

We run 300 paired generations on official LaViDa\(Li et al\.,[2025a](https://arxiv.org/html/2607.29079#bib.bib8)\), crossing its prefix\-cache switch with NFE\. At matched NFE 128, caching cuts mean time from 21\.33s to 4\.48s \(4\.77×4\.77\\times\) but lowers agreement with the uncached output to 0\.299 \(95% CI \[0\.265, 0\.334\]\); at NFE 64, uncached/cached agreement is 0\.510/0\.297 \(p<10−8p<10^\{\-8\}\)\. Within the cached regime, reducing NFE four\-fold leaves agreement at 0\.297–0\.300 \(Friedmanp=0\.544p=0\.544; extreme\-change CI \[−0\.025,\+0\.030\-0\.025,\+0\.030\]\)\. Thus this second model supports the diagnosis: enabling prefix caching coincides with a large trajectory shift, while NFE does not control its magnitude in the tested range\. LaViDa exposes no Fast\-dLLM\-style refresh interval, so this check supports the diagnosis, not the remedy\.

## 8\.RQ4: Can Smarter Refresh Policies Beat the Fixed Interval?

The natural next step after RQ2 is an*adaptive*refresh policy: monitor a cheap online signal and refresh only when needed, hoping to beat the fixed\-interval frontier of Table[3](https://arxiv.org/html/2607.29079#S6.T3)\. We tested five variant families and 12 configurations; the outcome is uniformly negative\. Table[5](https://arxiv.org/html/2607.29079#S8.T5)reports seven representative or strongest points plotted as grey×\\times’s in Figure[1](https://arxiv.org/html/2607.29079#S1.F1)b, each paired with the fixed\-interval setting it must beat; throughout, “matched compute” means the fixed\-interval reference has equal or lower measured wall\-clock per generation on identical hardware\.

Variants tested\.All variants run on the same 50 images atthreshold=0\.5, modifying only*when*or*how*the cache is refreshed\. The complete grid is: \(i\)*token budget*, refreshing afterB∈\{4,8,16,32\}B\\in\\\{4,8,16,32\\\}committed tokens; \(ii\)*confidence decay*, refreshing when mean commit confidence falls belowr∈\{0\.85,0\.95\}r\\in\\\{0\.85,0\.95\\\}of its post\-refresh level; \(iii\)*attention decay*, applying the samer∈\{0\.85,0\.95\}r\\in\\\{0\.85,0\.95\\\}rule to image\-token attention mass at a fixed middle layer; \(iv\)*chunked rotation*, recomputing one ofC∈\{16,8,4\}C\\in\\\{16,8,4\\\}image\-token chunks per step; and \(v\) one*joint per\-step recomputation*configuration, recomputing all image and generated\-text positions while freezing only the 47\-token generic system prefix\. Table[5](https://arxiv.org/html/2607.29079#S8.T5)reports the highest\-agreement token\-budget and decay points, all three chunk sizes, and the joint point; the unreported settings were also included in the statistical tests below\.

Table 5\.RQ4: seven reported adaptive/smoothed points \(of 12 tested\), each against a fixed\-interval setting of equal or greater speed\. No variant beats the frontier; exact\-match counts are 0/50 except joint per\-step \(7/50, vs\. 40/50 for interval 1\)\.Results\.The three signal\-triggered policies all fail against the fixed interval at matched compute: paired differences are indistinguishable \(allp\>0\.4p\>0\.4, confidence decay\) or significantly*worse*\(token budgetp=0\.024p=0\.024; attention decayp=0\.026p=0\.026\)\. The token\-budget failure is mundane \(near\-constant≈1\.1\{\\approx\}1\.1tokens/step makes token and step counts interchangeable\), but the other two are informative: neither the model’s confidence nor its attention to the image carries a usable early\-warning signal for drift\. The two smoothed\-refresh architectures also fail\. Chunked rotation is strictly dominated at every matched budget \(losing0\.0460\.046–0\.1000\.100Jaccard, pairedp≤0\.057p\\leq 0\.057atC=16/8/4C\{=\}16/8/4\), and joint per\-step recomputation – at compute comparable toprefix\_refresh\_interval=1\(50\.6s vs\. 55\.3s\) – reaches only Jaccard0\.6160\.616with7/507/50exact matches, versus0\.9870\.987and40/5040/50\(pairedp<10−7p<10^\{\-7\}\)\.

A feasibility probe for learned gating\.Could a*trained*gating policy succeed where hand\-designed triggers fail? Across 160 runs with*randomized*refresh schedules \(20 images×\\times8\), no logged trajectory feature predicts final agreement post hoc \(cross\-validatedR2≈0\.04R^\{2\}\\\!\\approx\\\!0\.04from schedule statistics;≈0\\approx\\\!0with all features\)\. Most tellingly, re\-running the*same*image at the*same*average refresh rate with a different realization of refresh*times*shifts final Jaccard by0\.110\.11on average \(max0\.530\.53\) – nearly the full between\-run standard deviation \(0\.1460\.146\) – leaving essentially nothing for a gating network to learn from\.

Interpretation\.Together these results support a simple reading: generation under confidence\-aware parallel decoding is highly sensitive to the exact realization of the refresh schedule, and reproducing the reference trajectory is brittle in a near all\-or\-nothing way – freezing 1\.2% of the sequence collapses exact reproduction from40/5040/50to7/507/50; adding an alternating image/text decomposition collapses agreement the rest of the way into the stale\-cache band \(0\.410\.41–0\.470\.47\)\. Across every variant we designed, agreement tracked one quantity only: how close the per\-step computation was to*exactly*the reference computation – never the cleverness of when, or how smoothly, an approximation was applied\. This gives the refresh interval’s dominance a structural explanation: among the parameters studied, it is the only one that buys agreement by making a fraction of steps exactly right rather than by introducing a new approximation whose deviation compounds\. This conclusion covers the signal\-triggered and smoothed\-refresh families tested here; it cannot rule out fundamentally different designs \(trained gating over richer features, retrieval\-style re\-grounding\)\. Within the design space we explored, the fixed interval was never beaten, and we propose it as the frontier for future work to beat\.

## 9\.Qualitative Analysis and Manual Verification

Word\-level Jaccard is coarse, so we manually classify 50 low\-similarity, image\-deduplicated pairs as*content substitution*,*benign paraphrase*, or*mixed*\. This targeted\-tail, single\-author audit finds 25/50 \(50%\) substitutions, 8/50 \(16%\) paraphrases, and 17/50 \(34%\) mixed; the substitution rate is stable between a 15\-case pilot \(47%\) and 35\-case extension\. Examples include a Roman street becoming unrelated ruins and an invented Hebrew translation, while “Fenders Dinner” versus “Fender’s Dinner” is benign\. Thus Jaccard conflates errors with rewording, but is not merely measuring lexical diversity; the blinded study below separately evaluates factuality against images\.

Semantic corroboration\.As a second, non\-lexical check, we compute BERTScore\-F1 \(RoBERTa\-large, rescaled\) on the paired 50\-image subset across all three sweeps \(900 generations\)\. It reproduces every qualitative finding: threshold\-insensitivity \(flat at0\.5680\.568–0\.5760\.576; Friedmanp=0\.76p=0\.76\), monotonic Fast\-dLLM recovery \(0\.576→0\.9910\.576\\to 0\.991; Page’sLL,p<10−10p<10^\{\-10\}\), and dLLM\-Cache’s small, insufficient recovery \(0\.407→0\.480\.407\\to 0\.48; Friedmanp=0\.006p=0\.006\)\.

Blinded image\-grounded evaluation\.The audit above classifies*differences*between paired outputs; it does not measure which side is right about the image\. Two independent computer\-science undergraduate annotators, neither involved in method development or experiments, therefore evaluated all 50 image pairs \(baseline vs\. default acceleration\) with system identities hidden, A/B order randomized per item, and judgments made against the image\. They recorded a five\-way verdict and the number of image\-inconsistent claims in each description\. The proportion containing at least one such claim is32%/32%32\\%/32\\%\(annotator 1\) and54%/52%54\\%/52\\%\(annotator 2\) for baseline/acceleration\. For each image, we average the two annotators’ accelerated\-minus\-baseline error\-count differences and test these 50 paired image\-level averages\. The mean is0\.000\.00\(95% bootstrap CI\[−0\.17,\+0\.17\]\[\-0\.17,\+0\.17\]; Wilcoxonp=0\.97p=0\.97\); strictly\-better vs\. strictly\-worse verdicts are7:77\{:\}7and8:78\{:\}7\. Five\-way agreement is low \(Cohen’sκ=0\.18\\kappa=0\.18;0\.210\.21–0\.260\.26after binarization\)\. Thus this study detects no factual\-error difference at the default setting, but its interval and annotator disagreement do not establish factual equivalence or rule out moderate effects\. Our main result remains about consistency with unaccelerated behavior\.

Ground\-truth spot\-check\.Five low\-similarity cases show why drift is not error\. The baseline is better on an illegible Hebrew plaque \(the accelerated output invents a translation\); acceleration is better on a ruins painting and church figures; and both outputs misidentify cities in two cases\. “Drift from baseline” therefore must not be read as “drift from truth”: our diagnosis concerns how content changes, not which side is more accurate\.

## 10\.Discussion and Limitations

Table 6\.Experiment ledger\. RQ1 uses 300 paired baselines; targeted studies reuse its fixed 50\-image subset\.Practical guidance\.For output stability, treat KV\-cache refresh interval rather than confidence threshold as the primary lever \(Table[3](https://arxiv.org/html/2607.29079#S6.T3)\): the default gives1010–12×12\\timesspeedup, while conservative intervals improve reproducibility and still retain1\.3×1\.3\\times\. This is not a safety control; it reproduces the baseline’s errors as well as its behavior \(Section[9](https://arxiv.org/html/2607.29079#S9)\)\. More generally, sweeping an exposed hyperparameter against a paired unaccelerated reference costs only a few hundred generations and should precede deployment\.

Scope\.Table[6](https://arxiv.org/html/2607.29079#S10.T6)itemizes 4,930 generations \(4,690 on RTX 6000 Ada; 240 A800 timing replications\)\. RQ1 uses 300 images; costlier mechanism and generalization studies use paired 50\-image subsets\.

We note several limitations\.

*First*, causal/remedy studies use LLaDA\-V; LaViDa supports the diagnosis but lacks the refresh knob needed to test the remedy\. Prompt generalization covers two additional single\-turn formats on an independent 50\-image sample, and a 20\-image MME yes/no pilot points the same way, but larger\-scale and multi\-turn tests remain future work\.

*Second*, Jaccard is lexical; a 50\-case audit and BERTScore corroborate each qualitative finding \(Section[9](https://arxiv.org/html/2607.29079#S9)\), but larger model\-judged studies would strengthen it\.

*Third*, dLLM\-Cache transfers the diagnosis but requires tightening both caches to recover agreement, eliminating its speed advantage \(Section[7](https://arxiv.org/html/2607.29079#S7)\)\.

*Fourth*, agreement is not ground truth\. Our blinded image\-grounded study covers only the default configuration and 50 images \(detectable effect roughly±0\.17\\pm 0\.17errors/description\), with low five\-way inter\-annotator agreement \(κ=0\.18\\kappa=0\.18\); broader factuality evaluation remains future work\.

*Fifth*, our Web\-facing motivation is not a Web\-specific evaluation\. MME is a general multimodal benchmark; we do not test rendered webpages, search or ranking outcomes, product/news pages, or end\-to\-end agent decisions\. Those settings motivate the diagnostic but are not measured deployment claims\.

## 11\.Conclusion

Within the parallelism range studied, confidence threshold does not control acceleration\-induced content drift despite being demonstrably operative\. KV\-cache refresh interval is the dominant observed consistency control for the tested Fast\-dLLM implementation, providing a continuous speed–agreement trade\-off; dLLM\-Cache reproduces the initial diagnosis but not the efficient remedy, and LaViDa provides an auxiliary cross\-model check\. Our target is consistency with unaccelerated behavior, not factuality: the limited blinded study detects no factual\-error difference but does not establish equivalence\. The paired diagnostic may be useful for evaluating other accelerated generators\.

## Ethical Considerations

This work analyzes publicly released, pretrained multimodal models and the public MME benchmark, and releases no new model weights\. Human involvement consisted of a bounded evaluation by two computer\-science undergraduates who were independent of method development and experiment execution\. They evaluated model\-generated descriptions against public benchmark images; the study analyzes the model outputs, not the annotators\. We draw no conclusions about annotator behavior, use no demographic or personal attributes as study variables, and report judgments only in aggregate\. Our drift reference is the model’s own unaccelerated output, not ground truth, so high agreement must not be interpreted as factual reliability\. Likewise, refresh interval is a consistency control, not a safety control: conservative refresh reproduces baseline behavior, including its errors\. Deployment in medical, legal, accessibility, surveillance, or other consequential settings requires task\-specific factuality evaluation and appropriate human oversight regardless of acceleration settings\. Our findings are diagnostic; we are not aware of additional dual\-use risk from characterizing this existing acceleration behavior\.

## References

- \(1\)
- Fu et al\.\(2023\)Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al\.2023\.MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models\.*arXiv preprint arXiv:2306\.13394*\(2023\)\.
- Israel et al\.\(2025\)Daniel Israel, Guy Van den Broeck, and Aditya Grover\. 2025\.Accelerating Diffusion LLMs via Adaptive Parallel Decoding\.*arXiv preprint arXiv:2506\.00413*\(2025\)\.
- Kang et al\.\(2025\)Wonjun Kang, Kevin Galim, Seunghyuk Oh, Minjae Lee, Yuchen Zeng, Shuibai Zhang, Coleman Hooper, Yuezhou Hu, Hyung Il Koo, Nam Ik Cho, and Kangwook Lee\. 2025\.ParallelBench: Understanding the Trade\-offs of Parallel Decoding in Diffusion LLMs\.*arXiv preprint arXiv:2510\.04767*\(2025\)\.
- Kasa et al\.\(2026\)Siva Rajesh Kasa, Yasong Dai, Sumit Negi, and Hongdong Li\. 2026\.Fast\-dLLM\+\+: Fréchet Profile Decoding for Faster Diffusion LLM Inference\.*arXiv preprint arXiv:2606\.02955*\(2026\)\.
- Koh et al\.\(2024\)Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Lim, Po\-Yu Huang, Graham Neubig, Shuyan Zhou, Russ Salakhutdinov, and Daniel Fried\. 2024\.VisualWebArena: Evaluating Multimodal Agents on Realistic Visually Grounded Web Tasks\. In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics*\. Association for Computational Linguistics, 881–905\.
- Li et al\.\(2026\)Duo Li, Zuhao Yang, Xiaoqin Zhang, Ling Shao, and Shijian Lu\. 2026\.A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion\-based Multimodal Large Language Models\. In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*\.
- Li et al\.\(2025a\)Shufan Li, Konstantinos Kallidromitis, Hritik Bansal, Akash Gokul, Yusuke Kato, Kazuki Kozuka, Jason Kuen, Zhe Lin, Kai\-Wei Chang, and Aditya Grover\. 2025a\.LaViDa: A Large Diffusion Model for Vision\-Language Understanding\. In*Advances in Neural Information Processing Systems*\.
- Li et al\.\(2025b\)Zhuowei Li, Haizhou Shi, Yunhe Gao, Di Liu, Zhenting Wang, Yuxiao Chen, Ting Liu, Long Zhao, Hao Wang, and Dimitris N\. Metaxas\. 2025b\.The Hidden Life of Tokens: Reducing Hallucination of Large Vision\-Language Models via Visual Information Steering\. In*Proceedings of the 42nd International Conference on Machine Learning**\(Proceedings of Machine Learning Research, Vol\. 267\)*\. PMLR, 35799–35819\.
- Liu et al\.\(2025\)Zhiyuan Liu et al\.2025\.dLLM\-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching\.*arXiv preprint arXiv:2506\.06295*\(2025\)\.
- Nie et al\.\(2025\)Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji\-Rong Wen, and Chongxuan Li\. 2025\.Large Language Diffusion Models\.*arXiv preprint arXiv:2502\.09992*\(2025\)\.
- Wu et al\.\(2025\)Chengyue Wu, Hao Zhang, Shuchen Xue, et al\.2025\.Fast\-dLLM: Training\-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding\.*arXiv preprint arXiv:2505\.22618*\(2025\)\.
- You et al\.\(2025\)Zebin You, Shen Nie, Xiaolu Zhang, Jun Hu, Jun Zhou, Yankai Lin, Ji\-Rong Wen, and Chongxuan Li\. 2025\.LLaDA\-V: Large Language Diffusion Models with Visual Instruction Tuning\.*arXiv preprint arXiv:2505\.16933*\(2025\)\.
- Yuan et al\.\(2026\)Yulin Yuan, Hongshuo Zhao, and Xiangming Meng\. 2026\.Visual\-Redundancy\-Controlled Parallel Decoding for Diffusion\-Based Multimodal Large Language Models\.*arXiv preprint arXiv:2605\.25820*\(2026\)\.
- Zheng et al\.\(2024\)Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su\. 2024\.GPT\-4V\(ision\) is a Generalist Web Agent, if Grounded\. In*Proceedings of the 41st International Conference on Machine Learning**\(Proceedings of Machine Learning Research, Vol\. 235\)*\. PMLR, 61349–61385\.