diffusion-language-models

The paper demonstrates a targeted bias injection attack on frozen diffusion language models (dLLMs) using a proportional-integral closed-loop controller that adapts steering strength during denoising, strongly increasing biased answers on benchmarks like BBQ and SocialStigmaQA, and argues bias audits must examine the serving stack rather than just the frozen model.

Mosaic is a constrained decoding framework for diffusion language models that significantly improves performance on function calling benchmarks like JevBench and is compatible with vLLM, SGLang and Hugging Face, supporting models such as DiffusionGemma and LLaDA2.

The paper introduces GoldiMask, a method for supervised fine-tuning of diffusion language models that selects context tokens via submodular optimization and weights prediction targets based on context benefit, achieving improved accuracy on reasoning and code generation benchmarks across multiple backbones.

The paper shows that scaling text embeddings (e.g., swapping T5-small for T5Gemma-2-270M) greatly improves continuous diffusion language models, and that distilling the scaled embeddings into a more connected latent space makes them easier to generate — achieving Gen. PPL 17.8 that outperforms GPT-2-M.

FluxLite introduces a training-free, inference-time proposal-control framework for discrete diffusion models that compensates jump-rate perturbations via a graph-divergence term in the Feynman-Kac potential, yielding two samplers (HEU and D-VCG) that substantially reduce reweighting variance and sampling error over standard SMC baselines.

论文提出 E-MoE（Enhanced Mixture-of-Experts），用 MoE 路由决策构成的离散共享隐变量来建模非因子化的反向过程，缓解连续隐变量的后验坍缩问题，并在无需增加活跃参数的情况下提升掩码扩散语言模型在少步生成场景下的样本质量。

ELF-REG scales continuous diffusion language models to reasoning tasks by using representation alignment and entanglement, achieving improved performance on benchmarks like GSM8K and HumanEval compared to other diffusion models.

The paper introduces LOCKR, a planner that uses hidden-state trajectories to detect and repair stable-but-wrong lock-in in diffusion language models, achieving absolute accuracy gains of 2.21–5.37 percentage points on mathematical reasoning benchmarks.

The paper introduces CaLR, a framework that reformulates reasoning as constrained latent optimization using causal topology to enhance diffusion language models, achieving state-of-the-art performance on complex benchmarks.

This paper introduces probe guidance, a new method for flow matching models in continuous diffusion language models, which achieves state-of-the-art performance on unconditional generation and improves multiple choice question answering benchmarks while providing insights into autoguidance.

The paper proposes Pill, an efficient adaptive-length infilling method for diffusion language models that improves performance on code and text infilling tasks while reducing inference time.

The paper proposes a survival-guided length predictor for diffusion language models that speeds up inference by up to 7x on reasoning and code-generation benchmarks without sacrificing accuracy.

GRAFT introduces a draft-tree construction framework for diffusion language model-based speculative decoding, optimizing edge selection and budget allocation to achieve 2.13×–6.36× speedup over autoregressive decoding with low overhead.

LoRA-Diffusion proposes a parameter-efficient fine-tuning method for diffusion-based language models by applying low-rank decomposition to the denoising trajectory rather than model weights, achieving competitive performance with only 1.2% trajectory adapter parameters.

This paper introduces Diffusion Language Models (DLMs) as a new inference paradigm for lossless text compression, aiming to overcome the throughput bottlenecks of autoregressive LLM-based compressors while achieving state-of-the-art compression ratios.

CORA-Diff is a training-free method that accelerates diffusion language model inference by using native confidence and persistence signals to accept residual positions early, skipping redundant dense denoising passes while preserving task quality.

Introduces Continuous Interaction Diffusion (CID), a diffusion-native model–runtime architecture that integrates tool interaction into iterative denoising, enabling asynchronous external reads while generation continues without waiting for discrete tool calls.

This paper identifies a 'representation confidence gap' in diffusion language models: internal states detect input noise accurately but reported confidence stays high and answer ranking degrades under noise. It introduces a lightweight, training-free extraction tool that leverages hidden states to improve ranking without modifying the base model.

Introduces Archer, a training-free KV caching method for diffusion language models that adaptively reuses cached hidden states to reduce recomputation while preserving rollback capabilities, achieving up to 2.95x speedup and improved generation quality.

The paper investigates whether retrieving more evidence helps visual retrieval-augmented generation with diffusion language models, finding that unconditionally expanding evidence hurts accuracy due to semantic conflict, and proposes a training-free Entropy-Based Candidate Filter (ECF) to selectively admit evidence, improving accuracy across benchmarks.