infilling

该论文将离散扩散的前向过程重新设计为可调的"信任轴"，通过区域非均匀的腐蚀核使模型能够检测并纠正被污染的条件上下文，在170M参数掩码扩散模型上以低于5小时的微调取得了显著的上下文腐蚀检测与infilling提升。

The paper proposes Pill, an efficient adaptive-length infilling method for diffusion language models that improves performance on code and text infilling tasks while reducing inference time.

Proposes ac-gpt, a simple modification to causal Transformers that enables evaluating and sampling from arbitrary conditionals (past, future, mixed) in a single forward pass while preserving left-to-right ordering and next-token prediction, allowing existing LLMs to be fine-tuned for arbitrary conditioning.

This paper introduces infilling extraction, a new method for extracting training data from diffusion language models by using arbitrary binary masks, showing that such models are more vulnerable to memorization attacks than previously thought.