Tag
Proposes ac-gpt, a simple modification to causal Transformers that enables evaluating and sampling from arbitrary conditionals (past, future, mixed) in a single forward pass while preserving left-to-right ordering and next-token prediction, allowing existing LLMs to be fine-tuned for arbitrary conditioning.
This paper introduces infilling extraction, a new method for extracting training data from diffusion language models by using arbitrary binary masks, showing that such models are more vulnerable to memorization attacks than previously thought.