Tag
Introduces HDD-RoPE, an extension of rotary positional embeddings that uses high-dimensional chunks and data-dependent rotation rates, showing faster convergence on TinyStories compared to xPos.
This paper demonstrates that text-to-image diffusion transformer models primarily rely on token merging and word order from text encoders rather than full contextual embeddings, suggesting that the image model itself decodes complex linguistic structures.
Bernini proposes a unified video generation and editing framework that combines multimodal large language models for semantic planning with diffusion models for pixel rendering, achieving state-of-the-art performance through semantic interface separation and enhanced positional embeddings.
EndPrompt proposes a method for extending the context window of large language models using only short training sequences, by anchoring a terminal prompt with target-length positional indices. It achieves strong benchmark results with substantially less computation than full-length fine-tuning.