Paper page - Keep-or-Drop? Adaptive Tokenizer for Compact Video Representation

Source: https://huggingface.co/papers/2608.24293 Published on Aug 25

·

Submitted byhttps://huggingface.co/yeon-kyeong

leeon Sep 1

Abstract

KATok is an adaptive transformer-based video tokenizer that selectively drops uninformative tokens to achieve data-dependent compression while preserving spatial consistency for diffusion-based generation.

Latent diffusion modelshave emerged as a dominant framework for high-fidelity image and video synthesis, operating in compact latent spaces withvariational autoencoders(VAEs) to enhance computational efficiency without compromising visual quality. However, conventional VAEs are suboptimal for video data as they employ fixed compression ratios that cannot adapt to the varying complexity of spatio-temporal content. We present KATok (Keep-or-Drop? Adaptive Tokenizer for Compact Video Representation), a transformer-based VAE that incorporates anadaptive token selectorwhich is jointly learned with latent tokens. By evaluating each token’s content-richness askeep-or-drop probability, the token selector effectively discards uninformative tokens, naturally allowing data-dependent compression. Applying adaptive tokenization to diffusion models may cause spatial misalignment, as token dropping can disturb the original spatio-temporal structure. To alleviate this issue, we propose twoposition-prediction strategies: cascaded andjoint generation, to ensure spatial consistency. We empirically show that our model achieves strong reconstruction and generation quality at a state-of-the-art compression ratio. Further analysis on video data reveals that this improvement is primarily achieved by reducingspatio-temporal redundancyand removing uninformative tokens, as supported by both quantitative and qualitative results.

View arXiv pageView PDFProject pageAdd to collection

Get this paper in your agent:

hf papers read 2608\.24293

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2608.24293 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2608.24293 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2608.24293 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.