LiFT: Loop Flow Transformers

Hugging Face Daily Papers Papers
diffusion-transformers looped-transformers flow-matching image-generation efficient-inference imagnet

Summary

LiFT introduces a family of looped generative models that scale computation by repeatedly applying a shared Diffusion Transformer (DiT) core, training each loop step on a continuous-depth flow-matching regression target that allows inference far beyond training depth. On ImageNet 256x256, LiFT-L/2 beats a dense DiT-XL/2 baseline by 3.34 FID points while using ~60% fewer parameters and 52% fewer inference FLOPs.

We introduce Loop Flow Transformers (LiFT), a family of looped generative models that scales computation by repeatedly applying a shared Diffusion Transformer (DiT) core, with only light changes to the standard architecture. Rather than asking every recurrent step for the final prediction, LiFT trains each step with a single regression target: a point on a straight path from the model's initial estimate to the flow-matching target. Because we index these targets by a continuous depth coordinate, a trained model can loop far beyond its training depth with no retraining, early exits, or other modifications. In our experiments, these longer rollouts improve generation, so inference computation can grow without adding parameters. On ImageNet at 256x256, LiFT-L/2 achieves an FID 3.34 points lower than our dense DiT-XL/2 baseline while using approximately 60% fewer parameters, 32% fewer training FLOPs, and 52% fewer inference FLOPs.
Original Article
View Cached Full Text

Cached at: 10/06/26, 12:54 PM

Paper page - LiFT: Loop Flow Transformers

Source: https://huggingface.co/papers/2610.05538

Abstract

WeintroduceLoopFlowTransformers(LiFT),afamilyofloopedgenerativemodelsthatscalescomputationbyrepeatedlyapplyingasharedDiffusionTransformer(DiT)core,withonlylightchangestothestandardarchitecture.Ratherthanaskingeveryrecurrentstepforthefinalprediction,LiFTtrainseachstepwithasingleregressiontarget:apointonastraightpathfromthemodel’sinitialestimatetotheflow-matchingtarget.Becauseweindexthesetargetsbyacontinuousdepthcoordinate,atrainedmodelcanloopfarbeyonditstrainingdepthwithnoretraining,earlyexits,orothermodifications.Inourexperiments,theselongerrolloutsimprovegeneration,soinferencecomputationcangrowwithoutaddingparameters.OnImageNetat256x256,LiFT-L/2achievesanFID3.34pointslowerthanourdenseDiT-XL/2baselinewhileusingapproximately60%fewerparameters,32%fewertrainingFLOPs,and52%fewerinferenceFLOPs.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2610\.05538

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2610.05538 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2610.05538 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2610.05538 in a Space README.md to link it from this page.

Collections including this paper1

Similar Articles

DeepLoop: Depth Scaling for Looped Transformers

arXiv cs.LG

DeepLoop introduces a residual scaling method for looped Transformers that adjusts for parameter visits, improving stability and performance when physical blocks are reused across multiple rounds.

Looped Diffusion Transformer: the same blocks run several times inside each denoising step, and loop depth can be changed at sampling time [R]

Reddit r/MachineLearning

介绍 Looped Diffusion Transformer 论文：通过在每个去噪步骤内重复运行共享的 transformer 块，并结合深度监督和自调制注意力，使 260M 模型以 4.9 倍更少的推理计算超越 6.5 倍大的模型，且循环深度可在采样时调节；作者还用 MIT 开源权重在 M5 Pro 上实测了不同深度的效果。

A new transformer passes its hidden state to the next token instead of recomputing it (25 minute read)

TLDR AI

The paper introduces LIFT (Latent Information Feedback Transformer), which propagates deep-layer hidden states across generation steps instead of relying solely on decoded tokens, using teacher-forced pretraining with states derived from an off-the-shelf LM's next-token distribution. Experiments on 135M–1B models show LIFT outperforms standard Transformers on language modeling, reasoning, and procedural tasks under token-matched budgets, with code and models released.