BA-T: An Iterative Transformer for Two-View Bundle Adjustment
Summary
BA-T is an iterative Transformer architecture for two-view bundle adjustment that improves 3D reconstruction accuracy and cross-view consistency using a lightweight design with only 16% of conventional decoder parameters, matching or surpassing larger models.
View Cached Full Text
Cached at: 06/03/26, 03:38 PM
Paper page - BA-T: An Iterative Transformer for Two-View Bundle Adjustment
Source: https://huggingface.co/papers/2606.03287
Abstract
BA-T is an iterative Transformer architecture that improves 3D reconstruction accuracy and cross-view consistency through structured updates inspired by bundle adjustment, using a lightweight design that requires only 16% of conventional decoder parameters.
Feed-forward modelsfor3D reconstructionhave achieved strong performance usingdeep cross-view attentionto exchange information across images. However, these approaches often depend on heavydecoder stacksand lack a structured mechanism forgeometry refinement, resulting in poor multi-view consistency. We address this by drawing inspiration from classicalbundle adjustment(BA), which can be viewed as an iterative information propagation process between poses and local geometry. Inspired by BA, we propose BA-T, aniterative Transformerthat implements BA-style structured updates as a repeatable layer inimplicit token space. Instead of relying on deep attention stacks, BA-T refines predictions based onlatent residualby a single lightweight layer. Experiments demonstrate that BA-T progressively improves pose and reconstruction accuracy across iterations, achieves strongercross-view consistencythan conventional decoders, and matches or surpasses substantially larger models while using only 16% of their decoder parameters. BA-T provides a compact, efficient, and structural alternative todepth-heavy attention, enabling accurate3D reconstructionwithin a lightweight architecture. The code will be made publicly at https://github.com/zhangganlin/BA-T.
View arXiv pageView PDFGitHub5Add to collection
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2606.03287 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2606.03287 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2606.03287 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
Lite3R: A Model-Agnostic Framework for Efficient Feed-Forward 3D Reconstruction
Lite3R is a model-agnostic framework that improves the efficiency of transformer-based 3D reconstruction using sparse linear attention and FP8-aware quantization. It reduces latency and memory usage by up to 2.4x while maintaining geometric accuracy on backbones like VGGT and DA3-Large.
ResBM: a new transformer-based architecture for low-bandwidth pipeline-parallel training, achieving 128× activation compression [R]
ResBM introduces a transformer-based architecture with residual encoder-decoder bottlenecks for pipeline-parallel training, achieving 128× activation compression while maintaining convergence. The work advances decentralized, internet-grade distributed training by reducing inter-stage communication overhead.
Track2View: 4D-Consistent Camera-Controlled Video Generation via Paired 3D Point Tracks
Track2View generates novel camera viewpoints from videos by conditioning a video diffusion transformer on paired 3D point tracks, achieving state-of-the-art visual quality and significant reductions in rotation and translation errors.
Precision Tracked Transformer via Kalman Filtering, Kriging and Process Noise
The paper introduces the Bayesian Filtering Transformer (BFT), which incorporates uncertainty into Transformers via precision-weighted attention and Kalman update residuals, improving performance on sequential recommendation and noisy LLM fine-tuning.
Linearizing Vision Transformer with Test-Time Training
This paper proposes a method to convert pretrained Softmax attention models into linear-complexity Test-Time Training (TTT) architectures, achieving comparable text-to-image quality to fine-tuned Softmax models while significantly accelerating inference. The approach is validated by linearizing Stable Diffusion 3.5, resulting in SD3.5-T^5 with 1.32x speedup at 1K resolution.