RenderFormer-V2: Neural Rendering with Heterogeneous Scene Primitives
Summary
RenderFormer-V2 is a transformer-based neural rendering model that handles diverse light-transport effects using a two-stage sequence-to-sequence architecture with improved attention mechanisms and support for heterogeneous scene primitives.
View Cached Full Text
Cached at: 09/10/26, 02:15 AM
Paper page - RenderFormer-V2: Neural Rendering with Heterogeneous Scene Primitives
Source: https://huggingface.co/papers/2609.05738
Abstract
RenderFormer-V2 is a transformer-based neural rendering model that handles diverse light-transport effects via a two-stage sequence-to-sequence architecture with improved attention and heterogeneous scene support.
We present ‘RenderFormer-V2’, a unified learnedtransformer-based neural renderingmodel, complementary to modern physics-based rendering systems, that can handle diverse light-transport effects such as caustics, volumetric scattering, environment lighting, textured and displaced surfaces and out-of-distribution materials without per-scene training or specialized code. RenderFormer-V2 modelsglobal light transportas asequence-to-sequence transformation. Following its predecessor, RenderFormer-V2 also employs a two stage process: a view-independent stage that resolves intra-scene primitive to primitive transport, and a view-dependent stage that transforms the internalneural scene representationinto image pixels. Different from RenderFormer, our model employs a novel combinedwindowed-attentionand rendering-informedattention sinkin the view-independent stage to improve scalability while maintaining render accuracy. To further improve versatility, RenderFormerV2 supportsheterogeneous scene primitives, including environment maps and participating media, and it employs amaterial encodingindependent of the underlying surface reflectance model that encodes material appearance via a novelneural embedding. We demonstrate the versatility of RenderFormer-V2 on a variety of scenes and perform an extensive ablation of the improved attention mechanism.
View arXiv pageView PDFProject pageGitHub1Add to collection
Get this paper in your agent:
hf papers read 2609\.05738
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2609.05738 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2609.05738 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2609.05738 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
RelightFormer: Feed-forward Generative Transformer for Multiview Object Relighting
RelightFormer introduces a feed-forward generative Transformer for direct single- and multi-view image relighting, using cross-attention for illumination injection and permutation-invariant encodings for unordered views, trained on a massive synthetic dataset to achieve state-of-the-art visual quality.
Neural Render Proxies for Interactive and Differentiable Lighting
Introduces a neural render proxy (NRP) for differentiable relighting of static scenes with fixed camera and materials at interactive rates, enabling fast gradient-based inverse workflows.
Walking in the Implicit: Interactive World Exploration via Neural Scene Representation
NeuWorld is a new interactive video generation system that uses compact neural implicit scene representations and a transformer VAE with diffusion transformer for trajectory-conditioned rendering, achieving long-horizon consistency.
RayDer: Scalable Self-Supervised Novel View Synthesis from Real-World Video
RayDer is a unified feed-forward transformer that consolidates camera estimation, scene reconstruction, and rendering for self-supervised novel view synthesis from real-world video, achieving clean power-law scaling and strong zero-shot performance.
Lite3R: A Model-Agnostic Framework for Efficient Feed-Forward 3D Reconstruction
Lite3R is a model-agnostic framework that improves the efficiency of transformer-based 3D reconstruction using sparse linear attention and FP8-aware quantization. It reduces latency and memory usage by up to 2.4x while maintaining geometric accuracy on backbones like VGGT and DA3-Large.