@bravo_abad: A principled construction for turning CNNs and transformers into resolution-agnostic neural operators Train a U-Net or …
Summary
The paper presents a principled construction to convert CNNs and transformers into resolution-agnostic neural operators, enabling models trained at one grid resolution to generalize to others, crucial for scientific ML applications like PDE solving.
View Cached Full Text
Cached at: 07/05/26, 12:33 PM
A principled construction for turning CNNs and transformers into resolution-agnostic neural operators
Train a U-Net or a ViT at one grid resolution, then run it at another, and performance usually falls apart. The receptive field and the tokenization are tied to pixel indices rather than physical coordinates, so changing the grid spacing quietly changes the operation the layer performs. Fine for vision, a real problem for scientific ML.
Many physical systems (fluid flow, heat transfer, electromagnetism) are governed by PDEs and live on continuous function spaces. The object you want to learn is an operator: a map from one function, say a forcing term or coefficient field, to another, the solution. But standard networks consume finite vectors, and simulation data arrive on meshes and point clouds whose resolution varies from run to run.
Julius Berner and coauthors distil clear principles for neural operators, models that map between function spaces and stay consistent across discretizations. The central move: figure out which continuous operator a layer is secretly approximating, then discretize it so it respects coordinates and quadrature weights instead of array indices. Sums over indices become integrals over the domain, and latent interfaces stay independent of input resolution.
The construction converts familiar blocks almost one to one: fully connected layers become integral transforms, convolutions become spectral operators (the FNO), attention becomes a global integral operator. On a Navier-Stokes benchmark, models trained only at resolution 128 tell the story: FNO and the OFormer transformer generalize from 64 up to 1024, while U-Net and ViT degrade sharply off the training grid. Quadrature weights matter too: drop them and aggregating irregularly spaced values overweights dense regions, so the output never converges.
For pipelines in drug discovery, materials development and energy modeling, the payoff is training once and deploying across resolutions: learn on cheap low-resolution simulations, reserve a few expensive high-resolution runs, and query the trained model at whatever fidelity the task needs.
Paper: Berner et al., Nature Machine Intelligence (2026), CC BY 4.0 | https://doi.org/10.1038/s42256-026-01267-z…
Client Challenge
Source: https://idp.nature.com/transit?redirect_uri=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs42256-026-01267-z&code=c0214b1a-7028-4d00-887a-9d6f76616272 A required part of this site couldn’t load. This may be due to a browser extension, network issues, or browser settings. Please check your connection, disable any ad blockers, or try using a different browser.
Similar Articles
A Robust Foundation Model for Conservation Laws: Injecting Context into Flux Neural Operators via Recurrent Vision Transformers
This paper proposes a new architecture that augments Flux Neural Operators with recurrent Vision Transformers to solve conservation laws as a foundation model. It demonstrates robust generalization and long-time prediction capabilities across diverse conservative systems without explicit access to governing equations.
@AnimaAnandkumar: This is something I have been emphasizing since we started our work on Neural Operators. We very quickly went from simp…
Anima Anandkumar highlights that neural operators, despite simple benchmarks, have achieved massive speedups (10,000–million times) in hard real-world problems like high-resolution AI weather modeling (FourCastNet) and nuclear fusion turbulence, referencing a new paper showing learned solvers become more cost-effective as PDE tasks get harder.
eCNNTO: A Highly Generalizable ConvNet for Accelerating Topology Optimization
This paper proposes eCNNTO, a CNN with residual connections to accelerate density-based topology optimization by predicting near-optimal densities from early iteration histories, achieving up to 97% reduction in iterations and strong generalization across different boundary conditions, geometries, and mesh resolutions.
ResBM: a new transformer-based architecture for low-bandwidth pipeline-parallel training, achieving 128× activation compression [R]
ResBM introduces a transformer-based architecture with residual encoder-decoder bottlenecks for pipeline-parallel training, achieving 128× activation compression while maintaining convergence. The work advances decentralized, internet-grade distributed training by reducing inter-stage communication overhead.
@rohanpaul_ai: New Microsoft paper argues that transformers generalize better when they learn compact internal states, not just next t…
Microsoft's NextLat paper proposes a self-supervised training method where transformers predict their next hidden state instead of just the next token, leading to more compact world models, better planning and reasoning, and up to 3.3x faster generation.