@snowboat84: This is the second part of the "When Physics Meets AI" series. The role of physics in AI can be divided into four layers: (1) The first layer is the bottommost, providing the computational skeleton—energy, entropy, and free energy are embedded into AI's training objectives. (2) The second layer is the middle layer, where physics shapes the network architecture—Hopfield's Ising energy function, CNN's translational symmetry, and renormalization group correspond to the hierarchical structure of deep networks.
Summary
This article explores the four layers of physics' role in AI, from the bottom computational skeleton to the methodological layer, arguing that physics' methodology is migrating from the natural world to the AI domain.
View Cached Full Text
Cached at: 06/05/26, 01:15 PM
This is the final installment of the When Physics Meets AI series. Physics contributes to AI at four main levels:
(1) The first and most fundamental level supplies the computational backbone—energy, entropy, and free energy are embedded into the training objectives of AI.
(2) The second, intermediate level shapes the structure of networks: Hopfield’s Ising energy function, the translational symmetry of CNNs, and the correspondence between renormalization groups and the hierarchical architecture of deep networks.
(3) The third, analogical level provides a language for understanding AI phenomena—physical tools such as critical scaling laws, spin glasses, and metastability are borrowed to analyze open problems in AI.
(4) The fourth, methodological level occasionally yields breakthroughs that can shift the AI paradigm, with diffusion models standing as one example.
Physics, as “the study of nature,” has reached its frontier—no experimental breakthrough beyond the Standard Model has emerged in half a century. But physics, as “a methodology for tackling complex systems,” has not lost its power. That methodology is now migrating from the natural world to the world of AI, moving physical tools over one by one, to see which can take root in new ground. Physics once asked the universe: “When countless individuals interact, what emerges as a whole?” The answers were statistical mechanics, thermodynamics, and phase transition theory. Now it asks the same question of AI. What the answer this time will be called, no one yet knows. But the tools for asking the question are already in hand.
This is the 45th article in my series of 100 long-form original pieces.
Similar Articles
@snowboat84: To add a supplementary note, regarding the phenomena emerging from AI—scaling laws, emergence, double descent, representation geometry—the papers discussing them are already numerous. But there is a big problem: they are all thinking in the way of computer scientists, not physicists. What is a computer sci…
The author comments that current AI research overuses the thinking style of computer science and lacks a physics-based approach, proposing the need to establish an ideal system like 'Cyber Space' to lay a theoretical foundation.
@snowboat84: https://x.com/snowboat84/status/2062686432335184321
This article explores the deep connections between physics and deep learning, analyzes the isomorphism of phenomena such as Scaling Law and emergence with concepts like critical scaling laws and phase transitions in physics, and reviews the current status and prospects of applying physical methodologies in AI.
@vincemask: Put together, this is the complete AI pipeline: Underlying principles → Model operation → Capability optimization → Product deployment. Breaking it into 4 layers makes it clear: 1. Principle layer: AI's foundation. Neural networks, tokenization, embeddings, attention, Transformer. Addresses: how models understand text, semantics, and context. ...
This post divides the complete AI pipeline into four layers: Principle layer, LLM operation layer, Optimization layer, and System layer, explaining respectively how models understand language, generate answers, optimize performance, and deliver products.
@snowboat84: https://x.com/snowboat84/status/2065215177029787705
This article is the middle part of the AI Engineering Landscape series, detailing core techniques such as inference optimization, model slimming (quantization, distillation, pruning, MoE), and speculative decoding, while reviewing the latest advances from hardware to the engineering stack.
@snowboat84: Today, let's discuss something hardcore. One question: what level of mathematics does AI use? From the perspective of tools and models themselves, the mathematics used by AI has an average age of 150 years, with most being from before the mid-19th century: matrix multiplication, gradient descent, chain rule, Fourier transform, inner product, probability — mostly content from the first two years of undergraduate studies. But some phenomena emerging from AI...
Discusses that the mathematics used by AI is mainly linear algebra, calculus, etc., from before the 19th century, but emerging phenomena such as Scaling Law, emergent abilities, double descent, in-context learning, and representation geometry lack mathematical explanation. Analogizes to the clouds in physics in 1900, suggesting it may drive the development of 21st-century mathematics.