Tag
The author comments that current AI research overuses the thinking style of computer science and lacks a physics-based approach, proposing the need to establish an ideal system like 'Cyber Space' to lay a theoretical foundation.
Discusses that the mathematics used by AI is mainly linear algebra, calculus, etc., from before the 19th century, but emerging phenomena such as Scaling Law, emergent abilities, double descent, in-context learning, and representation geometry lack mathematical explanation. Analogizes to the clouds in physics in 1900, suggesting it may drive the development of 21st-century mathematics.
This simulation study examines the double descent phenomenon for least-squares interpolation on contaminated data in linear regression, comparing the performance of the least-squares interpolator with robust alternatives.
OpenAI research reveals the 'double descent' phenomenon where test error exhibits a non-monotonic pattern as both model size and training steps increase, challenging traditional understanding of the bias-variance tradeoff in deep learning.