Tag
This article explores the deep connections between physics and deep learning, analyzes the isomorphism of phenomena such as Scaling Law and emergence with concepts like critical scaling laws and phase transitions in physics, and reviews the current status and prospects of applying physical methodologies in AI.
StreamMA introduces a streaming communication paradigm for multi-agent reasoning that pipelines intermediate results to reduce latency and improve effectiveness by leveraging more reliable early steps, outperforming baselines across benchmarks and revealing a step-level scaling law.
Huawei unveils the Tau Scaling Law, a chip architectural workaround to bypass US sanctions and achieve 1.4nm-equivalent transistor density by 2031, marking a significant step toward China's semiconductor self-sufficiency and altering the tech rivalry with Washington.
Discusses that the mathematics used by AI is mainly linear algebra, calculus, etc., from before the 19th century, but emerging phenomena such as Scaling Law, emergent abilities, double descent, in-context learning, and representation geometry lack mathematical explanation. Analogizes to the clouds in physics in 1900, suggesting it may drive the development of 21st-century mathematics.
In the interview, Yao Shunyu proposed a contrarian view that pre-training has not hit a wall and Scaling Law has not reached its limit, claiming that most people who say it has hit a wall have bugs in their code.
After being laid off from Meta, Yuandong announced a new direction, raising $650 million to found neolab Recursive_SI with a valuation of $4.65 billion. In an interview, he shared insights on AI trends, LLM limitations, reinforcement learning, and research freedom.