@SabrinaHalper: .@dwarkesh_sp's episode with @ericjang11 is awesome. Eric has a rare gift for making complicated ideas feel simple, whi…

X AI KOLs Timeline News
podcast deep-learning ai-history neural-networks interview

Summary

Sabrina Halper recommends Dwarkesh Patel's podcast with Eric Jang, who discusses how deep learning progress has been driven more by compute than by biological inspiration.

.@dwarkesh_sp's episode with @ericjang11 is awesome. Eric has a rare gift for making complicated ideas feel simple, which is exactly what struck me when I first met him at a dinner and immediately peppered him with questions. If you are looking to hear more from him, listen to our conversation from a couple years ago! "The history of neural networks and actually many things in AI have been inspired a lot by biology, like genetic algorithms, evolutionary algorithms, processes in both neuroscience and psychology and biology and so forth. I think drawing from nature is a great way to get inspiration... And this is going to get me a lot of flack from the scientific community, but I feel like in the last decade and a half-ish of deep learning progress, the vast majority of major contributions have come from people who did not really adhere to that way of thinking, but more like, how do I push as much data as fast as possible onto my GPU? It just empirically seems to me that people who are very attached to the idea of replicating a particular nature inspired architecture at the expense of enabling brute force compute... do not make the best algorithms. There's this essay by @sarahookr called the the hardware lottery where it's arguing that people who focused on maximizing and designing their algorithms to suit the hardware so that it could run the brute force thing as fast as possible tended to win. "
Original Article
View Cached Full Text

Cached at: 05/20/26, 04:25 AM

.@dwarkesh_sp’s episode with @ericjang11 is awesome. Eric has a rare gift for making complicated ideas feel simple, which is exactly what struck me when I first met him at a dinner and immediately peppered him with questions.

If you are looking to hear more from him, listen to our conversation from a couple years ago!

“The history of neural networks and actually many things in AI have been inspired a lot by biology, like genetic algorithms, evolutionary algorithms, processes in both neuroscience and psychology and biology and so forth. I think drawing from nature is a great way to get inspiration… And this is going to get me a lot of flack from the scientific community, but I feel like in the last decade and a half-ish of deep learning progress, the vast majority of major contributions have come from people who did not really adhere to that way of thinking, but more like, how do I push as much data as fast as possible onto my GPU? It just empirically seems to me that people who are very attached to the idea of replicating a particular nature inspired architecture at the expense of enabling brute force compute… do not make the best algorithms. There’s this essay by @sarahookr called the the hardware lottery where it’s arguing that people who focused on maximizing and designing their algorithms to suit the hardware so that it could run the brute force thing as fast as possible tended to win. “

Dwarkesh Patel (@dwarkesh_sp): New blackboard lecture w @ericjang11

He walks through how to build AlphaGo from scratch, but with modern AI tools.

Sometimes you understand the future better by stepping backward. AlphaGo is still the cleanest worked example of the primitives of intelligence: search, learning

Similar Articles