Do you agree with Judea that learning from data is not everything? [D]
Summary
Judea Pearl argues that there are mathematical limitations to learning solely from data, citing the inability to infer causation from correlation. The article prompts discussion on whether pure data-driven learning is sufficient.
Similar Articles
Data bottlenecks won't prevent an intelligence explosion (36 minute read)
The article argues that data bottlenecks will not prevent a fast intelligence explosion in AI, as software progress can overcome data limitations through improved algorithms and sample-efficient learning.
@alesfav: AI needs vastly more data than we do. One idea might close the gap: don't predict raw signals (tokens), predict your ow…
This thread presents a theoretical result showing that predicting abstract latent representations (as in JEPA and data2vec) instead of raw tokens can exponentially reduce the data gap between AI and human learning.
Rethinking the Data Moat (6 minute read)
The article discusses the evolving importance of data versus algorithmic progress in AI development, drawing from recent talks by researchers to highlight a shift in focus.
AI for science needs reasoning, not just data
This MIT Technology Review article argues that AlphaFold-style deep learning on massive datasets is not the ideal template for accelerating science, and that AI agents capable of reasoning and experimentation will drive future breakthroughs.
@omarsar0: If you aren't looking at the data, you aren't doing AI at all. It was true before and continues to be the case today.
Omar Saroo emphasizes that data inspection is crucial in AI development, illustrated by an example where ignoring data led to model degradation due to incorrect chat template usage.