The mathematics of AI uncertainty

**TL;DR:** AI systems often lack self-doubt, but probabilistic reasoning—key to human intelligence—is essential for building robust, trustworthy AI that understands its own limitations. ## The Importance of Uncertainty in Intelligent Systems The core argument presented is that true intelligence, whether biological or artificial, fundamentally requires the ability to handle uncertainty. Decision-making, a cornerstone of intelligence, occurs in a world where our perception of reality is always incomplete and the future is unpredictable. An intelligent system must be able to represent, update, and act upon its own state of uncertainty to make good decisions. Zoubin Ghahramani, Chief Scientist for AI at Google DeepMind and a Cambridge professor, has pioneered a type of intelligence based on the mathematics of uncertainty for over three decades. ## Types of Uncertainty Two primary types of uncertainty are discussed: 1. **Aleatoric Uncertainty (偶然不确定性):** This is inherent randomness in the world. An example given is the unpredictable direction a pedestrian on a street might turn, like the outcome of a coin toss. 2. **Epistemic Uncertainty:** This stems from encountering novel or unfamiliar situations—the "long-tail events." For an autonomous car, this could be navigating a hailstorm with a horse suddenly appearing in its path, a scenario not seen during its training. The crucial capability for an AI is to have a form of "self-awareness" about its own uncertainty. A self-driving car that recognizes it is in an uncharted situation (e.g., unsure if an object is a bicycle or a horse) could make a safe decision, like slowing down. From a mathematical perspective, the elegance is that all these forms of uncertainty can be mapped to probability. Probability theory provides the rules to update and reason under this uncertainty. ## Connection to Human Cognition This probabilistic framework aligns closely with models of human cognition. Our perception is inherently uncertain. When we hear a rustling in the leaves in the Costa Rican jungle, we use our senses (the sound) and prior beliefs (jaguar presence is possible there) to form a probabilistic inference. While humans are notoriously poor at estimating probabilities consciously (as shown by Kahneman and Tversky), our subconscious perceptual systems handle probabilistic inference effectively, which is vital for survival. Even infant learning is driven by implicit beliefs about how actions lead to outcomes, a process cognitive scientists model using the same formal probabilistic methods applied to AI. ## Correctness, Confidence, and Adversarial Attacks A critical distinction is made between correctness and confidence. A system can be correct but overconfident, or incorrect but appropriately uncertain. A famous example from image classification demonstrates this vulnerability: a neural network can be shown an image of a school bus, slightly altered by changing a few pixels in a way invisible to humans. The network will then incorrectly and *confidently* classify the altered image as a cheetah, monkey, or another animal. This phenomenon of "adversarial examples" shows that AI systems must not only be accurate but also have well-calibrated confidence. We cannot trust systems that are easily fooled or that express high confidence in incorrect answers. As large language models (LLMs) are deployed, this becomes a trust issue. An LLM might give a confident answer, then change it when questioned. A truly trustworthy intelligent system must recognize and communicate its own limitations and knowledge boundaries. ## Ghahramani's Journey in AI Zoubin Ghahramani's career provides historical context: * **1980s Revolution:** His entry into AI research began in 1986 with a summer job studying early neural networks ("Parallel Distributed Processing"). This was a revolution away from brittle expert systems towards flexible, data-driven learning, attracting interdisciplinary talent. * **Early Work on Neural Networks:** In 1989, he wrote a thesis on using neural networks for natural language parsing—early work akin to small language models. He worked on massive parallel computers like the Connection Machine. * **Shift to Probabilistic Models:** In the early 1990s, as the connection between neural networks and probability/statistics became clear, he pivoted to research on Bayesian machine learning and probabilistic models, though he never completely abandoned neural networks. * **2015 *Nature* Paper:** He co-authored a pivotal paper arguing that precise probabilistic representation of uncertainty is key to many aspects of intelligence, published alongside foundational deep learning and reinforcement learning work from his colleague Geoffrey Hinton. ## Probabilistic AI: The Bayesian Perspective Modern large language models *are* probabilistic models at their core—they predict the probability of the next token. However, the missing piece is an *explicit* representation of uncertainty. In a massive neural network, uncertainty is diffuse across billions of activations, not neatly packaged. Ghahramani advocates for a Bayesian approach. Bayesian inference is a formal framework for updating beliefs: 1. **Prior:** Start with an initial belief about the world, represented as a probability distribution (e.g., in a detective story, suspect probabilities). 2. **Evidence:** Observe new data. 3. **Posterior:** Update the belief using Bayes' rule, combining the prior and the likelihood of the evidence to form a new, updated probability distribution. This mathematically coherent process is the foundation for building AI that can genuinely understand and communicate what it does and does not know. **Source:** [The mathematics of AI uncertainty - Google DeepMind Podcast](https://www.youtube.com/watch?v=tBjgCj_dGZM)