@jennyzhangzt: general Intelligence requires rethinking exploration
Summary
This paper argues that exploration is essential for all learning systems, including supervised learning, and proposes a framework for generalized exploration to drive open-ended learning towards general intelligence.
View Cached Full Text
Cached at: 05/17/26, 01:25 AM
general Intelligence requires rethinking exploration
https://t.co/QA4uKB6oDl
General Intelligence Requires Rethinking Exploration
Source: https://arxiv.org/abs/2211.07819 View PDF
Abstract:We are at the cusp of a transition from “learning from data” to “learning what data to learn from” as a central focus of artificial intelligence (AI) research. While the first-order learning problem is not completely solved, large models under unified architectures, such as transformers, have shifted the learning bottleneck from how to effectively train our models to how to effectively acquire and use task-relevant data. This problem, which we frame as exploration, is a universal aspect of learning in open-ended domains, such as the real world. Although the study of exploration in AI is largely limited to the field of reinforcement learning, we argue that exploration is essential to all learning systems, including supervised learning. We propose the problem of generalized exploration to conceptually unify exploration-driven learning between supervised learning and reinforcement learning, allowing us to highlight key similarities across learning settings and open research challenges. Importantly, generalized exploration serves as a necessary objective for maintaining open-ended learning processes, which in continually learning to discover and solve new problems, provides a promising path to more general intelligence.
Submission history
From: Minqi Jiang [view email] **[v1]**Tue, 15 Nov 2022 00:46:15 UTC (869 KB)
Similar Articles
Learning to Explore: Scaling Agentic Reasoning via Exploration-Aware Policy Optimization
This paper proposes an exploration-aware reinforcement learning framework that enables LLM agents to adaptively explore only when uncertainty is high, improving performance on text-based and GUI-based benchmarks.
@rohanpaul_ai: AGI needs agents that actively explore what they do not know, not just models that answer better. This new large (111 p…
A new survey paper from top US and Chinese labs proposes that AGI requires agents that actively explore uncertainty via epistemic exploration, organized into five levels of AI progress.
Look Before You Leap: Autonomous Exploration for LLM Agents
This paper identifies autonomous exploration as a critical capability for LLM agents and proposes the Explore-then-Act paradigm, which decouples information gathering from task execution to improve adaptability and real-world performance. It also introduces Exploration Checkpoint Coverage as a verifiable metric for evaluating exploration breadth.
@lateinteraction: one of the least expected side effects of this effort is that it gives us a measurable definition of agent “intelligenc…
Jacob X. Li discusses a new perspective on continual learning for AI, emphasizing developing expertise from a corpus of documents, and suggests it provides a measurable definition of agent intelligence.
#Exploration: A study of count-based exploration for deep reinforcement learning
OpenAI researchers demonstrate that a simple count-based exploration approach using hash codes can achieve near state-of-the-art performance on high-dimensional deep RL benchmarks, challenging the assumption that count-based methods cannot scale to continuous state spaces.