@jennyzhangzt: general Intelligence requires rethinking exploration
Summary
This paper argues that exploration is essential for all learning systems, including supervised learning, and proposes a framework for generalized exploration to drive open-ended learning towards general intelligence.
View Cached Full Text
Cached at: 05/17/26, 01:25 AM
general Intelligence requires rethinking exploration
https://t.co/QA4uKB6oDl
General Intelligence Requires Rethinking Exploration
Source: https://arxiv.org/abs/2211.07819 View PDF
Abstract:We are at the cusp of a transition from “learning from data” to “learning what data to learn from” as a central focus of artificial intelligence (AI) research. While the first-order learning problem is not completely solved, large models under unified architectures, such as transformers, have shifted the learning bottleneck from how to effectively train our models to how to effectively acquire and use task-relevant data. This problem, which we frame as exploration, is a universal aspect of learning in open-ended domains, such as the real world. Although the study of exploration in AI is largely limited to the field of reinforcement learning, we argue that exploration is essential to all learning systems, including supervised learning. We propose the problem of generalized exploration to conceptually unify exploration-driven learning between supervised learning and reinforcement learning, allowing us to highlight key similarities across learning settings and open research challenges. Importantly, generalized exploration serves as a necessary objective for maintaining open-ended learning processes, which in continually learning to discover and solve new problems, provides a promising path to more general intelligence.
Submission history
From: Minqi Jiang [view email] **[v1]**Tue, 15 Nov 2022 00:46:15 UTC (869 KB)
Similar Articles
Learning to Explore: Scaling Agentic Reasoning via Exploration-Aware Policy Optimization
This paper proposes an exploration-aware reinforcement learning framework that enables LLM agents to adaptively explore only when uncertainty is high, improving performance on text-based and GUI-based benchmarks.
Look Before You Leap: Autonomous Exploration for LLM Agents
This paper identifies autonomous exploration as a critical capability for LLM agents and proposes the Explore-then-Act paradigm, which decouples information gathering from task execution to improve adaptability and real-world performance. It also introduces Exploration Checkpoint Coverage as a verifiable metric for evaluating exploration breadth.
#Exploration: A study of count-based exploration for deep reinforcement learning
OpenAI researchers demonstrate that a simple count-based exploration approach using hash codes can achieve near state-of-the-art performance on high-dimensional deep RL benchmarks, challenging the assumption that count-based methods cannot scale to continuous state spaces.
Some considerations on learning to explore via meta-reinforcement learning
OpenAI researchers introduce E-MAML and E-RL², two meta-reinforcement learning algorithms designed to improve exploration in tasks where discovering optimal policies requires significant exploration. The work demonstrates these algorithms' effectiveness on novel environments including Krazy World and maze tasks.
Exploring Spatial Intelligence from a Generative Perspective
Researchers introduce GSI-Bench, the first benchmark to quantify generative spatial intelligence in multimodal models by evaluating 3D spatial constraint compliance during image generation. Fine-tuning on their synthetic dataset boosts both spatial editing fidelity and downstream spatial understanding, showing generative training can strengthen spatial reasoning.