MAPLE: Multi-State Aggregated Policy Evaluation for AlphaZero in Imperfect-Information Games
Summary
This paper introduces MAPLE, a tree search method that aggregates policy and value evaluations from multiple sampled world states, extending AlphaZero to imperfect-information games. Experiments on Phantom Go and Dark Hex show Elo improvements of 291 and 136 over the PIMC-based AlphaZero baseline.
View Cached Full Text
Cached at: 05/26/26, 09:05 AM
# MAPLE: Multi-State Aggregated Policy Evaluation for AlphaZero in Imperfect-Information Games Source: [https://arxiv.org/abs/2605.24139](https://arxiv.org/abs/2605.24139) [View PDF](https://arxiv.org/pdf/2605.24139) > Abstract:Imperfect\-information games \(IIGs\) are challenging, as players must make decisions without fully observing the true game state\. While AlphaZero has achieved remarkable success in perfect\-information games, extending it to IIGs remains difficult\. Existing search\-based approaches, such as Perfect Information Monte Carlo \(PIMC\), suffer from strategy fusion, while Information Set Monte Carlo Tree Search \(IS\-MCTS\) incurs high computational cost when combined with neural networks\. In this paper, we propose Multi\-State Aggregated PoLicy Evaluation \(MAPLE\), a tree search method that aggregates policy and value evaluations from multiple sampled world states within a single search tree, combining the advantages of PIMC and IS\-MCTS while maintaining a controllable computational cost\. We further incorporate a Siamese\-based sampling strategy to select informative world states from the information set\. Experiments on Phantom Go and Dark Hex show that MAPLE significantly outperforms the PIMC\-based AlphaZero baseline, achieving Elo improvements of 291 and 136, respectively\. These results demonstrate that MAPLE is an effective approach for AlphaZero\-style learning in imperfect\-information games\. ## Submission history From: Qian\-Rong Li \[[view email](https://arxiv.org/show-email/2d3ffef9/2605.24139)\] **\[v1\]**Fri, 22 May 2026 18:59:01 UTC \(269 KB\)
Similar Articles
Towards Learning Representations of Policies in Two-Player Zero-Sum Imperfect-Information Games
This paper investigates learning useful policy representations (embeddings) in two-player zero-sum imperfect-information games, introducing methods for creating policy datasets, learning embeddings, and evaluating them on downstream tasks using Kuhn and Leduc Poker.
AlphaZero in Sparsely Rewarded Games: Limits and Auxiliary Supervision
This paper examines the gap between strong play and perfect play in AlphaZero for sparsely rewarded games, using Connect Four and Chomp as testbeds, and proposes an auxiliary loss (AZAL) to improve oracle consistency in optimal play.
Self-Play Reinforcement Learning under Imperfect Information in Big 2
This paper presents a self-play reinforcement learning framework for the four-player imperfect-information card game Big 2, comparing policy-gradient and value-based methods and finding that PPO with entropy regularization outperforms others.
What to expect from AlphaZero's value predictions [D]
The article analyzes how AlphaZero's value predictions are shaped by self-play training data and noise, questioning whether they reliably estimate win chances against opponents with different play styles despite AlphaZero's strong empirical performance.
BiPACE: Bisimulation-Guided Policy Optimization with Action Counterfactual Estimation for LLM Agents
BiPACE introduces a drop-in advantage estimator that fixes state-action credit mismatch in stepwise group-based RL for LLM agents, using bisimulation-guided state clustering and action counterfactual estimation, achieving significant performance gains on ALFWorld, WebShop, and TextCraft with Qwen2.5 models.