LLMs Are Not Good Strategists, Yet Memory-Enhanced Agency Boosts Reasoning

# LLMs Are Not Good Strategists, Yet Memory-Enhanced Agency Boosts Reasoning
Source: [https://arxiv.org/html/2608.12626](https://arxiv.org/html/2608.12626)
Yi WuThanks:Equal contribution\. The author order follows the alphabetical order of first names, and each author reserves the right to list their name first in their own records\. It was published at the ICLR 2025 Workshop on Reasoning and Planning for LLMs\.Affiliation:University of ChicagoEmail:[yiwu@uchicago\.edu](mailto:)Zhimin Hu11footnotemark:1

###### Abstract

Strategic reasoning in Large Language Models \(LLMs\) within long\-horizon environments is often limited by inconsistent subgoals\. In these settings, finite attention resources prevent the model from maintaining strategic coherence over thousands of steps\. This limitation leads to strategic drift, where localized decisions fail to sustain a coherent trajectory across reasoning\. To address this, we introduce EpicStar, a framework that enables agents to learn memory as policy to tackle long\-horizon reasoning\. Specifically, the agent maintains a bank of successful past episodes as a heuristic alongside a working memory to track short\-term environmental changes\. During inference, a dynamic gating mechanism determines whether to execute a retrieved action directly or to perform new reasoning through a contextual fusion of the retrieved episodes and current working memory\. Utilizing StarCraft II as the testbed, we evaluated EpicStar against diverse opponent styles\. It significantly outperforms baseline methods, achieving higher win rates while consuming an order of magnitude fewer tokens, and it maintains this advantage consistently across difficulty levels and opponent strategies\. Our findings provide compelling evidence that structured cross\-episode memory is essential for enabling LLM agents to perform robust, long\-term strategic execution in dynamic, autonomous settings\.

## 1Introduction

Strategic reasoning in dynamic and partially observable environments represents a formidable challenge for Large Language Models \(LLMs\), where agents must maintain a coherent long\-term trajectory while simultaneously adapting to rapid environmental shifts\. StarCraft II has long served as a testbed for such capabilities because it demands the simultaneous coordination of resource management, technological expansion, and unit control over thousands of time steps\. While LLMs have already achieved human\-level reasoning in short\-horizon settings\([13](https://arxiv.org/html/2608.12626#bib.bib14);[9](https://arxiv.org/html/2608.12626#bib.bib1)\), their performance often degrades in long\-range sequential reasoning tasks\. We argue that the failures are mainly due to the fact that the agent tends to progressively overfit to local observations and consequently loses sight of its global objectives\.

Existing efforts to mitigate these issues either rely on more prompting loops to compress information\([12](https://arxiv.org/html/2608.12626#bib.bib2)\)or introduce more rules to stabilize the reasoning\([15](https://arxiv.org/html/2608.12626#bib.bib5)\)\. In this paper, we argue that robust strategic reasoning requires a transition from reactive prompting to structured reuse of past experience\([10](https://arxiv.org/html/2608.12626#bib.bib15)\)\. We introduce EpicStar, an agentic framework that integrates episodic retrieval with situational modulation as a policy for reasoning\. EpicStar continuously stores successful gameplay episodes in a structured memory bank\. During inference, the agent retrieves relevant episodes, acting as a strategic heuristic to guide its reasoning\. To remain responsive to new situations, EpicStar maintains a working memory for tracking environmental changes and employs a dynamic gating mechanism to balance reusing past actions directly with new reasoning for situational adaptation\. Additionally, in situational adaptation, retrieved episodes are fused via a context\-aligned mechanism to provide high\-level context and structure for ongoing reasoning\.

![Refer to caption](https://arxiv.org/html/2608.12626v1/overview.png)Figure 1:Overview of the EpicStar architecture\. The agent is composed of distinct memory components and their interactions\. Episodic memory stores gameplay episodes for retrieval and long\-term learning\. On the left blue panel, working memory consists of a short\-term cache and gating mechanisms that manage exploration and exploitation, balancing coherent planning with dynamic adaptation\. Contextual fusion enables bidirectional modulation between working memory and episodic memory, facilitating the integration of immediate context with past strategic knowledge\.We evaluated EpicStar in StarCraft II using TextStarCraft II\([12](https://arxiv.org/html/2608.12626#bib.bib2)\)and chose its Chain of Summarization \(CoS\) method as our primary baseline, as it shares the most comparable assumptions and design scope with our method\. We focus on level 5 and 6 game difficulties, which roughly correspond to entry\-level and above\-average human performance and mark the threshold at which CoS begins to show signs of performance degradation\. We found that EpicStar consistently outperforms CoS across both levels, achieving higher win rates \(Table[1](https://arxiv.org/html/2608.12626#S4.T1)\) while consuming fewer tokens\. Further, it demonstrates notably improved adaptability under diverse strategic conditions \(Figure[2](https://arxiv.org/html/2608.12626#S5.F2)\)\.

In summary, our contributions are threefold: 1\) we introduce EpicStar, an LLM\-based agentic framework that incorporates structured episodic memory to support long\-horizon strategic reasoning\. EpicStar mitigates strategic drift by retrieving and adapting past trajectories to maintain coherence over time; 2\) we propose a situational modulation mechanism that balances prior experience with real\-time context\. This includes a dynamic gating module that integrates episodic recall with working memory, and a contextual fusion step that translates retrieved episodes into strategic guidance; 3\) we provide empirical evidence that both episodic and working memory are essential for strategic performance\. Even a small number of high\-quality episodic memories leads to marked improvements in win rate and adaptability\. Code can be found at[https://github\.com/ethanyiwu/EpicStar](https://github.com/ethanyiwu/EpicStar)\.

## 2Related Work

### 2\.1Game Agent

The world of LLM\-based game agents has been built on several fundamental ideas in agent design: 1\) agents can learn from their action failures and reflect on their reasoning\([21](https://arxiv.org/html/2608.12626#bib.bib8);[16](https://arxiv.org/html/2608.12626#bib.bib9)\); agents can be equipped with external knowledge\([11](https://arxiv.org/html/2608.12626#bib.bib16)\); agents can learn to use tools\([14](https://arxiv.org/html/2608.12626#bib.bib11)\)\. These advancements have led to the bloom of developing LLM agents in more complex game scenarios\. To begin with, Voyager\([18](https://arxiv.org/html/2608.12626#bib.bib7)\)autonomously builds a toolkit of procedural skills through executable code in Minecraft\. Conversely, PokeLLMon\([5](https://arxiv.org/html/2608.12626#bib.bib4)\)retrieves external knowledge and uses memory for in\-game reflection to refine high\-level strategy in Pokémon\.[3](https://arxiv.org/html/2608.12626#bib.bib17)and[6](https://arxiv.org/html/2608.12626#bib.bib18)utilize external gameplay datasets to train a language model to play Chess and Poker\.

### 2\.2StarCraft II Agent

Among various games, StarCraft II has been unique in testing long\-horizon strategic reasoning\. As a real\-time strategic game in partially observable environments, it requires players to reason under constraints from diverse sources of information and adapt strategies over extensive steps\. The complexity of this domain was first made accessible to agentic research through the PySC2 interface\([2](https://arxiv.org/html/2608.12626#bib.bib10)\), which provides low\-level state information and control\. This interface fueled the development of reinforcement learning\-based agents like AlphaStar\([17](https://arxiv.org/html/2608.12626#bib.bib3)\), which achieved superhuman performance via massive\-scale self\-play, albeit at a prohibitive computational cost and with limited generalization\. More recently, SwarmBrain\([15](https://arxiv.org/html/2608.12626#bib.bib5)\)applied LLMs to this low\-level interface\. While it attempted to learn from gameplay through a log of semantic descriptions of strategies used, its learning is fundamentally oriented towards low\-level control and lacks contextual richness\.

Afterwards, the TextStarCraft II interface\([12](https://arxiv.org/html/2608.12626#bib.bib2)\)was introduced, moving away from micro\-management and re\-framing the game as a high\-level strategic challenge\. Our work is situated within this paradigm\. Its CoS method serves as a strong baseline, using its sliding\-window summary as a working memory to distill complex, high\-dimensional state information into a concise, actionable representation\. However, a clear gap remains: no existing agent for high\-level strategic reasoning like StarCraft II learns from its own gameplay experience in a structured manner\.

## 3Reasoning with Memory

### 3\.1Task Formulation

Strategic reasoning within StarCraft II can be conceptualized as a sequential decision\-making process characterized by the tuple\(S,A,T,R,Z,O\)\(S,A,T,R,Z,O\)\.SSrepresents the actual state space of the world, whereS=\{s1,s2,…,sN\}S=\\left\\\{s\_\{1\},s\_\{2\},\\ldots,s\_\{N\}\\right\\\}is the set of all possible states the system can be in\.AArepresents the action space, andA=\{a1,a2,…,aM\}A=\\left\\\{a\_\{1\},a\_\{2\},\\ldots,a\_\{M\}\\right\\\}is the set of actions the agent can take\. In partially observable environments, the agent has access to an observation spaceO=\{o1,o2,…,oK\}O=\\left\\\{o\_\{1\},o\_\{2\},\\ldots,o\_\{K\}\\right\\\}\.TTacts as the state transition functionT⁡\(s′∣s,a\)=P⁡\(s′∣s,a\)T\\left\(s^\{\\prime\}\\mid s,a\\right\)=P\\left\(s^\{\\prime\}\\mid s,a\\right\)andZZis the observation function,Z⁡\(o∣s′,a\)Z\\left\(o\\mid s^\{\\prime\},a\\right\), represents the probability of receiving observationooafter the agent takes actionaaand transitions to states′s^\{\\prime\}\. In this framework, the agent does not receive a direct rewardR⁡\(s,a\)R\(s,a\)but rather observes the outcome of the game \- either a win or a loss\. Unlike traditional reinforcement learning, we do not directly estimate theTTandZZ, instead, we rely on LLM reasoning coupled with memory mechanisms to maximize the expected accumulated reward\.

Algorithm 1Working Memory Exploration and Exploitation \-W⁡\(⋅\)W\(\\cdot\)Exploration Cool Down Frame

ded\_\{e\}, Queue Pop Cool Down Frame

dqd\_\{q\}, Action Size

nn, Current Game Time

tt, Last Time to Explore

lel\_\{e\}, Last Time to Pop Queue

lql\_\{q\}

1:

\(e1,…,en\)←R​eE​C​\(t,ot\)\(e\_\{1\},\\dots,e\_\{n\}\)\\leftarrow Re^\{EC\}\(t,o\_\{t\}\)⊳\\trianglerightRetrievennepisodes

2:

at←E​x​t​r​a​c​t​A​c​t​i​o​n​\(\(,,,,,\)\)a\_\{t\}\\leftarrow ExtractAction\(\(e\_\{1\},\\dots,e\_\{n\}\)\)⊳\\trianglerightSelect the first action

3:if

t≥de\+let\\geq d\_\{e\}\+l\_\{e\}then

4:

le←tl\_\{e\}\\leftarrow t
5:

ot′=ot×Qo​\(t\)o^\{\\prime\}\_\{t\}=o\_\{t\}\\times Q\_\{o\}\(t\)
6:

\(a1,…,an\)←L​L​M​\(S,\(ot′\)\)\(a\_\{1\},\\dots,a\_\{n\}\)\\leftarrow LLM\(S,\(o^\{\\prime\}\_\{t\}\)\)⊳\\trianglerightProposennexploration actions

7:

Qa\.p​u​s​h​\(\(,,,,,\)\)Q\_\{a\}\.push\(\(a\_\{1\},\\dots,a\_\{n\}\)\)
8:endif

9:if

at=a\_\{t\}=EmptyActionand

Qa\.s​i​z​e​\(\)\>0Q\_\{a\}\.size\(\)\>0and

t≥lq\+dqt\\geq l\_\{q\}\+d\_\{q\}then

10:

lq=tl\_\{q\}=t
11:

at←Qa\.p​o​p​\(\)a\_\{t\}\\leftarrow Q\_\{a\}\.pop\(\)⊳\\trianglerightAdd the exploration action

12:endif

13:

t←t\+1t\\leftarrow t\+1
14:return

ata\_\{t\}

### 3\.2Retrieval from Episodic Memory

Given a set of gameplay data, we define a series of episodes\{\(ti,oi,ai\)∣i=1,2,…,e\}\\\{\(t\_\{i\},o\_\{i\},a\_\{i\}\)\\mid i=1,2,\\ldots,e\\\}, wheretit\_\{i\}represents the time in the game\. The episodic memory, denoted asME​C​\(t,o,a\)M^\{EC\}\(t,o,a\), consists of episodes corresponding to victory games\.

For each moment\(t,o\)\(t,o\)in a new game, we identify similar past moments by finding indices of episodesDi​n​d​e​xD\_\{index\}that closely match the current scenario fromME​CM^\{EC\}\. Initially, we searchME​CM^\{EC\}aroundttto retrieve a subset:

ME​Ct=BinarySearch⁡\(ME​C,t,tΔ\)\{M^\{EC\}\}\_\{t\}=\\operatorname\{BinarySearch\}\(M^\{EC\},t,t\_\{\\Delta\}\)\(1\)wheretΔt\_\{\\Delta\}is a predefined parameter representing a time range aroundttfor the search \(we settΔ=0t\_\{\\Delta\}=0for simplicity\)\. Next, we compute the differences between the current observationoto\_\{t\}and all previous observationsO′∈ME​CtO^\{\\prime\}\\in\{M^\{EC\}\}\_\{t\}, with each observationoto\_\{t\}represented as a Python dictionary \(with a unit as the key and a scalar value\)\. We calculate two metrics: \(1\) the number of items that changed, denoted asDi​t​e​mD\_\{item\}; and \(2\) the number of values that changed, denoted asDv​a​l​u​eD\_\{value\}\. We then obtain the topnnmemories based on the following criteria:

Di​n​d​e​x=Argsort⁡\(α​MinMax⁡\(Di​t​e​m\)CLOSEOPEN\+β​MinMax⁡\(Dv​a​l​u​e\)\)\\begin\{split\}D\_\{index\}=\\operatorname\{Argsort\}\(\\alpha\\operatorname\{MinMax\}\(D\_\{item\}\)\\\\ \+\\beta\\operatorname\{MinMax\}\(D\_\{value\}\)\)\\end\{split\}\(2\)where we perform min\-max normalization on bothDi​t​e​mD\_\{item\}andDv​a​l​u​eD\_\{value\}, then sortDDin ascending order\. We setn=3,α=0\.5,β=0\.5n=3,\\alpha=0\.5,\\beta=0\.5

Thus, the retrieval function is defined as:

ReE​C\(t,o\)=\{\{EmptyAction\},if​\|ME​Ct\|=0\{ME​Ct\(ti,oi,ai\)∣i∈Di​n​d​e​x\[:n\]\},otherwise\\begin\{split\}Re^\{EC\}\(t,o\)=\\begin\{cases\}\\Big\\\{\\text\{EmptyAction\}\\Big\\\},\\text\{if \}\|\{M^\{EC\}\}\_\{t\}\|=0\\\\ \\Big\\\{\{M^\{EC\}\}\_\{t\}\(t\_\{i\},o\_\{i\},a\_\{i\}\)\\mid i\\in D\_\{index\}\[:n\]\\Big\\\},\\\\ \\text\{otherwise\}\\end\{cases\}\\end\{split\}\(3\)

### 3\.3Working Memory and Gating Mechanism

For every time steptt, the observation queue,QoQ\_\{o\}, captures recentkm​a​xk\_\{max\}observations\. Inspired by frame skipping\([1](https://arxiv.org/html/2608.12626#bib.bib6)\), the working memory recalls the past observations by a frame interval ofLL:

Qo\(t\)=\{\(ot−k\)∣k=L,2L,…,kmaxL\}Q\_\{o\}\(t\)=\\left\\\{\\left\(o\_\{t\-k\}\\right\)\\mid k=L,2L,\\ldots,k\_\{\\max\}L\\right\\\}\(4\)whereot−ko\_\{t\-k\}represents observation at timet−kt\-k\. Before reasoning, we map the observationoto\_\{t\}to an augmented observation space defined as:ot′=ot×Qo​\(t\)o^\{\\prime\}\_\{t\}=o\_\{t\}\\times Q\_\{o\}\(t\)\. We setkm​a​x=4k\_\{max\}=4andL=24L=24\.

We define the combination of the vanilla system prompt and contextual fusion as semantic knowledgeSSand the querying of the LLM as the functionL​L​M​\(⋅\)LLM\(\\cdot\)\. We then engage in the exploration and exploitation process denoted asW⁡\(⋅\)W\(\\cdot\)and initiate it with an exploration action queue,QaQ\_\{a\}, as outlined in Algorithm[1](https://arxiv.org/html/2608.12626#alg1)\.

InW⁡\(⋅\)W\(\\cdot\),Qo​\(t\)Q\_\{o\}\(t\)provides short\-term historical information that allows the agent to analyze the state’s tendency\. In contrast,QaQ\_\{a\}stores actions for future interpolation into action sequences retrieved from episodic memory, enabling adaptive planning\.

Algorithm 2EpicStar AgentStarCraft II Game Environment

e​n​venv, Working Memory

W⁡\(⋅\)W\(\\cdot\)

1:

e​n​v\.i​n​i​t​i​a​l​i​z​e​\(\)env\.initialize\(\),

2:

Qa←∅Q\_\{a\}\\leftarrow\\emptyset,

le←0l\_\{e\}\\leftarrow 0,

t←0t\\leftarrow 0,

lq←0l\_\{q\}\\leftarrow 0
3:

ot←e​n​v\.o​b​s​e​r​v​a​t​i​o​n​\(\)o\_\{t\}\\leftarrow env\.observation\(\)⊳\\trianglerightInitial observation

4:while

e​n​venvis not terminateddo

5:

at←W⁡\(t,Qa,ot,Qo​\(t\),le,lq\)a\_\{t\}\\leftarrow W\(t,Q\_\{a\},o\_\{t\},Q\_\{o\}\(t\),l\_\{e\},l\_\{q\}\)
6:

t←t\+1t\\leftarrow t\+1
7:

ot=e​n​v\.s​t​e​p​\(at\)o\_\{t\}=env\.step\(a\_\{t\}\)⊳\\trianglerightNext frame’s observation

8:endwhile

9:return

e​n​v\.g​a​m​e​\_​r​e​s​u​l​t​\(\)env\.game\\\_result\(\)

### 3\.4Contextual Fusion

To make a fair comparison, we adapt the task instruction prompts from CoS methods\([12](https://arxiv.org/html/2608.12626#bib.bib2)\)as a backbone\. Additionally, we incorporate contextual fusion to connect the information between working memory and episodic memory to the prompts\. Specifically: 1\) We augment the prompt with additional instructions that encourage the retrieved actions to be executable within the current scenario\. This serves as a modulation from working memory to episodic memory\. 2\) To better align exploratory actions with episodic memory, we prompt the LLM to generate a high\-level description of the strategy underlying the retrieved episodic memory and concatenate this information with the prompt\. This serves as a modulation from episodic memory to working memory\. The prompts are provided in the Appendix[B](https://arxiv.org/html/2608.12626#A2)\. The full reasoning loop of our agent is presented in Algorithm[2](https://arxiv.org/html/2608.12626#alg2)\.

### 3\.5Learning Episodic Memory from Gameplay

Episodic memory typically encompasses both learning and retrieval processes\. To efficiently bootstrap the episodic memory, we use a rule\-based agent to collect successful trajectories\. Specifically, we gather game episodes as the agent competes against built\-in opponents at Levels 6 and 7 for 20 rounds, and retain episodes from five winning games, yielding a total of 4,592 episodes\. To ensure generalization and avoid overfitting to the data collection regime, we evaluate EpicStar on maps disjoint from those used during memory collection\.

### 3\.6Game Interface

Given the actionata\_\{t\}returned from Algorithm[2](https://arxiv.org/html/2608.12626#alg2), the game interface translates this action into specific procedures within StarCraft II\. We adapt the game interface from[12](https://arxiv.org/html/2608.12626#bib.bib2), keeping the same action spaces for fair comparison\. To improve generality, we removed overly specific operational routines hardcoded into thea​t​t​a​c​kattackandd​e​f​e​n​ddefendactions, allowing them to be invoked more flexibly in different contexts\.

## 4Experiments

We evaluated EpicStar by comparing it against built\-in agent opponents across varying difficulty levels, attack styles, and game maps in StarCraft II\.

### 4\.1Experimental Setup

Our evaluation was carried out in two phases: \(1\) a comparative analysis against baseline methods and \(2\) an ablation study to assess the impact of individual components\.

We used four closed\-source OpenAI models spanning two generations, gpt\-3\.5\-turbo, gpt\-4\-turbo, gpt\-4o\-mini, and gpt\-4o\([7](https://arxiv.org/html/2608.12626#bib.bib21)\), to assess whether the benefits of EpicStar are consistent across backbones of varying capability\. We conducted our experiments primarily at difficulty levels 5 and 6\. To ensure consistency, we adopted the results from[12](https://arxiv.org/html/2608.12626#bib.bib2)to mitigate potential performance degradation due to reproduction\. Also, we only usedgpt\-4\-turboin Level 5 to match the results from CoS\. Additional details on the experimental setup and metrics can be found in the Appendix[A](https://arxiv.org/html/2608.12626#A1)\.

### 4\.2Evaluation Metrics

We used the win rate as the primary indicator of agent performance\. To further assess the efficiency of resource management and technological advancement within the game, we employed additional metrics detailed in Results[5](https://arxiv.org/html/2608.12626#S5)and the Appendix[A](https://arxiv.org/html/2608.12626#A1)\.

DifficultyAgent TypeWin RatePBRRURAPUTRLevel 5CoS\(GPT\-3\.5\-Turbo\)0\.5500\.078178750\.76080\.4476CoS\(GPT\-4\-Turbo\)0\.6000\.033783060\.71940\.3452EpicStar\(GPT\-3\.5\-Turbo\)0\.5750\.1123113630\.79910\.2500EpicStar\(GPT\-4\-Turbo\)0\.7500\.1385114830\.84490\.2524EpicStar\(GPT\-4o\-mini\)0\.6750\.121198640\.81230\.2536EpicStar\(GPT\-4o\)0\.6500\.1175110200\.81070\.2524Level 6CoS\(GPT\-3\.5\-Turbo\)0\.0833\-\-\-\-EpicStar\(GPT\-3\.5\-Turbo\)0\.1500\.1043111910\.73780\.2375EpicStar\(GPT\-4o\-mini\)0\.3000\.1089109310\.78650\.2304EpicStar\(GPT\-4o\)0\.2750\.1176104490\.74040\.2220Table 1:Overall comparison of EpicStar with CoS baseline across different model backends against Level 5 and Level 6 built\-in agents\. Bold values indicate the highest win rates within each difficulty level\.
### 4\.3First Phase: Baseline Comparison

In the first phase of our experiments, we compared EpicStar against the CoS baseline\. The agent was evaluated against built\-in agents at difficulty levels 5 and 6, with 40 evaluation rounds conducted at each level \(exceptgpt\-4\-turbois tested with 20 rounds due to its high cost\)\. Specifically, we tested the agent against various built\-in game strategies—timing,rush,power,macro, andair—as outlined in Appendix[A](https://arxiv.org/html/2608.12626#A1)\. These tests were conducted on two newly introduced maps for the 2024 Season,Abyssal Reef LEandEver Dream LE\. Each combination of strategy and map was tested four times, resulting in a total of 40 rounds\.

### 4\.4Second Phase: Ablation Study

To evaluate the contribution of individual components within EpicStar, we selectively removed key elements while keeping other experimental conditions consistent with Phase 1 ongpt\-4o\-mini\. The first ablation, denoted w/o exploration in the tables and figures below, disables the exploration step using information from working memory, so that the agent relies solely on retrieved episodes for reasoning\. This allowed us to isolate and assess the role of exploration via LLMs within the framework\. In a subsequent ablation, we removed the contextual fusion to examine the influence of bidirectional modulation between memory components on the agent’s reasoning and planning abilities\. In EpicStar, ablating episodic memory reduces the agent to a configuration functionally similar to CoS\. Therefore, we omit the ablation of the episodic memory component\.

## 5Results

### 5\.1EpicStar v\.s\. Baseline

We present a comprehensive analysis of EpicStar’s performance with CoS against the built\-in agents, as shown in Table[1](https://arxiv.org/html/2608.12626#S4.T1)\. At Level 5, EpicStar achieves a win rate of 67\.5% ongpt\-4o\-miniand 75\.0% ongpt\-4\-turbo, while at Level 6, it secures 30\.0% withgpt\-4o\-mini\. In contrast, the strongest baseline, CoS \(gpt\-4\-turbo\), achieves a win rate of 60\.0% at Level 5\. CoS, however, struggles considerably at Level 6, with a win rate of just 8\.3% \(gpt\-3\.5\-Turbo\), and EpicStar nearly doubled the win rate to 15% with the same model\. Our method achieves a significant win rate gain compared to the baseline\. Beyond the increase in win rate, our token consumption is only 14\.5% of CoS when using the same models \(see more details in Appendix[C](https://arxiv.org/html/2608.12626#A3)\)

Beyond the win rate and token consumption, we observe additional metrics that validate EpicStar’s enhanced performance\. The Average Population Utilization \(APU\), which measures the efficiency of utilizing the population cap, reveals that EpicStar outperforms both baselines with values of 0\.7991\(gpt\-3\.5\-turbo\) and 0\.8449\(gpt\-4\-turbo\), compared to 0\.7608 for CoS \(gpt\-3\.5\-turbo\) and 0\.7194 for CoS \(gpt\-4\-turbo\)\. A higher APU indicates more effective macro management to facilitate units in the game\. We note occasional discrepancies among the Population Block Ratio \(PBR; the lower the better\), the Tech Rate \(TR; a neutral indicator\), and the Win Rate\. These discrepancies occur because when EpicStar defeats the built\-in agents, our agent does not accept their surrender\. Thus, the game is extended until EpicStar reaches the population cap, which may result in lower PBR values even after a win\.

Table 2:Overall comparison of EpicStar and two ablation agents against Level 5 and Level 6 built\-in agents\. We found that ablating either exploration using working memory or contextual fusion degrades the performance\.
### 5\.2Ablation Study

To understand the improved performance of EpicStar, we investigated the impacts of the exploration and contextual fusion components separately\. As shown in Table[2](https://arxiv.org/html/2608.12626#S5.T2), at Level 5, removing exploration decreases the win rate from 67\.5% \(EpicStar\) to 60\.0%, while removing contextual fusion slightly reduces it to 65\.0%\. Thus, both components contribute positively, with exploration playing a slightly more critical role\. At Level 6, this impact intensifies: the win rate significantly drops to 17\.5% without exploration and to 12\.5% without contextual fusion, compared to EpicStar’s 30\.0%\. These findings highlight that strategy coherence becomes increasingly important at higher difficulty levels, where misaligned exploration significantly impairs performance\. It is also worth noting that other metrics are very close to each other; this is due to the similarities among the three agents\.

Figure 2:Detailed win rate across built\-in strategies on EpicStar and two ablation agent types: agents without exploration and agents without contextual fusion\.Furthermore, as shown in Figure[2](https://arxiv.org/html/2608.12626#S5.F2), at Level 5 \(left\), the three agents exhibit comparable performance profiles across most strategies, indicating that the marginal benefit of exploration and contextual fusion is limited at this difficulty\. At Level 6 \(right\), however, the gap widens substantially, particularly in thePower,Rush, andTimingstrategies, where the ablated agents’ win rates drop close to zero while EpicStar retains a modest but non\-trivial win rate \(e\.g\., 37\.5% inPower, 12\.5% inRushandTiming\)\. It is worth noting that absolute win rates remain relatively low for all agents at this difficulty, suggesting that Level 6 continues to pose a substantial challenge even for the full EpicStar agent; the results should therefore be interpreted as evidence of relative robustness rather than mastery of the task\. This pattern suggests that the benefits of exploration and contextual fusion become more pronounced as task difficulty increases\.

### 5\.3Case Study

Besides the quantitative results showing the improved performance of EpicStar compared to the baseline, we conducted a case study on the effect of working memory\. Specifically, we compare EpicStar with and without exploration using working memory\. We select a victory case against Level 6 air style built\-in AI from each agent, and we find that EpicStar wins the game faster and achieves faster expansion, as illustrated\(shown time 1:00, 5:00, 8:00, and 11:00, respectively\), as shown in Figure[3](https://arxiv.org/html/2608.12626#S5.F3)\. This indicates that EpicStar reasons about its current strategic situation in the game and utilizes the limited resources more efficiently, and achieves better occupation, which leads to faster victory against the same opponent\.

![Refer to caption](https://arxiv.org/html/2608.12626v1/figure/case_study.jpg)Figure 3:Expansion condition on1,5,8,111,\\ 5,8,11minutes for EpicStar \(first row\) and EpicStar without exploration \(second row\)\. The green dots represent the occupied bases, and EpicStar expands faster than EpicStar without exploration\.

## 6Discussion & Future Work

Memory mechanisms have been explored in various short\-horizon complex tasks\. These studies primarily focus on utilizing memory as a storage for intermediate steps in multi\-step reasoning\([19](https://arxiv.org/html/2608.12626#bib.bib19)\)or as a means of grounding LLM inference with external knowledge\([20](https://arxiv.org/html/2608.12626#bib.bib13)\)\. While[4](https://arxiv.org/html/2608.12626#bib.bib12)argue that LLM agents should possess long\-term memory, the abundance of domain knowledge available online has led researchers to prioritize the efficient retrieval of existing knowledge over developing agents capable of autonomous learning and experiential integration\. We argue that while enabling agents to learn from experience presents significant challenges, it is essential for the long\-term advancement of autonomous intelligence\. Our results suggest that even a small, curated bank of past trajectories can act as an effective substitute for costly exploration at inference time, echoing findings in case\-based reasoning\([10](https://arxiv.org/html/2608.12626#bib.bib15)\)that a handful of well\-chosen precedents can carry more decision\-relevant signal than exhaustive search\. This reframes episodic memory not merely as a storage mechanism but as an implicit, non\-parametric policy that complements the LLM’s own reasoning\.

However, our work is not without limitations\. First, although we observed that incorporating exploration yields better performance than relying solely on episodic retrieval\-a clear sign of improvement\-there is currently limited empirical evidence to determine the extent to which this performance ceiling depends on the base model’s capabilities\. Meanwhile, our episodic memory was bootstrapped from a relatively small set of victories; we have not yet characterized how performance scales as the memory grows substantially larger, nor how the agent behaves when the bank contains noisy, redundant, or contradictory episodes\.

Looking forward, several directions could extend this work\. To enable more scalable memory use when reasoning, future work could investigate agentic frameworks that use specialized tokens to represent memory information, avoiding the growing prompt overhead of appending retrieved episodes in natural language as the memory bank scales\. Additionally, one could explore methods to integrate parameterized memory in such experiential learning\([8](https://arxiv.org/html/2608.12626#bib.bib20)\), which remains a promising direction\. Beyond StarCraft II, testing whether episodic\-memory\-as\-policy transfers to other long\-horizon, partially observable domains would help establish whether strategic drift and its mitigation via structured memory are general phenomena rather than artifacts of this particular testbed\. Finally, an important safety consideration for any system that reuses past successful behavior is the risk of overfitting to opponent styles seen during memory collection; developing principled ways to detect when a retrieved episode no longer applies, rather than relying solely on the gating heuristic used here, would make such agents more robust when deployed against genuinely novel adversaries\.

## 7Conclusion

In this paper, we introduce a novel perspective on the design of reasoning agents for long\-horizon tasks\. By integrating episodic and working memory systems, we enable agents to maintain coherent strategic trajectories and adapt dynamically to evolving game scenarios\. Empirical results demonstrate that our approach not only outperforms base methods in terms of performance but also has much lower token consumption, indicating that episodic memory can serve as an implicit mechanism for policy optimization, which enables more efficient reasoning\. We show that even limited high\-quality past experience can be maximized for significant performance gains when paired with a dynamic memory framework\. This research also underscores the importance of human cognitive mechanisms in developing AI that can navigate and excel in complex strategic environments, bridging the gap between artificial intelligence and cognitive science\.

## References

- Braylanet al\.\(2015\)A\. Braylan, M\. Hollenbeck, E\. Meyerson, and R\. MiikkulainenFrame skip is a powerful parameter for learning to play atari\.InAAAI Workshop: Learning for General Competency in Video Games,Cited by:[§3\.3](https://arxiv.org/html/2608.12626#S3.SS3.p1.1)\.
- DeepMind \(2017\)G\. DeepMindPySC2 \- starcraft ii learning environment\.Note:Accessed: 2024\-12\-27External Links:[Link](https://github.com/google-deepmind/pysc2)Cited by:[§2\.2](https://arxiv.org/html/2608.12626#S2.SS2.p1.1)\.
- Fenget al\.\(2023\)X\. Feng, Y\. Luo, Z\. Wang, H\. Tang, M\. Yang, K\. Shao, D\. Mguni, Y\. Du, and J\. WangChessgpt: bridging policy learning and language modeling\.Advances in Neural Information Processing Systems36,pp\. 7216–7262\.Cited by:[§2\.1](https://arxiv.org/html/2608.12626#S2.SS1.p1.1)\.
- Hataliset al\.\(2023\)K\. Hatalis, D\. Christou, J\. Myers, S\. Jones, K\. Lambert, A\. Amos\-Binks, Z\. Dannenhauer, and D\. DannenhauerMemory matters: the need to improve long\-term memory in llm\-agents\.InProceedings of the AAAI Symposium Series,Vol\.2,pp\. 277–280\.Cited by:[§6](https://arxiv.org/html/2608.12626#S6.p1.1)\.
- Huet al\.\(2024\)S\. Hu, T\. Huang, and L\. LiuPokeLLMon: a human\-parity agent for pokemon battles with large language models\.arXiv preprint arXiv:2402\.01118\.Cited by:[§2\.1](https://arxiv.org/html/2608.12626#S2.SS1.p1.1)\.
- Huanget al\.\(2024\)C\. Huang, Y\. Cao, Y\. Wen, T\. Zhou, and Y\. ZhangPokerGPT: an end\-to\-end lightweight solver for multi\-player texas hold’em via large language model\.arXiv preprint arXiv:2401\.06781\.Cited by:[§2\.1](https://arxiv.org/html/2608.12626#S2.SS1.p1.1)\.
- Hurstet al\.\(2024\)A\. Hurst, A\. Lerer, A\. P\. Goucher, A\. Perelman, A\. Ramesh, A\. Clark, A\. Ostrow, A\. Welihinda, A\. Hayes, A\. Radford,et al\.Gpt\-4o system card\.arXiv preprint arXiv:2410\.21276\.Cited by:[§4\.1](https://arxiv.org/html/2608.12626#S4.SS1.p2.1)\.
- Junet al\.\(2025\)J\. Jun, V\. Marupudi, R\. S\. Shah, and S\. VarmaA neural network model of complementary learning systems: pattern separation and completion for continual learning\.InProceedings of the Annual Meeting of the Cognitive Science Society,Vol\.47\.Cited by:[§6](https://arxiv.org/html/2608.12626#S6.p3.1)\.
- Kabraet al\.\(2025\)K\. Kabra, K\. Inani, V\. Marupudi, and S\. VarmaModeling understanding of story\-based analogies using large language models\.InProceedings of the Annual Meeting of the Cognitive Science Society,Vol\.47\.Cited by:[§1](https://arxiv.org/html/2608.12626#S1.p1.1)\.
- Klein \(2017\)G\. A\. KleinSources of power: how people make decisions\.MIT press\.Cited by:[§1](https://arxiv.org/html/2608.12626#S1.p2.1),[§6](https://arxiv.org/html/2608.12626#S6.p1.1)\.
- Lewiset al\.\(2020\)P\. Lewis, E\. Perez, A\. Piktus, F\. Petroni, V\. Karpukhin, N\. Goyal, H\. Küttler, M\. Lewis, W\. Yih, T\. Rocktäschel,et al\.Retrieval\-augmented generation for knowledge\-intensive nlp tasks\.Advances in neural information processing systems33,pp\. 9459–9474\.Cited by:[§2\.1](https://arxiv.org/html/2608.12626#S2.SS1.p1.1)\.
- Maet al\.\(2024\)W\. Ma, Q\. Mi, Y\. Zeng, X\. Yan, R\. Lin, Y\. Wu, J\. Wang, and H\. ZhangLarge language models play starcraft II:benchmarks and a chain of summarization approach\.InThe Thirty\-eighth Annual Conference on Neural Information Processing Systems,Cited by:[§A\.2](https://arxiv.org/html/2608.12626#A1.SS2.p1.1),[Appendix B](https://arxiv.org/html/2608.12626#A2.p1.1),[§1](https://arxiv.org/html/2608.12626#S1.p2.1),[§1](https://arxiv.org/html/2608.12626#S1.p3.1),[§2\.2](https://arxiv.org/html/2608.12626#S2.SS2.p2.1),[§3\.4](https://arxiv.org/html/2608.12626#S3.SS4.p1.1),[§3\.6](https://arxiv.org/html/2608.12626#S3.SS6.p1.1),[§4\.1](https://arxiv.org/html/2608.12626#S4.SS1.p2.1)\.
- Mondorf and Plank \(2024\)P\. Mondorf and B\. PlankComparing inferential strategies of humans and large language models in deductive reasoning\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 9370–9402\.Cited by:[§1](https://arxiv.org/html/2608.12626#S1.p1.1)\.
- Schicket al\.\(2023\)T\. Schick, J\. Dwivedi\-Yu, R\. Dessì, R\. Raileanu, M\. Lomeli, E\. Hambro, L\. Zettlemoyer, N\. Cancedda, and T\. ScialomToolformer: language models can teach themselves to use tools\.Advances in neural information processing systems36,pp\. 68539–68551\.Cited by:[§2\.1](https://arxiv.org/html/2608.12626#S2.SS1.p1.1)\.
- Shaoet al\.\(2024\)X\. Shao, W\. Jiang, F\. Zuo, and M\. LiuSwarmBrain: embodied agent for real\-time strategy game starcraft ii via large language models\.arXiv preprint arXiv:2401\.17749\.Cited by:[§1](https://arxiv.org/html/2608.12626#S1.p2.1),[§2\.2](https://arxiv.org/html/2608.12626#S2.SS2.p1.1)\.
- Shinnet al\.\(2024\)N\. Shinn, F\. Cassano, A\. Gopinath, K\. Narasimhan, and S\. YaoReflexion: language agents with verbal reinforcement learning\.Advances in Neural Information Processing Systems36\.Cited by:[§2\.1](https://arxiv.org/html/2608.12626#S2.SS1.p1.1)\.
- Vinyalset al\.\(2019\)O\. Vinyals, I\. Babuschkin, W\. M\. Czarnecki, M\. Mathieu, A\. Dudzik, J\. Chung, D\. Choi, R\. Powell,et al\.Grandmaster level in starcraft ii using multi\-agent reinforcement learning\.Nature575,pp\. 350 – 354\.Cited by:[§2\.2](https://arxiv.org/html/2608.12626#S2.SS2.p1.1)\.
- Wanget al\.\(2023\)G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. AnandkumarVoyager: an open\-ended embodied agent with large language models\.Transactions on Machine Learning Research\.Cited by:[§2\.1](https://arxiv.org/html/2608.12626#S2.SS1.p1.1)\.
- Wanget al\.\(2024\)S\. Wang, Z\. Wei, Y\. Choi, and X\. RenSymbolic working memory enhances language models for complex rule application\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,pp\. 17583–17604\.Cited by:[§6](https://arxiv.org/html/2608.12626#S6.p1.1)\.
- Wuet al\.\(2025\)J\. Wu, C\. Yang, Y\. Wu, S\. Mahns, C\. Wang, H\. Zhu, F\. Fang, and H\. XuAi realtor: towards grounded persuasive language generation for automated copywriting\.arXiv preprint arXiv:2502\.16810\.Cited by:[§6](https://arxiv.org/html/2608.12626#S6.p1.1)\.
- Yaoet al\.\(2022\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. R\. Narasimhan, and Y\. CaoReact: synergizing reasoning and acting in language models\.InThe eleventh international conference on learning representations,Cited by:[§2\.1](https://arxiv.org/html/2608.12626#S2.SS1.p1.1)\.

## Appendix AExperimental Setup and Metrics Description

### A\.1Experimental Setup

Agent and Opponent Configuration:To maintain a consistent and controlled testing environment, LLM agents are set to play as Protoss against built\-in agent\-controlled Zerg opponents\. This arrangement enables a systematic evaluation of strategic performance across varying difficulty levels\. The difficulty settings are listed below\.

Table 3:StarCraft II Built\-in Agent Levels and Approximate League EquivalentsParameter Configuration:The temperature parameter is set to 0\.1 to prioritize strategic decision\-making over random actions\.

Game Version:All experiments were conducted using the latest Patch5\.0\.14\.93333of StarCraft II\.

### A\.2Evaluation Metrics

Our evaluation framework for TextStarCraft II[12](https://arxiv.org/html/2608.12626#bib.bib2)builds upon StarCraft II’s established player performance analytics, incorporating tailored modifications to comprehensively assess LLM agent gameplay strategies\.

Win Rate:This is the primary performance indicator for the agent in the game\. It is calculated as the percentage of victories relative to the total games played\.

Population Block Ratio \(PBR\):PBR assesses the agent’s macro\-management skills, specifically its ability to allocate resources efficiently and sustain population growth\. It is defined as:

PBR=Time at Population CapGame Duration\\text\{PBR\}=\\frac\{\\text\{Time at Population Cap\}\}\{\\text\{Game Duration\}\}\(5\)
This metric represents the proportion of time the agent spends at maximum population capacity relative to the total game duration until it reaches the 200/200 supply cap for the first time\. A high PBR suggests ineffective macro\-strategic planning and suboptimal decision\-making\.

Resource Utilization Ratio \(RUR\):RUR measures how efficiently the agent manages in\-game resources over time\. It is computed as:

RUR=Total Minerals \+ Total Gas UsedGame Duration\\text\{RUR\}=\\frac\{\\text\{Total Minerals \+ Total Gas Used\}\}\{\\text\{Game Duration\}\}\(6\)
This metric evaluates the total resources expended relative to the game’s duration until the agent first reaches the maximum supply\. A high RUR may indicate poor resource utilization, reflecting suboptimal macro\-strategic decisions\.

Average Population Utilization \(APU\):APU quantifies how effectively the agent utilizes its available population capacity\. It is calculated as:

APU=1N​∑i=1N\(Used Population at​it​h​stepPopulation Cap at​it​h​step\)\\text\{APU\}=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\left\(\\frac\{\\text\{Used Population at \}i^\{th\}\\text\{ step\}\}\{\\text\{Population Cap at \}i^\{th\}\\text\{ step\}\}\\right\)\(7\)
This metric averages the ratio of the population used to total capacity across all time steps until the agent reaches full supply\. A higher APU indicates more efficient population management and better macro\-strategic execution\.

Technology Rate \(TR\):TR evaluates the agent’s inclination toward technological advancement by measuring its exploration of the technology tree\. It is defined as:

TR=Completed TechnologiesTotal Technologies Available\\text\{TR\}=\\frac\{\\text\{Completed Technologies\}\}\{\\text\{Total Technologies Available\}\}\(8\)
This metric calculates the fraction of completed technologies and structures relative to the total available from start to finish of the game\. TR reflects the agent’s tendency to pursue technological upgrades, although it does not necessarily correlate with overall performance\.

### A\.3StarCraft II Built\-in Agent Styles

1. 1\.Timing: Executes attacks at specific moments when a strategic advantage is perceived\.
2. 2\.Rush: Aims to overwhelm the opponent early by rapidly producing offensive units and launching swift attacks\.
3. 3\.Power: Focuses on building a strong economy and technological foundation before engaging in significant combat\.
4. 4\.Macro: Prioritizes long\-term economic growth and infrastructure development\.
5. 5\.Air: Prioritizes the development and deployment of air units\.

## Appendix BPrompt

We present the prompt used in the EpicStar and CoS baseline\([12](https://arxiv.org/html/2608.12626#bib.bib2)\)\. Prompt 1 \(in Figure[4](https://arxiv.org/html/2608.12626#A2.F4)\) serves as the backbone prompt of EpicStar\. In this prompt, the LLM will generate the strategy description based on the episodic memory retrieved\. Here we present the example of the Warpgate strategy description generated bygpt\-4o\. Prompt 2 \(in Figure[5](https://arxiv.org/html/2608.12626#A2.F5)\) is the CoS prompt and is used in our ablation study\.

Prompt 1: System prompt with contextual fusion from episodic memory \(e\.g\. Warpgate strategy\)\.You are an AI trained in analyzing and summarizing StarCraft II games\. You understand the nuances and strategies of the Protoss race\.Specifically, you are playing the WarpGate Strategy in StarCraft II, which is a core tactic for the Protoss race, leveraging their ability to instantly warp in units anywhere on the map using Pylons or Warp Prisms\. By converting Gateways into Warp Gates, Protoss players can quickly reinforce their army or apply pressure without needing to rally units across the map\. This strategy emphasizes map control, timing attacks, and flexibility, often paired with strong early\-game units like Zealots or Stalkers\. Proper use of Warp Gates can overwhelm opponents with rapid unit production and strategic positioning\.
Based on the summaries of multiple rounds in a game, we want you to analyze the game progression in a structured way\. Your analysis should include the following aspects:
1\. Game Overview: Provide a brief overview of the current situation based on all the rounds\.
2\. Current Game Stage: Determine the stage of the game based on the information of all rounds\. Is it the early game, mid\-game, or late game?
3\. Our Situation: Describe our current status in terms of:
3\.1 Units and Buildings: Analyze the state of our units and buildings\.
3\.2 Economy: Evaluate our economic condition, including resource collection and usage\.
3\.3 Technology: Describe the status of our technological research and what technologies we have unlocked so far\. Analyze our technology tree, indicating the available and potential upgrades or units\.
4\. Our Strategy: Infer our potential strategy based on our current situation and the information of all rounds\.
5\. Enemy’s Strategy: Infer the enemy’s potential strategy, based on the available information\.
6\. Key Information: Highlight the most important aspects from all rounds that have significantly influenced the game\.
For Protoss, keep an eye on Nexus’s energy to Chrono Boost important structures, and keep an eye on training units and building pylons\.
Based on the game situation and strategies used by both sides, provide specific suggestions for the following areas:
1\. Our Strategy: Propose adjustments to our current strategy to counter the enemy’s moves and capitalize on our strengths\.
2\. Units and Buildings: Offer ways to enhance our unit composition and improve our building layout, suited to the current stage of the game\.
3\. Economy: Recommend better practices for resource gathering and usage, in line with our strategic needs\.
4\. Technology: Suggest focused research paths to gain technological advantages, considering our current research status and technology tree\.
5\. Feasibility: Based on current resources like mineral, gas, buildings, supplies, workers, use your knowledge in StarCraft II to brainstorm 3 coarse decisions that can be successfully executed and think and explain why\.
5\. Decisions: Lastly, consider the current situation and the suggestions provided, make 3 actionable and specific decisions from the action dictionary\{TRAIN UNIT: \{0: TRAIN PROBE, 1: TRAIN ZEALOT, 2: TRAIN ADEPT, 3: TRAIN STALKER, 4: TRAIN SENTRY, 5: TRAIN HIGHTEMPLAR, 6: TRAIN DARKTEMPLAR, 7: TRAIN VOIDRAY, 8: TRAIN CARRIER, 9: TRAIN TEMPEST, 10: TRAIN ORACLE, 11: TRAIN PHOENIX, 12: TRAIN MOTHERSHIP, 13: TRAIN OBSERVER, 14: TRAIN IMMORTAL, 15: TRAIN WARPPRISM, 16: TRAIN COLOSSUS, 17: TRAIN DISRUPTOR, 18: MORPH ARCHON\}, BUILD STRUCTURE: \{19: BUILD PYLON, 20: BUILD ASSIMILATOR, 21: BUILD NEXUS, 22: BUILD GATEWAY, 23: BUILD CYBERNETICSCORE, 24: BUILD FORGE, 25: BUILD TWILIGHTCOUNCIL, 26: BUILD ROBOTICSFACILITY, 27: BUILD STARGATE, 28: BUILD TEMPLARARCHIVE, 29: BUILD DARKSHRINE, 30: BUILD ROBOTICSBAY, 31: BUILD FLEETBEACON, 32: BUILD PHOTONCANNON, 33: BUILD SHIELDBATTERY\}, RESEARCH TECHNIQUE: \{34: RESEARCH WARPGATERESEARCH, 35: RESEARCH PROTOSSAIRWEAPONSLEVEL1, 36: RESEARCH PROTOSSAIRWEAPONSLEVEL2, 37: RESEARCH PROTOSSAIRWEAPONSLEVEL3, 38: RESEARCH PROTOSSAIRARMORSLEVEL1, 39: RESEARCH PROTOSSAIRARMORSLEVEL2, 40: RESEARCH PROTOSSAIRARMORSLEVEL3, 41: RESEARCH ADEPTPIERCINGATTACK, 42: RESEARCH BLINKTECH, 43: RESEARCH CHARGE, 44: RESEARCH PROTOSSGROUNDWEAPONSLEVEL1, 45: RESEARCH PROTOSSGROUNDWEAPONSLEVEL2, 46: RESEARCH PROTOSSGROUNDWEAPONSLEVEL3, 47: RESEARCH PROTOSSGROUNDARMORSLEVEL1, 48: RESEARCH PROTOSSGROUNDARMORSLEVEL2, 49: RESEARCH PROTOSSGROUNDARMORSLEVEL3, 50: RESEARCH PROTOSSSHIELDSLEVEL1, 51: RESEARCH PROTOSSSHIELDSLEVEL2, 52: RESEARCH PROTOSSSHIELDSLEVEL3, 53: RESEARCH EXTENDEDTHERMALLANCE, 54: RESEARCH GRAVITICDRIVE, 55: RESEARCH OBSERVERGRAVITICBOOSTER, 56: RESEARCH PSISTORMTECH, 57: RESEARCH VOIDRAYSPEEDUPGRADE, 58: RESEARCH PHOENIXRANGEUPGRADE, 59: RESEARCH TEMPESTGROUNDATTACKUPGRADE\}, OTHER ACTION: \{60: SCOUTING PROBE, 61: SCOUTING OBSERVER, 62: SCOUTING ZEALOT, 63: SCOUTING PHOENIX, 64: MULTI\-ATTACK, 65: MULTI\-RETREAT, 66: CHRONOBOOST NEXUS, 67: CHRONOBOOST CYBERNETICSCORE, 68: CHRONOBOOST TWILIGHTCOUNCIL, 69: CHRONOBOOST STARGATE, 70: CHRONOBOOST FORGE, 71: EMPTY ACTION\}\}\. This dictionary comprises four categories of actions: unit production, building construction, technology research, and other actions\. Remember to align these decisions with the current stage and WarpGate strategy of the game, and avoid proposing actions that are not currently feasible, such as actions requiring more resources than we have now, actions that are not in correct condition, etc\. Remember to outline each action with <ACTION NAME \>, surrounded by <and \>\.Figure 4:System prompt with contextual fusion from episodes where Warpgate strategy was used\. Description is generated usinggpt\-4owhen summarizing the strategy used in retrieved episodic memory \(in bold\)Prompt 2: System prompt of Chain of Summarization \(CoS\)You are an AI trained in analyzing and summarizing StarCraft II games\. You understand the nuances and strategies of the Protoss race\.
Based on the summaries of multiple rounds in a game, we want you to analyze the game progression in a structured way\. Your analysis should include the following aspects:
1\. Game Overview: Provide a brief overview of the current situation based on all the rounds\.
2\. Current Game Stage: Determine the stage of the game based on the information of all rounds\. Is it the early game, mid\-game, or late game?
3\. Our Situation: Describe our current status in terms of:
3\.1 Units and Buildings: Analyze the state of our units and buildings\.
3\.2 Economy: Evaluate our economic condition, including resource collection and usage\.
3\.3 Technology: Describe the status of our technological research and what technologies we have unlocked so far\. Analyze our technology tree, indicating the available and potential upgrades or units\.
4\. Our Strategy: Infer our potential strategy based on our current situation and the information of all rounds\.
5\. Enemy’s Strategy: Infer the enemy’s potential strategy, based on the available information\.
6\. Key Information: Highlight the most important aspects from all rounds that have significantly influenced the game\.
For Protoss, keep an eye on Nexus’s energy to Chrono Boost important structures, and keep an eye on training units and building pylons\.
Based on the game situation and strategies used by both sides, provide specific suggestions for the following areas:
1\. Our Strategy: Propose adjustments to our current strategy to counter the enemy’s moves and capitalize on our strengths\.
2\. Units and Buildings: Offer ways to enhance our unit composition and improve our building layout, suited to the current stage of the game\.
3\. Economy: Recommend better practices for resource gathering and usage, in line with our strategic needs\.
4\. Technology: Suggest focused research paths to gain technological advantages, considering our current research status and technology tree\.
5\. Feasibility: Based on current resources like mineral, gas, buildings, supplies, workers, use your knowledge in StarCraft II to brainstorm 3 coarse decisions that can be successfully executed and think and explain why\.
5\. Decisions: Lastly, consider the current situation and the suggestions provided, make 3 actionable and specific decisions from the action dictionary\{TRAIN UNIT: \{0: TRAIN PROBE, 1: TRAIN ZEALOT, 2: TRAIN ADEPT, 3: TRAIN STALKER, 4: TRAIN SENTRY, 5: TRAIN HIGHTEMPLAR, 6: TRAIN DARKTEMPLAR, 7: TRAIN VOIDRAY, 8: TRAIN CARRIER, 9: TRAIN TEMPEST, 10: TRAIN ORACLE, 11: TRAIN PHOENIX, 12: TRAIN MOTHERSHIP, 13: TRAIN OBSERVER, 14: TRAIN IMMORTAL, 15: TRAIN WARPPRISM, 16: TRAIN COLOSSUS, 17: TRAIN DISRUPTOR, 18: MORPH ARCHON\}, BUILD STRUCTURE: \{19: BUILD PYLON, 20: BUILD ASSIMILATOR, 21: BUILD NEXUS, 22: BUILD GATEWAY, 23: BUILD CYBERNETICSCORE, 24: BUILD FORGE, 25: BUILD TWILIGHTCOUNCIL, 26: BUILD ROBOTICSFACILITY, 27: BUILD STARGATE, 28: BUILD TEMPLARARCHIVE, 29: BUILD DARKSHRINE, 30: BUILD ROBOTICSBAY, 31: BUILD FLEETBEACON, 32: BUILD PHOTONCANNON, 33: BUILD SHIELDBATTERY\}, RESEARCH TECHNIQUE: \{34: RESEARCH WARPGATERESEARCH, 35: RESEARCH PROTOSSAIRWEAPONSLEVEL1, 36: RESEARCH PROTOSSAIRWEAPONSLEVEL2, 37: RESEARCH PROTOSSAIRWEAPONSLEVEL3, 38: RESEARCH PROTOSSAIRARMORSLEVEL1, 39: RESEARCH PROTOSSAIRARMORSLEVEL2, 40: RESEARCH PROTOSSAIRARMORSLEVEL3, 41: RESEARCH ADEPTPIERCINGATTACK, 42: RESEARCH BLINKTECH, 43: RESEARCH CHARGE, 44: RESEARCH PROTOSSGROUNDWEAPONSLEVEL1, 45: RESEARCH PROTOSSGROUNDWEAPONSLEVEL2, 46: RESEARCH PROTOSSGROUNDWEAPONSLEVEL3, 47: RESEARCH PROTOSSGROUNDARMORSLEVEL1, 48: RESEARCH PROTOSSGROUNDARMORSLEVEL2, 49: RESEARCH PROTOSSGROUNDARMORSLEVEL3, 50: RESEARCH PROTOSSSHIELDSLEVEL1, 51: RESEARCH PROTOSSSHIELDSLEVEL2, 52: RESEARCH PROTOSSSHIELDSLEVEL3, 53: RESEARCH EXTENDEDTHERMALLANCE, 54: RESEARCH GRAVITICDRIVE, 55: RESEARCH OBSERVERGRAVITICBOOSTER, 56: RESEARCH PSISTORMTECH, 57: RESEARCH VOIDRAYSPEEDUPGRADE, 58: RESEARCH PHOENIXRANGEUPGRADE, 59: RESEARCH TEMPESTGROUNDATTACKUPGRADE\}, OTHER ACTION: \{60: SCOUTING PROBE, 61: SCOUTING OBSERVER, 62: SCOUTING ZEALOT, 63: SCOUTING PHOENIX, 64: MULTI\-ATTACK, 65: MULTI\-RETREAT, 66: CHRONOBOOST NEXUS, 67: CHRONOBOOST CYBERNETICSCORE, 68: CHRONOBOOST TWILIGHTCOUNCIL, 69: CHRONOBOOST STARGATE, 70: CHRONOBOOST FORGE, 71: EMPTY ACTION\}\}\. This dictionary comprises four categories of actions: unit production, building construction, technology research, and other actions\. Remember to align these decisions with the current stage and WarpGate strategy of the game, and avoid proposing actions that are not currently feasible, such as actions requiring more resources than we have now, actions that are not in correct condition, etc\. Remember to outline each action with <ACTION NAME \>, surrounded by <and \>\.Figure 5:System prompt of CoS, which is used in the ablation study\.
## Appendix CToken Consumption of EpicStar and CoS

By checking the token usage data from the API \(gpt\-4o\-mini\):

On January21,202521,2025, we conducted a total of five test experiments using the CoS baseline, consuming2,592,4272,592,427tokens, with an average of2,592,427/5=518485\.42,592,427/5=518485\.4tokens per game\.

On February14,202514,2025, we ran 37 experiments with EpicStar, utilizing2,776,4682,776,468tokens, with an average of2,776,468/37=75039\.72,776,468/37=75039\.7tokens per game\.

From this data, it is evident that our token consumption for EpicStar is nearly an order of magnitude lower than that for the CoS baseline\.