OPINE-World: Programmatic World Modeling with Ontology-error-Prioritized Interactive Exploration

# Programmatic World Modeling with Ontology-error-Prioritized Interactive Exploration
Source: [https://arxiv.org/html/2607.01531](https://arxiv.org/html/2607.01531)
David Courtis Wenhao Li Scott Sanner Department of Computer Science, University of Toronto david\.courtis@mail\.utoronto\.ca chriswenhao\.li@mail\.utoronto\.ca ssanner@mie\.utoronto\.ca

###### Abstract

Learning how an environment behaves from interaction is central to building agents that adapt to unfamiliar tasks\. World models learned with deep networks are flexible but data\-hungry and transfer poorly beyond their training distribution\. Program\-synthesized world models, written as source code by LLMs and refined through counterexample\-guided inductive synthesis \(CEGIS\), are instead data\-efficient and reusable, yet they have been demonstrated mainly on structured\-state worlds with a given object vocabulary, and a single program search does not scale to pixel\-rendered environments whose object structure must be hypothesized flexibly\. We introduce OPINE\-World, an LLM agent that learns an object\-centric programmatic world model online from interaction\. OPINE\-World couples two cooperating agents in a loop of hypothesis and test, one acting in the environment and one synthesizing the model in code with replay verification and model\-based planning, and it steers exploration with a Bayesian measure of object\-type adequacy we call ontology error\. We evaluate OPINE\-World on ARC\-AGI\-3, a benchmark for skill\-acquisition efficiency in which the object vocabulary, the goal, and the action semantics are withheld\. OPINE\-World solves 20 of 25 games without per\-game training and reaches an action\-efficiency score of 78\.4 against the human baseline\.

## 1Introduction

A world model predicts how an environment changes under each action, and an agent that holds one can plan and transfer to tasks that share a mechanism\(Sutton,[1991](https://arxiv.org/html/2607.01531#bib.bib36); Ha and Schmidhuber,[2018](https://arxiv.org/html/2607.01531#bib.bib37)\)\. In model\-based reinforcement learning, world models learned with deep networks are general but data\-hungry, and they transfer poorly beyond their training distribution\(Hafner and others,[2020](https://arxiv.org/html/2607.01531#bib.bib16); Schrittwieser and others,[2020](https://arxiv.org/html/2607.01531#bib.bib17); Hafneret al\.,[2025](https://arxiv.org/html/2607.01531#bib.bib38); Kaiseret al\.,[2020](https://arxiv.org/html/2607.01531#bib.bib41)\)\. A world model written as a program is the data\-efficient alternative, synthesized from logged transitions and refined by counterexample\-guided inductive synthesis \(CEGIS\), and it is inspectable and reusable\(Tanget al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib2); Ellis and others,[2025](https://arxiv.org/html/2607.01531#bib.bib3); Solar\-Lezamaet al\.,[2006](https://arxiv.org/html/2607.01531#bib.bib39)\)\. Such a program is most economical when it is factored by object type, with one transition rule shared across all objects of a type, so that few parameters support many predictions and a rule learned from one object transfers to the rest\(Diuket al\.,[2008](https://arxiv.org/html/2607.01531#bib.bib6); Guestrinet al\.,[2003](https://arxiv.org/html/2607.01531#bib.bib47); Džeroskiet al\.,[2001](https://arxiv.org/html/2607.01531#bib.bib48)\)\.

Factoring helps only when the object partition is approximately correct, and in an open environment that partition is not given and must be inferred from the same interaction the rules are learned from\(Kempet al\.,[2006](https://arxiv.org/html/2607.01531#bib.bib24); Tehet al\.,[2006](https://arxiv.org/html/2607.01531#bib.bib25)\)\. Two lines of work mark the extremes of when to commit to it\. Program\-synthesis\-and\-plan systems fix the partition at once and plan through the synthesized model, repairing it when no plan reaches reward\(Tanget al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib2); Ellis and others,[2025](https://arxiv.org/html/2607.01531#bib.bib3); Ahmed and others,[2025](https://arxiv.org/html/2607.01531#bib.bib4)\), so a partition fixed before the data identify it carries its error into every rule built on it\. Model\-free agentic systems fix no structure and act from a natural\-language interaction log with no synthesized model\(Foxet al\.,[2026](https://arxiv.org/html/2607.01531#bib.bib5); Yaoet al\.,[2023](https://arxiv.org/html/2607.01531#bib.bib73); Wanget al\.,[2023](https://arxiv.org/html/2607.01531#bib.bib74)\), keeping no reusable model and no record of what has been identified\. The program\-synthesis line has been shown on structured\-state inputs with a given object vocabulary, and a single program search does not scale to pixel\-rendered environments whose object structure must be recovered from observation\(Tanget al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib2); Ellis and others,[2025](https://arxiv.org/html/2607.01531#bib.bib3); Chollet,[2019](https://arxiv.org/html/2607.01531#bib.bib40)\)\.

We introduce OPINE\-World, an LLM agent that learns an object\-centric programmatic world model online from interaction\. OPINE\-World combines four mechanisms\. The model is factored by object type, so a rule is repaired one type at a time\. Two cooperating LLM agents run a hypothesis\-and\-test loop over one replay buffer\. The action agent proposes hypotheses about the dynamics and the goal and probes them in the environment, guided by a natural\-language world model it maintains, while the synthesizer turns each hypothesis into code and a critic challenges the fitted model for weak generalization\. A candidate program is admitted only when it reproduces every recorded transition exactly\. A planner then searches the verified model and validates each step against the environment\. A Bayesian measure of object\-type adequacy, the ontology error, steers exploration toward objects whose behavior the current types do not yet explain\.

On ARC\-AGI\-3\(ARC Prize Foundation,[2026](https://arxiv.org/html/2607.01531#bib.bib1)\), a benchmark for skill\-acquisition efficiency in which the object vocabulary, the goal, and the action semantics are withheld, OPINE\-World solves 20 of 25 games without per\-game training, surpasses a strong single\-agent coding baseline, and stays under the human action count on most of the games it solves, while program\-synthesis and neural latent world models solve none\.

#### Contributions\.

1. 1\.OPINE\-World, an LLM agent that learns object\-centric programmatic world models online from interaction, running a hypothesis\-and\-test loop of two cooperating LLM agents with counterexample\-guided synthesis and model\-based planning \(Section[3](https://arxiv.org/html/2607.01531#S3)\)\.
2. 2\.A method that discovers the object ontology from interaction, where prior program\-synthesized world models receive a structured state with a given object vocabulary, which extends them to pixel\-rendered environments with unknown object structure \(Section[3](https://arxiv.org/html/2607.01531#S3)\)\.
3. 3\.An evaluation on ARC\-AGI\-3 in which OPINE\-World solves 20 of 25 games and 160 of 183 levels without per\-game training, exceeding a strong single\-agent coding agent, while program\-synthesis and neural latent world models solve none \(Section[4](https://arxiv.org/html/2607.01531#S4)\)\.

## 2Problem Setting

#### Object\-oriented MDP\.

We model an interactive environment as an object\-oriented Markov decision process\(Diuket al\.,[2008](https://arxiv.org/html/2607.01531#bib.bib6)\)and instantiate it on ARC\-AGI\-3\. A game is a tuple\(𝒞,𝒜,T,R,s0\)\(\\mathcal\{C\},\\mathcal\{A\},T,R,s\_\{0\}\)with object classes𝒞\\mathcal\{C\}, a finite action set𝒜\\mathcal\{A\}, a deterministic transitionT:𝒮×𝒜→𝒮T:\\mathcal\{S\}\\times\\mathcal\{A\}\\to\\mathcal\{S\}, a rewardRR, and a level\-entry states0s\_\{0\}\. A structured states∈𝒮s\\in\\mathcal\{S\}is a finite collection of typed objectso=\(name,τ,𝐯\)o=\(\\text\{name\},\\tau,\\mathbf\{v\}\), whereτ\\tauis a type and𝐯\\mathbf\{v\}holds atomic attributes such as position, visibility, rotation, and a small pixel pattern\. The action set contains directional moves, a select action, and a pointer click\. The meaning of each action is not given, and the agent recovers it by acting\.

#### Perception and what is withheld\.

The engine exposes each frame as a raw64×6464\\times 64grid of color indices, with no sprite list, object identities, or types\. OPINE\-World synthesizes its own perception, an extractorα\\alphathat segments a frame into object records carrying a position, a visibility flag, and a small pixel pattern, and pairs them across time\. What is withheld is the rest\. The partition of the objects into mechanical types is unknown, the goal is unstated, the actions are anonymous, and no demonstration is provided\. The agent infers the type partition from observation, reads goal relevance from the reward it observes, and discovers action effects by trying actions in context\.

#### Reward and scoring\.

Reward is sparse\. The environment reports success on a level advance and is otherwise silent\. A game is a sequence of levels, and clearing the last level wins the game\. An agent is scored on the number of actions it spends, so an agent that re\-derives a mechanic it has already seen pays for the repetition\. We report the action count to clear a game and whether all levels were cleared\.

#### World model and verification\.

A world model is a Python transitionT^\\widehat\{T\}written by a language model and repaired by counterexample\-guided synthesis against the replay buffer𝒟t=\{\(si,ai,si′\)\}i≤t\\mathcal\{D\}\_\{t\}=\\\{\(s\_\{i\},a\_\{i\},s\_\{i\}^\{\\prime\}\)\\\}\_\{i\\leq t\}\. The acceptance test is exact replay,

ϕ\(𝒟t,T^\):T^\(si,ai\)=si′for all\(si,ai,si′\)∈𝒟t\.\\phi\(\\mathcal\{D\}\_\{t\},\\widehat\{T\}\):\\quad\\widehat\{T\}\(s\_\{i\},a\_\{i\}\)=s\_\{i\}^\{\\prime\}\\ \\text\{ for all \}\\ \(s\_\{i\},a\_\{i\},s\_\{i\}^\{\\prime\}\)\\in\\mathcal\{D\}\_\{t\}\.We do not synthesize a reward function in the optimism sense of earlier work\(Tanget al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib2); Brafman and Tennenholtz,[2002](https://arxiv.org/html/2607.01531#bib.bib9)\)\. Goal recognition uses the observed reward before a level is cleared, and a synthesized Boolean goal predicate after, checked by the same exact\-replay test\(Ahmed and others,[2025](https://arxiv.org/html/2607.01531#bib.bib4); Tanget al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib2)\)\.

###### Assumption 1\(Observable\-Markov determinism\)\.

There is a representation of the structured state under whichTTis a deterministic function of the state and action\. Conditioned on a correct partition into types and a sufficient local context, the effect of an action on each object is a single outcome\.

Assumption[1](https://arxiv.org/html/2607.01531#Thmassumption1)is what makes the exact\-replay test well posed\. Where it fails, for example under hidden state, the test still rejects wrong models, and the ontology error of Section[3](https://arxiv.org/html/2607.01531#S3)has a floor it cannot drive to zero\. We return to that case in Section[6](https://arxiv.org/html/2607.01531#S6)\.

## 3Method

OPINE\-World runs one loop over a growing record of interaction \(Figure[1](https://arxiv.org/html/2607.01531#S3.F1)\)\. The goal\-directed agent takes an action in the live game and appends the resulting transition to a shared buffer\. The world\-model agent reads the buffer and, when the current model has been contradicted, rewrites the object\-centric program so that it again reproduces every recorded transition\. Once a model has been admitted and at least one level has been cleared, a planner searches the model for a route to the goal, and the route is executed one step at a time against the live game\. A Bayesian measure of how well the current object types explain the recorded effects runs alongside the loop and steers where the agents look next\. The rest of this section describes these four mechanisms\.

![Refer to caption](https://arxiv.org/html/2607.01531v1/images/fig0_architecture.png)Figure 1:The OPINE\-World loop\. The goal\-directed agent acts in the live game and records each transition in a shared buffer\. The world\-model agent rewrites an object\-centric program from the buffer, and the program is admitted only when it reproduces every recorded transition exactly\. After a level has been cleared, a planner searches the admitted model, and its plan is executed one step at a time, with any mismatch returning a counterexample to the buffer\. The ontology errorηt\\eta\_\{t\}steers where the agents look next, and a periodic critic challenges the fitted model\.### 3\.1An object\-centric world model

#### A factored program\.

A structured state is a finite set of typed objects,

st=\{oti\}i∈It,oti=\(κti,τti,𝐯ti\),s\_\{t\}=\\\{o\_\{t\}^\{i\}\\\}\_\{i\\in I\_\{t\}\},\\qquad o\_\{t\}^\{i\}=\(\\kappa\_\{t\}^\{i\},\\tau\_\{t\}^\{i\},\\mathbf\{v\}\_\{t\}^\{i\}\),whereκ\\kappais a matching key,τ\\taua mechanical type, and𝐯\\mathbf\{v\}an attribute map over position, visibility, rotation, and a small pixel pattern\. The world model is a program organized by type: for each typeτ\\tauit holds a response rule

fτ:𝒪τ×𝒜×𝒳→𝒪τ,f\_\{\\tau\}:\\mathcal\{O\}\_\{\\tau\}\\times\\mathcal\{A\}\\times\\mathcal\{X\}\\to\\mathcal\{O\}\_\{\\tau\},a function from an object of typeτ\\tau, an action, and a read\-only local context𝒳\\mathcal\{X\}over the pre\-action state, to the updated object\. Pairing each before\-objectiiwith an after\-objectμt​\(i\)\\mu\_\{t\}\(i\), the rule predicts

ot\+1μt​\(i\)≈fτti​\(oti,at,uti\),uti=ψ​\(oti,st\),o\_\{t\+1\}^\{\\mu\_\{t\}\(i\)\}\\approx f\_\{\\tau\_\{t\}^\{i\}\}\\\!\\left\(o\_\{t\}^\{i\},\\,a\_\{t\},\\,u\_\{t\}^\{i\}\\right\),\\qquad u\_\{t\}^\{i\}=\\psi\(o\_\{t\}^\{i\},s\_\{t\}\),whereutiu\_\{t\}^\{i\}is a local context feature and the transitionT^\\widehat\{T\}applies the matchingfτf\_\{\\tau\}to every object and collects the results\. A program that places all behavior in one type degenerates to a single monolithic transition\. The factored form subsumes STRIPS operators, where the add and delete lists are the difference between an object before and after, with conditional effects organized by type rather than enumerated\(Fikes and Nilsson,[1971](https://arxiv.org/html/2607.01531#bib.bib7); Pednault,[1989](https://arxiv.org/html/2607.01531#bib.bib80); McDermottet al\.,[1998](https://arxiv.org/html/2607.01531#bib.bib81)\)\.

#### Why structure stays local\.

Organizing the program by type ties many predictions to few rules, and it lets a rule learned from one object apply to every object of its type\(Diuket al\.,[2008](https://arxiv.org/html/2607.01531#bib.bib6); Džeroskiet al\.,[2001](https://arxiv.org/html/2607.01531#bib.bib48); Battagliaet al\.,[2018](https://arxiv.org/html/2607.01531#bib.bib49)\)\. The benefit holds only when the grouping is close to correct\. A group that joins objects with different dynamics ties one rule across mismatched data, and the rule is biased wherever the objects differ\. OPINE\-World does not force one object per type and does not commit a partition in advance\. Objects in ARC\-AGI\-3 are often fragments of a single behavioral unit, such as a bridge drawn from several sprite ids or a counter built from separate glyphs, and forcing such fragments into separate types splits one rule across type boundaries\. Because each rule is attached to a type and is repaired one type at a time, a wrong grouping is corrected locally and does not spread through the rest of the model\.

### 3\.2Two cooperating agents

#### Acting and modeling are separate jobs\.

Decoupling the agent that gathers experience from the one that fits the model is a long\-standing design in reinforcement learning\(Sutton,[1991](https://arxiv.org/html/2607.01531#bib.bib36); Konda and Tsitsiklis,[1999](https://arxiv.org/html/2607.01531#bib.bib42); Horganet al\.,[2018](https://arxiv.org/html/2607.01531#bib.bib43)\), and cooperating language\-model agents with specialized roles have outperformed a single agent on multi\-step tasks\(Honget al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib44); Wuet al\.,[2023](https://arxiv.org/html/2607.01531#bib.bib45); Qianet al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib46)\)\. OPINE\-World runs two language\-model agents over the shared buffer\. The goal\-directed agent plays the live game\. It reads the interaction log with ordinary file tools, runs short scripts over it, and chooses the next action or short sequence of actions\. The world\-model agent writes and repairs the program\. It reads the same buffer and the diagnostics, proposes a program, and submits it to the verifier\. The two never write to each other’s artifacts, and they communicate only through the buffer and a short structured handoff\.

#### Proposal comes from the language model, guarantees from the verifier\.

The language model supplies what a discrete search cannot\. It proposes a partition of the scene into types, names candidate roles for objects, and writes code that generalizes a handful of transitions into a rule\(Chenet al\.,[2021](https://arxiv.org/html/2607.01531#bib.bib53); Austinet al\.,[2021](https://arxiv.org/html/2607.01531#bib.bib54)\)\. These proposals are cheap to generate and expensive to verify by enumeration, which is the regime where a language model is most useful\(Kambhampatiet al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib58); Wanget al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib57)\)\.

#### A loop of hypothesis and test\.

The two agents, together with a periodic critic, run a loop of hypothesis and test\(Piriyakulkijet al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib33); Tenenbaumet al\.,[2011](https://arxiv.org/html/2607.01531#bib.bib65); Wanget al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib57)\)\. The action agent proposes hypotheses about the dynamics and the goal, drawn from a natural\-language world model that the agents keep in shared notes and from the current synthesized model, and it explores to probe them\. The synthesizer turns a hypothesis into code, and the exact\-replay check accepts or refutes it\. A critic runs on a fixed cadence and challenges a fitted model for weak generalization, asking which parts are tailored to one level and would fail on a new layout, which presses the synthesizer to weigh competing hypotheses rather than commit to the first that fits\.

### 3\.3Verification and counterexample\-guided synthesis

#### Acceptance is an exact check\.

A proposed program is admitted only when it reproduces every transition in the buffer exactly,

ϕT\(T^,𝒟t\):T^\(sℓ,aℓ\)=sℓ\+1for all\(sℓ,aℓ,sℓ\+1\)∈𝒟t,\\phi\_\{T\}\(\\widehat\{T\},\\mathcal\{D\}\_\{t\}\):\\quad\\widehat\{T\}\(s\_\{\\ell\},a\_\{\\ell\}\)=s\_\{\\ell\+1\}\\ \\text\{ for all \}\\ \(s\_\{\\ell\},a\_\{\\ell\},s\_\{\\ell\+1\}\)\\in\\mathcal\{D\}\_\{t\},attribute for attribute and object for object\. Acceptance is a decision rather than a score\(Mitchell,[1982](https://arxiv.org/html/2607.01531#bib.bib62); Blumeret al\.,[1987](https://arxiv.org/html/2607.01531#bib.bib63)\)\. A program is also run twice on each transition, and a program whose two runs differ is rejected, which removes any model that depends on hidden state and would break a forward planner\. The goal predicate is checked the same way once reward has been observed, and a static check rejects a predicate that recognizes the goal by reading a cached future state rather than by testing the goal mechanic\.

#### Repair follows counterexamples\.

Synthesis does not run on a schedule\. The world\-model agent builds the first model once enough transitions exist, and after that it rewrites only when the live model mispredicts an observed transition\. A mispredicted transition is a counterexample\(Solar\-Lezamaet al\.,[2006](https://arxiv.org/html/2607.01531#bib.bib39); Jhaet al\.,[2010](https://arxiv.org/html/2607.01531#bib.bib59)\), and it serves both as a constraint for the next program and as a real observation that updates the diagnostics\. A short deferral window lets a few counterexamples accumulate before a rewrite, and a stall guard stops repeated rewrites that fail to improve accuracy until a fresh counterexample arrives\.

### 3\.4Object\-type diagnostics and ontology error

Alongside the program, OPINE\-World keeps a cheap Bayesian diagnostic that scores how well the current types explain what it has seen and steers exploration toward what they do not\. It is built from coarse effect signatures rather than exact deltas\.

#### Effect signatures\.

For a paired object letΔti⊆𝖠𝗍𝗍𝗋​\(τti\)\\Delta\_\{t\}^\{i\}\\subseteq\\mathsf\{Attr\}\(\\tau\_\{t\}^\{i\}\)be the attributes that changed; a quotient mapρ\\rhosends it to a categorical signature from an alphabetℰ\\mathcal\{E\}grown from what is observed,

Δti=\{b:𝐯t\+1μt​\(i\)​\(b\)≠𝐯ti​\(b\)\},eti=ρ​\(Δti\)∈ℰ,\\Delta\_\{t\}^\{i\}=\\\{\\,b:\\mathbf\{v\}\_\{t\+1\}^\{\\mu\_\{t\}\(i\)\}\(b\)\\neq\\mathbf\{v\}\_\{t\}^\{i\}\(b\)\\,\\\},\\qquad e\_\{t\}^\{i\}=\\rho\(\\Delta\_\{t\}^\{i\}\)\\in\\mathcal\{E\},recording*which*attributes changed and discarding the values \(Figure[2](https://arxiv.org/html/2607.01531#S3.F2)\)\.

![Refer to caption](https://arxiv.org/html/2607.01531v1/images/img2.png)Figure 2:Effect signatures\. For one transitionst→st\+1s\_\{t\}\\to s\_\{t\+1\}\(shown asO→O′O\\to O^\{\\prime\}\), each paired object’s changed\-attribute setΔti\\Delta\_\{t\}^\{i\}is sent by the quotient mapρ\\rhoto a single categorical signatureeti∈ℰe\_\{t\}^\{i\}\\in\\mathcal\{E\}\. The alphabetℰ\\mathcal\{E\}is grown from the signatures observed, such asno\_change,x,x,y,pixels,gone, andborn\. Exact deltas such asx:12→15x:12\\to 15are discarded\.
#### The local effect table\.

Each object\-transition is filed into a row keyed by type, action, and local context,j=\(τ,a,u\)j=\(\\tau,a,u\), and the table counts signatures per row,Ct​\(j,e\)C\_\{t\}\(j,e\)\. A symmetric Dirichlet prior yields a posterior over the row’s effect distribution and its mean,

qj∣𝒟t∼Dir⁡\(α0​𝟏m\+𝐜j\),q^j​\(e\)=α0\+Ct​\(j,e\)m​α0\+∑e′Ct​\(j,e′\)\.q\_\{j\}\\mid\\mathcal\{D\}\_\{t\}\\sim\\operatorname\{Dir\}\(\\alpha\_\{0\}\\mathbf\{1\}\_\{m\}\+\\mathbf\{c\}\_\{j\}\),\\qquad\\widehat\{q\}\_\{j\}\(e\)=\\frac\{\\alpha\_\{0\}\+C\_\{t\}\(j,e\)\}\{m\\alpha\_\{0\}\+\\sum\_\{e^\{\\prime\}\}C\_\{t\}\(j,e^\{\\prime\}\)\}\.A row that mixes signatures is under\-observed or missing a context feature, and adding a before\-state featureuucan split it into concentrated subrows \(Figure[3](https://arxiv.org/html/2607.01531#S3.F3)\)\.

![Refer to caption](https://arxiv.org/html/2607.01531v1/images/_img3_v.png)Figure 3:Row concentration by context refinement\. Pooling every local context of the player under one action gives a single mixed rowj=\(τ,a\)j=\(\\tau,a\)whose effect distributionq^j\\widehat\{q\}\_\{j\}spreads over many signatures\. Splitting the row by a before\-state context featureuu, here a hash of the object’s own pixels \(lit versus unlit\), separates the conflated effects: the unlit subrow becomes deterministic, while a subrow that stays mixed signals a context feature still missing\.
#### Ontology error\.

Two normalized entropies measure what is unresolved, the type uncertainty of an object and the effect uncertainty of its row,

Uitype=𝖧​\[Pr⁡\(τti∣𝒟t\)\]log⁡K,Ujrow=𝖧​\(q^j\)log⁡m∈\[0,1\],U\_\{i\}^\{\\rm type\}=\\frac\{\\mathsf\{H\}\[\\Pr\(\\tau\_\{t\}^\{i\}\\mid\\mathcal\{D\}\_\{t\}\)\]\}\{\\log K\},\\qquad U\_\{j\}^\{\\rm row\}=\\frac\{\\mathsf\{H\}\(\\widehat\{q\}\_\{j\}\)\}\{\\log m\}\\ \\in\[0,1\],and their noisy\-OR is the per\-object ontology error, averaged into the aggregate

ηti=1−\(1−Uitype\)​\(1−Ujtirow\),ηt=1Nt​∑ℓ<t∑i∈Iℓηℓi\.\\eta\_\{t\}^\{i\}=1\-\(1\-U\_\{i\}^\{\\rm type\}\)\(1\-U\_\{j\_\{t\}^\{i\}\}^\{\\rm row\}\),\\qquad\\eta\_\{t\}=\\frac\{1\}\{N\_\{t\}\}\\sum\_\{\\ell<t\}\\sum\_\{i\\in I\_\{\\ell\}\}\\eta\_\{\\ell\}^\{i\}\.A highηti\\eta\_\{t\}^\{i\}marks objects and contexts the current types do not yet explain, so the goal\-directed agent probes them and the synthesizer refines a type or adds a context feature, in line with exploring to reduce model uncertainty\(Schmidhuber,[2010](https://arxiv.org/html/2607.01531#bib.bib71); Pathaket al\.,[2017](https://arxiv.org/html/2607.01531#bib.bib70); Houthooftet al\.,[2016](https://arxiv.org/html/2607.01531#bib.bib69)\)\. A fallingηt\\eta\_\{t\}tracks the ontology settling over a run \(Figure[4](https://arxiv.org/html/2607.01531#S3.F4)\), while correctness is decided not byη\\etabut by the exact\-replay test below\. Appendix[F](https://arxiv.org/html/2607.01531#A6)gives the full construction\.

![Refer to caption](https://arxiv.org/html/2607.01531v1/images/_img4_v.png)Figure 4:Aggregate ontology errorηt\\eta\_\{t\}over one run \(ar25\), combining type and row uncertainty through the noisy\-ORηti=1−\(1−Uitype\)​\(1−Ujtirow\)\\eta\_\{t\}^\{i\}=1\-\(1\-U\_\{i\}^\{\\rm type\}\)\(1\-U\_\{j\_\{t\}^\{i\}\}^\{\\rm row\}\)\. After initializationηt\\eta\_\{t\}falls as types and rows resolve; it rises transiently while a wrong model is repaired by synthesis, then falls steadily as the committed model stabilizes\. Level completions are markedL​0,…,L​5L0,\\ldots,L5\.

### 3\.5Planning over the verified model

Once a model has been admitted and a level has been cleared, the world\-model agent writes a planner over the verified modelM^=\(𝒮,𝒜,T^,G^\)\\widehat\{M\}=\(\\mathcal\{S\},\\mathcal\{A\},\\widehat\{T\},\\widehat\{G\}\), a bounded forward search for an action sequence that reaches a goal state underT^\\widehat\{T\}and the goal predicateG^\\widehat\{G\}\. The planner is first checked offline by planning to reward from the entry states of the levels already cleared\. A plan that passes is executed against the live game one step at a time\. Each executed step is compared with the model’s prediction\. A match continues the plan, and a mismatch ends it, records the offending transition as a counterexample, and returns control to the goal\-directed agent\. A plan that reaches the goal confirms the win at its action count\.

## 4Experiments

#### Setup\.

We evaluate OPINE\-World on ARC\-AGI\-3\(ARC Prize Foundation,[2026](https://arxiv.org/html/2607.01531#bib.bib1)\), a benchmark for skill\-acquisition efficiency, using its public evaluation set of 25 games\. Each game is a sequence of levels with a sparse level\-advance reward, and an agent is scored on the actions it spends and on how many levels it clears\. OPINE\-World plays each game online under the live, on\-policy budget, with no reset to re\-sample a level\. We report the action count to clear a game, the levels cleared, and the outcome\. The synthesis and action agents run Claude Opus 4\.8 behind a filesystem sandbox, described in Appendix[A](https://arxiv.org/html/2607.01531#A1)\. Neither OPINE\-World nor baseline1 is trained, fine\-tuned, or shown demonstrations on the evaluation games\. The language model is a general pretrained model that is not updated, and the agent learns each game’s mechanics online during the scored run, from raw rendered frames\.

#### Baselines\.

The strongest comparison without per\-game training is a single\-agent object\-centric coding agent, called baseline1 here, which synthesizes an executable world model, verifies it against past transitions, and plans through it, all within one agent driven by a strong reasoning model\. As publicly reported, baseline1 runs GPT\-5\.5 at high reasoning effort, and OPINE\-World runs Claude Opus 4\.8, so the comparison is between systems that use different base models rather than a single controlled variable, and a same\-model study is left to future work\. We report two reference points that fail outright\. WorldCoder searches for one program that explains the whole game and repairs it under an optimism constraint\(Tanget al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib2)\), and a set of neural latent world models in the Dreamer and MuZero families learn dynamics in a latent space\(Hafner and others,[2020](https://arxiv.org/html/2607.01531#bib.bib16); Schrittwieser and others,[2020](https://arxiv.org/html/2607.01531#bib.bib17); Hafneret al\.,[2025](https://arxiv.org/html/2607.01531#bib.bib38)\)\. We list separately a continual\-learning vision agent, called Vision here, which explores the public game set offline and accumulates weights, then plays the scored run with those weights frozen\. Because Vision trains on the very games it is later scored on, its action counts measure performance after exposure to the evaluation games\. We mark it accordingly and keep it out of the no\-training comparison\.

Table 1:Aggregate performance on the ARC\-AGI\-3 public evaluation set \(25 games\)\. The mean levels\-cleared fraction averages cleared over total across the 25 games\.†Vision explores the evaluation games offline before the scored run, so its scores are not a no\-training comparison\. The Score column is the ARC\-AGI\-3 action\-efficiency score over all 25 games \(Figure[5](https://arxiv.org/html/2607.01531#S4.F5)\)\.
#### OPINE\-World leads on the benchmark’s action\-efficiency score\.

ARC\-AGI\-3 scores an agent by how few actions it spends against a human baseline\. Each level scores\(human/agent\)2\(\\text\{human\}/\\text\{agent\}\)^\{2\}capped at1\.151\.15, levels are weighted by their index, and the per\-game score is capped at100100, so a game reaches100100only when every level is cleared at or above human pace\. Figure[5](https://arxiv.org/html/2607.01531#S4.F5)reports the per\-game scores\. OPINE\-World reaches a total of78\.478\.4over all 25 games, where baseline1 reaches63\.863\.8and the pretrained Vision agent63\.263\.2\.

![Refer to caption](https://arxiv.org/html/2607.01531v1/images/scores_heatmap.png)Figure 5:Per\-game ARC\-AGI\-3 action\-efficiency score \(0–100100\) for OPINE\-World and baseline1, sorted by OPINE\-World score\. Each cell is a game score, the index\-weighted mean of per\-levelmin⁡\(1\.15,\(human/agent\)2\)\\min\(1\.15,\(\\text\{human\}/\\text\{agent\}\)^\{2\}\)over the levels the agent cleared, where brighter is higher\. Averaged over all 25 games OPINE\-World scores78\.478\.4and baseline163\.863\.8, while the pretrained Vision agent scores63\.263\.2\.
#### OPINE\-World clears more games than the baseline\.

Table[1](https://arxiv.org/html/2607.01531#S4.T1)summarizes the set, and Table[3](https://arxiv.org/html/2607.01531#S4.T3)reports every game\. OPINE\-World clears 20 of the 25 games\. baseline1 clears 14\. Of the 14 games baseline1 clears, OPINE\-World clears 13, the lone exception being ka59, and OPINE\-World additionally clears seven games that defeat baseline1 outright\. WorldCoder and the neural latent world models clear no game under the benchmark’s budget\. Figures[6](https://arxiv.org/html/2607.01531#S4.F6),[7](https://arxiv.org/html/2607.01531#S4.F7), and[8](https://arxiv.org/html/2607.01531#S4.F8)visualize the comparison, and Appendix[D](https://arxiv.org/html/2607.01531#A4)adds coverage, a per\-game scatter, and a levels\-cleared heatmap\.

#### The decisive margin is on the hard games\.

Six games defeat baseline1 entirely \(Table[2](https://arxiv.org/html/2607.01531#S4.T2)\)\. On re86, tn36, vc33, m0r0, sc25, and sp80, baseline1 exhausts its budget without clearing the game, after spending 10,874 actions in total across the six\. OPINE\-World clears all six, and it does so in 2,578 actions in total, about a quarter of what baseline1 spent before failing\. On the same six games the human reference uses 3,994 actions, so OPINE\-World clears them in about two thirds of the human budget\. OPINE\-World also wins dc22, a seventh game baseline1 fails, at a higher cost of 1,479 actions\. Two effects produce the margin\. OPINE\-World clears games that defeat a strong single agent, and it avoids spending thousands of actions on a game it cannot yet model\. The easy games both agents solve add little\.

![Refer to caption](https://arxiv.org/html/2607.01531v1/images/perlevel_hard.png)Figure 6:Per\-level action counts on three games baseline1 cannot clear\. baseline1 \(orange\) spends hundreds to over a thousand actions on a single level and still fails it, while OPINE\-World \(blue\) clears the same level in tens of actions\. The human reference is grey\.
#### On games both solve, the two are comparable\.

On the thirteen games that both OPINE\-World and baseline1 clear, the action counts are close\. OPINE\-World uses fewer actions on six of them and baseline1 uses fewer on seven, and the totals are within a fraction of a percent of each other\. OPINE\-World’s advantage over a strong single agent lies in covering harder games and in holding back action counts when a game is hard to model\.

![Refer to caption](https://arxiv.org/html/2607.01531v1/images/actions_common.png)Figure 7:Actions to clear each game that both OPINE\-World and baseline1 win, with the human reference in grey, ordered by human action count\. On games already within reach of a strong single agent the three are close, and OPINE\-World is at or below the human count on most of them\.
#### OPINE\-World is action\-efficient against the human reference\.

On 16 of its 20 wins, OPINE\-World clears the game in fewer actions than the human reference\. It uses more on four of them, by a single action on sb26 and by wider margins on ls20, tn36, and dc22\. Averaged over its wins, the ratio of human actions to OPINE\-World actions is 1\.7, with the largest margins on m0r0 \(4\.3\), lp85 \(3\.5\), and cn04 \(3\.0\)\.

![Refer to caption](https://arxiv.org/html/2607.01531v1/images/efficiency_human.png)Figure 8:Action efficiency relative to the human reference on the 20 games OPINE\-World wins, as the ratio of human actions to OPINE\-World actions\. Bars past the dashed parity line are games OPINE\-World clears in fewer actions than the human reference, which holds for 16 of 20\.
#### A note on Vision\.

The continual\-learning vision agent reaches a higher raw scorecard on the public set, near 63%, and it clears 12 of 25 games in the scored run\. Its offline phase plays the same public games and grows the weights it then freezes, so the action counts in its scored run follow earlier passes over those games\. We include it for completeness and do not treat it as a no\-training result\. OPINE\-World clears more games than Vision, 20 against 12, even though Vision trains on the evaluation games beforehand\.

#### Ablations\.

We compare against a strong single agent and against failing reference points\. Due to the heavy interdependency of each sub\-component, where a naive removal of any component in isolation will cause system instability, we leave more targeted per\-component ablations to future work\.

Table 2:Six games where baseline1 fails \(GO\) and OPINE\-World clears the game at a wide margin\. baseline1 exhausts its budget \(GO\) on every one, spending4\.2×4\.2\\timesmore actions in aggregate than OPINE\-World while clearing fewer levels\. OPINE\-World solves all six in0\.65×0\.65\\timesthe human action budget, and it also wins a seventh baseline1 failure, dc22, at a higher cost\.Table 3:Per\-game results on the ARC\-AGI\-3 public evaluation set \(25 games\)\. Each system cell reports actions, levels cleared, and outcome\. Outcomes are WIN \(all levels cleared\), GO \(game over, budget exhausted\)\. baseline1 and OPINE\-World use no per\-game training and play under the live on\-policy budget\.†Vision explores this evaluation set offline and freezes its weights before the scored run, so its action counts follow prior passes over these games and are not a no\-training comparison\. Rows shaded green mark six games that baseline1 fails and OPINE\-World clears at a wide margin\. A plus on a OPINE\-World level count marks a run that had cleared that many levels and was still progressing at game over\.

## 5Related Work

#### Program\-synthesized world models\.

WorldCoder and PoE\-World synthesize an executable world model, plan through it, and summarize what is known about the world as a binary distinction between learned and unlearned\(Tanget al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib2); Ellis and others,[2025](https://arxiv.org/html/2607.01531#bib.bib3)\)\. TheoryCoder adds a hand\-designed layer of PDDL predicates and operators above the synthesized dynamics and plans at two levels\(Ahmed and others,[2025](https://arxiv.org/html/2607.01531#bib.bib4)\)\. OPINE\-World keeps the synthesized executable model and the counterexample loop, and it differs in four ways\. It separates acting from modeling across two agents, it discovers the object and role layer from data rather than receiving it by hand, it admits a model only by exact replay rather than by an optimism or likelihood criterion, and it gates the planner on having cleared a level and validates each planned step against the live game\. It also drops the synthesized reward function and the optimism constraint that the earlier systems carry\.

#### Model\-free agentic exploration\.

A coding agent can play these games by reading a log of its own interaction and selecting the next action, with no synthesized dynamics\(Foxet al\.,[2026](https://arxiv.org/html/2607.01531#bib.bib5)\)\. The approach never commits to a wrong model, and it keeps no reusable model, no way to simulate a counterfactual, and no record of what has been pinned down\. OPINE\-World adopts the read\-and\-script pattern for its goal\-directed agent and restores the verified model and the diagnostic that a log\-only agent forgoes\.

#### Neural and latent world models\.

Latent\-dynamics models learn to predict and plan in a learned state space and have mastered arcade and control domains with large amounts of interaction\(Hafner and others,[2020](https://arxiv.org/html/2607.01531#bib.bib16); Schrittwieser and others,[2020](https://arxiv.org/html/2607.01531#bib.bib17); Hafneret al\.,[2025](https://arxiv.org/html/2607.01531#bib.bib38)\)\. Sample efficiency improves with explicit model learning\(Kaiseret al\.,[2020](https://arxiv.org/html/2607.01531#bib.bib41); Sutton,[1991](https://arxiv.org/html/2607.01531#bib.bib36)\), and the data these models need is still far beyond the action budget of a single ARC\-AGI\-3 game, while a learned latent state space transfers poorly to a new game with a new ontology, where object\-centric models with per\-entity rules generalize across layouts\(Kanskyet al\.,[2017](https://arxiv.org/html/2607.01531#bib.bib50); Kipfet al\.,[2020](https://arxiv.org/html/2607.01531#bib.bib51)\)\. OPINE\-World uses a program with shared per\-type rules, so a handful of transitions identifies a rule and that rule transfers to every object of its type\.

#### Bayesian structure and language\-model proposals\.

The object and role layer that OPINE\-World infers is a partition\-learning problem, and the Dirichlet effect counts and role posterior follow Bayesian structure learning and relational clustering\(Cooper and Herskovits,[1992](https://arxiv.org/html/2607.01531#bib.bib11); Kempet al\.,[2006](https://arxiv.org/html/2607.01531#bib.bib24); Tehet al\.,[2006](https://arxiv.org/html/2607.01531#bib.bib25)\)\. A growing line uses a language model to propose programs or theories that are then scored against data\(Curtis and others,[2025](https://arxiv.org/html/2607.01531#bib.bib30); Wong and others,[2023](https://arxiv.org/html/2607.01531#bib.bib32); Piriyakulkijet al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib33); Levy and others,[2025](https://arxiv.org/html/2607.01531#bib.bib29)\)\. These systems usually receive at least one structural prior that ARC\-AGI\-3 withholds, such as a fixed ontology, named actions, or a stated goal, and they score proposals by likelihood\. OPINE\-World composes the language\-model proposal with an exact replay test that the observable\-Markov assumption makes well posed\. For planning over the learned model, OPINE\-World uses bounded forward search in the style of width\-based planners over simulators\(Lipovetzky and Geffner,[2012](https://arxiv.org/html/2607.01531#bib.bib79); Lipovetzkyet al\.,[2015](https://arxiv.org/html/2607.01531#bib.bib26)\), and it follows the practice of writing a planner once rather than scoring each search node with a model call\(Corrêaet al\.,[2025](https://arxiv.org/html/2607.01531#bib.bib27); Zhou and others,[2024](https://arxiv.org/html/2607.01531#bib.bib28)\)\. The generate\-test\-repair loop is Reflexion\-style iteration\(Shinn and others,[2023](https://arxiv.org/html/2607.01531#bib.bib15); Madaanet al\.,[2023](https://arxiv.org/html/2607.01531#bib.bib84); Chenet al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib85)\), and the symbolic analogue of learning an action model inside a planning loop is classical action\-model learning\(Lamanna and others,[2021](https://arxiv.org/html/2607.01531#bib.bib18); Yanget al\.,[2007](https://arxiv.org/html/2607.01531#bib.bib82); Ainetoet al\.,[2019](https://arxiv.org/html/2607.01531#bib.bib83)\)\.

## 6Limitations

1. \(i\)Observable\-Markov scope\.The exact\-replay test and the epistemic measure assume that the state and action determine the next state under some representation \(Assumption[1](https://arxiv.org/html/2607.01531#Thmassumption1)\)\. Under hidden state the test still rejects wrong models, and the measure has a floor it cannot drive to zero, so the timing signal degrades\. Hidden\-state games are out of scope here\.
2. \(ii\)Inferred perception\.OPINE\-World works from raw frames and synthesizes its own object extractor, so pairing and the type partition are inferred rather than given\. Engine\-given object identities would make the pairingμt\\mu\_\{t\}exact and the initial type assignment informative, drivingUitype→0U\_\{i\}^\{\\rm type\}\\to 0so the ontology errorηti\\eta\_\{t\}^\{i\}collapses to the row uncertaintyUjtirowU\_\{j\_\{t\}^\{i\}\}^\{\\rm row\}alone, a sharper signal we leave to future work\.
3. \(iii\)Planner scale\.The synthesized planner is a bounded forward search, and on games with a high branching factor a naive search reaches its bound without a plan, which returns control to the goal\-directed agent\. A stronger synthesized search would extend the reach of the planning phase\.
4. \(iv\)Single run per game\.Each game is played once, so the table carries no variance estimate, and the headline counts could shift by a few games under resampling\.

## 7Conclusion

OPINE\-World learns an object\-centric programmatic world model online from interaction by running a hypothesis\-and\-test loop\. Two cooperating LLM agents propose and verify the model, counterexample\-guided synthesis keeps it consistent with every observed transition, and a planner searches the verified model to reach the goal\. On the ARC\-AGI\-3 public set OPINE\-World solves 20 of 25 games and 160 of 183 levels without per\-game training and exceeds a strong single\-agent coding agent on the benchmark’s action\-efficiency score, where program\-synthesis and neural latent world models solve none\. Extending the approach to stochastic and partially observed environments, and to raw\-pixel perception, is left to future work\.

The appendix records how the loop of Section[3](https://arxiv.org/html/2607.01531#S3)is realized \(Appendix[A](https://arxiv.org/html/2607.01531#A1)\), the design choices behind it \(Appendix[B](https://arxiv.org/html/2607.01531#A2)\), the integrity checks on the evaluation \(Appendix[C](https://arxiv.org/html/2607.01531#A3)\), and the full formal construction of the model and its ontology error, additional result visualizations \(Appendix[D](https://arxiv.org/html/2607.01531#A4)\), and the full formal construction \(Appendix[F](https://arxiv.org/html/2607.01531#A6)onward\)\.

## Appendix AImplementation and System Details

The components of Section[3](https://arxiv.org/html/2607.01531#S3)are realized in the running system as follows\. OPINE\-World runs two language\-model agents over one shared replay buffer of observed transitions, together with a planner that the second agent writes as code\. The goal\-directed action agent plays the live game\. The world\-model synthesis agent writes and repairs the executable model\. The planner searches that model after the model has been admitted and a level has been cleared\.

### A\.1The world model artifact

The synthesis agent maintains a single Python file,game\_engine\.py\. The file exposestransition\_function\(state, action\), which returns the predicted next state, andreward\_function\(state\), which returns the predicted reward and a goal flag for a state\. The file may also export aplanner\(\)hook and, in frames mode, anextract\_objects\(frame\)function that recovers an object representation from a raw64×6464\\times 64frame\. The synthesis agent is free to choose any internal organization that fits the observations, including plain functions, classes, or lookup tables\. The form is selected by the agent rather than fixed by the harness\.

### A\.2Per\-role responsibilities

The action agent reaches reward in the live environment using read\-only tools over a monotonic interaction log and a Python interpreter\. The agent reads and greps the log and runs short scripts over it, and the agent never edits the model\. The synthesis agent runs in the background and refinestransition\_function, and after a level has been cleared it also refinesreward\_function\. The planner runs a bounded forward search over the currenttransition\_functionandreward\_function, or over the synthesizer’s ownplanner\(\)hook when one is present\. Roles are separated by objective alone\. No orchestrator subagent sits above them\.

### A\.3Verification

A candidate model is admitted only after it passes the replay check\. Every transition in the buffer is replayed throughtransition\_function, and the predicted next state is compared for exact equality against the observed next state\. The predicted reward and goal flag are compared against the observed reward in the same pass\. A single mismatch rejects the candidate\. Each transition is also evaluated twice, and a candidate whose two outputs differ is rejected, which removes any model that carries hidden module\-level state, because the forward planner assumes a pure deterministic transition\. The verifier also applies a static check to the body ofreward\_function\. A predicate that recognizes the goal by reading the cached entry state of a later level is rejected by that check, because such a predicate passes the replay check trivially while encoding nothing about the goal mechanic\.

### A\.4Synthesis cadence

Synthesis is triggered by counterexamples rather than run on a fixed schedule\. The synthesis agent fires once to build the first model\. After that, it fires only when the live model mispredicts an observed transition\. A short deferral window lets a few moves and errors accumulate before the agent re\-fires, so a single surprising step does not restart synthesis on its own\. A stall guard stops the agent from re\-firing after two rounds that fail to improve transition accuracy, and the agent waits for a fresh counterexample before trying again\.

### A\.5Multi\-tick actions

A single action can animate over several internal engine ticks before the state settles, as happens with water flow, gravity, or a chain reaction\. The model predicts the settled after\-state, and the replay check compares against that settled state\. The intermediate per\-tick frames are surfaced to the synthesis agent as extra evidence, because a mechanic that plays out and reverts before settling leaves no signature in the before\-and\-after pair on its own\.

### A\.6Context discipline

Each agent runs under a uniform compaction discipline\. On crossing a per\-game token threshold, the active agent emits a structured handoff and its session is reset\. The successor agent reads the handoff and resumes from it\. Compaction is local to each role and is otherwise transparent to the formalism\.

### A\.7Sandboxing and resume

The synthesis agent and both playing agents run filesystem\-confined under bubblewrap or under Docker with a filtered network, so the ground\-truth game source cannot be read\. The dynamics are inferred from observation\. Runs are snapshotted, and a run resumes deterministically by resetting the environment and replaying the recorded action sequence, which reconstructs the exact environment state\.

## Appendix BDesign Rationale

The forms used in Section[3](https://arxiv.org/html/2607.01531#S3)are the outcome of a design loop that discarded several plausible alternatives\. We record the load\-bearing choices here so that a reader who asks why an obvious option was avoided has a referent\.

### B\.1No synthesized reward function in the WorldCoder sense

Earlier programmatic world\-model systems synthesize a reward function alongside the transition function and drive a planner with it\(Tanget al\.,[2024](https://arxiv.org/html/2607.01531#bib.bib2); Ellis and others,[2025](https://arxiv.org/html/2607.01531#bib.bib3)\)\. We carried this structure in early drafts and found that nothing in the system depends on a synthesized reward over hypothetical futures\. Action selection runs in the live environment on the environment’s own reward signal\. Goal relevance is read from reward observed on the trajectory\. A later system in the same line encodes goals as predicates over observed state rather than as a synthesized reward function\(Ahmed and others,[2025](https://arxiv.org/html/2607.01531#bib.bib4)\), and we reached the same conclusion\. Dropping the synthesized reward removes the optimism constraint that WorldCoder imposes, and it matches what the ARC\-AGI\-3 environment provides at observation time\. The synthesizedreward\_functionthat the system does maintain is a goal predicate over the structured state, fitted only after reward has been observed\.

### B\.2Forward search rather than regression

Regression planning presupposes a goal written as a conjunction of facts with operators that declare which facts they add and delete, the STRIPS form\(Fikes and Nilsson,[1971](https://arxiv.org/html/2607.01531#bib.bib7)\)\. The synthesized goal predicate is a Boolean function over the structured state with no such factored representation\. Backward chaining over it would require a second synthesis layer to recover an add\-and\-delete schema, which doubles the synthesis surface and adds a failure mode in which a wrong schema yields a wrong regression\. TheoryCoder pays this cost by hand\-writing the schema in PDDL\(Ahmed and others,[2025](https://arxiv.org/html/2607.01531#bib.bib4)\)\. We avoid the second layer\. Forward search is well posed directly over the artifact, and forward search over learned models is the dominant choice in this literature, from MuZero\(Schrittwieser and others,[2020](https://arxiv.org/html/2607.01531#bib.bib17)\)to width\-based arcade planners\(Lipovetzkyet al\.,[2015](https://arxiv.org/html/2607.01531#bib.bib26)\)\.

### B\.3Counterexample\-triggered synthesis rather than a fixed cadence

Re\-synthesizing on a fixed schedule spends model calls when the current model already predicts every observed transition\. Firing on counterexamples spends a call only when the live model has been contradicted by an observation, which is the only event that supplies new constraint for the verifier\. The deferral window and the stall guard of Appendix[A\.4](https://arxiv.org/html/2607.01531#A1.SS4)keep the loop from thrashing on a single surprising step or on a repair the agent cannot find, and they return control to the action agent for more exploration when synthesis stalls\. The generate\-test\-reflect shape follows Reflexion\-style iteration\(Shinn and others,[2023](https://arxiv.org/html/2607.01531#bib.bib15)\)\.

### B\.4Why the planner is gated and verified

The planner is allowed to fire only after at least one level has been cleared\. By the time the first level\-advance reward fires, the buffer has exercised the sprite types the goal mechanic touches, so the model has the evidence a plan would rely on\. Before the planner is used on the live game, it is verified offline by planning to reward from the entry states of completed levels\. A plan is then executed one real step at a time, and any mismatch between a predicted step and the observed step becomes a counterexample that returns control to the action agent and reopens synthesis\. The per\-step check is the runtime form of a plausibility test, so a separate pre\-flight probe before each invocation would catch the same failures at extra cost and is omitted\. There is no engine forward\-search fallback once the planner is in use\. Per\-node language\-model scoring of search states\(Zhou and others,[2024](https://arxiv.org/html/2607.01531#bib.bib28)\)is avoided because it pays one model call per expansion, which is infeasible at the branching factor of an ARC\-AGI\-3 level, and the heuristic\-once\-per\-domain posture\(Corrêaet al\.,[2025](https://arxiv.org/html/2607.01531#bib.bib27)\)motivates keeping search at full speed\.

### B\.5Why the model must be inferred and not read

The agents run filesystem\-confined so that the ground\-truth game source is unreadable, which forces the model to be inferred from observation\. A model\-free agent over a monotonic log is the read\-grep\-shell pattern this design borrows for the action role\(Foxet al\.,[2026](https://arxiv.org/html/2607.01531#bib.bib5)\), and the synthesized artifact restores the executable model and the exact replay check that such an agent forgoes\. Classical action\-model learning shares the planner\-in\-loop structure with symbolic synthesis in place of language\-model synthesis\(Lamanna and others,[2021](https://arxiv.org/html/2607.01531#bib.bib18)\)\.

## Appendix CIntegrity of the Evaluation

Because the agents run language models with shell access, we checked that no run read the ground\-truth game source, the web, or stored credentials\. The agents are filesystem\-confined so that the directories holding the environment source are never mounted to them\. A full audit of the action and synthesis transcripts across four sweeps found no successful access to game source and no network use\. Several explicit attempts to locate the source were present in the transcripts, and the filesystem confinement blocked all of them, returning only the run directory\. The synthesized models open only their own buffered state files\. One failure mode the audit surfaced is a goal predicate that recognizes a level advance by reading a cached entry state of the next level, which passes the replay test trivially\. A static check on the predicate body rejects that pattern, as recorded in Appendix[A\.3](https://arxiv.org/html/2607.01531#A1.SS3)\. The structured\-state results reported here therefore reflect dynamics inferred from interaction rather than copied from the environment\.

## Appendix DAdditional Result Visualizations

![Refer to caption](https://arxiv.org/html/2607.01531v1/images/coverage.png)Figure 9:Games won out of 25 by each system\. WorldCoder and the neural latent world models clear no game\. Vision pre\-trains on the evaluation set\.![Refer to caption](https://arxiv.org/html/2607.01531v1/images/hard_six.png)Figure 10:The six games baseline1 fails and OPINE\-World wins\. baseline1 \(hatched\) exhausts its budget on each, while OPINE\-World wins in far fewer actions\. The human reference is grey\.![Refer to caption](https://arxiv.org/html/2607.01531v1/images/scatter_actions.png)Figure 11:Per\-game actions, baseline1 against OPINE\-World, on log axes with the parity diagonal\. Points below the diagonal are games OPINE\-World clears in fewer actions\. The six games baseline1 fails \(orange\) sit far below, since OPINE\-World wins them at a fraction of the budget baseline1 spends before failing\.![Refer to caption](https://arxiv.org/html/2607.01531v1/images/levels_heatmap.png)Figure 12:Levels cleared as a fraction of each game’s total, per system\. OPINE\-World clears the full set on 19 games\.
## Appendix EPer\-Level Action Counts

TableLABEL:tab:per\-levelreports the actions each system spent on every level of every game, the per\-level data behind the figures in Section[4](https://arxiv.org/html/2607.01531#S4)\.

Table 4:Actions spent on each level, per game and system: the human reference, baseline1, and D3M\. Columns L1 to L10 index the levels in order\. A blank cell means the game has no such level, or, for baseline1, a level never reached after its budget was spent on an earlier one\. D3M per\-level counts are transitions recorded per level and may differ from the game total in Table[3](https://arxiv.org/html/2607.01531#S4.T3)by one initialisation step\. The D3M s5i5 run recorded only a game total\.GameSystemL1L2L3L4L5L6L7L8L9L10tu93Human19163442123801423111baseline1181619182928142129D3M223219373136142555sb26Human1828181931235818baseline11315151517193917D3M1347311517393517lp85Human17383116416026159baseline1141381913102057D3M81217161120817ar25Human325075378915923373baseline11714412230563747D3M17167524345511247tr87Human5458404571146baseline147190473647173D3M332926272274r11lHuman223351265249baseline152220139077D3M71335162333ft09Human431223286537baseline14714862323D3M6714165513cd82Human55841212323baseline116665141316D3M74620232017cn04Human295485300208113baseline114199222912460D3M325032394961su15Human224226115363184041baseline11889191475865419D3M23371699241875951re86Human264286108189139424241baseline12338124641505000D3M213652657170214321tn36Human32722640305562baseline111221413726410121598D3M132391111366263vc33Human718446113134152baseline1310541529000D3M101128175913675m0r0Human3011120326500237baseline12053479134081519D3M193576165657sc25Human366328314350baseline120563101900D3M325363250101sp80Human39582514896152baseline167800000D3M1030564385145wa30Human71119183983686879442415baseline1429121181972334842139205D3M4916980712455453440304g50tHuman78175179230965467baseline1581388599554871D3M907875134158122100ls20Human22123738496192186baseline122977410176216128D3M177510393129359183ka59Human28109515133132326baseline111266356193552180D3M45766616527104593dc22Human591026798324578baseline166469211415240D3M1327453120125758sk48Human6117710110323018112592baseline1144118355876168100D3M181166423666lf52Human32816071205148244109164225baseline182098158911512048000D3M11267104118bp35Human21484438338786131163baseline1201835336500000D3M19343141s5i5Human208910654162388683baseline117094839615140000D3M237412540376
## Appendix FObject\-Lifted MDP Formalization

The sections that follow give the detailed formal construction that Section[3](https://arxiv.org/html/2607.01531#S3)refers to, in self\-contained notation\. The account covers the object\-structured state, before\-and\-after pairing, effect signatures, the local effect table and its Dirichlet diagnostics, the role posterior, the ontology error, the synthesized model, and the planner\. It is the formalization the system was built from\. The notation here is local to the appendix and is introduced again where it is used\.

## Appendix GThe MDP Backbone

At the outermost level, ARC\-AGI\-3 is an interaction problem\. The agent observes a state, chooses an executable input action, the environment changes, and the agent receives a sparse success signal\. We model this as an MDP

M=\(𝒮,𝒜,T,R\)\.M=\(\\mathcal\{S\},\\mathcal\{A\},T,R\)\.The level has a particular initial state

s0∈𝒮,s\_\{0\}\\in\\mathcal\{S\},buts0s\_\{0\}is not included in the MDP tuple because it is one particular state, not the entire state space\.

The primitive actions are known as executable input bindings:

𝒜=\{up,down,left,right,space,click​\(x,y\),…\}\.\\mathcal\{A\}=\\\{\\text\{up\},\\text\{down\},\\text\{left\},\\text\{right\},\\text\{space\},\\text\{click\}\(x,y\),\\ldots\\\}\.The agent can execute these actions, but initially does not know their transition semantics\.

We treat ARC\-AGI\-3 dynamics as deterministic:

st\+1=T​\(st,at\)\.s\_\{t\+1\}=T\(s\_\{t\},a\_\{t\}\)\.A stochastic version can be obtained by replacingTTwith a transition kernelP​\(s′∣s,a\)P\(s^\{\\prime\}\\mid s,a\)\.

The reward is sparse:

R​\(st,at,st\+1\)=\{1,if the environment reports success,0,otherwise\.R\(s\_\{t\},a\_\{t\},s\_\{t\+1\}\)=\\begin\{cases\}1,&\\text\{if the environment reports success,\}\\\\ 0,&\\text\{otherwise\.\}\\end\{cases\}

## Appendix HObject\-Oriented State Space

### H\.1Why object states?

An ARC\-3 grid is not just an unstructured array\. A typical level naturally contains persistent entities: a player sprite, a key sprite, wall sprites, doors, hazards, buttons, and so on\. These entities have attributes such as position, visibility, rotation, and pixels\. This motivates an object\-oriented state representation\.

![Refer to caption](https://arxiv.org/html/2607.01531v1/images/img1.png)Figure 13:An ARC\-3 state decomposed into object records\. The grid naturally contains sprites, and each sprite becomes an objectoti=\(κti,τti,𝐯ti\)o\_\{t\}^\{i\}=\(\\kappa\_\{t\}^\{i\},\\tau\_\{t\}^\{i\},\\mathbf\{v\}\_\{t\}^\{i\}\)with a matching key, a type, and an attribute valuation such as position and color\. Objects that share a type share one transition rulefτf\_\{\\tau\}, here supplied by the game engine\.Thus the structured state at timettis written as a finite collection of objects:

st=\{oti:i∈It\}\.s\_\{t\}=\\\{o\_\{t\}^\{i\}:i\\in I\_\{t\}\\\}\.Each object is written as

oti=\(κti,τti,𝐯ti\)\.o\_\{t\}^\{i\}=\(\\kappa\_\{t\}^\{i\},\\tau\_\{t\}^\{i\},\\mathbf\{v\}\_\{t\}^\{i\}\)\.
Here𝐯ti\\mathbf\{v\}\_\{t\}^\{i\}is the object’s finite attribute valuation\. We use𝐯\\mathbf\{v\}, rather than𝐩\\mathbf\{p\}, to avoid conflict with probability notation later\.

For an object of typeτti\\tau\_\{t\}^\{i\}, the attribute valuation is a finite map

𝐯ti:𝖠𝗍𝗍𝗋​\(τti\)→𝖵𝖺𝗅\.\\mathbf\{v\}\_\{t\}^\{i\}:\\mathsf\{Attr\}\(\\tau\_\{t\}^\{i\}\)\\to\\mathsf\{Val\}\.For an attributeb∈𝖠𝗍𝗍𝗋​\(τti\)b\\in\\mathsf\{Attr\}\(\\tau\_\{t\}^\{i\}\), we write

𝐯ti​\(b\)\\mathbf\{v\}\_\{t\}^\{i\}\(b\)for its value\. For ARC\-3 sprites, attributes may include

𝖠𝗍𝗍𝗋​\(τ\)=\{x,y,visible,rotation,pixels,…\}\.\\mathsf\{Attr\}\(\\tau\)=\\\{\\text\{x\},\\text\{y\},\\text\{visible\},\\text\{rotation\},\\text\{pixels\},\\ldots\\\}\.

### H\.2Abstraction from raw observation to object state

The raw observation is denotedxtx\_\{t\}\. In the sprite\-based setting,xtx\_\{t\}may already contain engine sprite records\. In the spriteless setting,xtx\_\{t\}may be only the grid or rendered frame\. In both cases, the learner operates on an object\-structured state, so we define an abstraction map

α:xt↦st∈𝒮\.\\alpha:x\_\{t\}\\mapsto s\_\{t\}\\in\\mathcal\{S\}\.
Importantly,α\\alphais responsible not only for decomposing the observation into objects, but also for assigning the object\-level fields used later by the learner: the matching cueκti\\kappa\_\{t\}^\{i\}, the type/tagτti\\tau\_\{t\}^\{i\}, and the attribute valuation𝐯ti\\mathbf\{v\}\_\{t\}^\{i\}\.

#### Given sprite abstraction\.

In the current sprite\-based implementation, object records are not inferred by the LLM\. They come directly from the ARC engine\. The adapter reads the current level’s sprite list and stores fields such asname,tags,x,y,visible,rotation, andpixels\. Thus

αsprite​\(xt\)=st\\alpha\_\{\\rm sprite\}\(x\_\{t\}\)=s\_\{t\}is a deterministic engine\-provided abstraction\. In this regime, the object decomposition, object type/tag, and name\-based matching cue are all provided by the engine representation\.

A sprite record may look like:

\{

"name":"spr\_17",

"tags":\["blue\_player"\],

"x":2,

"y":2,

"visible":true,

"rotation":0,

"pixels":\[\[1,1\],\[1,1\]\]

\}

This record corresponds to

oti=\(κti,τti,𝐯ti\),o\_\{t\}^\{i\}=\(\\kappa\_\{t\}^\{i\},\\tau\_\{t\}^\{i\},\\mathbf\{v\}\_\{t\}^\{i\}\),where

κti=spr\_17,τti=blue\_player,\\kappa\_\{t\}^\{i\}=\\text\{spr\\\_17\},\\qquad\\tau\_\{t\}^\{i\}=\\text\{blue\\\_player\},and

𝐯ti=\{x↦2,y↦2,visible↦true,rotation↦0,pixels↦\[1111\]\}\.\\mathbf\{v\}\_\{t\}^\{i\}=\\\{\\text\{x\}\\mapsto 2,\\;\\text\{y\}\\mapsto 2,\\;\\text\{visible\}\\mapsto\\text\{true\},\\;\\text\{rotation\}\\mapsto 0,\\;\\text\{pixels\}\\mapsto\\begin\{bmatrix\}1&1\\\\ 1&1\\end\{bmatrix\}\\\}\.Equivalently, after fixing an attribute order, this valuation can be written as a vector:

𝐯ti=\(2,2,true,0,\[1111\]\)\.\\mathbf\{v\}\_\{t\}^\{i\}=\\left\(2,\\;2,\\;\\text\{true\},\\;0,\\;\\begin\{bmatrix\}1&1\\\\ 1&1\\end\{bmatrix\}\\right\)\.The dictionary view is usually clearer because ARC\-3 attributes are heterogeneous\.

#### Inferred object abstraction\.

In the spriteless regime, the LLM must infer an object decomposition from the raw grid or frame\. The output ofαLLM\\alpha\_\{\\rm LLM\}is still required to be a structured state

st=\{oti:i∈It\},oti=\(κti,τti,𝐯ti\)\.s\_\{t\}=\\\{o\_\{t\}^\{i\}:i\\in I\_\{t\}\\\},\\qquad o\_\{t\}^\{i\}=\(\\kappa\_\{t\}^\{i\},\\tau\_\{t\}^\{i\},\\mathbf\{v\}\_\{t\}^\{i\}\)\.In this case, the construction of object keys is part of the inferred abstraction problem\. In the current implementation,αLLM\\alpha\_\{\\rm LLM\}is a synthesizedextract\_objects\(frame\)function\. Each object it returns carries its ownname,id, orkeyfield, used as the keyκti\\kappa\_\{t\}^\{i\}; when none is present, the key defaults totype​\_​index\\text\{type\}\\\_\\text\{index\}, the object type joined with its enumeration index in the frame\. The typeτti\\tau\_\{t\}^\{i\}is read from the object’stype,kind,role, ortagfield, defaulting tovisual\_object\. Once these keys are assigned, the pairing policy reuses the sprite pairing heuristic below\.

### H\.3Pairing objects across time

In Figure[13](https://arxiv.org/html/2607.01531#A8.F13), imagine the player moves one cell to the right\. The object in the next frame should still be recognized as the same player\. Before we can say what changed, we need to pair before\-objects insts\_\{t\}with after\-objects inst\+1s\_\{t\+1\}\.

We write the before/after matching as a partial map

μt:It→It\+1∪\{⊥\}\.\\mu\_\{t\}:I\_\{t\}\\to I\_\{t\+1\}\\cup\\\{\\bot\\\}\.Ifμt​\(i\)=i′\\mu\_\{t\}\(i\)=i^\{\\prime\}, then before\-objectotio\_\{t\}^\{i\}is paired with after\-objectot\+1i′o\_\{t\+1\}^\{i^\{\\prime\}\}\. Ifμt​\(i\)=⊥\\mu\_\{t\}\(i\)=\\bot, then objectiiis treated as disappeared\.

In the current sprite\-based implementation, pairing is not inferred by the LLM\. It is a deterministic two\-pass heuristic over engine\-provided sprite records:

1. 1\.Group after\-objects by theirname\.
2. 2\.First pair unused before/after objects with exact \(name,x,y\)\(\\text\{name\},x,y\)matches\.
3. 3\.For each remaining unmatched before\-object, greedily pair it with the first unused after\-object with the samename, even if its position has changed\.
4. 4\.If no unused after\-object with the samenameremains, pair the before\-object with⊥\\bot\. This produces the effect signature

This handles simple movement: an object that changes position fails the exact\(name,x,y\)\(\\text\{name\},x,y\)pass but can still be matched by the same\-namepass\. However, the method is still heuristic\. Sprite names are not guaranteed to be unique, and if multiple same\-name sprites move simultaneously, greedy same\-name matching can be ambiguous\.

After\-only objects are not currently emitted as separate effect\-count samples by this pairing loop\. They still appear in the full after\-statest\+1s\_\{t\+1\}, so they are visible to the transition verifier\. If the Bayesian effect table is extended to count births, the natural signature is

counted under the after\-object’s type with context evaluated at the spawn location\.

#### Spriteless pairing\.

In the spriteless setting, the synthesized abstractionαLLM\\alpha\_\{\\rm LLM\}first extracts object records from each frame, and then the same two\-pass heuristic above is run over those records\. The only difference is the source of the keyκti\\kappa\_\{t\}^\{i\}\. In sprite mode it is the engine spritename; in spriteless mode it is thename,id, orkeyfield emitted byextract\_objects, or atype​\_​index\\text\{type\}\\\_\\text\{index\}fallback when the synthesizer emits none\. The exact\-position pass still pairs stationary objects, and the same\-name pass still recovers moved objects, provided the synthesized extractor assigns keys consistently across frames\. The same effect\-count tables and ontology\-error diagnostics are then produced in both modes\.

## Appendix IEffect Signatures

In the grid example, once the player before the action is paired with the player after the action, we can ask what changed\. If the player moved right, thexattribute changed\. If the player also changed appearance, then bothxandpixelschanged\. This motivates a coarse object\-level effect signature \(Figure[2](https://arxiv.org/html/2607.01531#S3.F2)\)\.

For a persistent objectii, whereμt​\(i\)≠⊥\\mu\_\{t\}\(i\)\\neq\\bot, define the changed\-attribute set

Δti=\{b∈𝖠𝗍𝗍𝗋​\(τti\):𝐯t\+1μt​\(i\)​\(b\)≠𝐯ti​\(b\)\}\.\\Delta\_\{t\}^\{i\}=\\left\\\{b\\in\\mathsf\{Attr\}\(\\tau\_\{t\}^\{i\}\):\\mathbf\{v\}\_\{t\+1\}^\{\\mu\_\{t\}\(i\)\}\(b\)\\neq\\mathbf\{v\}\_\{t\}^\{i\}\(b\)\\right\\\}\.The effect signature is a finite categorical symbol:

eti=ρ​\(Δti\)∈ℰ\.e\_\{t\}^\{i\}=\\rho\(\\Delta\_\{t\}^\{i\}\)\\in\\mathcal\{E\}\.
If an object disappears, so thatμt​\(i\)=⊥\\mu\_\{t\}\(i\)=\\bot, the current implementation assigns

eti=gone\.e\_\{t\}^\{i\}=\\text\{gone\}\.After\-only objects appear inst\+1s\_\{t\+1\}and are checked by the full transition verifier, but the current Bayesian effect\-count loop does not emit them as separate birth samples\. A natural extension is to add

etb=borne\_\{t\}^\{b\}=\\text\{born\}for after\-only objectsbb, counted under the after\-object’s type\.

The signature records which attributes changed, not the exact new values\. For example,xmeans thexx\-attribute changed, but does not say whether the object moved left or right\. The exact next\-state values remain in the full transition pair\(st,at,st\+1\)\(s\_\{t\},a\_\{t\},s\_\{t\+1\}\)\.

#### Example\.

If the player moves right and changes its pixels, then

Δti=\{x,pixels\},\\Delta\_\{t\}^\{i\}=\\\{\\text\{x\},\\text\{pixels\}\\\},and the implementation may represent the signature as

eti=pixels,x\.e\_\{t\}^\{i\}=\\text\{pixels,x\}\.This whole string is one categorical outcome\. It is not treated as two independent Bernoulli observations\.

## Appendix JObject\-Lifted Transition Rules

In the grid example, the player should not need a separate transition rule at every absolute board location\. Instead, the same type\-level rule can apply to all player instances, with behavior depending on local context\. For example, the actionrightmay move the player if the right cell is empty, and may do nothing if a wall is on the right\.

The environment transition is still the ordinary MDP transition:

T:𝒮×𝒜→𝒮\.T:\\mathcal\{S\}\\times\\mathcal\{A\}\\to\\mathcal\{S\}\.For a persistent objectii, whereμt​\(i\)≠⊥\\mu\_\{t\}\(i\)\\neq\\bot, the lifted hypothesis class shares transition rules by type:

ot\+1μt​\(i\)≈fτti​\(oti,at,uti\)\.o\_\{t\+1\}^\{\\mu\_\{t\}\(i\)\}\\approx f\_\{\\tau\_\{t\}^\{i\}\}\(o\_\{t\}^\{i\},a\_\{t\},u\_\{t\}^\{i\}\)\.Here

uti=ψ​\(oti,st\)u\_\{t\}^\{i\}=\\psi\(o\_\{t\}^\{i\},s\_\{t\}\)is the local context or state\-feature value used by the object rule\.

The word “lifted” means thatfτf\_\{\\tau\}is a type\-level rule, not a separate rule for every object instance\. Grounding the rule on objectotio\_\{t\}^\{i\}gives the object\-level prediction\.

#### Example\.

For a player object withτti=blue\_player\\tau\_\{t\}^\{i\}=\\text\{blue\\\_player\}, actionright, and local contextuti=empty\_rightu\_\{t\}^\{i\}=\\text\{empty\\\_right\}, the rule may predict thatxchanges\. If insteaduti=wall\_rightu\_\{t\}^\{i\}=\\text\{wall\\\_right\}, the same action may produce no change\. Thus the contextutiu\_\{t\}^\{i\}explains why the same type and action can have different effects in different local states\.

## Appendix KThe Local Effect Table

The grid example also motivates a count table\. If the player takesrightseveral times, we can count what effect was observed under each local context\. This table is not the executable transition modelTT, and it is not the synthesized modelT^\\widehat\{T\}\. It is a local diagnostic summarizing how objects of a given type change under an action and context\.

A table row is indexed by

whereτ\\tauis an object type,aais an action, anduuis a local context feature\. The columns are effect signaturese∈ℰe\\in\\mathcal\{E\}\. Thus the table has entries

Ct​\(j,e\)∈ℕ\.C\_\{t\}\(j,e\)\\in\\mathbb\{N\}\.
For each real transition\(st,at,rt,st\+1\)\(s\_\{t\},a\_\{t\},r\_\{t\},s\_\{t\+1\}\), and for each paired objectii, the system computes

uti=ψ​\(oti,st\),u\_\{t\}^\{i\}=\\psi\(o\_\{t\}^\{i\},s\_\{t\}\),then assigns the object transition to the row

jti=\(τti,at,uti\)\.j\_\{t\}^\{i\}=\(\\tau\_\{t\}^\{i\},a\_\{t\},u\_\{t\}^\{i\}\)\.It also computes the observed effect signature

eti=ρ​\(Δti\)\.e\_\{t\}^\{i\}=\\rho\(\\Delta\_\{t\}^\{i\}\)\.The table update is

Ct\+1​\(j,e\)=Ct​\(j,e\)\+∑i∈It𝟏​\{jti=j,eti=e\}\.C\_\{t\+1\}\(j,e\)=C\_\{t\}\(j,e\)\+\\sum\_\{i\\in I\_\{t\}\}\\mathbf\{1\}\\\{j\_\{t\}^\{i\}=j,\\;e\_\{t\}^\{i\}=e\\\}\.Equivalently, each object\-level observation increments the cell

Ct​\(jti,eti\)\.C\_\{t\}\(j\_\{t\}^\{i\},e\_\{t\}^\{i\}\)\.
#### Example before context refinement\.

The effect signatureetie\_\{t\}^\{i\}is computed after the transition fromΔti\\Delta\_\{t\}^\{i\}\. The context featureutiu\_\{t\}^\{i\}is computed before the transition fromsts\_\{t\}and is used to define or split the row\.

Suppose we initially use a coarse context valueu0u\_\{0\}for the player under actionright\. The local effect table may contain the following row\.

This row is mixed: sometimes the player changesxx, and sometimes nothing changes\. In a deterministic MDP, this usually means that the context feature is too coarse\. For example, the missing feature may be the object type at offset\(1,0\)\(1,0\), i\.e\., whether there is empty space or a wall to the right\.

After adding this feature, the same observations are split into two refined rows:

Both refined rows are concentrated\. This is what context refinement checks: not whether a new effect exists, but whether a before\-state feature explains why different effects were observed under the same type and action\.

#### Example with a joint effect signature\.

The table can also contain joint changed\-attribute signatures\. For instance, if a key\-like sprite sometimes moves and changes pixels, its row may contain a nonzero count in thepixels,xcolumn:

The entrypixels,xis one categorical outcome\. It is not treated as an independentpixelschange plus an independentxchange\.

## Appendix LDirichlet Row Diagnostics

The local effect table gives counts\. To measure whether a row is stable, we place a categorical distribution over effect signatures on each row:

qj​\(e\)=Pr⁡\(eti=e∣jti=j\)\.q\_\{j\}\(e\)=\\Pr\(e\_\{t\}^\{i\}=e\\mid j\_\{t\}^\{i\}=j\)\.Let

ℰ=\{e1,…,em\}\.\\mathcal\{E\}=\\\{e\_\{1\},\\ldots,e\_\{m\}\\\}\.The count vector for rowjjis

𝐜j=\(Ct​\(j,e1\),…,Ct​\(j,em\)\)\.\\mathbf\{c\}\_\{j\}=\(C\_\{t\}\(j,e\_\{1\}\),\\ldots,C\_\{t\}\(j,e\_\{m\}\)\)\.We place a symmetric Dirichlet prior:

qj∼Dir⁡\(α0​𝟏m\)\.q\_\{j\}\\sim\\operatorname\{Dir\}\(\\alpha\_\{0\}\\mathbf\{1\}\_\{m\}\)\.By conjugacy,

qj∣𝒟t∼Dir⁡\(α0​𝟏m\+𝐜j\)\.q\_\{j\}\\mid\\mathcal\{D\}\_\{t\}\\sim\\operatorname\{Dir\}\(\\alpha\_\{0\}\\mathbf\{1\}\_\{m\}\+\\mathbf\{c\}\_\{j\}\)\.The posterior mean is

q^j​\(e\)=α0\+Ct​\(j,e\)m​α0\+∑e′Ct​\(j,e′\)\.\\widehat\{q\}\_\{j\}\(e\)=\\frac\{\\alpha\_\{0\}\+C\_\{t\}\(j,e\)\}\{m\\alpha\_\{0\}\+\\sum\_\{e^\{\\prime\}\}C\_\{t\}\(j,e^\{\\prime\}\)\}\.
The Dirichlet posterior is not the world model\. It only summarizes which attributes changed\. The executable transition model is still

T^:𝒮×𝒜→𝒮,\\widehat\{T\}:\\mathcal\{S\}\\times\\mathcal\{A\}\\to\\mathcal\{S\},and it must satisfy full next\-state replay constraints:

T^​\(st,at\)=st\+1\.\\widehat\{T\}\(s\_\{t\},a\_\{t\}\)=s\_\{t\+1\}\.

## Appendix MRow Concentration and Context Refinement

The mixed player row from Section[K](https://arxiv.org/html/2607.01531#A11)illustrates the purpose of row concentration\. If the row contains bothxandno\_change, then the current type\-action\-context condition is not specific enough\. A refined context feature, such as whether the right neighbor is empty or a wall, can split the row into deterministic subrows \(Figure[3](https://arxiv.org/html/2607.01531#S3.F3)\)\.

The number of samples assigned to rowjjis

nj=∑e∈ℰCt​\(j,e\),n\_\{j\}=\\sum\_\{e\\in\\mathcal\{E\}\}C\_\{t\}\(j,e\),and the modal count is

sj=maxe∈ℰ⁡Ct​\(j,e\)\.s\_\{j\}=\\max\_\{e\\in\\mathcal\{E\}\}C\_\{t\}\(j,e\)\.The modal fraction is

modal​\_​frac​\(j\)=sjnj,nj\>0\.\\mathrm\{modal\\\_frac\}\(j\)=\\frac\{s\_\{j\}\}\{n\_\{j\}\},\\qquad n\_\{j\}\>0\.
A row is operationally identified when

nj≥nmin,modal​\_​frac​\(j\)≥mmin\.n\_\{j\}\\geq n\_\{\\min\},\\qquad\\mathrm\{modal\\\_frac\}\(j\)\\geq m\_\{\\min\}\.The full row uncertainty can also be measured by normalized entropy:

Ujrow=𝖧​\(q^j\)log⁡m∈\[0,1\]\.U\_\{j\}^\{\\rm row\}=\\frac\{\\mathsf\{H\}\(\\widehat\{q\}\_\{j\}\)\}\{\\log m\}\\in\[0,1\]\.
Returning to the example in Section[K](https://arxiv.org/html/2607.01531#A11), the coarse row

j=\(blue\_player,right,u0\)j=\(\\text\{blue\\\_player\},\\text\{right\},u\_\{0\}\)has

Ct​\(j,x\)=3,Ct​\(j,no\_change\)=2\.C\_\{t\}\(j,\\text\{x\}\)=3,\\qquad C\_\{t\}\(j,\\text\{no\\\_change\}\)=2\.Thus

nj=5,sj=3,modal​\_​frac​\(j\)=35\.n\_\{j\}=5,\\qquad s\_\{j\}=3,\\qquad\\mathrm\{modal\\\_frac\}\(j\)=\\frac\{3\}\{5\}\.This row is not concentrated\. After adding the right\-neighbor context feature, the refined row

j1=\(blue\_player,right,empty\_right\)j\_\{1\}=\(\\text\{blue\\\_player\},\\text\{right\},\\text\{empty\\\_right\}\)has

Ct​\(j1,x\)=3,nj1=3,modal​\_​frac​\(j1\)=1,C\_\{t\}\(j\_\{1\},\\text\{x\}\)=3,\\qquad n\_\{j\_\{1\}\}=3,\\qquad\\mathrm\{modal\\\_frac\}\(j\_\{1\}\)=1,and the refined row

j2=\(blue\_player,right,wall\_right\)j\_\{2\}=\(\\text\{blue\\\_player\},\\text\{right\},\\text\{wall\\\_right\}\)has

Ct​\(j2,no\_change\)=2,nj2=2,modal​\_​frac​\(j2\)=1\.C\_\{t\}\(j\_\{2\},\\text\{no\\\_change\}\)=2,\\qquad n\_\{j\_\{2\}\}=2,\\qquad\\mathrm\{modal\\\_frac\}\(j\_\{2\}\)=1\.The split is useful because a before\-state context feature explains the mixed effects in the original row\. In the implementation, minimum\-count thresholds may still require additional samples before a refined row is accepted as identified\.

## Appendix NTyping Uncertainty

In the running grid,blue\_playerandred\_keyare useful initial types\. But in the general case, the observed tag may not be the true mechanical role, or the object decomposition may be inferred from pixels\. The learner therefore maintains uncertainty over candidate mechanical types:

τti∈𝒞=\{1,…,K\}\.\\tau\_\{t\}^\{i\}\\in\\mathcal\{C\}=\\\{1,\\ldots,K\\\}\.
The analyzer or LLM supplies a prior:

πi​\(τ\)=Pr⁡\(τti=τ\)\.\\pi\_\{i\}\(\\tau\)=\\Pr\(\\tau\_\{t\}^\{i\}=\\tau\)\.For a candidate typeτ\\tau, objectii’s row at timettwould be

jti​\(τ\)=\(τ,at,uti\)\.j\_\{t\}^\{i\}\(\\tau\)=\(\\tau,a\_\{t\},u\_\{t\}^\{i\}\)\.The plug\-in likelihood of the observed effect is

ℓti​\(τ\)=q^jti​\(τ\)​\(eti\)\.\\ell\_\{t\}^\{i\}\(\\tau\)=\\widehat\{q\}\_\{j\_\{t\}^\{i\}\(\\tau\)\}\(e\_\{t\}^\{i\}\)\.Thus an approximate type posterior is

Pr⁡\(τti=τ∣𝒟t\)∝πi​\(τ\)​∏ℓ<t:i∈Iℓℓℓi​\(τ\)\.\\Pr\(\\tau\_\{t\}^\{i\}=\\tau\\mid\\mathcal\{D\}\_\{t\}\)\\propto\\pi\_\{i\}\(\\tau\)\\prod\_\{\\ell<t:\\,i\\in I\_\{\\ell\}\}\\ell\_\{\\ell\}^\{i\}\(\\tau\)\.
In the current ARC\-3 implementation, this posterior may be shared per sprite tag rather than stored independently for every object instance\. That is, all objects with the same engine tag share the same type posterior\.

The normalized typing uncertainty is

Uitype=𝖧​\[Pr⁡\(τti∣𝒟t\)\]log⁡K∈\[0,1\]\.U\_\{i\}^\{\\rm type\}=\\frac\{\\mathsf\{H\}\[\\Pr\(\\tau\_\{t\}^\{i\}\\mid\\mathcal\{D\}\_\{t\}\)\]\}\{\\log K\}\\in\[0,1\]\.

## Appendix OOntology Error

The local table can be uncertain for two different reasons\. First, the system may be unsure which type governs an object\. Second, even after choosing a type, the row may still mix multiple effects\. Ontology error combines these two sources of uncertainty\.

For objectiiassigned to rowjtij\_\{t\}^\{i\}, define

ηti=1−\(1−Uitype\)​\(1−Ujtirow\)\.\\eta\_\{t\}^\{i\}=1\-\(1\-U\_\{i\}^\{\\rm type\}\)\(1\-U\_\{j\_\{t\}^\{i\}\}^\{\\rm row\}\)\.This noisy\-OR score is high if either the object type is uncertain or the row is uncertain\. It is low only when both are resolved\.

The aggregate ontology error is

ηt=1Nt​∑ℓ<t∑i∈Iℓηℓi,\\eta\_\{t\}=\\frac\{1\}\{N\_\{t\}\}\\sum\_\{\\ell<t\}\\sum\_\{i\\in I\_\{\\ell\}\}\\eta\_\{\\ell\}^\{i\},where

Nt=∑ℓ<t\|Iℓ\|\.N\_\{t\}=\\sum\_\{\\ell<t\}\|I\_\{\\ell\}\|\.
IfUitypeU\_\{i\}^\{\\rm type\}is high, the system is unsure which type/role should govern objectii\. IfUjtirowU\_\{j\_\{t\}^\{i\}\}^\{\\rm row\}is high, the row mixes several effect signatures, which often means that the local contextutiu\_\{t\}^\{i\}is missing a relevant feature or that the current type groups together heterogeneous objects\.

Ontology error does not say that all world\-model errors are typing or row errors\. The synthesized transition code can still be wrong in exact values, pairing can be wrong, the abstractionα\\alphacan miss objects, or the goal predicate can be wrong\. Those failures are caught by replay verification, not by ontology error alone\.

## Appendix PCrystallization

The previous sections define uncertainty signals\. Crystallization is the point where the system decides that the object ontology is stable enough to commit to a synthesized model\.

LetℐG\\mathcal\{I\}^\{G\}be the set of goal\-relevant objects, tags, or groups, estimated after the first successful trajectory\. An idealized crystallization time is

t⋆=min\{t:some reward​rℓ=1​has been observed for​ℓ<t,Uitype≤ϵtype​for all goal\-relevant objects​i,nj≥nmin​and​modal​\_​frac​\(j\)≥mmin​for all visited goal\-relevant rows​j\.\}\.t^\{\\star\}=\\min\\left\\\{t:\\begin\{array\}\[\]\{l\}\\text\{some reward \}r\_\{\\ell\}=1\\text\{ has been observed for \}\\ell<t,\\\\\[2\.84526pt\] U\_\{i\}^\{\\rm type\}\\leq\\epsilon\_\{\\rm type\}\\text\{ for all goal\-relevant objects \}i,\\\\\[2\.84526pt\] n\_\{j\}\\geq n\_\{\\min\}\\text\{ and \}\\mathrm\{modal\\\_frac\}\(j\)\\geq m\_\{\\min\}\\text\{ for all visited goal\-relevant rows \}j\.\\end\{array\}\\right\\\}\.Att⋆t^\{\\star\}, the system commits to the current ontology and asks the synthesizer for a stable executable model\.

In the current ARC\-3 implementation, the operational trigger is simpler: successful level completion plus confident non\-decorative aliases\. The stricter row gate is best viewed as a general design target\.

## Appendix QSynthesized World Model

After commitment, the goal is not merely to know that a row usually changesx\. The system must synthesize executable code that predicts the full next state\. Therefore the learned world model is

T^:𝒮×𝒜→𝒮,G^:𝒮×𝒜×𝒮→\{0,1\}\.\\widehat\{T\}:\\mathcal\{S\}\\times\\mathcal\{A\}\\to\\mathcal\{S\},\\qquad\\widehat\{G\}:\\mathcal\{S\}\\times\\mathcal\{A\}\\times\\mathcal\{S\}\\to\\\{0,1\\\}\.In code,T^\\widehat\{T\}is implemented astransition\_function\(state, action\_id\), andG^\\widehat\{G\}is implemented asreward\_function\(state, action\_id, new\_state\)\.

The transition verifier checks

ϕT\(T^,𝒟t\):∀\(sℓ,aℓ,rℓ,sℓ\+1\)∈𝒟t,T^\(sℓ,aℓ\)=sℓ\+1\.\\phi\_\{T\}\(\\widehat\{T\},\\mathcal\{D\}\_\{t\}\):\\qquad\\forall\(s\_\{\\ell\},a\_\{\\ell\},r\_\{\\ell\},s\_\{\\ell\+1\}\)\\in\\mathcal\{D\}\_\{t\},\\;\\widehat\{T\}\(s\_\{\\ell\},a\_\{\\ell\}\)=s\_\{\\ell\+1\}\.After positive rewards exist, the goal verifier checks

ϕG\(G^,𝒟t\):∀\(sℓ,aℓ,rℓ,sℓ\+1\)∈𝒟t,G^\(sℓ,aℓ,sℓ\+1\)=rℓ\.\\phi\_\{G\}\(\\widehat\{G\},\\mathcal\{D\}\_\{t\}\):\\qquad\\forall\(s\_\{\\ell\},a\_\{\\ell\},r\_\{\\ell\},s\_\{\\ell\+1\}\)\\in\\mathcal\{D\}\_\{t\},\\;\\widehat\{G\}\(s\_\{\\ell\},a\_\{\\ell\},s\_\{\\ell\+1\}\)=r\_\{\\ell\}\.
A failed prediction,

T^​\(st,at\)≠st\+1,\\widehat\{T\}\(s\_\{t\},a\_\{t\}\)\\neq s\_\{t\+1\},is a counterexample for the next synthesis round\. It is also a real transition and therefore updates the Bayesian diagnostics through the same effect\-count mechanism\.

## Appendix RPlanning and Model Use

If a reliable model has been synthesized, it can be used for model\-based planning\. A planner searches inside the learned model

M^=\(𝒮,𝒜,T^,G^\)\.\\widehat\{M\}=\(\\mathcal\{S\},\\mathcal\{A\},\\widehat\{T\},\\widehat\{G\}\)\.Given the current observed statests\_\{t\}, it returns a predicted trace

τ^=\(s^t,at,s^t\+1,at\+1,…,s^t\+L\),\\widehat\{\\tau\}=\(\\widehat\{s\}\_\{t\},a\_\{t\},\\widehat\{s\}\_\{t\+1\},a\_\{t\+1\},\\ldots,\\widehat\{s\}\_\{t\+L\}\),where

s^t=st,s^k\+1=T^​\(s^k,ak\)\.\\widehat\{s\}\_\{t\}=s\_\{t\},\\qquad\\widehat\{s\}\_\{k\+1\}=\\widehat\{T\}\(\\widehat\{s\}\_\{k\},a\_\{k\}\)\.The first action of the trace gives the planner\-induced policy:

πplan​\(st\)=at\.\\pi\_\{\\rm plan\}\(s\_\{t\}\)=a\_\{t\}\.
Execution still occurs in the real environment\. If the real next state differs from the predicted next state, the transition becomes a counterexample:

st\+1≠T^​\(st,at\)\.s\_\{t\+1\}\\neq\\widehat\{T\}\(s\_\{t\},a\_\{t\}\)\.Thus model use and real\-environment validation remain separate\.

The current codebase implements such a planner as a synthesized module\. The synthesizer authors aplannerhook that searches over its owntransition\_functionandreward\_functionfor an action sequence that reaches reward under the model\. The planner runs only after at least one level has been completed and after the synthesized model is replay\-consistent, and it is first verified by planning to reward from the entry states of the completed levels before it is trusted\. Each planned action is then executed one step at a time in the real environment and compared against the predicted next state\. A mismatch aborts the plan, appends the real transition to the replay buffer as a counterexample, and blocks the planner until the next synthesis round\. The planner is a tool the action agent may consult, while the action agent still selects every move and checks model consistency through the replay buffer\.

## Appendix SDivision of Labor: Actor, Synthesizer, and Critic

Control and model construction are separated across two LLM\-driven roles that share one symbolic world modelM^=\(𝒮,𝒜,T^,G^\)\\widehat\{M\}=\(\\mathcal\{S\},\\mathcal\{A\},\\widehat\{T\},\\widehat\{G\}\)\.

#### Actor\.

The actor selects the primitive actionata\_\{t\}executed in the real environment at each step and records every real transition\(st,at,rt,st\+1\)\(s\_\{t\},a\_\{t\},r\_\{t\},s\_\{t\+1\}\)in the replay buffer𝒟t\\mathcal\{D\}\_\{t\}\. When a trusted model exists it may follow the planner\-induced policyπplan\\pi\_\{\\rm plan\}\(Section[R](https://arxiv.org/html/2607.01531#A18)\), but it selects every move itself\.

#### Synthesizer\.

The synthesizer authors and repairsT^\\widehat\{T\},G^\\widehat\{G\}, and the planner\. Repair is counterexample\-guided: any transition in𝒟t\\mathcal\{D\}\_\{t\}on which the verifierϕT\\phi\_\{T\}orϕG\\phi\_\{G\}\(Section[Q](https://arxiv.org/html/2607.01531#A17)\) fails is a counterexample for the next synthesis round\.

#### Critic\.

A critic call audits the current model against the buffer and the diagnostics of Section[O](https://arxiv.org/html/2607.01531#A15), surfacing the counterexamples and the high\-η\\etaobjects and contexts that the next synthesis round should address\. The critic scores the*consistency*of the model with observed transitions, not the*value*of the actor’s actions\.

#### Relation to actor–critic\.

This is a model\-based variant of the actor–critic pattern from reinforcement learning, with one substitution worth stating precisely\. In standard actor–critic the critic is a learned*value*function that supplies the actor a low\-variance reward signal\. Here the only extrinsic reward is the sparse environment successRR\(reaching the goal\); the critic instead scores the replay\-consistency of a symbolic world model\. Model consistency is thus an*instrumental*, intermediate objective: a model that replays the observed transitions is a prerequisite for planning to the sparse goal, so consistency acts as a dense signal in service of the terminal reward rather than an estimate of the policy’s value\. Operationally the synthesizer first seeks and resolves inconsistencies—treating each mispredicted transition as a causal hypothesis to test against𝒟t\\mathcal\{D\}\_\{t\}—and, once the model is replay\-consistent, plans toward reward; the trade\-off between further exploration and goal\-seeking is handled heuristically rather than by an explicit objective\. The artifact shared across all three roles isM^\\widehat\{M\}: the actor plans with it, the synthesizer repairs it, and the critic audits it, while the ontology errorηt\\eta\_\{t\}\(Section[O](https://arxiv.org/html/2607.01531#A15)\) is the compact signal that directs where each looks next\.

## Appendix TObject\-Based and Fluent\-Based Views

The main formalization is object\-based:

st=\{oti:i∈It\},oti=\(κti,τti,𝐯ti\)\.s\_\{t\}=\\\{o\_\{t\}^\{i\}:i\\in I\_\{t\}\\\},\\qquad o\_\{t\}^\{i\}=\(\\kappa\_\{t\}^\{i\},\\tau\_\{t\}^\{i\},\\mathbf\{v\}\_\{t\}^\{i\}\)\.This matches the implementation, which stores sprite/object records and computes object\-level effect signatures\.

The fluent\-based view is a field\-level expansion\. Ifbbis an attribute name in𝖠𝗍𝗍𝗋​\(τti\)\\mathsf\{Attr\}\(\\tau\_\{t\}^\{i\}\), then a grounded fluent is

meaning attributebbof objectii\. Its value is

st​\(y\)=𝐯ti​\(b\)\.s\_\{t\}\(y\)=\\mathbf\{v\}\_\{t\}^\{i\}\(b\)\.The object transition is equivalent to a bundle of fluent transitions:

𝐯ti​\(b\)↦𝐯t\+1μt​\(i\)​\(b\),b∈𝖠𝗍𝗍𝗋​\(τti\)\.\\mathbf\{v\}\_\{t\}^\{i\}\(b\)\\mapsto\\mathbf\{v\}\_\{t\+1\}^\{\\mu\_\{t\}\(i\)\}\(b\),\\qquad b\\in\\mathsf\{Attr\}\(\\tau\_\{t\}^\{i\}\)\.
We therefore use the object\-based view as the main formalization and the fluent\-based view only as an explanatory expansion\.

## Appendix USummary

The full flow is:

xt→𝛼st=\{oti:i∈It\},oti=\(κti,τti,𝐯ti\)\.x\_\{t\}\\xrightarrow\{\\alpha\}s\_\{t\}=\\\{o\_\{t\}^\{i\}:i\\in I\_\{t\}\\\},\\qquad o\_\{t\}^\{i\}=\(\\kappa\_\{t\}^\{i\},\\tau\_\{t\}^\{i\},\\mathbf\{v\}\_\{t\}^\{i\}\)\.The real environment transition gives

st\+1=T​\(st,at\)\.s\_\{t\+1\}=T\(s\_\{t\},a\_\{t\}\)\.The system pairs objects acrosssts\_\{t\}andst\+1s\_\{t\+1\}, computes effect signaturesetie\_\{t\}^\{i\}, assigns local effect\-table rows

jti=\(τti,at,uti\),j\_\{t\}^\{i\}=\(\\tau\_\{t\}^\{i\},a\_\{t\},u\_\{t\}^\{i\}\),updates Dirichlet row diagnostics, updates type uncertainty, and computes ontology error\. When the ontology is stable enough, the system commits and synthesizes an executable world modelT^\\widehat\{T\}\. The Bayesian diagnostic tracks row\-level abstraction adequacy; the executable model is checked by full next\-state equality\.

## References

- TheoryCoder: synthesizing world models for bilevel planning\.TMLR\.Cited by:[§B\.1](https://arxiv.org/html/2607.01531#A2.SS1.p1.1),[§B\.2](https://arxiv.org/html/2607.01531#A2.SS2.p1.1),[§1](https://arxiv.org/html/2607.01531#S1.p2.1),[§2](https://arxiv.org/html/2607.01531#S2.SS0.SSS0.Px4.p1.3),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px1.p1.1)\.
- D\. Aineto, S\. Jiménez, and E\. Onaindia \(2019\)Learning action models with minimal observability\.Artificial Intelligence275,pp\. 104–137\.Cited by:[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- ARC Prize Foundation \(2026\)ARC\-AGI\-3: a new challenge for frontier agentic intelligence\.External Links:2603\.24621,[Link](https://arxiv.org/abs/2603.24621)Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p4.1),[§4](https://arxiv.org/html/2607.01531#S4.SS0.SSS0.Px1.p1.1)\.
- J\. Austin, A\. Odena, M\. Nye, M\. Bosma, H\. Michalewski, D\. Dohan, E\. Jiang, C\. Cai, M\. Terry, Q\. Le, and C\. Sutton \(2021\)Program synthesis with large language models\.arXiv preprint arXiv:2108\.07732\.Cited by:[§3\.2](https://arxiv.org/html/2607.01531#S3.SS2.SSS0.Px2.p1.1)\.
- P\. W\. Battaglia, J\. B\. Hamrick, V\. Bapst, A\. Sanchez\-Gonzalez, V\. Zambaldi, M\. Malinowski, A\. Tacchetti, D\. Raposo, A\. Santoro,et al\.\(2018\)Relational inductive biases, deep learning, and graph networks\.arXiv preprint arXiv:1806\.01261\.Cited by:[§3\.1](https://arxiv.org/html/2607.01531#S3.SS1.SSS0.Px2.p1.1)\.
- A\. Blumer, A\. Ehrenfeucht, D\. Haussler, and M\. K\. Warmuth \(1987\)Occam’s razor\.Information Processing Letters24\(6\),pp\. 377–380\.Cited by:[§3\.3](https://arxiv.org/html/2607.01531#S3.SS3.SSS0.Px1.p1.2)\.
- R\. I\. Brafman and M\. Tennenholtz \(2002\)R\-MAX: a general polynomial time algorithm for near\-optimal reinforcement learning\.JMLR3,pp\. 213–231\.Cited by:[§2](https://arxiv.org/html/2607.01531#S2.SS0.SSS0.Px4.p1.3)\.
- M\. Chen, J\. Tworek, H\. Jun, Q\. Yuan, H\. P\. d\. O\. Pinto,et al\.\(2021\)Evaluating large language models trained on code\.arXiv preprint arXiv:2107\.03374\.Cited by:[§3\.2](https://arxiv.org/html/2607.01531#S3.SS2.SSS0.Px2.p1.1)\.
- X\. Chen, M\. Lin, N\. Schärli, and D\. Zhou \(2024\)Teaching large language models to self\-debug\.InICLR,Cited by:[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- F\. Chollet \(2019\)On the measure of intelligence\.arXiv preprint arXiv:1911\.01547\.Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p2.1)\.
- G\. F\. Cooper and E\. Herskovits \(1992\)A Bayesian method for the induction of probabilistic networks from data\.Machine Learning9,pp\. 309–347\.Cited by:[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- A\. B\. Corrêa, A\. Pollitt, A\. Jonsson, and J\. Seipp \(2025\)Classical planning with LLM\-generated heuristics\.External Links:2503\.18809Cited by:[§B\.4](https://arxiv.org/html/2607.01531#A2.SS4.p1.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- A\. Curtiset al\.\(2025\)LLM\-guided probabilistic program induction for POMDP model estimation\.InCoRL,Cited by:[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- C\. Diuk, A\. Cohen, and M\. L\. Littman \(2008\)An object\-oriented representation for efficient reinforcement learning\.InICML,Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p1.1),[§2](https://arxiv.org/html/2607.01531#S2.SS0.SSS0.Px1.p1.10),[§3\.1](https://arxiv.org/html/2607.01531#S3.SS1.SSS0.Px2.p1.1)\.
- S\. Džeroski, L\. De Raedt, and K\. Driessens \(2001\)Relational reinforcement learning\.Machine Learning43,pp\. 7–52\.Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p1.1),[§3\.1](https://arxiv.org/html/2607.01531#S3.SS1.SSS0.Px2.p1.1)\.
- K\. Elliset al\.\(2025\)PoE\-World: compositional world models via product of programmatic experts\.External Links:2505\.10819Cited by:[§B\.1](https://arxiv.org/html/2607.01531#A2.SS1.p1.1),[§1](https://arxiv.org/html/2607.01531#S1.p1.1),[§1](https://arxiv.org/html/2607.01531#S1.p2.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px1.p1.1)\.
- R\. E\. Fikes and N\. J\. Nilsson \(1971\)STRIPS: a new approach to the application of theorem proving to problem solving\.Artificial Intelligence2,pp\. 189–208\.Cited by:[§B\.2](https://arxiv.org/html/2607.01531#A2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2607.01531#S3.SS1.SSS0.Px1.p1.11)\.
- A\. Fox, J\. Wang, P\. Rosu, and B\. Dhingra \(2026\)Hill\-climbing ARC\-AGI\-3\.Cited by:[§B\.5](https://arxiv.org/html/2607.01531#A2.SS5.p1.1),[§1](https://arxiv.org/html/2607.01531#S1.p2.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px2.p1.1)\.
- C\. Guestrin, D\. Koller, R\. Parr, and S\. Venkataraman \(2003\)Efficient solution algorithms for factored MDPs\.JAIR19,pp\. 399–468\.Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p1.1)\.
- D\. Ha and J\. Schmidhuber \(2018\)Recurrent world models facilitate policy evolution\.InNeurIPS,Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p1.1)\.
- D\. Hafneret al\.\(2020\)Dream to control: learning behaviors by latent imagination\.InICLR,Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p1.1),[§4](https://arxiv.org/html/2607.01531#S4.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px3.p1.1)\.
- D\. Hafner, J\. Pasukonis, J\. Ba, and T\. Lillicrap \(2025\)Mastering diverse domains through world models\.Nature640\.Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p1.1),[§4](https://arxiv.org/html/2607.01531#S4.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px3.p1.1)\.
- S\. Hong, M\. Zhuge, J\. Chen, X\. Zheng, Y\. Cheng, C\. Zhang, J\. Wang, Z\. Wang, S\. K\. S\. Yau, Z\. Lin, L\. Zhou, C\. Ran, L\. Xiao, C\. Wu, and J\. Schmidhuber \(2024\)MetaGPT: meta programming for a multi\-agent collaborative framework\.InICLR,Cited by:[§3\.2](https://arxiv.org/html/2607.01531#S3.SS2.SSS0.Px1.p1.1)\.
- D\. Horgan, J\. Quan, D\. Budden, G\. Barth\-Maron, M\. Hessel, H\. van Hasselt, and D\. Silver \(2018\)Distributed prioritized experience replay\.InICLR,Cited by:[§3\.2](https://arxiv.org/html/2607.01531#S3.SS2.SSS0.Px1.p1.1)\.
- R\. Houthooft, X\. Chen, Y\. Duan, J\. Schulman, F\. De Turck, and P\. Abbeel \(2016\)VIME: variational information maximizing exploration\.InNeurIPS,Cited by:[§3\.4](https://arxiv.org/html/2607.01531#S3.SS4.SSS0.Px3.p1.3)\.
- S\. Jha, S\. Gulwani, S\. A\. Seshia, and A\. Tiwari \(2010\)Oracle\-guided component\-based program synthesis\.InICSE,Cited by:[§3\.3](https://arxiv.org/html/2607.01531#S3.SS3.SSS0.Px2.p1.1)\.
- L\. Kaiser, M\. Babaeizadeh, P\. Milos, B\. Osinski, R\. H\. Campbell, K\. Czechowski, D\. Erhan, C\. Finn, P\. Kozakowski, S\. Levine, A\. Mohiuddin, R\. Sepassi, G\. Tucker, and H\. Michalewski \(2020\)Model\-based reinforcement learning for Atari\.InICLR,Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p1.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px3.p1.1)\.
- S\. Kambhampati, K\. Valmeekam, L\. Guan, M\. Verma, K\. Stechly, S\. Bhambri, L\. P\. Saldyt, and A\. B\. Murthy \(2024\)Position: LLMs can’t plan, but can help planning in LLM\-modulo frameworks\.InICML,Cited by:[§3\.2](https://arxiv.org/html/2607.01531#S3.SS2.SSS0.Px2.p1.1)\.
- K\. Kansky, T\. Silver, D\. A\. Mély, M\. Eldawy, M\. Lázaro\-Gredilla, X\. Lou, N\. Dorfman, S\. Sidor, S\. Phoenix, and D\. George \(2017\)Schema networks: zero\-shot transfer with a generative causal model of intuitive physics\.InICML,Cited by:[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px3.p1.1)\.
- C\. Kemp, J\. B\. Tenenbaum, T\. L\. Griffiths, T\. Yamada, and N\. Ueda \(2006\)Learning systems of concepts with an infinite relational model\.InAAAI,Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p2.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- T\. Kipf, E\. van der Pol, and M\. Welling \(2020\)Contrastive learning of structured world models\.InICLR,Cited by:[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px3.p1.1)\.
- V\. R\. Konda and J\. N\. Tsitsiklis \(1999\)Actor\-critic algorithms\.InNeurIPS,Cited by:[§3\.2](https://arxiv.org/html/2607.01531#S3.SS2.SSS0.Px1.p1.1)\.
- L\. Lamannaet al\.\(2021\)Online learning of action models for PDDL planning\.InIJCAI,Cited by:[§B\.5](https://arxiv.org/html/2607.01531#A2.SS5.p1.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- G\. Levyet al\.\(2025\)WorldLLM: improving LLMs’ world modeling using curiosity\-driven theory\-making\.External Links:2506\.06725Cited by:[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- N\. Lipovetzky and H\. Geffner \(2012\)Width and serialization of classical planning problems\.InECAI,Cited by:[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- N\. Lipovetzky, M\. Ramirez, and H\. Geffner \(2015\)Classical planning with simulators: results on the Atari video games\.InIJCAI,Cited by:[§B\.2](https://arxiv.org/html/2607.01531#A2.SS2.p1.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- A\. Madaan, N\. Tandon, P\. Gupta, S\. Hallinan, L\. Gao, S\. Wiegreffe, U\. Alon, N\. Dziri, S\. Prabhumoye, Y\. Yang, S\. Gupta, B\. P\. Majumder, K\. Hermann, S\. Welleck, A\. Yazdanbakhsh, and P\. Clark \(2023\)Self\-refine: iterative refinement with self\-feedback\.InNeurIPS,Cited by:[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- D\. McDermott, M\. Ghallab, A\. Howe, C\. Knoblock, A\. Ram, M\. Veloso, D\. Weld, and D\. Wilkins \(1998\)PDDL — the planning domain definition language\.Technical reportTechnical ReportCVC TR\-98\-003,Yale Center for Computational Vision and Control\.Cited by:[§3\.1](https://arxiv.org/html/2607.01531#S3.SS1.SSS0.Px1.p1.11)\.
- T\. M\. Mitchell \(1982\)Generalization as search\.Artificial Intelligence18\(2\),pp\. 203–226\.Cited by:[§3\.3](https://arxiv.org/html/2607.01531#S3.SS3.SSS0.Px1.p1.2)\.
- D\. Pathak, P\. Agrawal, A\. A\. Efros, and T\. Darrell \(2017\)Curiosity\-driven exploration by self\-supervised prediction\.InICML,Cited by:[§3\.4](https://arxiv.org/html/2607.01531#S3.SS4.SSS0.Px3.p1.3)\.
- E\. P\. D\. Pednault \(1989\)ADL: exploring the middle ground between STRIPS and the situation calculus\.InKR,Cited by:[§3\.1](https://arxiv.org/html/2607.01531#S3.SS1.SSS0.Px1.p1.11)\.
- W\. T\. Piriyakulkij, C\. Langenfeld, T\. A\. Le, and K\. Ellis \(2024\)Doing experiments and revising rules with natural language and probabilistic reasoning\.InNeurIPS,Cited by:[§3\.2](https://arxiv.org/html/2607.01531#S3.SS2.SSS0.Px3.p1.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- C\. Qian, W\. Liu, H\. Liu, N\. Chen, Y\. Dang, J\. Li, C\. Yang, W\. Chen, Y\. Su, X\. Cong, J\. Xu, D\. Li, Z\. Liu, and M\. Sun \(2024\)ChatDev: communicative agents for software development\.InACL,Cited by:[§3\.2](https://arxiv.org/html/2607.01531#S3.SS2.SSS0.Px1.p1.1)\.
- J\. Schmidhuber \(2010\)Formal theory of creativity, fun, and intrinsic motivation \(1990–2010\)\.IEEE Transactions on Autonomous Mental Development2\(3\),pp\. 230–247\.Cited by:[§3\.4](https://arxiv.org/html/2607.01531#S3.SS4.SSS0.Px3.p1.3)\.
- J\. Schrittwieseret al\.\(2020\)Mastering Atari, Go, chess and shogi by planning with a learned model\.Nature588,pp\. 604–609\.Cited by:[§B\.2](https://arxiv.org/html/2607.01531#A2.SS2.p1.1),[§1](https://arxiv.org/html/2607.01531#S1.p1.1),[§4](https://arxiv.org/html/2607.01531#S4.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px3.p1.1)\.
- N\. Shinnet al\.\(2023\)Reflexion: language agents with verbal reinforcement learning\.InNeurIPS,Cited by:[§B\.3](https://arxiv.org/html/2607.01531#A2.SS3.p1.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- A\. Solar\-Lezama, L\. Tancau, R\. Bodík, S\. A\. Seshia, and V\. A\. Saraswat \(2006\)Combinatorial sketching for finite programs\.InASPLOS,Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p1.1),[§3\.3](https://arxiv.org/html/2607.01531#S3.SS3.SSS0.Px2.p1.1)\.
- R\. S\. Sutton \(1991\)Dyna, an integrated architecture for learning, planning, and reacting\.ACM SIGART Bulletin2,pp\. 160–163\.Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p1.1),[§3\.2](https://arxiv.org/html/2607.01531#S3.SS2.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px3.p1.1)\.
- H\. Tang, D\. Key, and K\. Ellis \(2024\)WorldCoder: building world models by writing code and interacting with the environment\.InNeurIPS,Cited by:[§B\.1](https://arxiv.org/html/2607.01531#A2.SS1.p1.1),[§1](https://arxiv.org/html/2607.01531#S1.p1.1),[§1](https://arxiv.org/html/2607.01531#S1.p2.1),[§2](https://arxiv.org/html/2607.01531#S2.SS0.SSS0.Px4.p1.3),[§4](https://arxiv.org/html/2607.01531#S4.SS0.SSS0.Px2.p1.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px1.p1.1)\.
- Y\. W\. Teh, M\. I\. Jordan, M\. J\. Beal, and D\. M\. Blei \(2006\)Hierarchical Dirichlet processes\.JASA101,pp\. 1566–1581\.Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p2.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- J\. B\. Tenenbaum, C\. Kemp, T\. L\. Griffiths, and N\. D\. Goodman \(2011\)How to grow a mind: statistics, structure, and abstraction\.Science331\(6022\),pp\. 1279–1285\.Cited by:[§3\.2](https://arxiv.org/html/2607.01531#S3.SS2.SSS0.Px3.p1.1)\.
- G\. Wang, Y\. Xie, Y\. Jiang, A\. Mandlekar, C\. Xiao, Y\. Zhu, L\. Fan, and A\. Anandkumar \(2023\)Voyager: an open\-ended embodied agent with large language models\.arXiv preprint arXiv:2305\.16291\.Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p2.1)\.
- R\. Wang, E\. Zelikman, G\. Poesia, Y\. Pu, N\. Haber, and N\. D\. Goodman \(2024\)Hypothesis search: inductive reasoning with language models\.InICLR,Cited by:[§3\.2](https://arxiv.org/html/2607.01531#S3.SS2.SSS0.Px2.p1.1),[§3\.2](https://arxiv.org/html/2607.01531#S3.SS2.SSS0.Px3.p1.1)\.
- L\. Wonget al\.\(2023\)From word models to world models: translating from natural language to the probabilistic language of thought\.External Links:2306\.12672Cited by:[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- Q\. Wu, G\. Bansal, J\. Zhang, Y\. Wu, B\. Li, E\. Zhu, L\. Jiang, X\. Zhang, S\. Zhang, J\. Liu, A\. H\. Awadallah, R\. W\. White, D\. Burger, and C\. Wang \(2023\)AutoGen: enabling next\-gen LLM applications via multi\-agent conversation\.arXiv preprint arXiv:2308\.08155\.Cited by:[§3\.2](https://arxiv.org/html/2607.01531#S3.SS2.SSS0.Px1.p1.1)\.
- Q\. Yang, K\. Wu, and Y\. Jiang \(2007\)Learning action models from plan examples using weighted MAX\-SAT\.Artificial Intelligence171\(2\-3\),pp\. 107–143\.Cited by:[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.
- S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. Cao \(2023\)ReAct: synergizing reasoning and acting in language models\.InICLR,Cited by:[§1](https://arxiv.org/html/2607.01531#S1.p2.1)\.
- A\. Zhouet al\.\(2024\)Language agent tree search unifies reasoning, acting, and planning in language models\.InICML,Cited by:[§B\.4](https://arxiv.org/html/2607.01531#A2.SS4.p1.1),[§5](https://arxiv.org/html/2607.01531#S5.SS0.SSS0.Px4.p1.1)\.