Hypothesis-guided discovery of cognitive algorithms via program refinement

# Hypothesis-guided discovery of cognitive algorithms via program refinement
Source: [https://arxiv.org/html/2610.02523](https://arxiv.org/html/2610.02523)
Huiwen Alex YangAffiliation:Department of PsychologyAffiliation:University of California, BerkeleyEmail:[hw\.yang@berkeley\.edu](mailto:)Bill D\. ThompsonAffiliation:Department of PsychologyAffiliation:University of California, Berkeley

###### Abstract

Developing cognitive models of algorithmic reasoning from behavioral data is a central problem in cognitive science that challenges current methods\. Traditional approaches to cognitive modeling are interpretable and benefit from human expertise, but lack flexibility and scalability\. Emerging techniques using large language models \(LLMs\) for de novo generation of cognitive models are scalable and flexible, but lack a role for human expertise and have mostly been applied to simpler tasks than algorithm recovery\. We propose a hybrid system that treats discovery of cognitive algorithms as a program refinement problem\. Human\-created cognitive models are expressed as probabilistic programs and provided to a system of LLM agents with a mandate to: identify mismatches between model and behavior; propose code\-level modifications within researcher\-specified constraints; and verify structural fidelity\. Revisions propagate to a probabilistic inference module that performs inference for latent variables and data likelihood computations\. We evaluate the pipeline on human behavior in a problem\-solving paradigm that exposes a variety of cognitive algorithms\. Revised models consistently improve model fit relative to ancestral models and reveal a small set of recurring innovations that capture meaningful behavioral variability in this task\.

## 1Introduction

An overarching goal of modern cognitive science and artificial intelligence is to advance the science of cognitive modeling\. Adaptive and scalable methods to induce interpretable cognitive models from behavioral data is a key frontier within this goal\. In particular, advances that support induction of models for*algorithmically\-structured*behaviors would enable innovation across many sub\-areas, and substantively expand the reach of cognitive science\. Recent advances in LLM\-supported modeling\([Rmus et al\., 2025](https://arxiv.org/html/2610.02523#bib.bib25);[Curtis et al\., 2025](https://arxiv.org/html/2610.02523#bib.bib13), e\.g\.\)establish new potential for scalable generation of cognitive models\. However, current methods \(1\) generally underutilize human expertise; \(2\) often produce uninteresting models that reflect the LLM’s prior; and \(3\) have mostly been applied at scale to simpler problems than algorithm recovery\. We propose a solution and evaluate it in depth on a single well\-suited problem\-solving paradigm\. Our approach views model discovery as*program refinement*: a scientist’s best hypothesized models are expressed as probabilistic programs and provided to a system of LLM agents with a mandate to identify targeted revisions that improve the fit between these initial proposals and human data; revised programs propagate to a probabilistic inference module that performs inference for latent variables and data likelihood computations\. Our approach offers systematic integration of human expertise into LLM\-supported discovery of algorithmic cognitive models\.

## 2Background

Standard approaches to cognitive modeling classically rely on model selection among a fixed set of models\([Farrell and Lewandowsky, 2018](https://arxiv.org/html/2610.02523#bib.bib7)\)\. This approach benefits from human expertise, strong interpretability, and normative standards for likelihood\-based inference\([Wilson and Collins, 2019](https://arxiv.org/html/2610.02523#bib.bib8)\)\. It has been successful in domains such as multi\-option choice or reward learning tasks, but involves fundamental limitations that become particularly apparent when applied to more complex settings; in particular, it is typically very challenging to capture the richness and variety of human thinking in*algorithmically\-structured*behaviors that unfold over time such as problem\-solving\([Allen et al\., 2020](https://arxiv.org/html/2610.02523#bib.bib31)\), planning\([Van Opheusden et al\., 2023](https://arxiv.org/html/2610.02523#bib.bib27)\), repeated games\([Brockbank and Vul, 2024](https://arxiv.org/html/2610.02523#bib.bib30)\), or social interactions\([Yu and Thompson, 2024](https://arxiv.org/html/2610.02523#bib.bib28);[Jha et al\., 2025](https://arxiv.org/html/2610.02523#bib.bib29)\)\. For example, imagine observing the actions of a skilled \(Rubik’s\) Cuber, and trying to reverse\-engineer a cognitive model of their underlying knowledge: the model would need to specify how the person’s actions depend not only on a complex set of cube features, but also potentially on a history of actions and their outcomes; in other words, the model would need to identify the structure of a latent*cognitive algorithm*\.

At least two challenges make this a very difficult problem\. One fundamental challenge is that the space of plausible cognitive algorithms is large and richly structured, making it difficult to anticipate relevant model variants\. Another fundamental challenge is that even when a model is generally a good hypothesis, human behavior is often not well\-captured by a single algorithm; even in domains where people learn canonical algorithms, such as mathematical reasoning\([Siegler and Shrager, 1984](https://arxiv.org/html/2610.02523#bib.bib38)\), people often exhibit systematic but idiosyncratic deviations from idealized procedures, including local execution errors, minor rule variations, and alternative tie\-breaking conventions\. Together, these challenges highlight a mismatch between classical approaches to model specification and the variability of real behavior\.

## 3Related Work

In response to the limitations of traditional cognitive modeling, efforts have focused on model*construction*\. For example, an influential framework views model construction as*program induction*\([Ellis et al\., 2023](https://arxiv.org/html/2610.02523#bib.bib23);[Jiang et al\., 2024](https://arxiv.org/html/2610.02523#bib.bib24);[Piantadosi et al\., 2016](https://arxiv.org/html/2610.02523#bib.bib11)\)\. These methods typically specify a domain\-specific language and perform statistical inference over expressible programs, extending a tradition of grammar\-induction\([O’Donnell et al\., 2009](https://arxiv.org/html/2610.02523#bib.bib9)\), with successful application to, e\.g\., language\([Yang and Piantadosi, 2022](https://arxiv.org/html/2610.02523#bib.bib15)\)and concept learning\([Piantadosi et al\., 2012](https://arxiv.org/html/2610.02523#bib.bib10);[Lake et al\., 2015](https://arxiv.org/html/2610.02523#bib.bib12);[Rule et al\., 2024](https://arxiv.org/html/2610.02523#bib.bib14)\)but often require specification of complex languages and sampling algorithms\. These difficulties motivated more general systems for program induction, driving development of probabilistic programming languages such as WebPPL\([Goodman and Stuhlmüller, 2014](https://arxiv.org/html/2610.02523#bib.bib16)\)that are natively enriched with probabilistic semantics and generic methods for statistical inference\. Probabilistic programming frameworks resolve some difficulties of domain\-specific methods, but remain primarily a tool for building hand\-crafted models\.

In parallel, advances have been made in automated model discovery\. For example, one approach generates models by training artificial neural networks on very large datasets\([Agrawal et al\., 2020](https://arxiv.org/html/2610.02523#bib.bib17)\)\. This approach has been examined in decision\-based problems\([Peterson et al\., 2021](https://arxiv.org/html/2610.02523#bib.bib18)\)as well as more algorithmic behaviors\([Ji\-An et al\., 2025](https://arxiv.org/html/2610.02523#bib.bib33);[Thompson et al\., 2022](https://arxiv.org/html/2610.02523#bib.bib21)\); it is scalable and can learn complex or counterintuitive models, but is not easily interpretable and is data\-intensive\. Prior theories can be integrated via initialization of network weights by pre\-training on synthetic data generated from structured cognitive models\([Bourgin et al\., 2019](https://arxiv.org/html/2610.02523#bib.bib19)\)\. More recently, emerging methods have leveraged the increasing capabilities of LLMs\. One class of approaches simply fine\-tunes LLMs on large volumes of human behavioral data\([Binz et al\., 2025](https://arxiv.org/html/2610.02523#bib.bib20)\); this is also data\-intensive and the resulting models are difficult to interpret\([Bowers et al\., 2025](https://arxiv.org/html/2610.02523#bib.bib22)\)\.

Another class of methods leverages the capabilities of LLMs for program synthesis\([Ellis et al\., 2022](https://arxiv.org/html/2610.02523#bib.bib4);[Pourcel et al\., 2025](https://arxiv.org/html/2610.02523#bib.bib39)\)\. Under this approach, cognitive models can be expressed as programs in, e\.g\., Python\. One stark proposal is to fully automate the larger enterprise of cognitive science\([Jagadish et al\., 2026b](https://arxiv.org/html/2610.02523#bib.bib5)\)\. Recent approaches have used LLMs to directly propose cognitive models\([Rmus et al\., 2025](https://arxiv.org/html/2610.02523#bib.bib25)\), integrating LLM proposals into model search or inference pipelines\([Colas et al\., 2025](https://arxiv.org/html/2610.02523#bib.bib26)\), using them to guide proposal generation\([Curtis et al\., 2025](https://arxiv.org/html/2610.02523#bib.bib13)\)or improve optimization over program spaces\([Piriyakulkij et al\., 2024](https://arxiv.org/html/2610.02523#bib.bib2)\), or leveraging natural language to bias synthesis toward more human\-like solutions\([Ellis, 2023](https://arxiv.org/html/2610.02523#bib.bib3)\)\. These approaches are most closely related to our framework, especially the approach of[Rmus et al\. \(2025\)](https://arxiv.org/html/2610.02523#bib.bib25)\.

However, these methods focus primarily on de novo generation of candidate programs – a highly under\-constrained problem that could benefit greatly from human expertise and the constraints of existing hypotheses or theories\. In our experience \(and commonly noted in the field\), the programs proposed by automated discovery can often seem uninteresting or overly influenced by an LLM’s prior over Python programs in general, being disconnected from the principles that researchers in the area would consider meaningful innovations, or e\.g\. simply enumerating contingencies\.

Another limitation of existing approaches is that they have primarily been applied to canonical problems that often lack rich algorithmic structure, such as multi\-option decision\-making\([Jagadish et al\., 2026a](https://arxiv.org/html/2610.02523#bib.bib40), e\.g\.\)or reward\-learning tasks\([Rmus et al\., 2026](https://arxiv.org/html/2610.02523#bib.bib41)\)or judgment problems\([Prystawski et al\., 2026](https://arxiv.org/html/2610.02523#bib.bib43)\)\. While there are many good reasons to begin with problems that have received much attention within cognitive science, this focus leaves open what we see as one of the most exciting opportunities presented by these methods: the opportunity to go*beyond*the problems we have focused on historically in part as a reflection of limited methods\. Recent advances in e\.g\. program synthesis\([Curtis et al\., 2025](https://arxiv.org/html/2610.02523#bib.bib13)\)and evolutionary coding\([Tang et al\., 2026](https://arxiv.org/html/2610.02523#bib.bib44)\)suggest that LLM discovery of rich algorithmic structure can be very effective, and that this approach can be applied in behavioral contexts\([Wang et al\., 2026](https://arxiv.org/html/2610.02523#bib.bib42)\)\.

## 4Key Contribution

We propose a system for*hypothesis\-guided*discovery of algorithmically structured cognitive models\. Our approach views model discovery as a program\-refinement problem\. We express existing models as probabilistic programs, which are provided along with human data to a system of LLM agents that iteratively adapt the programs to better account for observed behavioral trajectories\. LLM agents are instructed to identify mismatches between a program and behavior, and to propose targeted modifications as executable code\. In particular, the agents are given semantically rich instructions in natural language, such that their revisions can be precisely scoped\. Here we used this scoping to constrain the agents’ modifications to preserve the basic structure of the original model\. We represent this structure explicitly through structured annotated program components and revision specifications, which constrain how agents can introduce new decision rules, internal state, and other components\. An additional benefit is that agents can explain their revisions in natural language\.

The pipeline consists of four modules that operate sequentially\. Given a task specification, base model programs\(pt1,pt2,…,ptn\)\\left\(p\_\{t\_\{1\}\},p\_\{t\_\{2\}\},\\ldots,p\_\{t\_\{n\}\}\\right\), and a behavioral trajectorydd, the system produces and evaluates a revised programp^t\\hat\{p\}\_\{t\}\. Three modules, the Game, Coding, and Audit modules, are implemented using LLMs \(in this paper: OpenAI’s GPT\-5\.4\)\. A Game module interprets the trajectory and generates a structured update signal along with a natural language rationale; a Coding module maps this signal to program modifications; an Audit module verifies correctness and fidelity\. In contrast, the Inference module is not LLM\-based; it instead performs probabilistic program evaluation using standard methods for probabilistic inference, computing the likelihood of the observed behavior under the revised program\. We evaluate the system on human behavior in a novel problem\-solving task that admits multiple strategies and induces diverse behavioral patterns across experimental conditions\. The pipeline reliably produces revised programs that improve model fit while remaining consistent with the original strategy space\.

### 4\.1Cognitive models as probabilistic programs

Following previous work in cognitive science\([Griffiths et al\., 2024](https://arxiv.org/html/2610.02523#bib.bib1)\)and hierarchical reinforcement learning\([Andre et al\., 2002](https://arxiv.org/html/2610.02523#bib.bib34)\), we use probabilistic programs as a representation for cognitive models that capture structured problem\-solving strategies\. Probabilistic programs combine the expressivity of structured programs with the flexibility of probabilistic models to provide a general \(indeed, universal\) representation for sequential decision\-making\([van de Meent et al\., 2021](https://arxiv.org/html/2610.02523#bib.bib37)\)\. For example, a probabilistic program can represent a hierarchical policy in which a high\-level controller probabilistically decides between executing different algorithms instantiated as low\-level policies \(e\.g\., turning on colors in a blue\-green\-red order versus green\-blue\-red order\)\. The current work uses the FlipPy Python library\([Ho and Correa, 2026](https://arxiv.org/html/2610.02523#bib.bib6)\), which takes a subset of Python as a “host” language and combines it with sampling and conditioning statements to express probabilistic programs\. One of the benefits of using FlipPy, as opposed to other probabilistic programming languages such as Church\([Goodman et al\., 2014](https://arxiv.org/html/2610.02523#bib.bib36)\)\(LISP\-based\) or WebPPL\([Goodman and Stuhlmüller, 2014](https://arxiv.org/html/2610.02523#bib.bib16)\)\(JavaScript\-based\), is that LLMs demonstrate strong biases to program in Python\([Twist et al\., 2025](https://arxiv.org/html/2610.02523#bib.bib35)\)\.

### 4\.2Strategy Discovery and Evaluation System

Our system transforms base model programs into revised programs: given base model program\(pt\)\(p\_\{t\}\), participant trajectorydd, and a task specification, the system produces a revised programp^t\\hat\{p\}\_\{t\}and evaluates its likelihoodPr⁡\(d∣p^t\)\\Pr\(d\\mid\\hat\{p\}\_\{t\}\)\. The system consists of four modules: theGame Module\(GG\),Coding Module\(CC\),Audit Module\(AA\), andInference Module\(II\)\. These modules operate in a structured sequence: each stage transforms the output of the previous one, jointly producing validated revised programs\. TheGame ModuleandCoding Moduleare separated to distinguish behavioral reasoning from code generation\. The Game Module has access to the task specification and participant trajectories\. The Coding Module specializes in translating these revisions into executable FlipPy code using the FlipPy library\. This separation also reduces context size for each module, improving output reliability\. Fig\.[1](https://arxiv.org/html/2610.02523#S4.F1)provides a high\-level overview of this system\.

1. 1\.Researchers’ input to the system\.The system is initialized with a set of human\-written cognitive models as inputs: base strategy model programs\(pt1,pt2,pt3\)\\left\(p\_\{t\_\{1\}\},p\_\{t\_\{2\}\},p\_\{t\_\{3\}\}\\right\), participant dataddfrom datasetDD, and the task specification\. These inputs define the initial state of the system and are provided to the Game ModuleGG\.
2. 2\.Game Module \(GG\)\.For every participant dataddand base strategy programptp\_\{t\}, the Game ModuleGGgenerates a strategy revision signalt′t^\{\\prime\}, representing a structured transformation of the base strategy program class induced by the observed behavior from this participant\. The revision signal specifies targeted modifications to the decision policy, including rule\-level changes, added latent variables and parameters, and operational constraints that preserve the original program structure while introducing behaviorally motivated revisions\.
3. 3\.Coding Module \(CC\)\.Conditioned ont′t^\{\\prime\}and the FlipPy library,CCimplements a program synthesis step that maps the updated strategy specification to an executable programpt′p^\{\\prime\}\_\{t\}\.
4. 4\.Audit Module \(AA\)\.The audit moduleAAperforms verification overpt′p^\{\\prime\}\_\{t\}under the FlipPy runtime environment\. It checks for syntactic and runtime correctness\. The output is a validated programp^t\\hat\{p\}\_\{t\}\. In case of failure,AAapplies a bounded repair procedure using execution error signals and re\-verification\.
5. 5\.Inference Module \(II\)\.The inference module evaluatesp^t\\hat\{p\}\_\{t\}on participant dataddand computes the likelihoodPr⁡\(d∣p^t\)\\Pr\(d\\mid\\hat\{p\}\_\{t\}\)under the revised strategy model\.

Figure 1:Strategy discovery and evaluation system
### 4\.3Generation, selection, and validation of revised strategies

We ran the full system twice per participant per base strategy model, producing two independently generated revised programs for each participant\-base strategy pair\. Programs produced by the Coding Module \(CC\) were passed through the Audit Module \(AA\), which allowed up to five iterative repair attempts to resolve syntactic or runtime errors\. Programs that remained non\-executable were excluded from subsequent analyses\. The resulting audited programs constitute the candidate revised\-strategy set used in subsequent evaluation\. Because different runs can produce different revisions of the same base strategy, we subsequently grouped candidate programs into broader classes of qualitatively similar revisions\. This clustering and selection procedure was applied separately within each base\-strategy family to obtain a compact set of representative revised strategy models for evaluation\. After generating and selecting the representative revised strategy models, we evaluated whether they provided improved fit to participants’ behavioral data\. We first evaluated the representative programs on the original dataset and compared their fit with that of the corresponding base strategy models\. We then conducted the same evaluation on a held\-out set of trials that was not used during program generation or selection, allowing us to assess whether improvements in fit generalized beyond the trajectories used to develop the revised strategies\.

### 4\.4Human problem\-solving algorithms

We applied our pipeline to human behavior in a problem\-solving paradigm that examines how people develop algorithmic strategies to solve structured tasks\([Yang et al\., 2026](https://arxiv.org/html/2610.02523#bib.bib32)\)\. The dataset contains trial\-level data from 150 participants across multiple task environments, providing an ideal testbed because participants developed a variety of strategies, producing heterogeneous behavior\. Some participants’ behavior is well captured by existing cognitive models of varying structural complexity, whereas other participants exhibit behavior that is not adequately captured by these models, providing an opportunity for our pipeline to discover more accurate algorithms\.

#### 4\.4\.1Task Structure

Participants completed a problem\-solving task involving a grid of colored lights \(Fig\.[2](https://arxiv.org/html/2610.02523#S4.F2)A\)\. The goal is simple: turn on all the lights\. Each light can be toggled by clicking\. Clicking a light could, however, turn off other lights that were already on\. The task was defined by a set of colors and their dependency relations, where a dependency from coloriito colorjjmeant that clicking any light of coloriiturned off all currently active lights of colorjj\. These dependencies formed a directed acyclic graph \(DAG\) – e\.g\. blues turn off pinks, greens turn off blues\. Grid size, number of colors, and the DAG structure varied across conditions\. Participants were not told the underlying dependencies and had to discover an effective way to solve the task\. See\([Yang et al\., 2026](https://arxiv.org/html/2610.02523#bib.bib32)\)for further detail\.

Under these dynamics, the task can be solved by activating colors in an order consistent with the DAG\. Specifically, any*topological sort*of the DAG, a permutation of its colors in which every source color precedes the colors it turns off, defines a valid color ordering\. For example, if blue and yellow both turn off green, either blue–yellow–green or yellow–blue–green is a valid ordering\. Participants therefore did not necessarily need to recover the full DAG; identifying a valid ordering was sufficient to solve the task efficiently\. In the studies reported here, participants are not informed of the DAG’s structure: they must find a way to solve the problem \(turn on all the lights\) efficiently, which could involve trying to establish the structure and using that knowledge to complete the task, or simply turning on lights at random, or any one of innumerable algorithms between these extremes; our goal as modelers is to characterize the structure of the algorithms developed by participants\.

![Refer to caption](https://arxiv.org/html/2610.02523v1/design.png)Figure 2:Behavioral experiment\. \(A\) Task Structure: first five actions in an example trial\. \(B\) Procedure: instructions, two practice trials, and five scored trials, followed by a strategy description\. \(C\) Experimental conditions defined by board size \(small vs\. large\) and task structure \(simple vs\. complex DAG\), yielding three conditions\. Representative grids and DAGs for each condition, along with a set of working orderings\.

### 4\.5Base Strategy Models

We begin with the 3 strategy models developed in the original study to characterize participants’ behavior in this task\([Yang et al\., 2026](https://arxiv.org/html/2610.02523#bib.bib32)\)\. These models were specified*a priori*to capture qualitatively distinct ways of approaching the task, ranging from surface\-level spatial heuristics to strategies that infer and exploit the task’s latent structure\. In the present work, these models serve as the starting points for program refinement\. Each model has a fixed program structure with latent parameters inferred during model fitting, and we implemented them using the probabilistic programming library FlipPy\([Ho and Correa, 2026](https://arxiv.org/html/2610.02523#bib.bib6)\)\.

##### Spatial \(SP\)

Lights are turned on using simple spatial patterns: e\.g\. row\-by\-row, column\-by\-column, or back\-and\-forth zigzag sequences\. Pattern and lapse rate are latent parameters inferred during model fitting\. Does not consider color or turn\-off relations\. This strategy reflects a failure to recognize or utilize the relevance of color\-based structure altogether, instead relying on generic spatial heuristics that would be effective in tasks without latent dependencies\.

##### Color Blocking \(CB\)

Systematically turns on all lights of one color before moving to the next, without considering dependencies between colors\. The color order and a probabilistic lapse rate are treated as latent parameters inferred during model fitting\. This strategy reflects a partial discovery of task structure: participants recognize that color is a relevant organizing feature and exploit this regularity, but do not infer or utilize the underlying dependency relationships between colors\.

##### Hierarchical Two\-Phase Filling \(TP\)

This model represents a hierarchical strategy in which a valid topological ordering is first inferred and then used to complete the board\. In phase one, the model explores lights across colors, only testing lights not yet activated, thereby accumulating constraints on the set of possible orderings\. In phase two, it commits to a single ordering drawn from the set of topological orders still consistent with the observed turn\-offs\. For example, if a yellow light is observed to turn off a green light, then any ordering placing yellow before green is eliminated from the feasible set\. As more such constraints are observed in phase one, the set of admissible orderings becomes increasingly restricted, making correct orderings more likely\. This strategy captures participants who infer color dependencies and attempt to exploit them in a heuristic, potentially imperfect manner\. Two latent parameters are inferred: the transition point between phases and the phase\-two ordering used to complete the board\.

## 5Results

![Refer to caption](https://arxiv.org/html/2610.02523v1/improvement.png)Figure 3:Base strategies provide adequate baseline fit, and revised strategies yield improved model fit for a substantial subset of participants\. \(A\) Kernel density estimates of per\-step log marginal likelihood \(log ML\) for the best\-fitting base strategy, shown by condition \(x\-axis shared with panel C\)\. \(B\) Improvement from revised strategies by condition, shown as per\-step log ML gain \(gain = revised \- base\)\. Half\-violins show the distribution, points show individual participants, and dashed lines indicate condition means\. The y\-axis is truncated to \[0, 0\.35\] for clarity \(9/139 participants have higher gain and exceed this range\)\. \(C\) Participant\-level comparison of best base vs\. best overall \(base or revised\) fit; points above the diagonal indicate improvement\. Color denotes the family of the best revised strategy; shape denotes the best base strategy\. \(D\) Same comparison as \(C\), broken down by strategy family and condition, with the y\-axis showing the best revised variant within each family\.### 5\.1Baseline Model Fit and Benchmark Quality

To establish a benchmark for evaluating refined programs, we first assessed fit for three base models\. We computed data likelihood for each model on the final correctly completed trial for each participant\. Participants who did not complete any trial correctly were excluded, leaving 49 trials in*small\-complex*, 51 trials in*small\-simple*, and 41 trials in*large\-simple*\. Tab\.[2](https://arxiv.org/html/2610.02523#A2.T2)\(Appendix[B](https://arxiv.org/html/2610.02523#A2)\) summarizes trial\-level model fit using per\-step log marginal likelihood\. The baseline models produced numerically stable and broadly computable fits \(finite\-fit coverage: 95–100% across conditions\), with relatively tight bootstrap confidence intervals around condition\-level means and medians\. Fig\.[3](https://arxiv.org/html/2610.02523#S5.F3)A shows the distribution of best\-fitting base\-strategy log ML across conditions\. The results reveal meaningful variation in fit across participants: some are well explained by the base strategies, while others are less well captured, leaving room for improvement through model refinement\.

### 5\.2Discovery of Revised Strategies

We first generated candidate revisions by running the pipeline twice for each participant and each of the 3 base strategy models\. This produced 846 candidate revised programs in total\. The Audit Module validated 733 programs \(86\.6%\); we excluded the rest because they remained non\-executable after the allowed repair attempts\. Within each base\-strategy family, we clustered audited revisions according to the semantic similarity of their natural\-language revision proposals\. The largest clusters captured recurring types of modifications to the corresponding base strategies \(Table[1](https://arxiv.org/html/2610.02523#S5.T1)\)\.

Across the 3 base strategy families, we identified 8 recurring revision clusters: 3 within Color Blocking, 2 within Spatial, and 3 within Two\-Phase\. Cluster size varied substantially, with some revisions recurring across many participants and others appearing more rarely\. We selected 3 representative audited programs closest to the centroid of each cluster, yielding 24 prototypical revised models for the subsequent analyses\.

### 5\.3Improvement in Model Fit from Revised Models

Fig\.[3](https://arxiv.org/html/2610.02523#S5.F3)B reports the improvement in per\-step log marginal likelihood \(log ML\) for each participant\. The improvement is computed as the difference between the per\-step log ML of the best\-fitting strategy \(selected over both base and revised strategies\) and the per\-step log ML of the best\-fitting base strategy alone\. For a substantial subset of participants 53\.2% \(74/139\), the inclusion of revised strategies yields meaningful improvements in fit\. This indicates that, for these participants, the best\-explaining model is not contained within the original base strategy set, but instead lies within the space of revised programs\.

Fig\.[3](https://arxiv.org/html/2610.02523#S5.F3)C,D further characterize these gains\. Panel C shows that for 53\.2% \(74/139\) of participants, a revised strategy provides the overall best fit\. Panel D complements this analysis by evaluating improvements within each strategy family, rather than only the best\-fitting strategy\. As a result, improvements are observed across all families \(ColorBlocking: 54%; Spatial: 76\.9%; TwoPhase: 80\.4%\), including cases where a revised variant improves the fit of a non\-winning base model without changing the participant’s overall best\-fitting model\. Together, these results demonstrate that the program refinement system reliably generates structured variants of base models that better account for observed behavior across participants\.

Fig\.[4](https://arxiv.org/html/2610.02523#S5.F4)illustrates a representative case\. This participant is reasonably well fit by the base ColorBlocking \(CB\) algorithm, but exhibits improved fit under a revised variant,CB\_damaged\_2\. While the base CB algorithm captures the participant’s tendency to complete one color at a time, it fails to account for behavior following a*damage event*\(i\.e\. a turn\-off event, where previously\-on lights are turned off by an action that violates the DAG\)\. Specifically, after observing that activating a yellow light turns off the greens, the participant completes the yellow lights and then immediately returns to repair the previously ‘damaged’ green lights\. The revised strategyCB\_damaged\_2captures this damage\-aware repair behavior, resulting in improved fit for this participant by proposing a meaningful qualitative innovation to the base algorithm\.

![Refer to caption](https://arxiv.org/html/2610.02523v1/example.png)Figure 4:A representative case of a participant who is reasonably well fit by the base ColorBlocking strategy but exhibits improved fit under a revised variant\. \(A\) Core logic of the base CB strategy, which predicts the behavior to complete one color at a time\. \(B\) Participant behavior data\. \(C\) Core logic of the revised strategy variantCB\_damaged\_2\. Annotations highlight key modifications to the base strategy\. Modifications capture the participant’s behavior to repair previously damaged colors\.
### 5\.4Generalization and synthetic recovery

To assess whether these improvements generalize beyond the data used for model refinement, we additionally evaluated the 24 prototypical revised strategy programs on the held\-out dataset\. In the held\-out data, 52\.6% \(72/137\) of trials showed improved fit under a revised program\. Thus, the improvement observed during refinement was not restricted to the data used to generate and select revisions\. The persistence of these improvements on held\-out data suggests that the revised programs capture behavioral regularities that extend beyond the trajectories on which they were developed\. Details on held\-out analyses are provided in Appendix[E](https://arxiv.org/html/2610.02523#A5)\.

As a complementary synthetic recovery test, we generated synthetic behavioral trajectories from theCB\_damaged\_2strategy and applied our pipeline to individual trajectories\. Of the 215 valid revised programs generated by our pipeline, 111 \(51\.6%\) achieved a higher mean held\-out log likelihood across 40 held\-out trials than the base Color Blocking program, while 18 \(8\.4%\) recovered at least 75% of the predictive gap between the two models\. 104 \(48\.4%\) did not achieve an improvement in mean held\-out log likelihood over the base model over held\-out trials\. The variability in recovery may in part reflect that the pipeline sometimes produces revised programs that overfit to idiosyncratic features of the single behavioral trajectory used to generate each revision, rather than recovering the underlying strategy\. Overall, these results provide evidence that the procedure can recover predictive consequences of a richer strategy, while also illustrating the limitations of trajectory\-level strategy modeling\. Further synthetic recovery analyses are provided in Appendix[F](https://arxiv.org/html/2610.02523#A6)\.

### 5\.5Revised models reveal a small set of recurring algorithmic innovations

To better understand the structure of the revisions produced by the system, we analyze their organization in semantic space\. Clustering identifies recurring themes and yields a compact set of representative variants; Tab\.[1](https://arxiv.org/html/2610.02523#S5.T1)reveals two robust regularities in the structure of revisions\. First, revisions are highly clustered in semantic space: across 462 total revision proposals, behavior concentrates into a small number of dominant groups \(6 clusters for ColorBlocking, 3 for Spatial, and 6 for TwoPhase\), with a small subset of clusters accounting for the majority of revisions within each strategy family\. Second, the dominant clusters are strikingly consistent across strategy families and reflect an unanticipated feature:*locally grounded behavioral adjustments*\. Themes related to damage repair, local corrective actions, and deterministic tie\-breaking appear as the largest cluster\(s\) in all three strategy families\. These account for approximately 73% of ColorBlocking revisions \(161/222\), 67% of Spatial revisions \(8/12\), and 79% of TwoPhase revisions \(180/228\)\.

Table 1:Top three revision clusters per strategy family, ranked by size \(\# revisions within the cluster\)\. Cluster descriptions are LLM\-generated for interpretation only and are not used in modeling or analysis\. Full results are in Appendix[C](https://arxiv.org/html/2610.02523#A3)\.Strategy FamilySizeLLM\-generated cluster descriptionColorBlocking161Prioritize recent\-damage repair before resuming default strategyColorBlocking44Damage\-aware color\-blocking with flexible partial completionColorBlocking8Within\-color tie\-breaking with consistent index orderSpatial8Spatial scanning with temporary damage\-repair modeSpatial3Probabilistic local repair and repeat after eventsTwoPhase106Two\-phase damage\-aware repair and priority policyTwoPhase74Short\-term repair prioritization after observed damageTwoPhase29Stateful color\-focused exploration with immediate repair

## 6Limitations and future work

Our analysis focused on a single problem\-solving paradigm, using just a single iteration of the propose\-revise loop and a single underlying LLM\. This*narrow and deep*approach revealed important new findings relative to*broad and shallow*approaches, but it is clearly important to explore whether other algorithmically\-structured problem settings reveal further insight and to explore potential gains from alternative models\. In future work, we envisage extending our approach to a larger multi\-agent process or other canonical sampling algorithms such as MCMC or SMC procedures, and further exploring aspects of the harness that could support improved exploration in program space\. Perhaps the most important issue not addressed here or in other work concerns the potential for program revisions to over\-fit individual participants, trials, or tasks: methods for effective partial pooling and program compression across contexts are a clear priority for future work\. Similarly, no current approach \(including ours\) offers a systematic way to assess how discovered programs relate to broader theoretical frameworks beyond lower\-level procedural constraints; improved data\-likelihood alone is an insufficient indicator of meaningful meaningful innovation, which here we assessed only heuristically–this broader question will be central as the area advances\.

## 7Conclusion

We proposed a program refinement framework for cognitive modeling that combines structured probabilistic programs with an LLM\-based program revision system\. We applied this system to the more challenging domain of algorithmically\-structured sequential decision\-making models\. By enabling targeted revisions while preserving hypothesized structure and interpretability, our framework offers one principled way to integrate the increasing capabilities of frontier LLMs with \(1\) human expertise and \(2\) probabilistic statistical inference, and shows that these methods successfully extend to recovery of algorithmic models, a key frontier in cognitive science\.

## Ethics statement

The experimental protocol was reviewed and approved by the institutional review board \(IRB\) at our institution, and all participants provided informed consent prior to participation\. Participants received a base payment of $3 and a bonus of $0\.4 per correctly completed trial\.

## Reproducibility statement

All code, data, and experimental materials associated with this work are available in the supplementary materials and in the accompanying anonymous OSF repository\. The main paper and appendix provide detailed descriptions of the model refinement pipeline, base and revised strategy programs, model\-fitting and inference procedures, and held\-out evaluation\. The appendix further provides complete descriptions of the revision clusters and additional analyses\. The OSF repository contains the source code, experimental data, model programs, and materials necessary to reproduce the analyses and results reported in this work\.

Code, data, and experimental materials associated with this work are not included with this arXiv version\.

## References

- Agrawalet al\.\(2020\)M\. Agrawal, J\. C\. Peterson, and T\. L\. GriffithsScaling up psychology via scientific regret minimization\.Proceedings of the National Academy of Sciences117\(16\),pp\. 8825–8835\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p2.1)\.
- Allenet al\.\(2020\)K\. R\. Allen, K\. A\. Smith, and J\. B\. TenenbaumRapid trial\-and\-error learning with simulation supports flexible tool use and physical reasoning\.Proceedings of the National Academy of Sciences117\(47\),pp\. 29302–29310\.Cited by:[§2](https://arxiv.org/html/2610.02523#S2.p1.1)\.
- Andreet al\.\(2002\)D\. Andre S\. J\. Russellet al\.State abstraction for programmable reinforcement learning agents\.InAaai/iaai,pp\. 119–125\.Cited by:[§4\.1](https://arxiv.org/html/2610.02523#S4.SS1.p1.1)\.
- Binzet al\.\(2025\)M\. Binz, E\. Akata, M\. Bethge, F\. Brändle, F\. Callaway, J\. Coda\-Forno, P\. Dayan, C\. Demircan, M\. K\. Eckstein, N\. Éltető,et al\.A foundation model to predict and capture human cognition\.Nature644\(8078\),pp\. 1002–1009\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p2.1)\.
- Bourginet al\.\(2019\)D\. D\. Bourgin, J\. C\. Peterson, D\. Reichman, S\. J\. Russell, and T\. L\. GriffithsCognitive model priors for predicting human decisions\.InInternational conference on machine learning,pp\. 5133–5141\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p2.1)\.
- Bowerset al\.\(2025\)J\. S\. Bowers, G\. Puebla, S\. Thorat, K\. Tsetsos, and C\. J\. LudwigCentaur: a model without a theory\.OSF\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p2.1)\.
- Brockbank and Vul \(2024\)E\. Brockbank and E\. VulRepeated rock, paper, scissors play reveals limits in adaptive sequential behavior\.Cognitive Psychology151,pp\. 101654\.Cited by:[§2](https://arxiv.org/html/2610.02523#S2.p1.1)\.
- Colaset al\.\(2025\)C\. Colas, T\. Mills, B\. Prystawski, M\. H\. Tessler, N\. Goodman, J\. Andreas, and J\. B\. TenenbaumLanguage and experience: a computational model of social learning in complex novel tasks\.InProceedings of the Annual Meeting of the Cognitive Science Society,Vol\.47\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p3.1)\.
- Curtiset al\.\(2025\)A\. Curtis, H\. Tang, T\. Veloso, K\. Ellis, J\. B\. Tenenbaum, T\. Lozano\-Pérez, and L\. P\. KaelblingLlm\-guided probabilistic program induction for pomdp model estimation\.InConference on Robot Learning,pp\. 3137–3184\.Cited by:[§1](https://arxiv.org/html/2610.02523#S1.p1.1),[§3](https://arxiv.org/html/2610.02523#S3.p3.1),[§3](https://arxiv.org/html/2610.02523#S3.p5.1)\.
- Elliset al\.\(2022\)K\. Ellis, A\. Albright, A\. Solar\-Lezama, J\. B\. Tenenbaum, and T\. J\. O’DonnellSynthesizing theories of human language with Bayesian program induction\.Nature Communications13\(1\),pp\. 5024\(en\)\.External Links:ISSN 2041\-1723,[Link](https://www.nature.com/articles/s41467-022-32012-w),[Document](https://dx.doi.org/10.1038/s41467-022-32012-w)Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p3.1)\.
- Elliset al\.\(2023\)K\. Ellis, L\. Wong, M\. Nye, M\. Sable\-Meyer, L\. Cary, L\. Anaya Pozo, L\. Hewitt, A\. Solar\-Lezama, and J\. B\. TenenbaumDreamcoder: growing generalizable, interpretable knowledge with wake–sleep bayesian program learning\.Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences381\(2251\)\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p1.1)\.
- Ellis \(2023\)K\. EllisHuman\-like Few\-Shot Learning via Bayesian Reasoning over Natural Language\.arXiv\.Note:arXiv:2306\.02797 \[cs\]External Links:[Link](http://arxiv.org/abs/2306.02797),[Document](https://dx.doi.org/10.48550/arXiv.2306.02797)Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p3.1)\.
- Farrell and Lewandowsky \(2018\)S\. Farrell and S\. LewandowskyComputational modeling of cognition and behavior\.Cambridge University Press\.Cited by:[§2](https://arxiv.org/html/2610.02523#S2.p1.1)\.
- Goodman and Stuhlmüller \(2014\)N\. D\. Goodman and A\. StuhlmüllerThe Design and Implementation of Probabilistic Programming Languages\.Note:[http://dippl\.org](http://dippl.org/)Accessed: 2026\-5\-6Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p1.1),[§4\.1](https://arxiv.org/html/2610.02523#S4.SS1.p1.1)\.
- Goodmanet al\.\(2014\)N\. Goodman, V\. Mansinghka, D\. M\. Roy, K\. Bonawitz, and J\. B\. TenenbaumChurch: a language for generative models\.External Links:1206\.3255,[Link](https://arxiv.org/abs/1206.3255)Cited by:[§4\.1](https://arxiv.org/html/2610.02523#S4.SS1.p1.1)\.
- Griffithset al\.\(2024\)T\. L\. Griffiths, N\. Chater, and J\. B\. TenenbaumBayesian Models of Cognition: Reverse Engineering the Mind\.MIT Press\(en\)\.Note:Google\-Books\-ID: FEb6EAAAQBAJExternal Links:ISBN 978\-0\-262\-38104\-8Cited by:[§4\.1](https://arxiv.org/html/2610.02523#S4.SS1.p1.1)\.
- Ho and Correa \(2026\)M\. K\. Ho and C\. G\. CorreaFlipPy: pythonic probabilistic programming\.GitHub\.Note:GitHub repository[https://github\.com/codec\-lab/flippy](https://github.com/codec-lab/flippy)Cited by:[§4\.1](https://arxiv.org/html/2610.02523#S4.SS1.p1.1),[§4\.5](https://arxiv.org/html/2610.02523#S4.SS5.p1.1)\.
- Jagadishet al\.\(2026a\)A\. K\. Jagadish, Y\. Strittmatter, N\. Jacoby, G\. Kachergis, E\. Schulz, N\. Daw, S\. H\. Chandramouli, and T\. L\. GriffithsClosing the loop to discover psychological theories with an automated cognitive scientist\.arXiv preprint arXiv:2606\.26448\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p5.1)\.
- Jagadishet al\.\(2026b\)A\. K\. Jagadish, M\. Rmus, K\. Witte, M\. Mathony, M\. Binz, and E\. SchulzCan we automatize scientific discovery in the cognitive sciences?\.arXiv\.Note:arXiv:2603\.20988 \[cs\]External Links:[Link](http://arxiv.org/abs/2603.20988),[Document](https://dx.doi.org/10.48550/arXiv.2603.20988)Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p3.1)\.
- Jhaet al\.\(2025\)K\. Jha, A\. Y\. Huang, E\. Ye, N\. Jaques, and M\. Kleiman\-WeinerModeling others’ minds as code\.arXiv preprint arXiv:2510\.01272\.Cited by:[§2](https://arxiv.org/html/2610.02523#S2.p1.1)\.
- Ji\-Anet al\.\(2025\)L\. Ji\-An, M\. K\. Benna, and M\. G\. MattarDiscovering cognitive strategies with tiny recurrent neural networks\.Nature644\(8078\),pp\. 993–1001\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p2.1)\.
- Jianget al\.\(2024\)G\. Jiang, M\. Hofer, J\. Mao, L\. Wong, J\. B\. Tenenbaum, and R\. P\. LevyFinding structure in logographic writing with library learning\.arXiv preprint arXiv:2405\.06906\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p1.1)\.
- Lakeet al\.\(2015\)B\. M\. Lake, R\. Salakhutdinov, and J\. B\. TenenbaumHuman\-level concept learning through probabilistic program induction\.Science350\(6266\),pp\. 1332–1338\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p1.1)\.
- O’Donnellet al\.\(2009\)T\. J\. O’Donnell, J\. B\. Tenenbaum, and N\. D\. GoodmanFragment grammars: exploring computation and reuse in language\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p1.1)\.
- Petersonet al\.\(2021\)J\. C\. Peterson, D\. D\. Bourgin, M\. Agrawal, D\. Reichman, and T\. L\. GriffithsUsing large\-scale experiments and machine learning to discover theories of human decision\-making\.Science372\(6547\),pp\. 1209–1214\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p2.1)\.
- Piantadosiet al\.\(2012\)S\. T\. Piantadosi, J\. B\. Tenenbaum, and N\. D\. GoodmanBootstrapping in a language of thought: a formal model of numerical concept learning\.Cognition123\(2\),pp\. 199–217\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p1.1)\.
- Piantadosiet al\.\(2016\)S\. T\. Piantadosi, J\. B\. Tenenbaum, and N\. D\. GoodmanThe logical primitives of thought: empirical foundations for compositional cognitive models\.\.Psychological review123\(4\),pp\. 392\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p1.1)\.
- Piriyakulkijet al\.\(2024\)W\. T\. Piriyakulkij, C\. Langenfeld, T\. A\. Le, and K\. EllisDoing Experiments and Revising Rules with Natural Language and Probabilistic Reasoning\.arXiv\.Note:arXiv:2402\.06025 \[cs\]External Links:[Link](http://arxiv.org/abs/2402.06025),[Document](https://dx.doi.org/10.48550/arXiv.2402.06025)Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p3.1)\.
- Pourcelet al\.\(2025\)J\. Pourcel, C\. Colas, and P\. OudeyerSelf\-improving language models for evolutionary program synthesis: a case study on arc\-agi\.arXiv preprint arXiv:2507\.14172\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p3.1)\.
- Prystawskiet al\.\(2026\)B\. Prystawski, K\. Mukherjee, D\. Wurgaft, L\. Nasvytis, M\. Y\. Li, N\. D\. Goodman, and M\. C\. FrankAuto\-psych: automating the science of mind using agent\-driven theory discovery and experimentation\.arXiv preprint arXiv:2606\.26460\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p5.1)\.
- Rmuset al\.\(2025\)M\. Rmus, A\. K\. Jagadish, M\. Mathony, T\. Ludwig, and E\. SchulzGenerating computational cognitive models using large language models\.arXiv preprint arXiv:2502\.00879\.Cited by:[§1](https://arxiv.org/html/2610.02523#S1.p1.1),[§3](https://arxiv.org/html/2610.02523#S3.p3.1)\.
- Rmuset al\.\(2026\)M\. Rmus, A\. K\. Jagadish, M\. Mathony, T\. Ludwig, and E\. SchulzGenerating computational cognitive models using large language models\.Advances in Neural Information Processing Systems38,pp\. 87796–87833\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p5.1)\.
- Ruleet al\.\(2024\)J\. S\. Rule, S\. T\. Piantadosi, A\. Cropper, K\. Ellis, M\. Nye, and J\. B\. TenenbaumSymbolic metaprogram search improves learning efficiency and explains rule learning in humans\.Nature Communications15\(1\),pp\. 6847\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p1.1)\.
- Siegler and Shrager \(1984\)R\. S\. Siegler and J\. ShragerA model of strategy choice\.InOrigins of Cognitive Skills,C\. Sophian \(Ed\.\),pp\. 229–293\.Cited by:[§2](https://arxiv.org/html/2610.02523#S2.p2.1)\.
- Tanget al\.\(2026\)Z\. Tang, X\. Yin, W\. Chen, Z\. Chen, Y\. Zheng, W\. Ye, K\. Wang, and L\. LinAlphaagentevo: evolution\-oriented alpha mining via self\-evolving agentic reinforcement learning\.InThe Fourteenth International Conference on Learning Representations,Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p5.1)\.
- Thompsonet al\.\(2022\)B\. Thompson, B\. Van Opheusden, T\. Sumers, and T\. GriffithsComplex cognitive algorithms preserved by selective social learning in experimental populations\.Science376\(6588\),pp\. 95–98\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p2.1)\.
- Twistet al\.\(2025\)L\. Twist, J\. M\. Zhang, M\. Harman, D\. Syme, J\. Noppen, and D\. D\. NauckLLMs love python: a study of llms’ bias for programming languages and libraries\.CoRRabs/2503\.17181\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2503.17181),[Link](https://arxiv.org/abs/2503.17181)Cited by:[§4\.1](https://arxiv.org/html/2610.02523#S4.SS1.p1.1)\.
- van de Meentet al\.\(2021\)J\. van de Meent, B\. Paige, H\. Yang, and F\. WoodAn introduction to probabilistic programming\.External Links:1809\.10756,[Link](https://arxiv.org/abs/1809.10756)Cited by:[§4\.1](https://arxiv.org/html/2610.02523#S4.SS1.p1.1)\.
- Van Opheusdenet al\.\(2023\)B\. Van Opheusden, I\. Kuperwajs, G\. Galbiati, Z\. Bnaya, Y\. Li, and W\. J\. MaExpertise increases planning depth in human gameplay\.Nature618\(7967\),pp\. 1000–1005\.Cited by:[§2](https://arxiv.org/html/2610.02523#S2.p1.1)\.
- Wanget al\.\(2026\)C\. Wang, D\. Kasenberg, K\. Stachenfeld, and P\. S\. CastroDiscovering differences in strategic behavior between humans and llms\.arXiv preprint arXiv:2602\.10324\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p5.1)\.
- Wilson and Collins \(2019\)R\. C\. Wilson and A\. G\. CollinsTen simple rules for the computational modeling of behavioral data\.elife8,pp\. e49547\.Cited by:[§2](https://arxiv.org/html/2610.02523#S2.p1.1)\.
- Yanget al\.\(2026\)H\. A\. Yang, M\. K\. Ho, and B\. D\. ThompsonSpontaneous meta\-learning of efficient problem\-solving algorithms\.InProceedings of the 47th Annual Conference of the Cognitive Science Society,Note:Accepted for publicationCited by:[§F\.1](https://arxiv.org/html/2610.02523#A6.SS1.p1.1),[§4\.4\.1](https://arxiv.org/html/2610.02523#S4.SS4.SSS1.p1.1),[§4\.4](https://arxiv.org/html/2610.02523#S4.SS4.p1.1),[§4\.5](https://arxiv.org/html/2610.02523#S4.SS5.p1.1)\.
- Yang and Piantadosi \(2022\)Y\. Yang and S\. T\. PiantadosiOne model for the learning of language\.Proceedings of the National Academy of Sciences119\(5\),pp\. e2021865119\.Cited by:[§3](https://arxiv.org/html/2610.02523#S3.p1.1)\.
- Yu and Thompson \(2024\)D\. Yu and B\. ThompsonPeople balance joint reward, fairness and complexity to develop social norms in a two\-player game\.InProceedings of the annual meeting of the cognitive science society,Vol\.46\.Cited by:[§2](https://arxiv.org/html/2610.02523#S2.p1.1)\.

## Appendix AParticipant Instructions

All participants received the same instructions prior to beginning the task, reproduced below:

> On each trial, you will play a game\. You will see a grid of lights like the one below\. Your goal is simple: turn on all the lights\! Every square on the grid is a light\. You can turn a light on or off by clicking on it\. You will complete 2 practice trial\(s\)\. In these trials, every trial is the same game\. After you complete the practice trials, you will complete 5 scored trial\(s\)\. In these trials, every trial will be a different game\. You will get a bonus payment for every trial you complete \($bonus per trial\)\. If you really cannot complete a trial, you can skip it \(but you will not receive the bonus for that trial\)\. Good luck\!

## Appendix BBase strategy model fit summary

Table[2](https://arxiv.org/html/2610.02523#A2.T2)summarizes the performance of the best\-fitting base strategy model across conditions using per\-step log\-likelihoods\. Overall, the base models produce stable and well\-defined fits, with high finite\-fit coverage \(95–100%\) across all conditions, indicating that inference is numerically reliable for the vast majority of trials\. The relatively tight bootstrap confidence intervals further suggest that condition\-level estimates are well\-constrained\. There are also systematic differences across conditions: Conditions*small\-complex*and*small\-simple*exhibit similar fit quality, whereas Condition*large\-simple*shows substantially more negative per\-step log\-likelihood\. This pattern is consistent with the increased trajectory length in Condition*large\-simple*, reflecting the increased task scale and associated demands\. Taken together, these results indicate that the base models provide a stable but imperfect account of behavior, establishing a meaningful baseline against which improvements from revised strategies can be evaluated\.

Table 2:Descriptive summary of best\-fitting base model performance by condition\. Values reported here are per\-step log\-likelihoods, computed as the log marginal likelihoods of the best\-fitting base strategy model normalized by trajectory length\. 95% confidence intervals are bootstrap percentile intervals \(B=2000B=2000\)\.ConditionNNFinite fits \(%\)Mean \[CI95\]SDAvg\. traj\. length*small\-complex*49100\.0−1\.544\-1\.544\[−1\.680\-1\.680,−1\.414\-1\.414\]0\.4890\.48939\.739\.7*small\-simple*51100\.0−1\.464\-1\.464\[−1\.598\-1\.598,−1\.335\-1\.335\]0\.4790\.47938\.438\.4*large\-simple*4195\.1−2\.577\-2\.577\[−2\.693\-2\.693,−2\.465\-2\.465\]0\.3730\.373125\.6125\.6
## Appendix CClusters of revisions

Table[3](https://arxiv.org/html/2610.02523#A3.T3)provides a comprehensive summary of revision clusters across all strategy families\. Several regularities emerge\. First, revisions are highly concentrated: within each strategy family, a small number of dominant clusters account for the majority of proposals\. For example, in the ColorBlocking family, the largest cluster \(size 161\) substantially outweighs all others, and a similar skew is observed in the TwoPhase family, where the top two clusters \(sizes 106 and 74\) dominate\. In contrast, the Spatial family contains fewer and smaller clusters overall, reflecting both lower revision volume and a more compact set of variation patterns\.

Second, the dominant clusters across families reflect closely related functional themes\. In particular, revisions centered on damage detection and repair, as well as short\-term prioritization of recently affected lights, appear as the largest clusters in all three families\. Deterministic tie\-breaking and ordering strategies also recur, though typically at smaller scales\. This suggests that, despite differences in the underlying base strategies, revisions are organized around a shared set of locally grounded behavioral adjustments\.

Table 3:Summary of revision clusters across strategy families\. For each family, all clusters, ranked by size \(number of revisions in the cluster\), are shown\. Descriptions are LLM\-generated for interpretive purposes and are not used in the system or analysis\.SizeStrategy FamilyLLM\-generated cluster descriptionColorBlocking Strategy Family161Prioritize recent\-damage repair before resuming default strategyAfter a turn\-off/damage event, temporarily focus on repairing or avoiding the most recently damaged color\(s\) using a stable within\-color click order, then resume the default color\-blocking policy\.44Damage\-aware color\-blocking with flexible partial completionA damage\-aware color\-blocking strategy that maintains memory of recently damaged colors and flexibly switches between continuing a color block and repairing or reordering priorities\.8Within\-color tie\-breaking with consistent index orderWhen selecting within a color, ties among same\-color OFF lights are resolved using a deterministic spatial/index sweep instead of random selection\.4Occasional post\-completion or late\-stage exploratory actionsRare exploratory deviations from strict termination, such as additional clicks after completion or near\-completion, to capture behavioral variability\.3Selection stickiness — persist with chosen color until ON\_COLORA persistence bias that keeps selecting the same color until it is fully ON, only switching when external state changes require it\.2Optional exploratory lapse augmenting ColorBlocking policyA low\-probability lapse mechanism that occasionally overrides the main policy, producing rare exploratory or erroneous clicks\.Spatial Strategy Family8Spatial scanning with temporary damage\-repair modeSpatial scanning with a conditional repair mode that activates after damage events, prioritizing restoration before resuming scanning\.3Probabilistic local repair and repeat after eventsA stochastic mechanism that occasionally repeats or locally repairs recent actions after specific outcome events\.1Structured\-prior bias toward deterministic orderingA strong bias toward deterministic spatial ordering and reduced randomness in action selection\.TwoPhase Strategy Family106Two\-phase damage\-aware repair and priority policyA two\-phase strategy that prioritizes repairing damaged colors before proceeding with the exploit phase\.74Short\-term repair prioritization after observed damageTemporary prioritization of recently damaged colors before returning to the main policy\.29Stateful color\-focused exploration with immediate repairStateful exploration that maintains color focus while inserting immediate repair steps when damage is observed\.15Deterministic spatial tie\-breaking within color selectionDeterministic spatial ordering replaces random selection within color choices during execution\.2Turn\-off\-triggered repair prioritization in exploitationExploitation phase is temporarily interrupted after damage events to prioritize repair\.2Deterministic within\-color spatial ordering strategyA deterministic within\-color ordering rule applied during the exploit phase\.
## Appendix DHeterogeneity of best\-fitting revised strategy implementations

Table[4](https://arxiv.org/html/2610.02523#A4.T4)shows how best\-fitting revised strategies are distributed across strategy families, clusters, and specific variants\. Several patterns emerge\. First, even within a given cluster, multiple variants are selected across participants, indicating that no single implementation uniformly dominates\. For example, within the TwoPhase “damaged” cluster, all 3 variants are preferred by some participants, and similar splits appear in ColorBlocking clusters\. This provides direct evidence of within\-cluster heterogeneity in how shared high\-level revision ideas are instantiated\. Second, while certain variants account for a larger fraction of participants \(e\.g\., TP\_repair\_1 with 22/75\), the overall distribution remains diffuse, with a substantial portion of participants assigned to lower\-frequency variants\. This long\-tailed pattern indicates that model fit is distributed across many implementations rather than concentrated on a single dominant strategy\. Finally, these patterns are consistent across strategy families\. In all three families, clusters such as repair and damage handling contain multiple viable variants, reinforcing the idea that common functional theoretical revisions admit multiple realizations on a program level\. Together, these results support the conclusion that, although revisions are organized around shared cluster\-level principles, the specific implementations that best explain behavior vary systematically across participants\.

Table 4:Number of participants best\-fitted by each revised strategy, grouped by base strategy family, cluster type, and specific variant \(N = 75\)\.Strategy FamilyClusterVariantCount%TwoPhase \(TP\)TPrepairTP\_repair\_12229\.3%TP\_repair\_322\.7%TPdamagedTP\_damaged\_2912\.0%TP\_damaged\_334\.0%TP\_damaged\_134\.0%TPlocalTP\_local\_179\.3%ColorBlocking \(CB\)CBdamagedCB\_damaged\_21114\.7%CB\_damaged\_134\.0%CBrepairCB\_repair\_322\.7%CB\_repair\_122\.7%CBindexCB\_index\_356\.7%CB\_index\_111\.3%Spatial \(SP\)SPrepairSP\_repair\_322\.7%SP\_repair\_122\.7%SPdamagedSP\_damaged\_311\.3%
## Appendix EHeldout Analyses

### E\.1Heldout dataset

Each participant completed five scored trials in the experiment\. In the model refinement process and analyses reported in the main text, we used only the final correctly completed trial from each participant\. The held\-out dataset consists of the second\-to\-last correctly completed trial for each participant\. Participants who did not complete any trial correctly, or who had only one correctly completed trial, were therefore excluded from the held\-out analysis\. This resulted in 49 trials in the*small\-complex*condition, 50 trials in the*small\-simple*condition, and 39 trials in the*large\-simple*condition\.

### E\.2Held\-out Evaluation

We evaluated the held\-out trials using the same model\-fitting procedure as in the main analysis\. First, we fit each of the 138 held\-out trials to the 3 base strategies and selected the base strategy with the highest per\-step log marginal likelihood for each trial\. We then fit the same trials to the 24 prototypical revised strategies identified from the 3 strategy families and selected the strategy with the highest per\-step log marginal likelihood among all base and revised strategies\.

One held\-out trial could not be evaluated because all 27 strategies produced a log marginal likelihood of−∞\-\\infty\. This trial contained 270 behavioral steps, but we could not determine whether the non\-finite likelihoods were attributable to sequence length or to another numerical or model\-specific issue\. We therefore excluded this trial from the comparison, leaving 137 evaluable held\-out trials\.

For each held\-out trial, we quantified improvement as the difference between the per\-step log marginal likelihood of the best\-fitting strategy among the base and revised strategies and that of the best\-fitting base strategy alone\. A positive value therefore indicates that at least one revised strategy provides a better fit to the held\-out trial than any of the three base strategies\. Across the 137 held\-out trials, 72 \(52\.6%\) showed positive improvement under a revised strategy\.

Similar to the main analysis, held\-out improvements are observed across all three strategy families \(ColorBlocking: 52\.6%; Spatial: 86\.7%; TwoPhase: 83\.2%\), including cases in which a revised variant improves fit relative to the corresponding base strategy without becoming the best\-fitting strategy overall\.

## Appendix FSynthetic Analyses

### F\.1Synthetic Dataset

To evaluate recovery in a setting where the underlying behavioral strategy is known, we generated a synthetic dataset from theCB\_damaged\_2strategy\. This strategy variant was selected because the original behavioral study identified damage\-repair behavior as a common extension of the ColorBlocking strategy, with participants often returning to repair colors that had been damaged by subsequent actions\([Yang et al\., 2026](https://arxiv.org/html/2610.02523#bib.bib32)\)\. Thus,CB\_damaged\_2provides a concrete example of a behavioral regularity observed in participants and allows us to ask whether the discovery pipeline can recover such a strategy when its generating process is known\.

To make the synthetic dataset resemble the empirical data as closely as possible, we used 30 board configurations that participants encountered in the behavioral experiment and generated 5 independent synthetic behavioral trajectories for each configuration, yielding 150 trajectories in total\. Within each board configuration, the 5 trajectories were generated with lapse probabilities of 0\.0, 0\.0, 0\.1, 0\.1, and 0\.2, respectively\. These values were chosen to approximate the distribution of lapse probabilities estimated when fitting the original ColorBlocking model to participants: most participants had low estimated lapse rates, and approximately half had an estimated lapse rate of 0\.0\. This design therefore preserves the empirical task configurations and approximates the level of stochasticity observed in the original behavioral data while allowing the underlying strategy to be controlled\.

### F\.2Generating Revised Programs

For each synthetic behavioral trajectory, we independently ran our program refinement pipeline twice, resulting in two candidate revised programs per synthetic trial\. Of the 300 generated programs, 287 were validated by the Audit Module\. Among those programs, 215 had the expected program interface and could be evaluated using the same likelihood\-based inference procedure as the base CB model\. The remaining programs were excluded because their generated interfaces required additional arguments or otherwise could not be evaluated under the standardized procedure\. Thus, the synthetic analysis reports results for 215 refined programs\.

### F\.3Synthetic analysis results

We evaluated each discovered program on 40 independently generated held\-out trajectories from the sameCB\_damaged\_2strategy and task configurations that were not included in the generation of revised programs\. For each program, we compared its mean held\-out log likelihood with those of the original ColorBlocking \(CB\) model and the generatingCB\_damaged\_2model\. Across the 215 evaluated programs, 111 \(51\.6%\) achieved higher mean held\-out log likelihood than CB, while 2 \(0\.9%\) matched or exceeded the generating model\. To quantify partial predictive recovery, we computed the fraction of the predictive gap between CB and CB\-damaged recovered by each revised program:

R=LL¯​revised−LL¯​CBLL¯​CB​\-​damaged​\-​2−LL¯​CB\.R=\\frac\{\\overline\{\\mathrm\{LL\}\}\{\\mathrm\{revised\}\}\-\\overline\{\\mathrm\{LL\}\}\{\\mathrm\{CB\}\}\}\{\\overline\{\\mathrm\{LL\}\}\{\\mathrm\{CB\\text\{\-\}damaged\}\\text\{\-\}2\}\-\\overline\{\\mathrm\{LL\}\}\{\\mathrm\{CB\}\}\}\.The median recovery was 0\.334\. Overall, 109 programs \(50\.7%\) recovered a positive fraction of the predictive gap, 18 \(8\.4%\) recovered at least 75%, and 2 \(0\.9%\) matched or exceeded the generating model\. These results indicate that the discovery procedure can recover predictive consequences of a richer strategy, but recovery is heterogeneous: because revisions are generated from individual trajectories, they can also overfit trajectory\-specific features and fail to generalize even when the underlying data\-generating strategy is known\. Importantly, predictive recovery does not imply exact recovery of the generating computational mechanism; multiple distinct implementations of damage\-repair behavior were produced by the discovery procedure\.