PolyBridgeBench: Benchmarking Multimodal LLMs for Physics-Grounded Bridge Design

# PolyBridgeBench: Benchmarking Multimodal LLMs for Physics-Grounded Bridge Design
Source: [https://arxiv.org/html/2609.21493](https://arxiv.org/html/2609.21493)
###### Abstract

Multimodal large language models, or MLLMs, perform well at visual understanding and structured generation, yet these capabilities do not establish whether an engineering design will work when executed\. Existing benchmarks assess spatial reasoning, structural validity, or physics\-grounded construction, but they do not determine whether MLLMs can synthesize complete load\-bearing structures and repair them after simulator execution exposes a failure\. We introduce PolyBridgeBench, an executable benchmark for multimodal bridge design\. A model receives a visual scene and structured engineering constraints and generates a complete node–member–material topology\. Deterministic legality checks gate execution in a native dynamic physics simulation\. Following an execution failure, the benchmark returns temporal visual evidence from the failed rollout and evaluates repair under a fixed interaction budget\. Separate measurements of deterministic validity, dynamic functional success, and post\-failure recovery identify the stage at which design fails\. Experiments with six representative MLLMs across 189 levels expose a substantial gap between deterministic validity and dynamic success, pronounced sensitivity to material budgets, and limited post\-failure recovery under the primary strict\-budget setting\.

1Fudan University

2Shanghai Innovation Institute

25213050507@m\.fudan\.edu\.cn, xuyinghui@fudan\.edu\.cn

![Refer to caption](https://arxiv.org/html/2609.21493v1/figure1_1.png)Figure 1:Overview of PolyBridgeBench\. The input combines a visual scene with structured materials, loads, geometry, and budget constraints\. Each model call emits a complete topology\. Invalid candidates receive textual constraint feedback and return to synthesis without entering physics; valid candidates are executed in the native simulator\. A physical failure returns temporal frames from that rollout, whereas functional success requires all vehicles to reach their targets\. Both repair paths share a fixed interaction budget\.## Introduction

A bridge may obey every explicit construction rule and still collapse when a vehicle enters it\. An engineering artifact is useful only if it performs its intended function under execution, not merely if it conforms to a symbolic specification\. The central evaluation question is whether an MLLM can translate a multimodal scene and engineering constraints into a complete structure that works under native dynamics, then revise that structure when execution exposes a flaw\.

Existing benchmarks expose relevant capabilities but stop short of this joint evaluation\. Spatial and physical reasoning suites probe compositional relations, metric scene understanding, event prediction, and action selection\([Johnson et al\. 2017](https://arxiv.org/html/2609.21493#bib.bib1);[Chen et al\. 2024](https://arxiv.org/html/2609.21493#bib.bib7);[Yang et al\. 2025](https://arxiv.org/html/2609.21493#bib.bib8);[Bakhtin et al\. 2019](https://arxiv.org/html/2609.21493#bib.bib10);[Bear et al\. 2021](https://arxiv.org/html/2609.21493#bib.bib11);[Chow et al\. 2025](https://arxiv.org/html/2609.21493#bib.bib12)\)\. ORIGAMISPACE evaluates multi\-step spatial reasoning under explicit mathematical constraints\([Xu et al\. 2025](https://arxiv.org/html/2609.21493#bib.bib6)\), DreamHouse and BrDSL study structural compliance and analysis\([Yang et al\. 2026](https://arxiv.org/html/2609.21493#bib.bib20);[Maharjan and Chun 2026](https://arxiv.org/html/2609.21493#bib.bib21)\), and BuildArena evaluates language\-driven construction in a physics\-grounded environment\([Xia et al\. 2026](https://arxiv.org/html/2609.21493#bib.bib22)\)\. No existing protocol follows a candidate from complete structure synthesis through separate legality and functionality tests to recovery based on evidence from its own failed execution\.

Reference similarity cannot define correctness because structural design is one\-to\-many: a level may admit several functionally valid topologies\. This limitation also motivates executable, test\-based CAD evaluation\([Alrashedy et al\. 2025](https://arxiv.org/html/2609.21493#bib.bib16);[Mallis et al\. 2026](https://arxiv.org/html/2609.21493#bib.bib19)\)\. Legality must also remain distinct from functionality\. Deterministic rules can verify connectivity, geometry, materials, and cost, but they cannot establish whether a legal topology will survive dynamic loading and permit traversal\. Repair introduces a further measurement problem\. Without fixed limits on model calls and physics attempts, higher final success may merely reflect more test\-time computation\. Diagnostic embodied\-agent benchmarks likewise show that final success alone cannot localize the missing capability\([Li et al\. 2024](https://arxiv.org/html/2609.21493#bib.bib24)\)\. Bridge construction concentrates these evaluation problems in a controlled setting with dense constraints, non\-unique solutions, and failures that unfold over time\.

We therefore evaluate a generated structure as an executable hypothesis rather than a static answer\. The benchmark accepts any design that is legal and works under native dynamics without matching it to a reference bridge\. When execution fails, evidence from that same rollout supports a revision within the remaining budget\. This staged protocol admits multiple valid solutions, measures the gap between compliance and functionality directly, and makes repair cost explicit\.

PolyBridgeBench instantiates this evaluation across 189 levels from 27 structural families\. A model generates a complete node–member–material topology from a scene image and structured engineering constraints\. Each candidate undergoes deterministic validation before serialization into Poly Bridge for native physics execution\. After a failure, the model may revise the full topology under fixed budgets for model calls and physics attempts; the post\-failure input includes temporal visual evidence from the rollout\. The protocol records legality, physics success, and recovery separately\. We evaluate six representative MLLMs under strict and loose material budgets and analyze their performance across parameterized and compositional structural conditions\.

This work makes the following contributions:

- •We introduce PolyBridgeBench, an executable benchmark that formulates multimodal bridge construction as complete topology generation under explicit engineering constraints and native dynamic execution\.
- •We develop a layered diagnostic protocol that measures deterministic validity, single\-call and budgeted dynamic success, and post\-failure recovery separately\. The resulting measurements identify where rule\-compliant designs cease to function\.
- •Evaluation across 189 levels and six MLLMs exposes a substantial gap between deterministic validity and dynamic success, pronounced sensitivity to material budgets, and distinct model\-specific failure bottlenecks\.

Figure 2:Composition of PolyBridgeBench\. Panel a shows the 149 Core Structural Suite levels from 21 parameterized structural families\. Panel b shows the 40 Compositional Challenge Suite levels from six families that combine multiple engineering constraints\. Panel c reports the bridge\-span distribution over all 189 levels\. Sector areas and bar lengths are proportional to level counts\. The suites are complementary evaluation strata, and all six models are evaluated on all 189 levels\.
## Related Work

### Multimodal Spatial and Physical Reasoning

CLEVR, NLVR2, and StepGame established controlled tests of compositional and spatial relations\([Johnson et al\. 2017](https://arxiv.org/html/2609.21493#bib.bib1);[Suhr et al\. 2019](https://arxiv.org/html/2609.21493#bib.bib2);[Shi et al\. 2022](https://arxiv.org/html/2609.21493#bib.bib3)\)\. Newer benchmarks cover metric estimation, reference\-frame grounding, video\-based spatial memory, and sequential assembly\([Chen et al\. 2024](https://arxiv.org/html/2609.21493#bib.bib7);[Song et al\. 2025](https://arxiv.org/html/2609.21493#bib.bib9);[Yang et al\. 2025](https://arxiv.org/html/2609.21493#bib.bib8);[Stogiannidis et al\. 2025](https://arxiv.org/html/2609.21493#bib.bib4);[Tang et al\. 2025](https://arxiv.org/html/2609.21493#bib.bib5)\)\. ORIGAMISPACE adds foldability constraints and crease\-pattern generation\([Xu et al\. 2025](https://arxiv.org/html/2609.21493#bib.bib6)\)\. These benchmarks expose failures hidden by static visual question answering, but their outputs remain answers, plans, predictions, or compilable geometry rather than load\-bearing topologies whose functionality requires dynamic execution\.

Physical\-reasoning benchmarks test commonsense, object attributes, event prediction, or intervention in predefined scenes\([Bisk et al\. 2020](https://arxiv.org/html/2609.21493#bib.bib13);[Wang et al\. 2023](https://arxiv.org/html/2609.21493#bib.bib14);[Bakhtin et al\. 2019](https://arxiv.org/html/2609.21493#bib.bib10);[Bear et al\. 2021](https://arxiv.org/html/2609.21493#bib.bib11);[Chow et al\. 2025](https://arxiv.org/html/2609.21493#bib.bib12)\); LLMPhy uses a simulator for parameter estimation and dynamics prediction\([Cherian et al\. 2024](https://arxiv.org/html/2609.21493#bib.bib15)\)\. They assess existing scenes rather than artifact synthesis\. PolyBridgeBench instead requires creating the artifact that determines subsequent dynamics, using execution as both the functional criterion and evidence for revision\.

### Constrained Structural Generation and Engineering Analysis

Executable design pipelines directly verify structured artifacts\. CAD\-Assistant executes and adapts VLLM\-planned FreeCAD actions\([Mallis et al\. 2025](https://arxiv.org/html/2609.21493#bib.bib17)\); CADCodeVerify uses rendered execution feedback\([Alrashedy et al\. 2025](https://arxiv.org/html/2609.21493#bib.bib16)\); Text2BIM combines programmatic authoring with model checking\([Du et al\. 2026](https://arxiv.org/html/2609.21493#bib.bib18)\); and CADTestBench tests geometric and topological requirements\([Mallis et al\. 2026](https://arxiv.org/html/2609.21493#bib.bib19)\)\. DreamHouse checks timber\-frame geometry, structure, constructability, and code compliance, while BrDSL connects bridge specifications to finite\-element analysis\([Yang et al\. 2026](https://arxiv.org/html/2609.21493#bib.bib20);[Maharjan and Chun 2026](https://arxiv.org/html/2609.21493#bib.bib21)\)\. These criteria emphasize compilation, compliance, or scoped analysis\. PolyBridgeBench instead accepts any legal topology that completes vehicle traversal under native dynamics\.

### Executable Engineering Construction and Interactive Evaluation

Language\-model agents interleave reasoning with actions, while PlanBench and Embodied Agent Interface expose planning or diagnostic failures beyond final success\([Yao et al\. 2023](https://arxiv.org/html/2609.21493#bib.bib25);[Valmeekam et al\. 2023](https://arxiv.org/html/2609.21493#bib.bib23);[Li et al\. 2024](https://arxiv.org/html/2609.21493#bib.bib24)\)\. BuildArena is the closest physics\-grounded construction benchmark, evaluating language\-driven 3D assembly through transport, support, and lift tasks\([Xia et al\. 2026](https://arxiv.org/html/2609.21493#bib.bib22)\)\. PolyBridgeBench changes the evaluation unit from an action sequence to a complete graph: validation and native execution expose distinct failures, and evidence from a failed rollout tests whether a subsequent graph recovers within the remaining budget\.

## PolyBridgeBench

Figure[1](https://arxiv.org/html/2609.21493#S0.F1)shows two diagnostic paths under one call budget\. Invalid graphs,D⁡\(Gt,x\)=0D\(G\_\{t\};x\)=0, receive validator feedback without consuming a physics attempt; valid but unsuccessful graphs,D⁡\(Gt,x\)=1D\(G\_\{t\};x\)=1andP⁡\(Gt,x\)=0P\(G\_\{t\};x\)=0, receive rollout evidence\. Both paths require a new complete topology\.

### Task Formulation

Each level isx=\(I0,T0,C,E\)x=\(I\_\{0\},T\_\{0\},C,E\), comprising an initial editor image, native\-anchor topology, task constraints \(materials, budget, vehicles, and loads\), and environment constraints \(terrain and clearance regions\)\. At calltt, the model receivesItI\_\{t\},Tt−1T\_\{t\-1\}, fixed conditions\(C,E\)\(C,E\), and prior feedbackH<tH\_\{<t\}, then outputs

Gt=\(Vt,Mt\),G\_\{t\}=\(V\_\{t\},M\_\{t\}\),\(1\)whereVtV\_\{t\}contains nodes with coordinates and kinematic attributes, whileMtM\_\{t\}contains members with endpoints, materials, and connectivity\. Every call returns the complete structure rather than recommendations or incremental edits, enabling independent parsing, validation, execution, and replay\.

Within a fixed interaction budget, the objective is a design satisfying legalityD⁡\(Gt,x\)=1D\(G\_\{t\};x\)=1and dynamic functionalityP⁡\(Gt,x\)=1P\(G\_\{t\};x\)=1, where functionality requires vehicle traversal under native physics\. This accepts multiple working topologies rather than one reference bridge\.

### Benchmark Composition

PolyBridgeBench contains 189 executable levels from 27 procedural families\. The 149\-level Core Structural Suite varies span, slope, bank geometry, anchors, platforms, materials, vehicles, and loads\. The 40\-level Compositional Challenge Suite combines clearance, material, load, vehicle, platform, and support constraints to test performance when several requirements apply at once\.

Figure[2](https://arxiv.org/html/2609.21493#Sx1.F2)visualizes the benchmark composition across structural families, compositional subtypes, and bridge spans\.

The suites are complementary strata, not an in\-/out\-of\-distribution split; every model runs on all levels\. Core measures fundamental variations, while Compositional isolates combined constraints\.

The generator retains family identifiers, parameters, and tags for offline analysis but hides them from models\. Inputs expose only native anchors, available materials, vehicle configuration, applicable clearance regions, and a per\-level budget calibrated from a manually built reference design\. The primary evaluation enforces this budget as a hard constraint\.

### Structured Bridge Representation

Bridges use node–member–material graphs\. Nodes store identifiers, 2D coordinates, and fixed\-anchor status; members store endpoints, materials, and save\-format geometry\. Native anchors are immutable, generated nodes are non\-kinematic, and materials have distinct length limits and costs; each level may further restrict admissible materials\. The graph supports diverse topologies, deterministic parsing, and direct game\-save serialization; the appendix provides the schema, coordinate conventions, and prompt\.

### Deterministic Validation and Dynamic Execution

Before physics, deterministic validation checks complete topology, immutable anchors, valid references, anchor\-connected dynamic components, member lengths, materials, cost, duplicate edges, isolated nodes, and forbidden\-region crossings\. It defines the legal action space but neither completes designs nor predicts stability\.

Invalid candidates consume a call but no physics attempt and receive a deterministic textual error\. Valid candidates are serialized and executed in the native simulator\. A unified detector checks whether all vehicles complete their required traversal within a fixed window; otherwise the run is a physical failure or timeout without an inferred cause\. Topologies, responses, frames, and resource records are retained for audit\.

After physical failure, selected rollout frames, failure status, and the current topology form the next observation\. The model regenerates a complete graph within the remaining three\-call/three\-execution budget; an episode may include validator repair, physical failures, and later success\.

Frames provide observational evidence, not an automatic diagnosis: selection removes repeated post\-termination frames but labels no causal breakpoint\. Together with failure status and the current topology, they form the standard post\-failure observation\. Recovery therefore measures this complete feedback package rather than any single modality\.

## Evaluation Protocol

### Layered Metrics

Let the benchmark containNNlevels\. For levelii, letvit∈\{0,1\}v\_\{i\}^\{t\}\\in\\\{0,1\\\}indicate whether the candidate from thett\-th LLM call passes deterministic validation\. If that candidate enters physics execution,yit∈\{0,1\}y\_\{i\}^\{t\}\\in\\\{0,1\\\}indicates whether it succeeds functionally\. We define

Valid@k=1N∑i=1N𝕀\[∃t≤k,vit=1\],\\mathrm\{Valid@\}k=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbb\{I\}\\\!\\left\[\\exists t\\leq k,\\ v\_\{i\}^\{t\}=1\\right\],\(2\)which measures whether the model produces at least one deterministically valid topology withinkkcalls\. We measure end\-to\-end physical success as

Success@k=1N∑i=1N𝕀\[∃t≤k,yit=1\]\.\\mathrm\{Success@\}k=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\mathbb\{I\}\\\!\\left\[\\exists t\\leq k,\\ y\_\{i\}^\{t\}=1\\right\]\.\(3\)Success@1 requires the first model output to pass validation and succeed on its first physics execution\. If the model never produces a valid design, the level cannot enter physics execution and is counted as unsuccessful for Success@k\.

To quantify recovery after execution failure, letℛk\\mathcal\{R\}\_\{k\}contain the levels that experience a physical failure before thekk\-th call and retain at least one LLM call and one physics attempt after that failure\. We define

Recovery@k=1\|ℛk\|∑i∈ℛk𝕀\[∃tf<ts≤k,yitf=0∧yits=1\]\.\\mathrm\{Recovery@\}k=\\frac\{1\}\{\|\\mathcal\{R\}\_\{k\}\|\}\\sum\_\{i\\in\\mathcal\{R\}\_\{k\}\}\\mathbb\{I\}\\\!\\left\[\\exists t\_\{f\}<t\_\{s\}\\leq k,\\ y\_\{i\}^\{t\_\{f\}\}=0\\land y\_\{i\}^\{t\_\{s\}\}=1\\right\]\.\(4\)Recovery@k is computed per level rather than per failure event\. A trajectory ofF→F→SF\\\!\\rightarrow\\\!F\\\!\\rightarrow\\\!Stherefore counts as one successful recovery, with no additional weight for the second failed attempt\. Because the conditional denominator can differ across models, Recovery@3 is interpreted as a conditional diagnostic rather than another aggregate success rate\. Parsing failures, validator\-error categories, physics attempts, and model calls provide additional evidence about where failures occur\.

### Models and Unified Settings

The primary evaluation covers six representative MLLMs: Gemini\-3\-Flash\-Preview, Qwen3\-VL\-Plus, Gemma\-4\-31B\-IT, Kimi\-K2\.5, GPT\-5\.2, and Claude\-Sonnet\-5\. Every model receives semantically equivalent inputs under the same topology\-output specification\. The appendix documents each model’s thinking or reasoning controls, exact API snapshot, access date, and maximum output length\.

Each episode permits at most three LLM calls and three physics attempts\. We set the temperature to 0\.2 when the provider exposes this control; otherwise, we retain the provider default and report it in the appendix\. Physics execution uses a 12\-second simulation window and samples raw frames at 4 fps\. Four frames from each failed rollout form a contact sheet for temporal visual input\. Experiments use individually calibrated per\-level budgets under strict enforcement and a looser uniform\-budget setting\.

## Experiments and Analysis

The analysis is organized around three research questions\. RQ1 asks whether current MLLMs can generate executable bridges and recover under bounded interaction\. RQ2 locates failures in structured generation, deterministic legality, or dynamic functionality\. RQ3 examines the material and interaction costs of successful designs\.

Table 1:Main results on the Core Structural Suite, denoted Level 1 with 149 levels; the Compositional Challenge Suite, denoted Level 2 with 40 levels; and all 189 levels\. V@k and S@k denote Valid@k and Success@k withinkkmodel calls\. R@3 denotes recovery among repair\-eligible physical failures\. Loose budget relaxes material cost, whereas strict budget enforces the calibrated per\-level limit\. All entries are percentages; higher is better\.### Experimental Setup

We evaluate Gemini\-3\-Flash\-Preview, Qwen3\-VL\-Plus, Gemma\-4\-31B\-IT, Kimi\-K2\.5, GPT\-5\.2, and Claude\-Sonnet\-5 on all 189 levels\. Level 1 denotes the 149 levels in the Core Structural Suite, and Level 2 denotes the 40 levels in the Compositional Challenge Suite\. Every model receives semantically equivalent scene information, engineering constraints, and topology\-output specifications and follows the common protocol for model calls, physics execution, and success detection described above\.

We compare loose\- and strict\-budget settings\. The strict setting enforces the material budget calibrated for each level from a manually constructed reference design as a hard deterministic constraint\. The loose setting relaxes the material\-cost restriction so that topology generation and physical execution can be examined when budget is not the primary bottleneck\. Both settings allow at most three LLM calls and three physics attempts per episode\. An invalid design consumes one model call but does not enter simulation; only a validated design is written into the game and executed\.

We report results for Level 1, Level 2, and Overall\. For each suite, we report Success@1 and Success@3; for Overall, we additionally report Valid@1, Valid@3, and Recovery@3\. Strict budget is the primary setting for assessing executable design under constraints\. Loose budget is a diagnostic condition that separates material\-budget failures from limitations in topology generation and dynamic load bearing\. All table entries are percentages, and Recovery@3 uses the eligible episodes defined in the evaluation protocol as its conditional denominator\.

### Main Results

Table[1](https://arxiv.org/html/2609.21493#Sx5.T1)reports performance under both budget settings\. The gap between Valid and Success measures how often deterministic compliance fails to yield dynamic functionality\. The change from @1 to @3 measures the benefit of bounded interaction\. Strict budget is the primary benchmark condition; loose budget is a diagnostic setting that removes material cost as the dominant bottleneck\.

Figure 3:Composition of recorded error events over all three interaction rounds under loose and strict budgets\. Each bar is normalized separately within a model and budget setting, so segment height is a within\-model event share rather than a level failure rate or an absolute error count\.Figure 4:Success–efficiency trade\-offs under loose and strict budgets\. Colors identify models, squares denote strict budget, hollow circles denote loose budget, and arrows point from strict to loose\. The horizontal axis is Overall Success@3 across all 189 levels\. The left panel reports successful\-design cost normalized by the corresponding per\-level strict limit; its shaded region is budget compliant\. The right panel reports model calls averaged over successful episodes\. Both vertical axes are conditioned on success\.Recovery@3 has a conditional denominator and should not be read as another aggregate success rate\. It measures recovery only among levels that experienced a physical failure while retaining sufficient budget for another model call and physics attempt\.

#### Strict budgets change model rankings\.

Under loose budget, Gemini\-3\-Flash\-Preview obtains the highest Overall Success@3 at 84\.7%\. Under strict budget, Claude\-Sonnet\-5 ranks first with 50\.8%\. High success under the loose setting therefore does not establish that a model can generate executable structures under constrained engineering conditions\. The per\-level material budget is part of the task definition, not an auxiliary cost measure\.

#### Deterministic validity is not a reliable proxy for dynamic functionality\.

Under loose budget, Qwen3\-VL\-Plus reaches 92\.1% Valid@3 but only 12\.2% Success@3\. A similar gap remains under strict budget\. Models can use multiple rounds to produce topologies that satisfy explicit rules without ensuring vehicle traversal under native dynamics\. Validator pass rates and physical success must therefore be reported separately\.

#### Compositional constraints amplify the functional bottleneck\.

Level 2 is more difficult for every model\. Under strict budget, only Gemini\-3\-Flash\-Preview and Claude\-Sonnet\-5 achieve nonzero Success@3 on this subset, and the best score is 15\.0%\. Combining material restrictions, clearance requirements, platforms, loads, and support conditions sharply reduces the feasible design space\. Models that solve some core structural levels do not retain the same functional performance when several constraints apply simultaneously\.

#### Multi\-round gains are not equivalent to recovery from physical failure\.

Success@3 exceeds Success@1 for every model under both budget settings, whereas the highest strict\-budget Recovery@3 is only 22\.0%\. The increase in Success@3 also includes legality corrections prompted by validator feedback and cannot be attributed entirely to effective repair after execution failure\. Success@3 and Recovery@3 must therefore be interpreted jointly rather than treating every multi\-round gain as physical repair\.

### Where Does the Capability Break Down?

The main results establish an aggregate gap between deterministic legality and dynamic functionality but do not identify the errors that constitute this gap\. Figure[3](https://arxiv.org/html/2609.21493#Sx5.F3)aggregates error events over all three interaction rounds and normalizes them within each model\. Deterministic errors cover member length, budget, duplicate edges, clearance, and isolated nodes\.RUN\_FALSE\_AFTER\_VALIDdenotes a candidate that passed validation but did not succeed during physics execution\. Because one candidate may trigger several validator errors, the figure describes the composition of recorded error events rather than the level failure rate; absolute model performance should still be read from Table[1](https://arxiv.org/html/2609.21493#Sx5.T1)\.

The error composition already separates two model behaviors\. Claude\-Sonnet\-5 is dominated by post\-validation physics failures in both settings, whereas strict budget shifts most other models toward deterministic errors that prevent simulation\.

Claude\-Sonnet\-5 fails primarily during dynamic execution\.RUN\_FALSE\_AFTER\_VALIDaccounts for 93\.2% and 94\.3% of its recorded errors under loose and strict budgets, respectively, while deterministic errors are rare\. This pattern agrees with its high Valid@1 and Valid@3: Claude usually produces complete topologies that satisfy explicit constraints, but converting a legal structure into one that carries the vehicle across remains the bottleneck\. Qwen3\-VL\-Plus shows the same tendency under loose budget, explaining why its Valid@3 is high while its Success@3 remains low\.

GPT\-5\.2 and Kimi\-K2\.5 more often violate local geometric constraints before reaching physics execution\. Under loose budget,EDGE\_TOO\_LONGaccounts for 50\.9% and 60\.0% of their error events, respectively, and remains the largest category under strict budget\. Gemma\-4\-31B\-IT also exhibits substantial member\-length errors together with more duplicate edges\. The validator can detect these failures without dynamic inference\. Their primary difficulty is therefore converting coordinates, material\-specific length limits, and connectivity requirements into a legal graph topology\.

Strict budget shifts errors toward deterministic validation\. For Gemini\-3\-Flash\-Preview,EDGE\_TOO\_LONGandOVER\_BUDGETtogether account for 93\.5% of recorded errors, whereasRUN\_FALSE\_AFTER\_VALIDaccounts for only 4\.2%\. Gemma\-4\-31B\-IT and GPT\-5\.2 also produce substantial over\-budget errors\. The reduced share of physics failures does not indicate better dynamic reasoning; more candidates are blocked by length or cost constraints before they enter the game\. This selection effect helps explain why strict budget lowers both Valid@1 and Success@3\.

The bottleneck is therefore model dependent\. Some models are limited first by representation and constraint satisfaction, whereas others reliably enter simulation but fail to produce functional structures\. Validator feedback identifies the former errors directly, butRUN\_FALSE\_AFTER\_VALIDreveals only that execution was unsuccessful, not how the failure unfolded\. This decomposition localizes failures without isolating the causal value of any feedback component; temporal frames remain part of the standard post\-failure package rather than a separately varied intervention\.

### How Many Resources Does Success Require?

Figure[4](https://arxiv.org/html/2609.21493#Sx5.F4)analyzes the efficiency of successful designs along material and interaction dimensions\. The horizontal axis is Overall Success@3 across all 189 levels\. Material efficiency is the cost of a successful design divided by the strict budget for that level and then averaged over successful levels\. Interaction efficiency is the mean number of LLM calls among successful episodes\. Squares and hollow circles distinguish strict and loose budgets, and arrows connect the two settings for the same model\. Both vertical axes are conditioned on success, so failed levels and levels that never enter physics execution are not assigned zero cost\.

A lower\-right position combines broader benchmark success with lower resource use among successful episodes\. This is a conditional trade\-off, not an unconditional cost ranking, because models can succeed on subsets of different size and difficulty\.

More material does not guarantee functional success\. Under loose budget, Gemini\-3\-Flash\-Preview achieves 84\.7% Success@3 with a mean cost ratio of 1\.40\. GPT\-5\.2 uses more material but succeeds much less often\. Differences between models cannot be explained by material expenditure alone; structural organization still determines whether a design survives dynamic execution\.

Similar budget utilization can correspond to markedly different functional performance\. Under strict budget, the mean cost ratios of Claude\-Sonnet\-5 and Gemini\-3\-Flash\-Preview differ by only 0\.01, but their Success@3 scores differ by 25\.4 percentage points\. Budget compliance is necessary for an admissible design, but it does not remove differences in structural planning and dynamic load\-bearing ability\.

Interaction rounds indicate how strongly successful trajectories depend on later revision\. Successful Claude\-Sonnet\-5 episodes use 1\.3 rounds on average under both budget settings, suggesting that its strict\-budget successes do not primarily depend on repeated repair\. Gemini\-3\-Flash\-Preview requires one additional LLM call on average under strict budget than under loose budget, indicating greater reliance on later revisions\. Other models show related tendencies, but differences in the size and difficulty of their successful subsets prevent attributing round\-count differences solely to the budget constraint\.

All\-episode resource use remains necessary for a complete efficiency comparison because a model with low success may solve only easier levels and obtain a low success\-conditioned cost\. Model calls and physics attempts over all episodes must therefore be reported separately\. Overall, neither additional material nor additional repair rounds consistently corresponds to higher physical success\.

## Conclusion

PolyBridgeBench tests whether MLLMs transform visual scenes and engineering constraints into bridge topologies that survive native dynamic execution\. Deterministic checks and simulation distinguish rule compliance from physical functionality, while fixed budgets quantify repair\. Valid@k, Success@k, and Recovery@k separately measure validity, functionality, and post\-failure recovery\. It provides a reproducible, auditable test of structures that function physically rather than appear plausible or satisfy explicit rules\.

## References

- Alrashedyet al\.\(2025\)K\. Alrashedy, P\. Tambwekar, Z\. H\. Zaidi, M\. Langwasser, W\. Xu, and M\. GombolayGenerating CAD code with vision\-language models for 3d designs\.InInternational Conference on Learning Representations,Cited by:[Introduction](https://arxiv.org/html/2609.21493#Sx1.p3.1),[Constrained Structural Generation and Engineering Analysis](https://arxiv.org/html/2609.21493#Sx2.SSx2.p1.1)\.
- Bakhtinet al\.\(2019\)A\. Bakhtin, L\. van der Maaten, J\. Johnson, L\. Gustafson, and R\. GirshickPHYRE: a new benchmark for physical reasoning\.InAdvances in Neural Information Processing Systems,Vol\.32\.Cited by:[Introduction](https://arxiv.org/html/2609.21493#Sx1.p2.1),[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p2.1)\.
- Bearet al\.\(2021\)D\. M\. Bear, E\. Wang, D\. Mrowca, F\. J\. Binder, H\. F\. Tung, R\. T\. Pramod, C\. Holdaway, S\. Tao, K\. Smith, F\. Sun, L\. Fei\-Fei, N\. Kanwisher, J\. B\. Tenenbaum, D\. L\. K\. Yamins, and J\. E\. FanPhysion: evaluating physical prediction from vision in humans and machines\.InAdvances in Neural Information Processing Systems Datasets and Benchmarks Track,Cited by:[Introduction](https://arxiv.org/html/2609.21493#Sx1.p2.1),[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p2.1)\.
- Bisket al\.\(2020\)Y\. Bisk, R\. Zellers, R\. Le Bras, J\. Gao, and Y\. ChoiPIQA: reasoning about physical commonsense in natural language\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.34,pp\. 7432–7439\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v34i05.6239)Cited by:[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p2.1)\.
- Chenet al\.\(2024\)B\. Chen, Z\. Xu, S\. Kirmani, B\. Ichter, D\. Sadigh, L\. Guibas, and F\. XiaSpatialVLM: endowing vision\-language models with spatial reasoning capabilities\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 14455–14465\.Cited by:[Introduction](https://arxiv.org/html/2609.21493#Sx1.p2.1),[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p1.1)\.
- Cherianet al\.\(2024\)A\. Cherian, R\. Corcodel, S\. Jain, and D\. RomeresLLMPhy: complex physical reasoning using large language models and world models\.External Links:2411\.08027Cited by:[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p2.1)\.
- Chowet al\.\(2025\)W\. Chow, J\. Mao, B\. Li, D\. Seita, V\. C\. Guizilini, and Y\. WangPhysBench: benchmarking and enhancing vision\-language models for physical world understanding\.InInternational Conference on Learning Representations,Cited by:[Introduction](https://arxiv.org/html/2609.21493#Sx1.p2.1),[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p2.1)\.
- Duet al\.\(2026\)C\. Du, S\. Esser, S\. Nousias, and A\. BorrmannText2BIM: generating building models using a large language model\-based multiagent framework\.Journal of Computing in Civil Engineering40\(2\),pp\. 04025142\.External Links:[Document](https://dx.doi.org/10.1061/JCCEE5.CPENG-6386)Cited by:[Constrained Structural Generation and Engineering Analysis](https://arxiv.org/html/2609.21493#Sx2.SSx2.p1.1)\.
- Johnsonet al\.\(2017\)J\. Johnson, B\. Hariharan, L\. van der Maaten, L\. Fei\-Fei, C\. L\. Zitnick, and R\. GirshickCLEVR: a diagnostic dataset for compositional language and elementary visual reasoning\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition,pp\. 2901–2910\.Cited by:[Introduction](https://arxiv.org/html/2609.21493#Sx1.p2.1),[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p1.1)\.
- Liet al\.\(2024\)M\. Li, S\. Zhao, Q\. Wang, K\. Wang, Y\. Zhou, S\. Srivastava, C\. Gokmen, T\. Lee, L\. E\. Li, R\. Zhang, W\. Liu, P\. Liang, L\. Fei\-Fei, J\. Mao, and J\. WuEmbodied agent interface: benchmarking LLMs for embodied decision making\.InAdvances in Neural Information Processing Systems,Vol\.37\.Cited by:[Introduction](https://arxiv.org/html/2609.21493#Sx1.p3.1),[Executable Engineering Construction and Interactive Evaluation](https://arxiv.org/html/2609.21493#Sx2.SSx3.p1.1)\.
- Maharjan and Chun \(2026\)S\. Maharjan and P\. ChunA large language model\-driven framework for automated bridge specification generation and simulation validation\.Computer\-Aided Civil and Infrastructure Engineering41,pp\. 100014\.External Links:[Document](https://dx.doi.org/10.1016/j.cacaie.2026.100014)Cited by:[Introduction](https://arxiv.org/html/2609.21493#Sx1.p2.1),[Constrained Structural Generation and Engineering Analysis](https://arxiv.org/html/2609.21493#Sx2.SSx2.p1.1)\.
- Malliset al\.\(2025\)D\. Mallis, A\. S\. Karadeniz, S\. Cavada, D\. Rukhovich, N\. Foteinopoulou, K\. Cherenkova, A\. Kacem, and D\. AouadaCAD\-Assistant: tool\-augmented VLLMs as generic CAD task solvers\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 7284–7294\.Cited by:[Constrained Structural Generation and Engineering Analysis](https://arxiv.org/html/2609.21493#Sx2.SSx2.p1.1)\.
- Malliset al\.\(2026\)D\. Mallis, M\. Wang, A\. S\. Karadeniz, E\. Ricci, A\. Kacem, and D\. AouadaText\-to\-CAD evaluation with CADTests\.External Links:2605\.07807Cited by:[Introduction](https://arxiv.org/html/2609.21493#Sx1.p3.1),[Constrained Structural Generation and Engineering Analysis](https://arxiv.org/html/2609.21493#Sx2.SSx2.p1.1)\.
- Shiet al\.\(2022\)Z\. Shi, Q\. Zhang, and A\. LipaniStepGame: a new benchmark for robust multi\-hop spatial reasoning in texts\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.36,pp\. 11321–11329\.External Links:[Document](https://dx.doi.org/10.1609/aaai.v36i10.21383)Cited by:[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p1.1)\.
- Songet al\.\(2025\)C\. H\. Song, V\. Blukis, J\. Tremblay, S\. Tyree, Y\. Su, and S\. BirchfieldRoboSpatial: teaching spatial understanding to 2d and 3d vision\-language models for robotics\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 15768–15780\.Cited by:[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p1.1)\.
- Stogiannidiset al\.\(2025\)I\. Stogiannidis, S\. McDonagh, and S\. A\. TsaftarisMind the gap: benchmarking spatial reasoning in vision\-language models\.External Links:2503\.19707Cited by:[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p1.1)\.
- Suhret al\.\(2019\)A\. Suhr, S\. Zhou, A\. Zhang, I\. Zhang, H\. Bai, and Y\. ArtziA corpus for reasoning about natural language grounded in photographs\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,Florence, Italy,pp\. 6418–6428\.External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1644)Cited by:[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p1.1)\.
- Tanget al\.\(2025\)K\. Tang, J\. Gao, Y\. Zeng, H\. Duan, Y\. Sun, Z\. Xing, W\. Liu, K\. Lyu, and K\. ChenLEGO\-puzzles: how good are MLLMs at multi\-step spatial reasoning?\.External Links:2503\.19990Cited by:[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p1.1)\.
- Valmeekamet al\.\(2023\)K\. Valmeekam, M\. Marquez, A\. Olmo, S\. Sreedharan, and S\. KambhampatiPlanBench: an extensible benchmark for evaluating large language models on planning and reasoning about change\.InAdvances in Neural Information Processing Systems,Vol\.36\.Cited by:[Executable Engineering Construction and Interactive Evaluation](https://arxiv.org/html/2609.21493#Sx2.SSx3.p1.1)\.
- Wanget al\.\(2023\)Y\. Wang, J\. Duan, D\. Fox, and S\. SrinivasaNEWTON: are large language models capable of physical reasoning?\.InFindings of the Association for Computational Linguistics: EMNLP 2023,pp\. 9743–9758\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.652)Cited by:[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p2.1)\.
- Xiaet al\.\(2026\)T\. Xia, T\. Gao, W\. Deng, L\. Wei, X\. Qian, C\. Yu, and T\. WuBuildArena: a physics\-aligned interactive benchmark of LLMs for engineering construction\.InProceedings of the Forty\-Third International Conference on Machine Learning,Cited by:[Introduction](https://arxiv.org/html/2609.21493#Sx1.p2.1),[Executable Engineering Construction and Interactive Evaluation](https://arxiv.org/html/2609.21493#Sx2.SSx3.p1.1)\.
- Xuet al\.\(2025\)R\. Xu, D\. Lu, Z\. Zhao, X\. Tan, X\. Wang, S\. Yuan, J\. Chen, and Y\. XuORIGAMISPACE: benchmarking multimodal LLMs in multi\-step spatial reasoning with mathematical constraints\.InAdvances in Neural Information Processing Systems,Vol\.38\.Cited by:[Introduction](https://arxiv.org/html/2609.21493#Sx1.p2.1),[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p1.1)\.
- Yanget al\.\(2025\)J\. Yang, S\. Yang, A\. W\. Gupta, R\. Han, L\. Fei\-Fei, and S\. XieThinking in space: how multimodal large language models see, remember, and recall spaces\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 10632–10643\.Cited by:[Introduction](https://arxiv.org/html/2609.21493#Sx1.p2.1),[Multimodal Spatial and Physical Reasoning](https://arxiv.org/html/2609.21493#Sx2.SSx1.p1.1)\.
- Yanget al\.\(2026\)L\. Yang, Y\. Dai, A\. Yan, V\. Prabhu, R\. Xu, and Z\. ChenHow far are vision\-language models from constructing the real world? a benchmark for physical generative reasoning\.External Links:2603\.24866Cited by:[Introduction](https://arxiv.org/html/2609.21493#Sx1.p2.1),[Constrained Structural Generation and Engineering Analysis](https://arxiv.org/html/2609.21493#Sx2.SSx2.p1.1)\.
- Yaoet al\.\(2023\)S\. Yao, J\. Zhao, D\. Yu, N\. Du, I\. Shafran, K\. Narasimhan, and Y\. CaoReAct: synergizing reasoning and acting in language models\.InInternational Conference on Learning Representations,Cited by:[Executable Engineering Construction and Interactive Evaluation](https://arxiv.org/html/2609.21493#Sx2.SSx3.p1.1)\.

## Appendix ABenchmark Construction and Composition

This section documents how the 189 executable levels are constructed, which information is retained for analysis, and how the strict and loose material budgets are defined\. The benchmark contains two complementary evaluation strata\. The*Core Structural Suite*contains 149 levels from 21 parameterized structural families\. The*Compositional Challenge Suite*contains 40 levels from six families that combine clearance, material, load, vehicle, platform, and support constraints\. These strata are not an in\-distribution/out\-of\-distribution split: every evaluated model is run on every level\.

### A\.1Generation and Export Pipeline

Each family is defined by a deterministic generator and a finite parameter grid\. The grid varies bridge span, bank elevation, anchor layout, center terrain, admissible materials, vehicle count, and vehicle weight\. Once a level specification is fixed, the generator exports three synchronized artifacts:

- •<level\>\.json, the complete native level layout, including terrain, vehicles, targets, resources, and the encoded initial save;
- •<level\>\_inner\.json, the decoded initial editable topology containing the native kinematic anchors; and
- •<level\>\_context\.json, the task\-facing context and the analysis\-only metadata described in Section[A\.6](https://arxiv.org/html/2609.21493#A1.SS6)\.

The three files share the same stable level name\. The controller refuses to load an episode if any member of this triplet is absent or if the context does not contain materials, budget, and vehicle\-weight information\.

### A\.2Core Structural Suite

Tables[2](https://arxiv.org/html/2609.21493#A1.T2)and[3](https://arxiv.org/html/2609.21493#A1.T3)enumerate the 21 mutually exclusive Core families\. The family label is used only for aggregation\. Within a family, levels vary one or more continuous or discrete parameters while preserving the stated structural motif\. “RW” denotes a Road/Wood\-only material set; all other rows expose Road, Wood, and Steel\. Vehicle weight is expressed in the game’s passenger\-car weight unit, denoted PG in the released context\.

Table 2:Core Structural Suite, part I: basic connectivity and anchor\-layout families\. Budget ranges are the per\-level limits used in the strict\-budget condition\.Table 3:Core Structural Suite, part II: terrain, clearance, and dynamic\-load families\.
### A\.3Compositional Challenge Suite

The six Compositional families deliberately combine constraints that appear separately in the Core suite\. They are therefore reported as a second evaluation stratum rather than as evidence of training\-time distribution shift\. Table[4](https://arxiv.org/html/2609.21493#A1.T4)lists their composition\.

Table 4:The six families in the 40\-level Compositional Challenge Suite\.
### A\.4Strict and Loose Material Budgets

Material cost is computed from member length rather than from member count\. For a generated topologyGGwith membersmm, the controller computes

Cost⁡\(G\)=∑m∈Gcτ⁡\(m\)​‖pa⁡\(m\)−pb⁡\(m\)‖2,\\operatorname\{Cost\}\(G\)=\\sum\_\{m\\in G\}c\_\{\\tau\(m\)\}\\left\\lVert p\_\{a\(m\)\}\-p\_\{b\(m\)\}\\right\\rVert\_\{2\},\(5\)whereτ⁡\(m\)\\tau\(m\)is the material type,cτ⁡\(m\)c\_\{\\tau\(m\)\}is its per\-meter cost, anda⁡\(m\)a\(m\)andb⁡\(m\)b\(m\)are the member endpoints\. Costs are accumulated before rounding the reported total to two decimal places\.

The strict limitBiB\_\{i\}for each level was calibrated by manually loading that level into Poly Bridge, constructing and executing a successful reference bridge, and recording a level\-specific admissible budget\. Calibration was performed before model evaluation\. Only the scalar limit is exposed to a model; the reference topology is not part of the model input and is not used as an evaluation target\. The resulting 189 limits contain 26 distinct values and range from $6,500 to $42,000\. The strict condition rejects a candidate whenCost⁡\(G\)\>Bi\\operatorname\{Cost\}\(G\)\>B\_\{i\}\. The loose condition preserves the same level geometry and task constraints but uses a uniform $50,000 limit, making material cost unlikely to be the first gating failure\. The loose condition is diagnostic and does not replace the strict condition as the primary benchmark setting\.

### A\.5Clearance\-Region Calibration

The rendered rock sprite does not expose metric collision geometry\. We therefore define a deterministic triangular forbidden region from the stored sprite position\(xr,yb\)\(x\_\{r\},y\_\{b\}\)and scale\(w,h\)\(w,h\)\. Its vertices are

\(xr\+w,yb\+3\.5​h\),\(xr−6​w,yb\),\(xr\+6​w,yb\)\.\(x\_\{r\}\+w,\\;y\_\{b\}\+3\.5h\),\\qquad\(x\_\{r\}\-6w,\\;y\_\{b\}\),\\qquad\(x\_\{r\}\+6w,\\;y\_\{b\}\)\.\(6\)For the benchmark rock scale\(w,h\)=\(0\.5,2\.0\)\(w,h\)=\(0\.5,2\.0\), this gives a base extending 3 m to either side and an apex 7 m above the stored bottom coordinate, shifted 0\.5 m inxx\. The mapping is fixed before evaluation and is identical for all models\. The polygon, rather than the raw sprite scale, is supplied insemantic\_environment\.forbidden\_regionsand used by the deterministic clearance validator\.

### A\.6Information Boundary and Analysis Metadata

Table[5](https://arxiv.org/html/2609.21493#A1.T5)distinguishes task information from offline annotations\. The controller constructs the model\-facing level context from exactly two objects:task\_contextandsemantic\_environment\. The context retainslevelNameas an episode identifier, but it does not append the separateanalysis\_metadata\.family\_id, generator parameter dictionary, or level tags\. Family\-level and tag\-level breakdowns are computed only after an episode is complete\.

Table 5:Boundary between model\-visible task information and metadata retained for generation or offline analysis\.

## Appendix BTask Interface and Structured Representation

### B\.1Observation and Action Contract

At the first call, the model receives \(i\) an editor screenshot, \(ii\) the model\-visible level context from Table[5](https://arxiv.org/html/2609.21493#A1.T5), and \(iii\) the current decoded save containing the native anchors\. At a later call, the same fields are augmented with the previous candidate topology and the most recent deterministic or physical feedback\. After a validator failure, the exact invalid draft and validator message are returned\. After a physical failure, the current topology, a conservativefailure\_or\_timeoutsummary, and selected temporal frames are returned\. Each call must regenerate a complete topology; the model cannot issue an imperative action such as “add one beam” without returning the resulting full graph\.

The primary benchmark uses no manually authored or trajectory\-derived skill instructions\. Thus, the optional skill field in the controller is empty in the reported evaluation\.

### B\.2Node–Member–Material Graph

A candidate bridge is a graphG=\(V,M\)G=\(V,M\)\. Every nodev∈Vv\\in Vcontains an integer identifier, a two\-dimensional position\(x,y\)\(x,y\)in game\-world meters, and anisKinematicflag\. Native anchors are kinematic and must retain their original identifiers and coordinates\. Model\-created nodes are dynamic and must setisKinematicto false\. Every memberm∈Mm\\in Mcontains an integer identifier, a material type, the identifiers of its two endpoint nodes, and aratefield retained for save\-format compatibility\.

The benchmark levels reported in the paper expose the three materials in Table[6](https://arxiv.org/html/2609.21493#A2.T6)\. A particular level may restrict this set to Road and Wood\. Material strength values are supplied as design context, but the controller does not analytically predict capacity from them; native game execution remains the only functional test\.

Table 6:Material identifiers and deterministic limits in the released 189\-level benchmark\.The required model output is one raw JSON object with the following shape\. Thethought\_processfield is a brief decision summary used for trajectory audit; it is not treated as privileged model reasoning and is not scored\.

```
{
  "action": "modify",
  "thought_process":
    "brief decision summary",
  "nodes": [
    {"id": 1, "x": 0.0, "y": 0.0,
     "isKinematic": true},
    {"id": 2, "x": 2.0, "y": 0.0,
     "isKinematic": false}
  ],
  "edges": [
    {"id": 3, "type": 1,
     "anchorAID": 1, "anchorBID": 2,
     "rate": 0}
  ]
}
```

Node coordinates are rounded to four decimal places during conversion\. A node becomes a save object withtype=0\. A member becomes a save object whosetypeis the material identifier and whose stored\(x,y\)\(x,y\)coordinate is the midpoint of its endpoints\. The endpoint IDs remain inanchorAIDandanchorBID; consequently, the graph can be validated before it is serialized into the native save format\.

### B\.3Model\-Visible Context Schema

The following abridged example illustrates the exact nesting of the model\-facing context\. Fields with empty lists remain present so that all models receive the same schema\.

```
{
  "task_context": {
    "levelName": "<episode identifier>",
    "available_materials":
      ["Road", "Wood", "Steel"],
    "budget_limit": 13000,
    "vehicle_count": 2,
    "vehicle_weights_pg": [3.0, 3.0],
    "vehicle_weight_pg": 3.0,
    "material_profile": "full"
  },
  "semantic_environment": {
    "terrain_affordances": [
      {
        "type": "center_platform",
        "shape": "rectangular_terrain",
        "x_start": 6.0,
        "x_end": 10.0,
        "surface_y": 2.0,
        "bottom_y": -5.0,
        "drivable": true,
        "support_anchor_ids": [3, 4]
      }
    ],
    "forbidden_regions": [
      {
        "type": "rock",
        "shape": "triangle",
        "vertices": [
          {"x": 5.0, "y": 2.0},
          {"x": 1.5, "y": -5.0},
          {"x": 7.5, "y": -5.0}
        ],
        "applies_to":
          ["Road", "Wood",
           "Steel", "Hydraulics"]
      }
    ]
  }
}
```

The structured context is authoritative for task constraints\. The image is used to ground the visible scene and current editor state, while the decoded save is authoritative for exact node identifiers and coordinates\. When sources appear inconsistent, the prompt instructs the model to follow the structured constraint and save representations rather than infer hidden generator parameters\.

### B\.4Prompt Specification

The static instruction below is a line\-wrapped transcription of the prompt used for the primary benchmark\. Line wrapping is changed for typesetting; the wording and operative constraints are preserved\. The concrete JSON example embedded in the runtime prompt follows the schema already shown in Section[B\.2](https://arxiv.org/html/2609.21493#A2.SS2)\.

```
# Role
You are an expert Structural Engineer and Level
Designer
for Poly Bridge. Generate or repair a complete bridge
topology from image + save code + level context.

Use engineering heuristics with material stats/costs;
native game execution alone determines physical
success.
Some levels may contain 2 or 3 vehicles. Read
task_context.vehicle_count and
task_context.vehicle_weights_pg; design for every
vehicle.

Level context is intentionally scoped to task_context
and
semantic_environment; raw generator parameters are not
part of the task.

If semantic_environment.terrain_affordances includes
center platforms/islands, use the screenshot and
kinematic anchor
positions to distinguish unsupported gaps from solid
drivable platform terrain. The vehicle path must be
continuous, but road edges are only necessary over
unsupported gaps unless a structural reason requires
them.

If semantic_environment.forbidden_regions is
non-empty,
keep Road/Wood/Steel/Hydraulics geometry outside those
regions. Treat forbidden-region vertices as the
geometry
used by the controller validator.

When temporal physics frames are provided, compare
their
progression and use observed deformation, collapse, or
vehicle obstruction to guide repair; treat causal
explanations as hypotheses. failure_or_timeout means
the controller did not confirm success within its
fixed window;
it does not prove an exact failure cause.

# Material Encyclopedia (for reasoning)
WARNING: You cannot use all materials every level.
Only materials in task_context.available_materials are
legal.
- Road (Type 1): strength 900PN, max length 2m,
  cost $200/m.
- Wood (Type 2): strength 800PN, max length 2m,
  cost $180/m.
- Steel (Type 3): strength 2000PN, max length 4m,
  cost $450/m.

# Generation Protocols (Strict Rules)
1. Nodes and edges
- Preserve every original isKinematic=true base node
  from
  Current Save Code with the same id and coordinates.
- New suspended nodes must set isKinematic=false.
  Never
  invent a new isKinematic=true node.
- IDs for newly added nodes must continue
  sequentially from
  the highest existing ID in Current Save Code.
- Edge type is the material identifier. Never use
  type 0
  for an edge; type 0 is reserved for nodes.

2. Strict length limits
- Calculate Euclidean endpoint distance.
- Road/Wood <= 2.0m; Steel <= 4.0m.
- If a limit is exceeded, insert an intermediate node.
- Do not duplicate the same material between the same
  two
  nodes; repeated identical edges are invalid.

3. Evidence roles and constraints
- task_context and semantic_environment are
  authoritative
  for structured constraints.
- Save Code is authoritative for current topology and
  exact
  node geometry.
- Validation Feedback is authoritative for
  deterministic
  violations in the previous draft.
- Use editor images for scene/current-state grounding
  and
  temporal frames as execution evidence for repair.
- Treat visual causal inferences as hypotheses and
  discard
  conflicting History.
- Use only materials in
  task_context.available_materials.
- If task_context.budget_limit is provided, keep
  estimated
  material cost <= that limit.
- If validation or physics feedback lists errors,
  repair
  them in the returned topology.
- The controller automatically validates and runs
  every
  legal topology.

# Output Format (STRICT)
Return one valid raw JSON object only: no Markdown
block or extra text. Always return action="modify".
The required
fields are thought_process, nodes, and edges.

thought_process is a brief decision summary, not a
step-by-step derivation. For an initial generation,
state the structural concept and how loads reach the
original anchors; if forbidden regions exist, state
the clearance route. For repair, state the observed
validation or physics evidence, mark uncertain
physical causes as hypotheses, and
name the concrete topology changes.
```

For every call, the user message inserts the currenttask\_context/semantic\_environmentJSON and decoded save\. Depending on the previous outcome, it then appends one of the following:

- •the exact validator message and invalid draft topology;
- •a compact physical\-execution summary and the selected temporal evidence; or
- •neither, for the initial call\.

The message ends with labeled visual inputs\. The labeleditor\_staticidentifies the current editable state;physics\_t\_<time\>identifies an individual frame; andphysics\_contact\_sheet\_<index\>identifies a timestamped frame grid\. In the primary setting, four uniformly sampled frames are packaged as one contact sheet\.

## Appendix CDeterministic Validation and Native Execution

### C\.1Deterministic Legality Checks

The validator defines the legal action space before native execution\. It is not a bridge generator, structural optimizer, or physics oracle\. It can reject a graph for a factual schema, geometry, connectivity, material, clearance, or cost violation, but it does not estimate load capacity, stability, deformation, or successful vehicle traversal\.

Table[7](https://arxiv.org/html/2609.21493#A3.T7)lists the hard checks\. All applicable violations found in a candidate are returned together\. A rejected candidate consumes its LLM call, is never written into the live game save, and consumes no physics attempt\.

Table 7:Deterministic checks applied before native physics execution\.For connectivity, the implementation constructs a disjoint\-set forest over all nodes and unions the endpoints of every referenced material member\. A dynamic node is valid only if its component root matches the root of at least one native kinematic anchor\. Clearance uses standard point\-in\-polygon and segment–polygon intersection tests; nodes or segments on the polygon boundary are treated as violations\.

### C\.2Feedback Taxonomy

Validator messages are retained verbatim for the next repair call and are also normalized into the audit codes in Table[8](https://arxiv.org/html/2609.21493#A3.T8)\. Normalization is used for aggregate failure analysis; it does not replace the original factual feedback shown to the model\. A candidate can contribute multiple deterministic codes\. By contrast,RUN\_FALSE\_AFTER\_VALIDmarks an unsuccessful native execution after all deterministic checks passed and is not a validator violation\.

Table 8:Normalized error taxonomy used in the diagnostic analysis\.
### C\.3Save Serialization and Game Loading

Once a candidate passes validation, the controller saves the decodedDisplayName/ObjectsJSON for audit\. It then serializes the compact JSON to UTF\-8, applies zlib compression, and Base64\-encodes the result using the game’s save representation\. The encoded payload replaces the first entry of the target level’ssaveFileslist; all other level metadata is preserved\. The game\-loading helper then reloads this layout in the editor\. This conversion is deterministic and does not alter the graph selected by the model\.

### C\.4Native Physics Execution and Success Detection

Table[9](https://arxiv.org/html/2609.21493#A3.T9)gives the primary execution settings\. The controller records an editor\-state frame immediately before starting the simulation, presses the native run control, and samples the screen at 4 fps for a fixed 12\-second window\. The final screenshot is inspected at a fixed interface pixel\. A green\-channel value above the threshold corresponds to the game’s success state, which is reached only when the level’s required vehicle traversal completes\. Otherwise the controller recordsfailure\_or\_timeout\. This label deliberately makes no claim about whether the bridge collapsed, blocked a vehicle, or merely failed to finish within the observation window\.

Table 9:Controller and native\-execution settings for the primary benchmark\.
### C\.5Temporal Evidence and Repair Loop

For an unsuccessful execution, the controller first removes repeated frames after the game displays its terminal failure marker\. It scans the captured sequence chronologically and, if the red channel at pixel\(1145,637\)\(1145,637\)exceeds 150, retains frames only through the first marked frame\. If no marker is detected, the full 12\-second sequence remains eligible; this preserves timeout evidence rather than imposing an inferred failure time\.

Four frames are then selected at uniformly spaced indices from the eligible sequence, including its endpoints\. Each frame is annotated with its frame index and elapsed time and placed in a single contact sheet\. The next model call receives this sheet together with \(i\) a new editor screenshot after returning from simulation, \(ii\) the full current topology, and \(iii\) a compact summary containing the outcome, duration, detector, selected timestamps, and cutoff metadata\. The prompt explicitly states that causal explanations based on the frames are hypotheses\.

The overall episode controller is summarized below\. LLM calls and physics attempts have separate counters: validator rejection advances only the former, whereas every valid candidate advances both before native execution\.

```
for call = 1, 2, 3:
    observation <- scene + context + current topology
                   + latest available feedback
    candidate <- MLLM(observation)

    if candidate cannot be parsed or converted:
        return deterministic feedback on next call
        continue

    violations, usage <-
        deterministic_validator(candidate)
    if violations are non-empty:
        return violations and invalid draft
        on next call
        continue

    write candidate to the native game save
    result, raw_frames <- execute for up to 12 seconds
    if result is success:
        terminate episode as successful

    if no call or physics budget remains:
        terminate episode as unsuccessful

    frames <- cutoff terminal repeats, then sample 4
    feedback <- failure_or_timeout summary
                + contact sheet
terminate when the call budget is exhausted
```

For audit and replay, the controller retains the model response, parsed topology, decoded design JSON, encoded save payload, validator feedback, material\-usage report, raw frames, selected\-frame metadata, contact sheet, per\-round timing, and cumulative resource counters\. These records support the layered validity, functionality, recovery, error, and efficiency analyses without treating model\-written explanations as ground\-truth failure causes\.

## Appendix DEvaluation and Reproducibility Details

### D\.1Evaluation Unit and Interaction Accounting

The atomic evaluation unit is one model–level–budget\-condition episode\. A paper\-level result contains one episode for each of the 189 levels under the specified model and budget condition\. The interaction horizon is indexed by LLM calls rather than by physics attempts\. Callttis thett\-th complete topology generation or repair request, regardless of whether its output passes validation\. A validator rejection therefore advancesttbut not the physics\-attempt counter\. A candidate that passes validation advances both counters when it is executed\. In the released controller,rounds\_usedandllm\_calls\_usedconsequently have the same value\.

Each episode has at most three LLM calls and three physics attempts\. The two limits are enforced independently\. For example, two validator\-invalid outputs followed by one valid execution consume three calls but only one physics attempt\. Conversely, three valid but unsuccessful candidates consume the full allowance of both resources\. An episode terminates immediately on confirmed physical success or when no LLM call remains; the physics limit is also checked after every execution\.

### D\.2Metric Edge Cases

The main\-paper definitions of Valid@kk, Success@kk, and Recovery@kkare implemented directly from the round\-aligned trajectory\. A round is valid when it produces a candidate admitted to native execution\. A round is successful only when that execution returns confirmed success\. Table[10](https://arxiv.org/html/2609.21493#A4.T10)makes the accounting explicit for common edge cases\.

Table 10:Call\-, physics\-, validity\-, and recovery\-accounting rules\.#### Valid@kk\.

An episode contributes one if at least one candidate from calls1,…,k1,\\ldots,kenters physics\. Parsing and validator failures contribute zero\. The main results usek∈\{1,3\}k\\in\\\{1,3\\\}; call\-2 values can be derived from the same trajectory for cumulative curves\.

#### Success@kk\.

An episode contributes one if a physics execution associated with a call no later thankksucceeds\. Success@1 is therefore strict one\-call end\-to\-end success: the first response must be parseable, legal, and physically functional\. It differs from*first\-physics\-attempt success*\. If call 1 is validator\-invalid and call 2 is the first candidate to enter physics and succeed, first\-physics\-attempt success is true but Success@1 is false\.

#### Recovery@3\.

A level is eligible only if it experiences an unsuccessful physical execution before call 3 and before physics attempt 3, so that at least one call and one physics attempt remain\. Recovery is computed per level, not per failure event\. LetFFdenote an unsuccessful physical execution,SSa successful one, andVVa validator rejection\. ThenF→SF\\\!\\rightarrow\\\!SandF→F→SF\\\!\\rightarrow\\\!F\\\!\\rightarrow\\\!Seach count as one successful recovery;V→SV\\\!\\rightarrow\\\!Sis not a physical recovery; and anFFon call 3 is not eligible\. The denominator is model\- and condition\-dependent and must be reported together with the numerator\. Recovery@3 is consequently a conditional diagnostic, not another aggregate success rate\.

#### Failure and error counts\.

Result statuses are mutually exclusive at the episode level, but validator error codes are multi\-label at the candidate level\. One rejected candidate can therefore contribute bothEDGE\_TOO\_LONGandOVER\_BUDGET\. Normalized error\-composition figures divide by the number of recorded error events for that model and condition; they do not estimate the fraction of failed levels\. Absolute event counts and the number of levels reaching physics are retained alongside normalized shares\.

### D\.3Resource and Efficiency Metrics

Resource use is reported with both an all\-episode denominator and a success\-conditioned denominator\. For episodeii, letCiC\_\{i\}be the number of LLM calls,AiA\_\{i\}the number of physics attempts,YiY\_\{i\}the final\-success indicator,MiM\_\{i\}the cost of its successful design whenYi=1Y\_\{i\}=1, andBiB\_\{i\}the strict calibrated budget for that level\. We report

AvgCallsall\\displaystyle\\mathrm\{AvgCalls\}\_\{\\mathrm\{all\}\}=1N​∑i=1NCi,\\displaystyle=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}C\_\{i\},AvgAttemptsall\\displaystyle\\mathrm\{AvgAttempts\}\_\{\\mathrm\{all\}\}=1N​∑i=1NAi,\\displaystyle=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}A\_\{i\},\(7\)AvgCallssucc\\displaystyle\\mathrm\{AvgCalls\}\_\{\\mathrm\{succ\}\}=∑iYi​Ci∑iYi,\\displaystyle=\\frac\{\\sum\_\{i\}Y\_\{i\}C\_\{i\}\}\{\\sum\_\{i\}Y\_\{i\}\},AvgAttemptssucc\\displaystyle\\mathrm\{AvgAttempts\}\_\{\\mathrm\{succ\}\}=∑iYi​Ai∑iYi,\\displaystyle=\\frac\{\\sum\_\{i\}Y\_\{i\}A\_\{i\}\}\{\\sum\_\{i\}Y\_\{i\}\},\(8\)BudgetUtilizationsucc\\displaystyle\\mathrm\{BudgetUtilization\}\_\{\\mathrm\{succ\}\}=1∑iYi∑i:Yi=1MiBi\.\\displaystyle=\\frac\{1\}\{\\sum\_\{i\}Y\_\{i\}\}\\sum\_\{i:Y\_\{i\}=1\}\\frac\{M\_\{i\}\}\{B\_\{i\}\}\.\(9\)Failed episodes contribute their actual consumed calls and attempts to the all\-episode means\. In particular, an episode that never passes validation contributes zero physics attempts rather than a missing value\. Material cost is success\-conditioned because an unsuccessful structure has no meaningful functional\-efficiency interpretation\. The successful subsets can differ in size and difficulty across models; material\-efficiency plots are therefore interpreted jointly with Success@3 rather than as standalone rankings\. A common\-success subset may additionally be released for paired cost comparisons, but it is not substituted for the full benchmark result\.

Validation\-repair calls count rounds ending in a parsing, geometry, or deterministic validation failure\. Physics\-repair calls count model requests that receive feedback from a previous native execution\. These quantities are separate because the former spend calls without entering physics, whereas the latter spend both forms of test\-time interaction if their repaired topology is valid\.

### D\.4Suite, Family, and Tag Aggregation

Overall metrics use all 189 levels\. Core metrics use the 149 non\-G families, and Compositional metrics use the 40 G\-prefixed levels\. The storedfamily\_idassigns each level to exactly one of the 27 families and supports mutually exclusive family\-level comparisons\. By contrast,level\_tagssuch asslope,multi\_vehicle,rock\_clearance, androad\_wood\_onlyoverlap\. Tag\-level breakdowns describe performance conditional on a constraint and must not be summed to reconstruct suite totals\. Neither family IDs nor tags are used as evaluation targets\.

### D\.5Models and Request Controls

All models receive the task contract in Appendix[B](https://arxiv.org/html/2609.21493#A2)and the same image ordering\. Requests are made through an OpenAI\-compatible gateway with SDK\-level automatic retries disabled; the experiment wrapper records each explicit HTTP attempt\. Table[11](https://arxiv.org/html/2609.21493#A4.T11)documents the model aliases and request controls used by the current benchmark harness\. “Provider default” means that no thinking intensity or binary override is sent\. For Qwen and Kimi the gateway supports a binary reasoning switch, but the primary setting leaves it unset\. Gemma exposes no compatible reasoning control through this interface\.

Table 11:Model identifiers and request controls\. Provider\-default output limits are not replaced by a common artificial cap\. Access timestamps and the resolved request configuration are retained per request\.The common transport configuration uses a 10\-second connect timeout, 60\-second write timeout, 30\-second pool timeout, and observable exponential backoff from 2 to 10 seconds\. Claude\-Sonnet\-5 receives a longer read timeout and fewer attempts because a single generation can exceed the default 300\-second window; other models retain the common policy\. Transport retries do not increase the benchmark’s LLM\-call counter: the counter advances once per logical model request, while HTTP\-attempt telemetry is reported separately\.

Model aliases can be mutable at the provider\. We therefore define the access time as the recorded request\-tracestarted\_attimestamp rather than a filesystem modification time\. Each released request trace contains the model identifier, gateway endpoint, start and completion timestamps, temperature, maximum\-output setting, reasoning\-control keys, timeout policy, HTTP\-attempt outcomes, latency, and token usage when returned by the provider\. Credentials are never included\. A release\-level manifest maps every table row to its constituent session IDs and access timestamps, allowing future changes behind an alias to be distinguished from the evaluated snapshot\.

### D\.6Frozen Run Inclusion Rule

The repository contains pilot and debugging sessions in addition to paper runs\. Paper aggregation is therefore defined by configuration fields, not by placing every file under one directory into the denominator\. A primary\-run episode is eligible for aggregation only when its frozen session manifest matches all of the following:

- •skill\_mode=no\_skill;
- •at most three LLM calls and three physics attempts;
- •a 12\-second simulation window sampled at 4 fps;
- •uniform selection of four frames, packaged as a contact sheet;
- •the failure\-marker cutoff specified in Section[C\.5](https://arxiv.org/html/2609.21493#A3.SS5);
- •the common prompt and deterministic validator versions; and
- •the declared strict or loose budget map for the condition\.

Sessions generated with earlier execution windows, alternative feedback modes, partial level subsets, ad hoc prompts, or skill injection are pilot runs and are excluded\. Before aggregation, the report generator verifies one record per model–level–condition key, the expected level count, and the presence of the associated trajectory\. This rule prevents an incomplete rerun or a copied session directory from silently changing a paper result\.

### D\.7Recorded Artifacts and Metric Reconstruction

The experiment output is hierarchical\. A session\-levelsummary\.jsonrecords the frozen controller configuration and a copy of all level results\. Each level directory containsresult\.jsonandtrajectory\.json\. The former stores final outcome, budgets, calls, physics attempts, material usage, timestamps, and aggregated HTTP telemetry; the latter stores one record per logical LLM call, including the response, parsed topology, feedback status, evidence metadata, and per\-phase timing\. Round directories retain decoded designs, encoded saves, editor screenshots, raw physics frames, and repair contact sheets\.

All headline metrics can be reconstructed fromtrajectory\.jsonwithout trusting a precomputed table\. The report script restricts events to the three\-call horizon, derives valid and successful call indices, derives the Recovery@3 eligibility and success flags, then aggregates the resulting level records\. Model\-authoredthought\_processstrings are not used to decide validity, success, recovery, cost, or error category\.

Table 12:Strict\-condition performance by coarse capability group\. V@3 and S@3 denote Valid@3 and Success@3; all entries are percentages\. Group sizes are shown in the headers so that each percentage has an explicit denominator\.Table 13:Recovery and resource accounting under strict and loose conditions\. Callsallaverages logical LLM calls over all 189 episodes, including failures\. Costsuccis the mean realized material cost over successful episodes, in thousands of dollars; the parenthesized value is the number of successful episodes contributing to that mean\.

## Appendix EAdditional Quantitative Results

This section reports two compact diagnostics beyond the suite\-level results in the main paper\. To avoid unstable percentages from families containing only two to six levels, Table[12](https://arxiv.org/html/2609.21493#A4.T12)aggregates the 27 families into four mutually exclusive, pre\-defined capability groups\. The strict condition is used because it is the primary benchmark setting\. Model names are shortened in the tables; their exact API identifiers appear in Table[11](https://arxiv.org/html/2609.21493#A4.T11)\.

### E\.1Coarse\-Grained Capability Breakdown

The four groups are Anchor and Connectivity \(T1B–T1H\), Terrain and Geometry \(T2A–T2K\), Dynamic Load \(T4B–T4D\), and Compositional Constraints \(G1–G6\)\. Their sample sizes are 59, 64, 26, and 40 levels, respectively\. These groups are used only for descriptive diagnosis; they are not additional test splits and are not used to claim statistical significance\.

Claude has the highest Success@3 in every group, while Gemini is the strongest remaining model on Anchor and Connectivity, Terrain and Geometry, and Compositional Constraints\. The breakdown also sharpens the validity–success gap\. For example, Qwen reaches 73\.1% Valid@3 on Dynamic Load and 85\.0% on Compositional Constraints, but records no physical success in either group\. Likewise, Gemma reaches 62\.5% validity on the Compositional group without a successful traversal\. Passing deterministic checks therefore remains insufficient evidence of dynamic functionality across qualitatively different structural conditions\.

### E\.2Recovery and Resource Use

Table[13](https://arxiv.org/html/2609.21493#A4.T13)reports the Recovery@3 numerator and denominator, LLM calls averaged over all 189 episodes, and the realized material cost averaged over successful episodes\. A cost cell has the form “mean cost in thousands of dollars \(number of successful episodes\)\.” It reports the material used by the successful design, not the level’s initial budget limit and not a cost\-to\-limit ratio\.

Recovery must be interpreted with its conditional denominator\. Claude recovers on 20 of 91 eligible strict episodes and 31 of 62 eligible loose episodes, whereas Qwen’s loose rate is based on a much larger set of 148 eligible episodes\. Callsallincludes failed episodes that exhaust the three\-call allowance, so it complements the success\-conditioned interaction measure in the main paper\. Costsuccis also conditional: Qwen’s low strict mean, for example, is based on only three successful levels and must not be read as a standalone efficiency ranking\. Per\-level records and trajectories are included in the artifact inventory in Appendix[G\.3](https://arxiv.org/html/2609.21493#A7.SS3)\.

## Appendix FPaired Three\-Round Repair Traces

This section makes the trajectory\-level meaning of Recovery@3 concrete with two deliberately selected Claude\-Sonnet\-5 traces\. Both come from the same strict\-condition session and the same G1 rock\-clearance family, use a 14 m span with Road, Wood, and Steel available, and consume three model calls and three physics attempts\. Every candidate in both traces passes deterministic validation and enters the native simulator, so neither trajectory contains a validator\-repair round\. The examples are illustrative rather than representative: they contrast one recovered episode with one episode that exhausts the interaction budget, and they do not estimate the marginal causal effect of temporal frames\.

### F\.1Paired Case Context

The successful trace isG1\_Clearance\_Multi2\_14m\_R2m, which contains two vehicles with weight 3\.0 each\. The unsuccessful trace isG1\_Clearance\_HeavyW3\_14m\_R2m, which contains one vehicle with weight 9\.0\. Table[14](https://arxiv.org/html/2609.21493#A6.T14)summarizes how the two episodes enter the Recovery@3 accounting from Section[D\.2](https://arxiv.org/html/2609.21493#A4.SS2)\. Here,FFdenotes a valid topology followed byfailure\_or\_timeout, andSSdenotes simulator\-confirmed success\.

Table 14:Overview of the paired three\-round traces\. Both episodes are Recovery@3\-eligible after their first physical failure\. Multi2 contributes to both the numerator and denominator, whereas HeavyW3 contributes only to the denominator\.
Figure[5](https://arxiv.org/html/2609.21493#A6.F5)pairs the initial state of every executed candidate with a diagnostic or terminal frame from the corresponding rollout\. The edit labels summarize the stored topology difference and the model\-written decision summary\. They describe the action taken by the model, not a verified ground\-truth diagnosis of the physical failure\.

![Refer to caption](https://arxiv.org/html/2609.21493v1/Figures/appendix_g_paired_traces.png)Figure 5:Two three\-round repair traces under the standard feedback package\. The upper row shows Multi2 progressing from two valid physical failures to simulator\-confirmed success\. The lower row shows HeavyW3 changing failure signatures across two redesigns but ending in terminal collapse\. Each panel pairs the candidate att=0t=0with selected temporal evidence or the terminal frame\. Action labels are reconstructed from stored design differences and model\-authored decision summaries; they are not causal annotations\.
### F\.2Successful Trace: Progressive Strengthening

Multi2 begins with a fixed 20\-node, 37\-edge geometry containing ten Road segments and a non\-road web of 21 Wood and six Steel members\. The first rollout remains visually coherent for several seconds before a late failure near the right side\. The model treats a left–right strength asymmetry as a hypothesis and changes six apex and right\-side Wood members to Steel\. The second candidate retains every node position and edge endpoint, now with 15 Wood and 12 Steel web members, but produces a similar late physical failure\.

For the third call, the model again preserves the complete geometry and upgrades the remaining fifteen Wood chord and vertical members\. The resulting candidate has ten Road and 27 Steel members and carries both vehicles to their targets\. ThisF→F→SF\\\!\\rightarrow\\\!F\\\!\\rightarrow\\\!Strajectory is therefore a successful Recovery@3 case: the episode recovers under the complete feedback package described in Section[C\.5](https://arxiv.org/html/2609.21493#A3.SS5)\. It does not by itself show that the model’s stated explanation was correct or that temporal frames alone caused the recovery\.

### F\.3Failed Trace: Non\-Monotonic Redesign

HeavyW3 initially uses a 20\-node, 37\-edge design whose ten Road segments are supported by 27 Steel members\. The first rollout visibly collapses by approximately 3\.26 s\. In response, the model changes the suspended\-node geometry and expands the structure to 45 edges, producing a deeper and more densely cross\-braced Steel truss\. The redesigned bridge remains visibly intact through the 12 s window, but the vehicle remains near the left approach and the controller does not confirm task completion\.

The model then hypothesizes that the steep entry profile, rather than global structural instability, prevented completion\. Its third candidate replaces the road profile with a gentler arch and reduces the topology to 37 edges\. The final execution collapses and terminates in failure, exhausting both the three\-call and three\-physics\-attempt allowances\. Thus the feedback changes the observed failure signature and the model’s repair direction, but the edits are not monotonically improving\. Together with the successful trace, this case illustrates why deterministic validity, physical success, and recovery are reported separately, while retaining the causal qualifications in Appendix[G\.1](https://arxiv.org/html/2609.21493#A7.SS1)\.

## Appendix GExtended Limitations and Artifact Documentation

### G\.1Claim Scope

PolyBridgeBench evaluates executable structural synthesis in a controlled two\-dimensional game environment\. It is not a civil\-engineering design or certification system\. The benchmark supports the claim that a generated graph satisfies the stated game constraints and succeeds under the native simulator; it does not establish real\-world safety, code compliance, fatigue life, constructability, or reliability under unmodeled loads\. The central scientific object is the gap between deterministic rule compliance and dynamic functionality, with bridge construction serving as a compact, falsifiable testbed\.

The standard post\-failure observation is a package containing the failure status, current topology, editor image, temporal contact sheet, and interaction history\. Recovery@3 measures recovery under this complete package\. Unless a paired feedback ablation is reported, it does not identify the marginal causal effect of temporal frames, nor does a model\-written failure explanation prove that the model inferred the physical cause correctly\.

### G\.2Threats to Validity

Table 15:Primary threats to validity, controls, and residual limitations\.#### Coverage and generalization\.

The 189 levels cover span, slope, bank asymmetry, bottom anchors, center platforms, clearance, material restriction, multiple vehicles, and elevated loads, but they remain a finite sample from hand\-designed generator families\. The Compositional Challenge Suite combines known constraint types; it is not a training/test OOD split\. Family\-level results diagnose robustness across parameterized configurations and should not be described as proof that a model learned a transferable engineering strategy\.

#### Automation and detector dependence\.

The controller interacts with a desktop game through save\-file replacement, keyboard/mouse automation, screenshots, and fixed\-pixel UI detection\. This choice preserves native physics but introduces dependencies on game version, screen resolution, interface layout, input focus, and rendering latency\. A 12\-second window makes the criterion reproducible but merges slow completion with physical failure\. Raw frames and detector pixels are therefore retained so questionable episodes can be audited without relying only on the final Boolean result\.

#### Budget and clearance calibration\.

Strict budgets are admissibility thresholds derived from successful manual designs, not certified optima\. A model can construct a cheaper valid bridge, and no geometric similarity to the reference is required\. Similarly, the rock triangle is an explicit benchmark convention\. It makes clearance deterministic but does not claim pixel\-perfect equivalence to the simulator’s internal collision shape\. Future versions could replace it with exported collision geometry and evaluate sensitivity to the polygon boundary\.

#### Provider and sampling variance\.

The primary table reports one trajectory for each model–level–condition key\. It therefore measures the realized behavior of the accessed model snapshot under a fixed low\-temperature protocol, not the expectation over repeated stochastic samples\. Access timestamps and raw responses make the realization auditable, but confidence intervals would require repeated complete runs\. In addition, provider\-default reasoning and output limits are not necessarily matched in internal compute across model families\.

### G\.3Artifact Inventory

The intended anonymous artifact separates source, benchmark definitions, and generated run data as shown in Table[16](https://arxiv.org/html/2609.21493#A7.T16)\. Paths are repository\-relative\. Local game paths, API credentials, caches, and unrelated pilot outputs are excluded\.

Table 16:Artifact components and their roles\.
### G\.4Release Manifest and Integrity Checks

The artifact root includes a machine\-readable manifest containing the source commit, benchmark and budget\-map hashes, prompt and validator hashes, game version, operating system, display resolution, Python and package versions, model aliases, gateway endpoints, access timestamps, and the session IDs used for every reported model–condition row\. Each session entry records the expected number of levels and the frozen configuration predicate from Section[D\.6](https://arxiv.org/html/2609.21493#A4.SS6)\. File hashes cover the per\-level result and trajectory files; raw images may be packaged in a separate archive with its own checksum\.

The manifest is considered valid only when it contains no duplicate model–level–condition keys, missing benchmark levels, missing trajectories, inconsistent budget maps, or sessions whose call limit, physics limit, execution window, feedback mode, or skill mode differs from the declared paper protocol\. Aggregate CSV and LaTeX tables are treated as derived artifacts: the JSON trajectories, not manually edited tables, are the source of truth\.

Secrets and machine\-specific configuration are removed before release\. Specifically,\.env, API keys, local Poly Bridge layout paths, and provider authentication headers are never included\. The benchmark does not require personal data or human\-subject annotations\. Poly Bridge itself is a proprietary dependency and is not redistributed; reproducing native execution requires a legally obtained compatible installation\. Generated structured levels, prompts, controller code, metrics, and non\-proprietary audit metadata can be released independently subject to the final artifact license\.