Object Aligner: A Configurable JSON Schema Similarity Score for Graphs, Applied to LLM Prompt Optimization

# Object Aligner: A Configurable JSON Schema Similarity Score for Graphs, Applied to LLM Prompt Optimization
Source: [https://arxiv.org/html/2607.01972](https://arxiv.org/html/2607.01972)
Jan DrchalJ\. Drchal is with the Artificial Intelligence Center, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic \(e\-mail: drchajan@fel\.cvut\.cz\)\.This work was created with the state support of the Technology Agency of the Czech Republic within the Sigma Programme, Project No\. TQ01000100\.Preprint\. This is a submitted version of a manuscript under review at IEEE Access; it has not been peer reviewed\.

###### Abstract

Large language models \(LLMs\) are often asked to produce JSON conforming to a fixed schema, powering information extraction, tool calling, agentic planning, and knowledge\-graph construction\. Measuring how closely an output matches a gold reference is essential yet surprisingly hard: exact match is brittle, text similarity ignores structure, and an LLM judge is expensive, opaque, and non\-deterministic\. We address this with*Object Aligner*\(OA\), an open\-source Python library that scores two JSON objects deterministically by recursively aligning their trees \(the Hungarian algorithm for unordered collections, sequence alignment for ordered ones\) and awarding partial credit at the granularity the schema declares\. TheObject Aligneris configured entirely through a set of JSON Schema extensions, so adapting it to a new task involves annotating a schema rather than writing code\. Complex structured data, however, are rarely flat trees: records may form graphs or hypergraphs keyed by arbitrary identifiers, breaking the assumptions of prior similarity metrics\. Our central contribution,*referential alignment*, closes this gap by inferring a bijection between gold and candidate identifiers and scoring every reference through it, so the score is invariant to relabeling\. Since recovering this bijection exactly is graph isomorphism, theObject Alignerapproximates it with Weisfeiler–Leman color refinement\. An order\-sensitive*sequence*regime targets ranking and planning\. Since the same alignment localizes every mismatch, theObject Aligneremits ranked repair suggestions at no extra cost\. Used as a reward inside the GEPA prompt optimizer,Object Alignerhelps or stays neutral across all datasets\.

## IIntroduction

Comparing complex structured data, i\.e\., measuring how closely two structured objects agree, is a basic operation in many settings: automated testing, change detection between versions, record linkage, and the evaluation of machine\-learning systems against reference outputs\. It has become more urgent with large language models \(LLMs\)\. Along with free\-form text, an LLM is often asked to return data conforming to a fixed schema \(most often JSON\) so that its output can be parsed, stored, and consumed directly by downstream systems: information extraction, document understanding, tool and function calling, agentic planning, and knowledge\-graph construction all rely on it\. JSON is among the most widely used data formats in practice, and although it is essentially a tree, cross\-references between records allow it to encode general graphs and hypergraph structures \(Fig\.[1](https://arxiv.org/html/2607.01972#S1.F1)\)\. Judging whether two such objects agree \(for instance a model prediction against a gold reference\) is therefore both common and hard: the comparison must look past cosmetic differences such as reordering or identifier renumbering and credit what is structurally correct\.

schema=\{"type":"object","properties":\{

"people":\{"type":"array","order":"align",

"items":\{"type":"object","keyImportance":0\.0,"valueImportance":1\.0,"properties":\{

"id":\{"type":"integer","idScope":"person"\},

"name":\{"type":"string","score":"exact","valueWeight":2\.0\},

"role":\{"type":"string","score":"exact","enum":\["Manager","Engineer","Intern"\]\}\}\}\},

"mentorships":\{"type":"array","order":"align","ignoreExcess":true,

"items":\{"type":"object","properties":\{

"mentor":\{"type":"integer","ref":"person"\},"mentee":\{"type":"integer","ref":"person"\}\}\}\},

"agenda":\{"type":"array","order":"fixed","items":\{"type":"string","score":"jaro","threshold":0\.3\}\},

"period":\{"type":"array","prefixWeights":\[1,1\],

"prefixItems":\[\{"type":"integer","score":"invdiff"\},\{"type":"integer","score":"invdiff"\}\]\}\}\}

gold=\{

"people":\[

\{"id":1,"name":"Alice","role":"Manager"\},

\{"id":2,"name":"Bob","role":"Engineer"\},

\{"id":3,"name":"Carol","role":"Engineer"\},

\{"id":4,"name":"Dave","role":"Engineer"\},

\{"id":5,"name":"Dave","role":"Engineer"\},

\{"id":6,"name":"Eve","role":"Intern"\}\],

"mentorships":\[

\{"mentor":1,"mentee":4\},

\{"mentor":2,"mentee":3\},

\{"mentor":3,"mentee":5\}\],

"agenda":\["intro","reviews","closing"\],

"period":\[2023,2025\]\}

candidate=\{

"people":\[

\{"id":90,"name":"Dave","role":"Engineer"\},

\{"id":91,"name":"Alice","role":"Manager"\},

\{"id":92,"name":"Dave","role":"Engineer"\},

\{"id":93,"name":"Carol","role":"Engineer"\},

\{"id":94,"name":"Bob","role":"Engineer"\}\],

"mentorships":\[

\{"mentor":91,"mentee":92\},

\{"mentor":94,"mentee":93\},

\{"mentor":93,"mentee":90\}\],

"agenda":\["reviews","intro","closin"\],

"period":\[2023,2026\]\}

Figure 1:Motivating example, used as the running example throughout\. The schema \(top\) declarespeopleids andmentorshipsthat reference them;Object Alignerextensions are inorange\(Appendix[A](https://arxiv.org/html/2607.01972#A1)\)\.goldandcandidateencode the*same*org chart even thoughcandidaterenumbers every id and reorders the people list\. The twoDave/Engineerrecords are property\-identical*twins*: only graph structure tells them apart \(Section[III\-E](https://arxiv.org/html/2607.01972#S3.SS5)\)\. The candidate also omits the internEve, transposes the first twoagendaitems and drops a character in the third, and is a year off onperiod\. Under the identifier bijectionπ=\{1↦91,2↦94,3↦93,4↦92,5↦90,6↦⊥\}\\pi=\\\{1\\\!\\mapsto\\\!91,\\,2\\\!\\mapsto\\\!94,\\,3\\\!\\mapsto\\\!93,\\,4\\\!\\mapsto\\\!92,\\,5\\\!\\mapsto\\\!90,\\,6\\\!\\mapsto\\\!\\bot\\\}everymentorshipmatches; the residual errors yields=0\.77\\mathrm\{s\}=0\.77\.When the scoring procedure itself must be controlled—reproducible and auditable—LLM\-based judging is not a solution\. LLM\-as\-a\-judge\[[26](https://arxiv.org/html/2607.01972#bib.bib27)\]replaces a human evaluator with another model, but it inherits the very difficulty we are trying to escape: the judge must itself be prompted, calibrated, and validated; it is costly, noisy, and non\-deterministic; it is prone to position and verbosity bias; and its verdicts are opaque, which makes them hard to audit\. Generic text\-similarity metrics\[[64](https://arxiv.org/html/2607.01972#bib.bib24),[48](https://arxiv.org/html/2607.01972#bib.bib25),[45](https://arxiv.org/html/2607.01972#bib.bib26)\]avoid these costs but discard the structure entirely\. What is needed instead is a deterministic, schema\-aware score that compares a prediction directly against a gold object and credits the parts it gets right\.

We introduce*Object Aligner*\(OA\), a highly configurable similarity score111Following common usage in the evaluation literature, we use the terms “score” and “metric” interchangeably, even thoughObject Aligneris not a metric in the strict mathematical sense\.for structured outputs, computed by a recursive, schema\-driven alignment of gold and candidate trees that matches each node’s children with the appropriate optimal\-alignment algorithm: the Hungarian algorithm for unordered collections and a sequence\-alignment dynamic program for ordered ones \(such as our example in Fig\.[1](https://arxiv.org/html/2607.01972#S1.F1)\)\. By design, the score is deterministic, decomposable, and schema\-aware, so partially correct objects receive partial credit at the granularity declared by the schema\. The comparison is driven entirely by a schema expressed as a small set of JSON Schema extensions \(per\-field weights, leaf comparators, and ordering and reference semantics\) so adaptingObject Alignerto a new task amounts to annotating a schema rather than writing code\.Object Alignerships as an open\-source Python module\.

Such a score has many uses; this study concentrates on one of them, prompt optimization\. Coaxing an LLM to produce the*correct*structured output is difficult: performance is highly sensitive to how the task is phrased, which examples are shown, and how the schema and its constraints are explained\. Therefore building a reliable pipeline is increasingly an exercise in prompt engineering\. Prompt\-optimization \(PO\) frameworks automate this search by iteratively proposing and refining prompts, and can match or surpass careful manual tuning at a fraction of the human effort\[[44](https://arxiv.org/html/2607.01972#bib.bib14)\]\.

Regardless of the method, the reward that*measures*candidate quality determines what the PO search can achieve: no search can optimize for quality its reward cannot see\. That reward is computed over a labeled dataset and evaluated for many candidate prompts across the loop\. This is exactly where theObject Alignerfits—a deterministic, decomposable structural reward—though prompt optimization is far from its only use\. The common alternative of using an LLM to score candidates or to reflect on their failures is computationally demanding and non\-deterministic, expensive to run at the scale a search requires, hard to reproduce and audit, and sensitive to prompt and position bias—the same problems that motivate a controlled score in the first place\.

The core idea—side\-by\-side recursive Hungarian matching of gold and candidate trees—was described concurrently by STED\[[56](https://arxiv.org/html/2607.01972#bib.bib2)\]and ExtractBench\[[14](https://arxiv.org/html/2607.01972#bib.bib3)\]and implemented in Stickler\[[2](https://arxiv.org/html/2607.01972#bib.bib8)\]\. Our publicly recorded implementation222[https://github\.com/aic\-factcheck/prompt\_opt](https://github.com/aic-factcheck/prompt_opt), committed 2024\-12\-20\.predates all of them, and we treat this part of the work as a simultaneous discovery\. Taking this shared idea as our starting point, we chose to extend it; hence our main contributions are:

- •An open\-source Python library implementingObject Aligner333[https://github\.com/aic\-factcheck/object\_aligner](https://github.com/aic-factcheck/object_aligner), configured entirely through JSON Schema extensions and deployable as a drop\-in reward for existing prompt\-optimization frameworks such as DSPy\[[20](https://arxiv.org/html/2607.01972#bib.bib9)\], GEPA\[[1](https://arxiv.org/html/2607.01972#bib.bib4)\], and TextGrad\[[60](https://arxiv.org/html/2607.01972#bib.bib11)\]\.
- •The application of a deterministic structural score as the reward signal of a prompt optimizer\. It is the first PO pipeline driven by schema\-aware partial credit over nested structures, to our knowledge\.
- •*Referential alignment*for \(hyper\)graphs: A scoring regime invariant to the relabeling of identifiers, which infers a bijection between gold and candidate identifiers and scores every reference through it, approximated with Weisfeiler–Leman color refinement\[[58](https://arxiv.org/html/2607.01972#bib.bib35)\]since recovering it exactly is graph isomorphism \(Section[III\-E](https://arxiv.org/html/2607.01972#S3.SS5)\)\.
- •*Per\-list sequence semantics*: A per\-list choice between order\-agnostic matching, a monotone, insertion/deletion\-aware regime suited to ranking and planning tasks, and*positional tuples*—fixed\-arity sequences whose slots carry position\-specific meaning, each with its own importance and comparator \(Section[III\-C](https://arxiv.org/html/2607.01972#S3.SS3)\)\.
- •*Deterministic ranked feedback*for prompt optimization: The same gold–candidate alignment that produces the score also pinpoints where the candidate departs from the gold, and emits these mismatches as repair operations—edits that would bring the candidate closer to the gold—ranked by the exact amount of score each recovers, so the optimizer is pointed at the changes that matter most, with no LLM call \(Section[III\-F](https://arxiv.org/html/2607.01972#S3.SS6)\)\.
- •An empirical study on both synthetic data, which allows us to isolate and control the specific properties of the score, and real\-world datasets\.444The code for all experiments is available at[https://github\.com/aic\-factcheck/object\_aligner\_paper](https://github.com/aic-factcheck/object_aligner_paper), ensuring the reproducibility\.

The remainder of the paper is organized as follows\. Section[II](https://arxiv.org/html/2607.01972#S2)reviews the structural similarity metrics, concurrent structured\-output scorers, and prompt optimization\. Section[III](https://arxiv.org/html/2607.01972#S3)defines theObject Alignerand its extensions\. Section[IV](https://arxiv.org/html/2607.01972#S4)describes the datasets, and Section[V](https://arxiv.org/html/2607.01972#S5)reports the experiments\. Section[VI](https://arxiv.org/html/2607.01972#S6)discusses the findings, Section[VII](https://arxiv.org/html/2607.01972#S7)the limitations, and Sections[VIII](https://arxiv.org/html/2607.01972#S8)and[IX](https://arxiv.org/html/2607.01972#S9)conclude and outline future work\.

## IIRelated Work

Object Alignerbuilds on a long line of work on comparing structured objects and is motivated by a recent one: prompt optimizers that consume cheap, reliable reward signals\. We review structural similarity first, then the concurrent structured\-output scorers, and close with the prompt\-optimization frameworks that define the use case\.

### II\-ASimilarity Scoring for Structured Data

#### Tree edit distance\.

Comparing trees is classically cast as tree edit distance \(TED\): the minimum\-cost script of node insertions, deletions, and relabelings that turns one tree into the other\[[52](https://arxiv.org/html/2607.01972#bib.bib43)\]\. For ordered trees the Zhang–Shasha algorithm computes the TED in polynomial time\[[62](https://arxiv.org/html/2607.01972#bib.bib44)\], with APTED being the current state of the art\[[42](https://arxiv.org/html/2607.01972#bib.bib46)\]\. For unordered trees, the problem is NP\-hard, and Zhang’s*constrained*edit distance restores tractability by requiring disjoint subtrees to map to disjoint subtrees, which reduces the matching of each node’s children to a bipartite assignment\[[63](https://arxiv.org/html/2607.01972#bib.bib45)\]: the closest classical relative ofObject Aligner’s recursion\. X\-Diff applies the same restriction to unordered XML change detection\[[57](https://arxiv.org/html/2607.01972#bib.bib48)\]\. All of these return an edit cost under uniform, hand\-set operation costs: they have no notion of a schema, per\-field importance, or graded leaf similarity\.

#### Optimal assignment as an evaluation primitive\.

The Hungarian algorithm\[[22](https://arxiv.org/html/2607.01972#bib.bib34)\]has a long history in evaluation\. In coreference resolution, CEAF scores a system by maximum\-weight bipartite matching between gold and predicted entity clusters\[[29](https://arxiv.org/html/2607.01972#bib.bib49)\], in contrast to link\-based MUC\[[55](https://arxiv.org/html/2607.01972#bib.bib50)\]and LEA\[[31](https://arxiv.org/html/2607.01972#bib.bib52)\], and mention\-based B3\[[3](https://arxiv.org/html/2607.01972#bib.bib51)\]\. Template\-filling evaluation extends CEAF by one level of recursion—templates are matched on the aggregate similarity of their slot fillers\[[12](https://arxiv.org/html/2607.01972#bib.bib31),[10](https://arxiv.org/html/2607.01972#bib.bib32)\]—but remains specific to flat MUC\-style templates\. In computer vision, DETR uses the same matching as training loss for set prediction\[[8](https://arxiv.org/html/2607.01972#bib.bib33)\]\. Most directly, the bipartite approximation of graph edit distance reduces an NP\-hard comparison to a single linear assignment over node edit costs\[[46](https://arxiv.org/html/2607.01972#bib.bib47)\]\.Object Alignerapplies similar idea recursively to schema\-typed trees, with the cost matrix computed bottom\-up from configurable leaf comparators\.

#### Semantic\-graph metrics\.

Invariance to identifier renumbering—the property motivating referential alignment \(Section[III\-E](https://arxiv.org/html/2607.01972#S3.SS5)\)—has been studied mainly for semantic graphs\. Smatch scores two Abstract Meaning Representation \(AMR\) graphs by searching for a one\-to\-one alignment of their variables with a restart\-dependent hill\-climbing heuristic\[[7](https://arxiv.org/html/2607.01972#bib.bib53)\]\. S2match replaces binary concept equality with graded embedding similarity\[[37](https://arxiv.org/html/2607.01972#bib.bib28)\], and Smatch\+\+ solves the alignment optimally as an integer linear program\[[38](https://arxiv.org/html/2607.01972#bib.bib30)\]\. WWLK instead compares AMR graphs through Weisfeiler–Leman \(WL\) node embeddings and optimal transport\[[36](https://arxiv.org/html/2607.01972#bib.bib29)\]\. Discrete shared WL colors underlie the optimal\-assignment graph kernel WL\-OA\[[21](https://arxiv.org/html/2607.01972#bib.bib37)\], the closest prior use of joint refinement for matching vertices \(cf\. Section[III\-E](https://arxiv.org/html/2607.01972#S3.SS5)\)\. These metrics achieve exactly the renumbering invarianceObject Alignertargets, but for a single formalism: the Smatch family operates on a flat triple set, none recurses into nested values, and none exposes schema\-level configuration\. We make this relationship concrete by evaluatingObject Aligneron AMR \(Section[IV\-E](https://arxiv.org/html/2607.01972#S4.SS5)\), where Smatch’s variable alignment is the same problem thatObject Alignersolves as referential matching from a schema\.

#### Practical comparison of JSON and records\.

JSON diff libraries such as DeepDiff recurse through nested objects and can ignore list order, but they pair list items greedily rather than by optimal assignment and expose only an unweighted distance with no field importance or graded leaf similarity\[[11](https://arxiv.org/html/2607.01972#bib.bib60)\]\. Probabilistic record linkage is a mature precedent for*declarative per\-field comparison*: Fellegi–Sunter\-based tools such as Splink attach fuzzy comparators and weights to each field of a flat record\[[13](https://arxiv.org/html/2607.01972#bib.bib59),[25](https://arxiv.org/html/2607.01972#bib.bib58)\]\.Object Alignergeneralizes this declarative pattern to trees of arbitrary depth\. The idea of identifying records through their references rather than their attributes is rooted in the relational model\. Chen’s entity–relationship model represents a relationship as a relation keyed by the primary keys of the entities it links and, when attributes alone cannot identify a record, resolves it recursively through its relationships with other records\[[9](https://arxiv.org/html/2607.01972#bib.bib62)\]—the reference\-driven disambiguation that referential alignment performs up to identifier relabeling \(Section[III\-E](https://arxiv.org/html/2607.01972#S3.SS5)\)\. For LLM outputs specifically, function\-calling benchmarks check predicted calls by exact AST matching, recursing into argument values for type and equality but offering no graded leaf similarity, field importance, or alignment of records by reference\[[41](https://arxiv.org/html/2607.01972#bib.bib57)\], while the flexible alternative, LLM\-as\-a\-judge\[[26](https://arxiv.org/html/2607.01972#bib.bib27)\], reintroduces the cost, noise, and calibration burden discussed in Section[I](https://arxiv.org/html/2607.01972#S1)\. Recent surveys likewise call for a more rigorous, reproducible LLM evaluation methodology\[[49](https://arxiv.org/html/2607.01972#bib.bib23)\]\.

### II\-BConcurrent Structured\-Output Scorers

To our knowledge, three systems developed concurrently with theObject Aligner\(Section[I](https://arxiv.org/html/2607.01972#S1)\) target the same problem\. Stickler\[[2](https://arxiv.org/html/2607.01972#bib.bib8)\], an AWS Labs library \(without an accompanying paper\), scores LLM outputs against Pydantic models with per\-field comparators and weights; lists of structured models are Hungarian\-matched, but pairs scoring below a match threshold are not recursed into and are counted as false detections in a confusion\-matrix report aimed at document\-processing audits\. STED\[[56](https://arxiv.org/html/2607.01972#bib.bib2)\]computes a tree edit distance with Hungarian child matching at every level and embedding\-based leaf costs, controlled by a handful of global weights with no schema or per\-field configuration, targeting the consistency benchmarking of repeated generations\. ExtractBench\[[14](https://arxiv.org/html/2607.01972#bib.bib3)\]is primarily a benchmark; its evaluator reads a per\-field scoring configuration from an annotated schema but aligns arrays with an LLM call rather than an assignment solver\.

Table[I](https://arxiv.org/html/2607.01972#S2.T1)summarizes these differences, with Smatch\+\+ included as the closest non\-JSON relative\. Several axes have no full counterpart in the three concurrent systems\. First, none is invariant to identifier renumbering: renumbering the identifiers of Fig\.[1](https://arxiv.org/html/2607.01972#S1.F1)changes their score even though the encoded graph is unchanged, whereas referential alignment \(Section[III\-E](https://arxiv.org/html/2607.01972#S3.SS5)\) scores every reference through an inferred identifier bijection: among these systems, only Smatch\+\+ offers the same guarantee \(but limited to AMR graphs\)\. Second, none lets the schema declare, per list, whether order is part of the answer: STED matches children order\-agnostically throughout, Stickler always Hungarian\-matches structured lists, and ExtractBench delegates ordering to the judge LLM, whereasObject Aligner’sorderkeyword selects per sequence between the order\-agnostic \(optimal bipartite\) and order\-sensitive \(monotone, insertion/deletion\-aware\) regimes \(Section[III\-C](https://arxiv.org/html/2607.01972#S3.SS3)\)\. Relatedly, none treats a fixed\-arity sequence as a*positional tuple*—slots whose meaning differs by position, each with its own importance and comparator, aligned one\-to\-one by position \(Section[III\-C](https://arxiv.org/html/2607.01972#S3.SS3)\): STED and ExtractBench match such arrays interchangeably like any other list\. Third, onlyObject Aligneremits optimizer\-ready feedback: repair operations ranked by exact score deltas derived from the match tree \(Section[III\-F](https://arxiv.org/html/2607.01972#S3.SS6)\); Stickler’s reports target human auditors, and STED returns a bare scalar\. Finally, two design choices make theObject Alignersuitable for the inner loop of prompt optimization, its motivating use case\. The score is continuous, with no threshold gating \(unlike Stickler\), and scoring requires no embedding or LLM calls \(unlike STED and ExtractBench\); rollouts can thus be scored deterministically and reproducibly, without the per\-call model inference those scorers require\.

TABLE I:Design\-axis comparison ofObject Alignerwith the concurrent structured\-output scorers and the closest semantic\-graph metric\. \(✓\\checkmark\) denotes partial support; “dependency\-free” means no embedding or LLM calls at scoring time; superscript letters refer to the notes below\.aPydantic\-first; JSON Schema via vendor\-prefixed extensions\.bHungarian matching for lists of structured models only; recursion gated by a match threshold\.cArray alignment via the LLM call\.dOptimal ILP alignment over a flat triple set; not recursive\.eDeterministic comparators available; embedding/LLM comparators common\.fVia AMR variable alignment; formalism\-specific\.gHTML reports intended for human audit\.hTriples are fixed⟨\\langlesource, relation, target⟩\\rangleslots, matched whole once a one\-to\-one node alignment is chosen; Smatch\+\+ adds a per\-triple weight and one graded*concept*\-slot comparator, but no configurable comparator or weight per position\.

### II\-CPrompt Optimization

A prominent line of automatic prompt optimization uses LLMs as optimizers: APE generates instruction candidates and keeps the best\-scoring\[[66](https://arxiv.org/html/2607.01972#bib.bib12)\], OPRO iteratively proposes prompts conditioned on the trajectory scored so far\[[59](https://arxiv.org/html/2607.01972#bib.bib13)\], evolutionary variants maintain a mutating population\[[15](https://arxiv.org/html/2607.01972#bib.bib15),[17](https://arxiv.org/html/2607.01972#bib.bib16)\], and error\-driven refinement clusters failure cases into prompt updates\[[40](https://arxiv.org/html/2607.01972#bib.bib22)\]\. DSPy makes the reward interface explicit: a multi\-stage LM program is compiled against an arbitrary user\-supplied metric\[[20](https://arxiv.org/html/2607.01972#bib.bib9)\], which its MIPROv2 optimizer maximizes by Bayesian optimization over instructions and few\-shot demonstrations\[[39](https://arxiv.org/html/2607.01972#bib.bib10)\]\. TextGrad instead propagates natural\-language “gradients”—textual critiques of intermediate outputs—through compound systems\[[60](https://arxiv.org/html/2607.01972#bib.bib11)\]\. Across all of these frameworks, the metric is supplied by the user\. Where prompt optimizers have been applied to structured\-output tasks, the reward has been exact\-match triple F1 for knowledge\-graph construction\[[30](https://arxiv.org/html/2607.01972#bib.bib17)\], exact\-match parse accuracy for semantic parsing\[[47](https://arxiv.org/html/2607.01972#bib.bib18)\], or, at most, a graded span\-overlap measure for flat entity extraction\[[54](https://arxiv.org/html/2607.01972#bib.bib19)\]; graded*structural*rewards such as Smatch or graph edit distance have appeared only as reinforcement\-learning signals for weight tuning\[[34](https://arxiv.org/html/2607.01972#bib.bib20),[51](https://arxiv.org/html/2607.01972#bib.bib21)\]\. To the best of our knowledge, no published prompt\-optimization pipeline has driven the search with schema\-driven partial credit over nested structures\.

We adopt GEPA\[[1](https://arxiv.org/html/2607.01972#bib.bib4)\]as the optimizer in all the experiments for two reasons\. First, it is sample\-efficient: candidate mutations are screened on minibatches rather than full validation sweeps, and new candidates are drawn from a Pareto frontier over individual training instances, preserving diversity\. The authors report that it outperforms MIPROv2 and matches or beats GRPO\-style reinforcement learning at a fraction of the rollouts\. Second, it is reflective: like TextGrad and other feedback\-driven optimizers, it updates prompts from natural\-language critique, but here that critique is returned by the metric itself: each mutation is proposed by an LLM that reads execution traces together with*textual feedback*from the metric, so its metric interface is a \(score, feedback\) pair rather than a bare scalar\.Object Alignerexactly fills this interface: a deterministic score for Pareto bookkeeping and structurally faithful repair feedback for reflection \(Section[III\-F](https://arxiv.org/html/2607.01972#S3.SS6)\)\.

## IIIObject Aligner

This section provides a description of theObject Aligner\. We first fix the data model and the two\-phase scoring it uses \(Section[III\-A](https://arxiv.org/html/2607.01972#S3.SS1)\), and then instantiate it for the primitive, sequence, and map nodes \(Sections[III\-B](https://arxiv.org/html/2607.01972#S3.SS2)–[III\-D](https://arxiv.org/html/2607.01972#S3.SS4)\)\. We then develop three extensions that are the focus of this paper: order\-sensitive sequence alignment with insertion/deletion support \(Section[III\-C](https://arxiv.org/html/2607.01972#S3.SS3)\), referential alignment for \(hyper\)graphs, which compares structured outputs up to identifier renumbering \(Section[III\-E](https://arxiv.org/html/2607.01972#S3.SS5)\), and deterministic, optimizer\-shaped feedback that turns the score into a learning signal \(Section[III\-F](https://arxiv.org/html/2607.01972#S3.SS6)\)\. We close with the formal properties and cost of the procedure \(Section[III\-G](https://arxiv.org/html/2607.01972#S3.SS7)\)\.

### III\-AOverview and data model

TheObject Alignertakes a*gold*objectgg, a*candidate*\(predicted\) objectpp, and a*schema*SS, and returns a similarity*score*s​\(g,p;S\)∈\[0,1\]\\mathrm\{s\}\(g,p\\,;\\,S\)\\in\[0,1\], where11denotes a perfect match and0a complete mismatch\. By design, the score is \(i\)*deterministic*—the same inputs always yield the same number555User\-defined primitive type comparators, such as semantic embedding\-based scores, may break determinism\.; \(ii\)*schema\-aware*—the comparison is driven entirely bySS, so partially correct objects receive partial credit at the granularity the schema declares; and \(iii\)*decomposable*—the top\-level score is an explicit weighted aggregate of child scores, which makes feedback possible \(Section[III\-F](https://arxiv.org/html/2607.01972#S3.SS6)\)\.

We model bothggandppas value trees\. A value is either a*primitive*\(a string, a number, or a boolean\), the empty value⊥\\bot, a*sequence*\(an ordered tuple of values\), or a*map*\(a JSONobject, a finite set of key–value pairs with distinct keys\)\. The schemaSSis an annotated tree of the same shape: each node fixes the expected type and the local scoring choices—which primitive comparator to use, how to weight children, whether a sequence is order\-sensitive, and whether a primitive acts as an identifier\. Our implementation represents the value trees as JSONs, whereas the schema takes the form of JSON Schema extensions documented in Appendix[A](https://arxiv.org/html/2607.01972#A1)\.

The alignment descendsgg,pp, andSSin lockstep, producing a score at every node and aggregating upward to the root\. Fig\.[1](https://arxiv.org/html/2607.01972#S1.F1)shows a concrete schema for the running example, a small organization chart: sixpeoplein one identifier scope, directedmentorshipsbetween them, an order\-sensitiveagenda, and aperiodyear tuple\. The schema also exercises severalObject Aligner\-specific extensions beyond the plain JSON Schema \(e\.g\.,keyImportance,valueWeight,enum,prefixItems, and fuzzyscorecomparators\)\.

The scoring at every internal node proceeds in two phases\. In the*alignment*phase,Object Alignerfixes a correspondence between the children ofggand those ofpp—a partial matching that says which gold child is compared against which candidate child\. In the*scoring*phase, it aggregates the per\-pair child scores over the fixed correspondence into a single number in\[0,1\]\[0,1\]\.

The cost that drives the matching depends on the node type and schema configuration\. For*sequences*, the elements are matched by either the Hungarian algorithm or the order\-sensitive dynamic programming approach\. For a*map*, matching involves only the Hungarian algorithm over*key similarity*\. Moreover, referential alignment \(Section[III\-E](https://arxiv.org/html/2607.01972#S3.SS5)\) introduces identifiers and references, analogous to primary and foreign keys in relational databases\. Primitive types \(leaves\) are scored directly\. A node where one or both sides are the empty value⊥\\botis also scored directly: two empties agree \(score11\), whereas a value present on only one side scores the schema’snullScore\(default0\)\. Algorithm[1](https://arxiv.org/html/2607.01972#alg1)states the dispatch, and the following subsections describe each case\.

Algorithm 1Recursive alignment and scorings​\(g,p;S\)\\mathrm\{s\}\(g,p\\,;\\,S\)0:gold value

gg, candidate

pp, schema node

SS
0:score in

\[0,1\]\[0,1\]
1:if

SSis an identifier or reference fieldthen

2:returnsymbolic id/reference score \{Section[III\-E](https://arxiv.org/html/2607.01972#S3.SS5)\}

3:elseif

g=⊥g=\\botor

p=⊥p=\\botthen

4:return

11if both empty, elsenullScore\(default

0\)

5:elseif

SSis a primitivethen

6:return

σ​\(g,p\)\\sigma\(g,p\)
7:elseif

SSis a sequencethen

8:alignchildren by

Mi​j=s​\(gi,pj\)M\_\{ij\}=\\mathrm\{s\}\(g\_\{i\},p\_\{j\}\):

9:order\-agnostic \(Hungarian\) or order\-sensitive \(DP\)

10:return

1D​∑\(i,j\)∈ℳMi​j\\tfrac\{1\}\{D\}\\sum\_\{\(i,j\)\\in\\mathcal\{M\}\}M\_\{ij\}\{Eq\. \([2](https://arxiv.org/html/2607.01972#S3.E2)\)\}

11:elseif

SSis a mapthen

12:alignkeys by

Ki​j=κ​\(ki,kj′\)K\_\{ij\}=\\kappa\(k\_\{i\},k^\{\\prime\}\_\{j\}\)\{keys only\}

13:scorematched pairs:

vi​j←s​\(g​\[ki\],p​\[kj′\]\)v\_\{ij\}\\leftarrow\\mathrm\{s\}\(g\[k\_\{i\}\],p\[k^\{\\prime\}\_\{j\}\]\)
14:returnweighted aggregate of

\{Ki​j\}\\\{K\_\{ij\}\\\},

\{vi​j\}\\\{v\_\{ij\}\\\}\{Eq\. \([7](https://arxiv.org/html/2607.01972#S3.E7)\)\}

15:endif

### III\-BPrimitive types

A primitive node carries a similarity functionσ:𝒱×𝒱→\[0,1\]\\sigma:\\mathcal\{V\}\\times\\mathcal\{V\}\\to\[0,1\]on primitive values𝒱\\mathcal\{V\}\. For strings,σ\\sigmais a configurable comparator: exact match𝟙​\[g=p\]\\mathbbm\{1\}\[g=p\], a normalized edit\- or sequence\-distance similarity, such as Jaro\[[18](https://arxiv.org/html/2607.01972#bib.bib40)\]or Levenshtein\[[24](https://arxiv.org/html/2607.01972#bib.bib41)\]\. For numbers, we may choose an exact match, inverse difference \(1/\(1\+\|g−p\|\)1/\(1\+\|g\-p\|\)\); Booleans are compared exactly\. Any primitive type can be scored using a user\-supplied comparator function \(e\.g\., semantic embedding similarity for text\)\.

### III\-CSequences

A sequence node compares two tuplesg1,…,gng\_\{1\},\\dots,g\_\{n\}andp1,…,pmp\_\{1\},\\dots,p\_\{m\}whose elements share a child schema\. The per\-pair matching cost is the full recursive scoreMi​j=s​\(gi,pj\)M\_\{ij\}=\\mathrm\{s\}\(g\_\{i\},p\_\{j\}\), so whatever similarity the children carry \(nested maps, further sequences\) propagates into the correspondence\. The Object Aligner implements two regimes:

#### Order\-agnostic\.

When the element order is irrelevant \(a set of entities\), the alignment is the*optimal*bipartite matching: padMMto a squared×dd\\times dmatrix \(d=max⁡\(n,m\)d=\\max\(n,m\)\) with zero rows/columns for absent elements and choose

π⋆=arg​maxπ​∑i=1dMi,π​\(i\),\\pi^\{\\star\}=\\operatorname\*\{arg\\,max\}\_\{\\pi\}\\sum\_\{i=1\}^\{d\}M\_\{i,\\pi\(i\)\},\(1\)the maximum\-weight assignment, solved optimally using the Hungarian algorithm\[[22](https://arxiv.org/html/2607.01972#bib.bib34)\]\. Writingℳ=\{\(i,π⋆​\(i\)\):Mi,π⋆​\(i\)\>0\}\\mathcal\{M\}=\\\{\(i,\\pi^\{\\star\}\(i\)\):M\_\{i,\\pi^\{\\star\}\(i\)\}\>0\\\}for the genuinely matched pairs, the score is

s​\(g,p;S\)=1D​∑\(i,j\)∈ℳMi​j\.\\mathrm\{s\}\(g,p\\,;\\,S\)=\\frac\{1\}\{D\}\\sum\_\{\(i,j\)\\in\\mathcal\{M\}\}M\_\{ij\}\.\(2\)An unmatched gold element is*missing*, and an unmatched candidate element is*excess*; each occupies a slot scoring0\. The denominatorD=\|ℳ\|\+nmiss\+nexcD=\|\\mathcal\{M\}\|\+n\_\{\\mathrm\{miss\}\}\+n\_\{\\mathrm\{exc\}\}counts the matched pairs plus the penalized unmatched elements, and whether the missing or excess elements count towardDDis schema\-configurable\.

#### Order\-sensitive\.

When the order is itself part of the answer \(steps of a plan, ranked items\), admissible matchings are restricted to the*monotone*ones, which preserve the left\-to\-right order while allowing insertions and deletions\. The optimal monotone matching is computed by the standard sequence\-alignment dynamic program and scored under the same denominator convention as in Eq\. \([2](https://arxiv.org/html/2607.01972#S3.E2)\)\. Unlike the order\-agnostic regime, a transposition breaks the monotone path and is penalized on both elements\.

#### Tuples and prefixes\.

The two regimes above treat a sequence as unbounded\. In practice, we often work with*tuples*: a fixed number of positional slots whose meaning differs by position \(a coordinate triple, a labeled header\), so each slot deserves its own importance rather than being matched interchangeably\. We unify both as a*prefix*: a fixed\-arity positional head followed by an optional variable\-length tail\. The prefix elements are aligned one\-to\-one by position; with per\-position importancesv1,…,vav\_\{1\},\\dots,v\_\{a\}\(aathe prefix arity, uniform by default\) their scores are combined as

spre=∑k=1avk​s​\(gk,pk;S\)∑k=1avk,s\_\{\\mathrm\{pre\}\}=\\frac\{\\sum\_\{k=1\}^\{a\}v\_\{k\}\\,\\mathrm\{s\}\(g\_\{k\},p\_\{k\}\\,;\\,S\)\}\{\\sum\_\{k=1\}^\{a\}v\_\{k\}\},\(3\)whereas the tail is aligned by one of the two regimes above intosrests\_\{\\mathrm\{rest\}\}\(a pure tuple is just the empty\-tail case\)\. The two blend as

s​\(g,p;S\)=wp​spre\+wr​srestwp\+wr,\\mathrm\{s\}\(g,p\\,;\\,S\)=\\frac\{w\_\{p\}\\,s\_\{\\mathrm\{pre\}\}\+w\_\{r\}\\,s\_\{\\mathrm\{rest\}\}\}\{w\_\{p\}\+w\_\{r\}\},\(4\)with prefix and tail importanceswp,wrw\_\{p\},w\_\{r\}\.

### III\-DMaps

The map is aligned*by its keys alone*\. Given gold keysk1,…,knk\_\{1\},\\dots,k\_\{n\}and candidate keysk1′,…,km′k^\{\\prime\}\_\{1\},\\dots,k^\{\\prime\}\_\{m\}, the alignment is optimal bipartite matching \(Eq\. \([1](https://arxiv.org/html/2607.01972#S3.E1)\)\) on the key\-similarity matrixKi​j=κ​\(ki,kj′\)K\_\{ij\}=\\kappa\(k\_\{i\},k^\{\\prime\}\_\{j\}\)whereκ\\kappais a key comparator \(exact, fuzzy, or user\-defined\)\. Note that the*values are not consulted when choosing which keys pair*: maps are matched by their labels only\. This is deliberate\. The keys already identify what each entry represents, so they suffice to decide what is compared to what, and matching values would force the user to weight keys against values: two scales with no obvious, schema\-independent trade\-off\. Keys fix the correspondence, the values then grade it\. If the key\-only rule is too limiting, the collection can be modelled as an order\-agnostic sequence of⟨key,value⟩\\langle\\text\{key\},\\text\{value\}\\rangletuples \(Section[III\-C](https://arxiv.org/html/2607.01972#S3.SS3)\), recovering the relational view of a map as a matched set of typed pairs\.

Given the matched key pairsℳ=\{\(i,π⋆​\(i\)\):Ki,π⋆​\(i\)\>0\}\\mathcal\{M\}=\\\{\(i,\\pi^\{\\star\}\(i\)\):K\_\{i,\\pi^\{\\star\}\(i\)\}\>0\\\}, the*scoring*phase recurses on the paired values,vi​j=s​\(g​\[ki\],p​\[kj′\]\)v\_\{ij\}=\\mathrm\{s\}\(g\[k\_\{i\}\],p\[k^\{\\prime\}\_\{j\}\]\)\(unmatched keys contribute0\), and aggregates a key term and a value term:

skey\\displaystyle s\_\{\\mathrm\{key\}\}=1d​∑\(i,j\)∈ℳKi​j,\\displaystyle=\\tfrac\{1\}\{d\}\\textstyle\\sum\_\{\(i,j\)\\in\\mathcal\{M\}\}K\_\{ij\},\(5\)sval\\displaystyle s\_\{\\mathrm\{val\}\}=∑\(i,j\)∈ℳωi​vi​j∑i=1dωi,\\displaystyle=\\frac\{\\sum\_\{\(i,j\)\\in\\mathcal\{M\}\}\\omega\_\{i\}\\,v\_\{ij\}\}\{\\sum\_\{i=1\}^\{d\}\\omega\_\{i\}\},\(6\)s​\(g,p;S\)\\displaystyle\\mathrm\{s\}\(g,p\\,;\\,S\)=wK​skey\+wV​svalwK\+wV,\\displaystyle=\\frac\{w\_\{K\}\\,s\_\{\\mathrm\{key\}\}\+w\_\{V\}\\,s\_\{\\mathrm\{val\}\}\}\{w\_\{K\}\+w\_\{V\}\},\(7\)with key weightwKw\_\{K\}, value weightwVw\_\{V\}, and per\-property weightsωi\\omega\_\{i\}\. TheObject Aligner’s defaultwK=0w\_\{K\}=0\(schema keywordkeyImportance\) treats keys as fixed*scaffolding*therefore, only the values are scored\. SettingwK\>0w\_\{K\}\>0suits open\-vocabulary extraction, where the model also*chooses*the keys and obtaining a key right is itself part of the task\.

### III\-EReferential alignment

Many structured outputs are really*\(hyper\)graphs*: some records carry an identifier at which other parts point back\. The identifier values are meaningless; two correct extractions may number their records differently \(Fig\.[1](https://arxiv.org/html/2607.01972#S1.F1)\) and a primitive\-by\-primitive comparison would punish this as an error\. We want a score*invariant to relabeling of identifiers*\. Referential alignment does this: the schema marks one primitive field as an*identifier*and others as*references*to it, andObject Alignerinfers a bijection between gold and candidate identifiers and then scores every reference through it\.

#### Scopes, records, and references\.

The schema keywords areidScopeandref\(Fig\.[1](https://arxiv.org/html/2607.01972#S1.F1)\), which behave like primary and foreign keys\. AnidScopeon a primitive inside a sequence names a*scope*whose items are its*records*; arefof the same type points to a named scope from anywhere\. In the example, thepersonscope has six records \(thepeople, keyed byid\), and eachmentorshipreferences two of them \(mentorandmentee\)\. Identifiers and references are matched*symbolically*through the bijection, not by raw values, so their primitive comparator is ignored\.

#### Deriving the bijection\.

To recover the bijection for a scope, we align its gold and candidate records as an order\-agnostic sequence \(Eq\. \([2](https://arxiv.org/html/2607.01972#S3.E2)\)\): writingdid\_\{i\}for the gold records anddj′d^\{\\prime\}\_\{j\}for the candidate ones, form the record cost matrixCi​j=s​\(di,dj′\)C\_\{ij\}=\\mathrm\{s\}\(d\_\{i\},d^\{\\prime\}\_\{j\}\), solve the assignment \(Eq\. \([1](https://arxiv.org/html/2607.01972#S3.E1)\)\), and read the bijectionπ:gold ids→candidate ids\\pi:\\text\{gold ids\}\\to\\text\{candidate ids\}off the optimal pairs \(partial if the sides differ in size—unmatched gold ids map to⊥\\botand references to them score0; excess candidate ids stay outside the bijection’s image, so any reference pointing at one cannot match its gold counterpart and scores0\)\. Deriving the bijection involves two steps\. First, we*mask*the scope’s own id field: its value is arbitrary, so comparing it would only penalize the right pairing\. Instead, we treat it as a perfect match \(score11\), so it drops out of the comparison, and records are paired by everything*except*their id—in the example, by name and role, which already pair each gold person with the right candidate\. Second, we never compare a reference by value, while the scope it points to is still unresolved, as that would use a bijection we have not yet computed\. Thus, we resolve scopes in dependency order \(topological sort, lexicographic tie\-break for determinism\)\. A reference into the*same*scope, and any cycle of mutually referential scopes, has no such order, so there we warn and fall back to property\-only alignment\.

#### Breaking ties by structure\.

Masking ids can leave the assignment underdetermined: when two records agree on all their*other*attributes \(the twoDave/Engineer*twins*in the example\) their rows inCCare identical and attributes give no reason to prefer one pairing\. The twins may still differ in*structure*: one Dave is mentored byAlice, and the other byCarol\(Fig\.[2](https://arxiv.org/html/2607.01972#S3.F2)\)\. Resolving this exactly is again graph isomorphism, far costlier than the cubic Hungarian assignment that we already pay\. We therefore use a near\-linear11\-dimensional Weisfeiler–Leman \(1\-WL\) color refinement\[[58](https://arxiv.org/html/2607.01972#bib.bib35),[50](https://arxiv.org/html/2607.01972#bib.bib36)\]as a tractable approximation to that test\. It gives each node a color and repeatedly refreshes it from its neighbors’ colors until structurally distinct nodes are separated\.

The colors must be comparable*across*the two sides, and refining each graph on its own does not give that: 1\-WL numbers colors in the order signatures appear, so gold’s color33and candidate’s color33are unrelated\. We therefore refine*jointly*: one graph per side—a node per record, and a directed edge for each reference linking two records, labeled by the scalar fields of the reference\-bearing record \(its*carrier*\); in the example eachmentorshipis a singlementor→\\tomenteeedge between twopersonnodes—over their*disjoint union*𝒢g⊔𝒢c\\mathcal\{G\}\_\{g\}\\sqcup\\mathcal\{G\}\_\{c\}\(gold, candidate; Fig\.[2](https://arxiv.org/html/2607.01972#S3.F2)\), each round recomputing a vertex’s signature from its own color and the sorted multiset of its incident\-edge descriptors,

sigt\+1​\(v\)=\(ct​\(v\),sort​\{\{\(δ,f,ℓ,ct​\(u\)\)\}\}\),\\mathrm\{sig\}^\{t\+1\}\(v\)=\\big\(\\,c^\{t\}\(v\),\\ \\mathrm\{sort\}\\,\\\{\\\!\\\!\\\{\\,\(\\delta,f,\\ell,c^\{t\}\(u\)\)\\,\\\}\\\!\\\!\\\}\\,\\big\),\(8\)where the multiset ranges over all incident edges\(v,u,f,ℓ\)∈ℰ\(v,u,f,\\ell\)\\in\\mathcal\{E\}in both directionsδ∈\{in,out\}\\delta\\in\\\{\\text\{in\},\\text\{out\}\\\}, each contributing its directionδ\\delta\(for a mentorship, whethervvis thementoror thementeeend\), the relationship’s labelff\(herementor→\\tomentee\), an edge labelℓ\\ell, and the neighbor’s colorct​\(u\)c^\{t\}\(u\)\. The labelℓ\\ellis compared by*exact*equality, so it carries only what is already settled on both sides—the carrier’s own scalar fields together with any of its references into already\-resolved scopes—while fuzzy attributes666Scalars compared by a graded similarity rather than exact equality \(e\.g\., Jaro string distance or inverse numeric difference\), whose partial scores have no place in WL’s exact\-equality color refinement\.and references into the still\-unresolved current scope are left out\. In the running example, amentorshipholds nothing but its two references, soℓ\\ellis empty and the twinDaves separate only through their neighbors’ colors\. Were each link to also record asinceyear—sayAlice→\\toDavesince2019andCarol→\\toDavesince2021—their two incoming edges would carryℓ=2019\\ell=\\texttt\{2019\}versusℓ=2021\\ell=\\texttt\{2021\}, splitting the look\-alikeDaves by this exact\-equal scalar in a single round, without the extra refinement their mentors’ differing colors otherwise require\.

A single*intern*table shared by both sides \(Relabel, Algorithm[2](https://arxiv.org/html/2607.01972#alg2)\) maps each round’s signatures to integer colorsct\+1​\(v\)c^\{t\+1\}\(v\), so identical signatures get the same color on gold and candidate—comparable by construction, with no seed, anchor, or learned embedding, and without ever consulting the bijection we derive \(no circularity\)\. This joint relabeling is the standard Weisfeiler–Leman construction,*concordant*across graphs by design\[[50](https://arxiv.org/html/2607.01972#bib.bib36)\]\. The closest prior method to use such discrete shared colors for*matching*vertices is the optimal\-assignment kernel WL\-OA\[[21](https://arxiv.org/html/2607.01972#bib.bib37)\]\. We depart from it in two ways: our graphs carry directed, relation\- and scalar\-labeled edges with references into already\-resolved scopes, and the colors enter only as an infinitesimal tie\-break \(below\) rather than as the assignment objective itself\. In Fig\.[2](https://arxiv.org/html/2607.01972#S3.F2), our construction pins the twins, and in the final round, eachcandidateDave carries the color of its gold match\.

In the default*tie\-break*mode, the colors enter the cost matrix only as an infinitesimal bonusϵ​bi​j\\epsilon\\,b\_\{ij\}, withbi​j=𝟙​\[colors agree\]b\_\{ij\}=\\mathbbm\{1\}\[\\,\\text\{colors agree\}\\,\]andϵ\\epsilonbelow the smallest gap between distinct attribute costs; therefore, the structure breaks*only*exact ties and never overrides what the attributes decide\.Object Aligneralso implements an optional*blend*mode that seeds the refinement from each record’s attributes and mixes structure into the cost at a fixed weight rather than as a tie\-break, which suits cases where attributes already separate most records\. We omit its details and experiments for brevity\.

Algorithm 2Relabel: intern signatures to comparable integer colors0:a signature

sig​\[v\]\\mathrm\{sig\}\[v\]for every vertex

v∈Vv\\in Vof the union

𝒢g⊔𝒢c\\mathcal\{G\}\_\{g\}\\sqcup\\mathcal\{G\}\_\{c\}
0:one integer color per vertex, comparable across both sides

1:

D←Sort​\(\{sig​\[v\]:v∈V\}\)D\\leftarrow\\textsc\{Sort\}\\big\(\\\{\\,\\mathrm\{sig\}\[v\]:v\\in V\\,\\\}\\big\)\{distinct signatures, canonical order\}

2:

color←\\mathrm\{color\}\\leftarrowempty map

3:for

i=0,…,\|D\|−1i=0,\\dots,\|D\|\-1do

4:

color​\[Di\]←i\\mathrm\{color\}\[D\_\{i\}\]\\leftarrow i\{next free integer, reused for repeats\}

5:endfor

6:return

\{v↦color​\[sig​\[v\]\]:v∈V\}\\\{\\,v\\mapsto\\mathrm\{color\}\[\\,\\mathrm\{sig\}\[v\]\\,\]:v\\in V\\,\\\}

goldcandidateround 0round 1round 2round 0round 1round 2![Refer to caption](https://arxiv.org/html/2607.01972v1/x1.png)![Refer to caption](https://arxiv.org/html/2607.01972v1/x2.png)![Refer to caption](https://arxiv.org/html/2607.01972v1/x3.png)![Refer to caption](https://arxiv.org/html/2607.01972v1/x4.png)![Refer to caption](https://arxiv.org/html/2607.01972v1/x5.png)![Refer to caption](https://arxiv.org/html/2607.01972v1/x6.png)

Figure 2:Joint*tie\-break*1\-WL refinement over the disjoint union𝒢g⊔𝒢c\\mathcal\{G\}\_\{g\}\\sqcup\\mathcal\{G\}\_\{c\}\(goldleft,candidateright; rounds left\-to\-right, the last is stable\); a color change marks a refinement step\. The relabeling is shared, so equal colors are comparable across the two sides: by the final round everycandidaterecord carries the color of itsgoldcounterpart, which pins theDave/Engineertwin bijection\.
#### Scoring references\.

With the bijection in hand, the main alignment pass \(Algorithm[1](https://arxiv.org/html/2607.01972#alg1)\) treats identifier fields as labels worth11and scores a reference as correct if and only if it points, through the inferred mapping, at the same record the gold reference does \(and that target exists among the candidate ids; otherwise it scores0\)\. In Fig\.[1](https://arxiv.org/html/2607.01972#S1.F1)the masked name/role comparison, with the twins split by structure, recoversπ=\{1↦91,2↦94,3↦93,4↦92,5↦90,6↦⊥\}\\pi=\\\{1\\\!\\mapsto\\\!91,2\\\!\\mapsto\\\!94,3\\\!\\mapsto\\\!93,4\\\!\\mapsto\\\!92,5\\\!\\mapsto\\\!90,6\\\!\\mapsto\\\!\\bot\\\}, under which everymentorshipmatches exactly despite every id differing; the object still loses credit for the omittedEveand the perturbedagenda/period, for a finals=0\.77\\mathrm\{s\}=0\.77\.

### III\-FOptimizer feedback

A scalar reward tells an optimizer*how*well a prompt does, but not*what*to change\. BecauseObject Aligner’s score is decomposable, we can answer the second question deterministically and*without an LLM*: the same alignment that produced the score also localizes the deficit, which we render as a compact, prescriptive feedback string suitable for the natural\-language reflection slot of a reflective prompt optimizer such as GEPA\[[1](https://arxiv.org/html/2607.01972#bib.bib4)\]or TextGrad\[[60](https://arxiv.org/html/2607.01972#bib.bib11)\]—in DSPy\[[20](https://arxiv.org/html/2607.01972#bib.bib9)\], for instance, GEPA is exposed as an optimizer whose metric returns this textual feedback alongside the scalar score\. This contrasts sharply with LLM\-as\-judge feedback\[[26](https://arxiv.org/html/2607.01972#bib.bib27)\], which is noisy, non\-deterministic, and must be prompted and calibrated\.

Walking the match tree, every imperfect leaf or structural mismatch becomes a*repair operation*with an associated*score delta*—the amount of total score recovered by fixing it\. For a child contributing with effective weightcwc\_\{w\}and value scoresws\_\{w\}, the delta isΔ​\(op\)=cw​\(1−sw\)\\Delta\(\\mathrm\{op\}\)=c\_\{w\}\\,\(1\-s\_\{w\}\), and, under the fixed assignment chosen by alignment, summing these deltas over the full set𝒪\\mathcal\{O\}of repair operations accounts for the entire score deficit exactly,∑op∈𝒪Δ​\(op\)=1−s​\(g,p;S\)\\sum\_\{\\mathrm\{op\}\\in\\mathcal\{O\}\}\\Delta\(\\mathrm\{op\}\)=1\-\\mathrm\{s\}\(g,p\\,;\\,S\), that is, the gap between a perfect score and the one achieved\. Operations are typed \(primitive replacement, missing/extraneous key, fuzzy key rename, missing/excess sequence element, wrong reference, whole\-subtree replacement, etc\.\) and each carries a path to its location\.

Operations are ranked byΔ\\Delta, the topKKare kept \(those below a minimum delta are dropped\), and each is rendered through a per\-type template\. A trailing*synthesis*line names the dominant error category when one type accounts for most of the displayed deficit\. The per\-type templates are configurable\. For the running example of Fig\.[1](https://arxiv.org/html/2607.01972#S1.F1)\(scored at0\.770\.77\), the feedback rendered by the default templates is shown in Fig\.[3](https://arxiv.org/html/2607.01972#S3.F3)\. Each line is a deterministic template substitution, and the deltas are derived from the exact decomposition described above\.

The prediction scored 0\.77 \(deficit 0\.23\)\. Top 5 of 5 fix locations: 1\. /agenda/0: extraneous list item ’reviews’\. Removing it recovers \+0\.062\. 2\. /agenda/2: missing list item ’reviews’\. Adding it recovers \+0\.062\. 3\. /period/1: expected 2025, got 2026\. Fixing this recovers \+0\.062\. 4\. /people: list is missing item \{’id’: 6, ’name’: ’Eve’, ’role’: ’Intern’\}\. Adding it recovers \+0\.042\. 5\. /agenda/3: expected ’closing’, got ’closin’\. Fixing this recovers \+0\.003\. The deficit is spread across multiple issue types \(missing\-list\-item, primitive\-value, extraneous\-list\-item\)\.

Figure 3:Deterministic GEPA\-style optimizer feedback for the running example \(Fig\.[1](https://arxiv.org/html/2607.01972#S1.F1),s=0\.77\\mathrm\{s\}=0\.77\): fix locations ranked by score delta, each rendered from a per\-type template, closing with the synthesis line\. The reportedΔ\\Deltavalues come straight from the deficit decomposition, so the feedback is faithful to the score by construction\.
### III\-GProperties and cost

Object Aligneris*deterministic*and reproducible: every choice point \(assignment, WL relabeling, tie\-breaks\) is resolved by a canonical, seed\-independent rule\. The score is*bounded*in\[0,1\]\[0,1\]and awards partial credit at schema granularity\. It is*decomposable*, which precisely enables the attribution and feedback of Section[III\-F](https://arxiv.org/html/2607.01972#S3.SS6)\.

The cost is dominated by per\-node work\. A sequence or map alignment of arityddsolves one assignment problem,O​\(d3\)O\(d^\{3\}\), over anO​\(d2\)O\(d^\{2\}\)\-entry matrix whose*sequence*entries are recursive alignments \(a*map*instead scores cheap key comparisons, recursing only on the≤d\\leq dmatched pairs\); order\-sensitive sequences replace the assignment with anO​\(n​m\)O\(nm\)dynamic program\. Referential alignment adds one assignment per scope plus WL refinement, which costsO​\(R​\(\|V\|\+\|E\|\)\)O\\\!\\big\(R\\,\(\|V\|\+\|E\|\)\\big\)withR≤\|V\|R\\leq\|V\|rounds over the scope’s reference graph\. Summed over the tree, the cost is dominated by the assignments, each cubic in its collection’s local aritydd, whereddis the number of matched siblings—fixed and small for an object’s keys, so only list lengths grow with the document\. A single such list is alreadyO​\(n3\)O\(n^\{3\}\), and nesting growing lists multiplies this further, but in practice these lists stay short and shallow, so the cost remains a non\-issue\.

## IVDatasets

The following datasets are used in Section[V](https://arxiv.org/html/2607.01972#S5)to demonstrate the Object Aligner properties, focusing mainly on Referential Alignment \(RA\) and order semantics\. The list contains both synthetic \(Org2Graph and Facts2Order\) and real\-world data\. We briefly introduce each dataset; worked examples, schemas, and native task metrics are deferred to the per\-dataset appendices \(Appendix[B](https://arxiv.org/html/2607.01972#A2), native metrics in Appendix[E](https://arxiv.org/html/2607.01972#A5)\)\. Real\-world datasets were converted to JSON under an OA schema and, in some cases, reframed for our needs\.

### IV\-AOrg2Graph

In natural information extraction corpora, the difficulty of recovering*referential structures*is tangled up with parsing, domain vocabulary, and annotation noise, so they cannot isolate*why*referential alignment helps or fails\. Org2Graph \(O2G\) is a synthetic generator that converts this difficulty into controlled variables\. Each instance is a templated natural\-language paragraph describing a fictional organization, and the task is to extract the graph it describes as JSON with five scopes:peopleandcompaniesare records carrying identifiers, andemployment,acquaintance, andpartnershiprecords link them through references, foreign\-key style\. Records and links carry one categorical property each \(title,industry,role, and the two relation types\)\. Because the paragraph is rendered from the gold graph \(every link surfaces as exactly one sentence\) extraction is unambiguous by construction\. Fig\.[13](https://arxiv.org/html/2607.01972#A3.F13)\(Appendix[C](https://arxiv.org/html/2607.01972#A3)\) shows a small instance including gold JSON output\.

Three parameters control where the difficulty lies; each scope draws its categorical values from a fixed*codebook*\(its vocabulary of allowed entries, or*codes*\)\.*Twin density*sets the fraction of records that are*property\-twins*—identical in every scored attribute, thus telling them apart, and hence routing references correctly, requires the reference structure itself\. Within a scope, exactly one set of twins shares identical scored attributes, whereas every other record maintains a distinct combination of them\.*Value obfuscation*replaces the readable codebook entries with opaque codes under a fixed legend never shown to the model \(the paragraph says “engineer,” the gold value isT07\), so an optimizer can recover the vocabulary only by aggregating evidence across examples\.*Vocabulary width*sets each scope’s codebook size \(24/20/20/12/1224/20/20/12/12vs\.6/6/6/3/36/6/6/3/3\); narrow codebooks make property\-twins frequent\.

### IV\-BFacts2Order

Validating*order*sensitivity requires a task whose correct order is unique, stated explicitly in the input, and inexpensive to corrupt by a measured amount\. Facts2Order \(F2O\) is a synthetic generator built to satisfy these properties\. Each instance listsNNshort sentences, each stating one sortable key for a named entity \(an integer quantity, a date, or an ordinal word\)\. The task is to output the index permutation that sorts the items by their key \(Fig\.[14](https://arxiv.org/html/2607.01972#A3.F14)\)\. The generator exposes a few independent parameters, and the study instantiates several configurations that differ in their settings\. The*number of items*NNfixes the length of the permutation to recover; we use different values ofNNacross experiments \(exact ranges in Section[V\-A](https://arxiv.org/html/2607.01972#S5.SS1)and Section[V\-D](https://arxiv.org/html/2607.01972#S5.SS4)\)\. The*key type*is an integer quantity, a date, or an ordinal word\. Each instance may carry a variable number of*irrelevant distractor sentences*that state no sortable key, and merely pad the input\. Finally, the*sort key*is presented in one of two regimes:*stated*, where the key is the lone numeric clause on each item, or*hidden*, where each item additionally carries22–33*numeric decoy*clauses on other attributes \(e\.g\., a price in dollars, a length in centimeters\) whose values are unconstrained, with the numeric clauses shuffled per item, so the key is never identifiable as the lone number and the model must discover*which*measure to sort on\. Keys are drawn distinctly, so the gold order is unique, while the distractor sentences and decoy clauses leave it untouched\. Because gold and candidate are index permutations, disorder is exactly measurable via the Kendall distance\[[19](https://arxiv.org/html/2607.01972#bib.bib42)\]\(the minimum number of adjacent transpositions between two orderings\), so corruptions can be dosed\.

### IV\-CSciERC

SciERC\[[27](https://arxiv.org/html/2607.01972#bib.bib64)\]annotates500500computer\-science abstracts with scientific entities, coreference clusters, and typed relations, which is our natural\-text extraction benchmark, where difficulty arises from parsing and domain vocabulary\. The target output is a*graph*:entitiesas nodes joined by reference\-linkedrelations, extracted from the abstract as JSON\. A worked instance is in Fig\.[8](https://arxiv.org/html/2607.01972#A2.F8)\(Appendix[B\-A](https://arxiv.org/html/2607.01972#A2.SS1)\)\. A*simplified*form is used\. Because reproducing exact character offsets is poorly suited to the text\-generation nature of decoder\-only LLMs\[[61](https://arxiv.org/html/2607.01972#bib.bib63)\], we drop all span offsets and keep only surface text: each coreference cluster becomes one entity whosementionsarray holds its surface forms \(its closed\-vocabularytype, one of six, by majority vote\)\. The raw annotations carry no entity identifiers, so we assign each entity a syntheticid\(e0,e1, …\) in the cluster first\-occurrence order purely to link the relation endpoints\. The original mention\-pair relations are projected to the cluster level \(one of the seven closedpredicates\), discarding any relation whose endpoints fall outside a cluster\. We use original splits with no subsampling \(349/50/100349/50/100train/val/test\)\.

### IV\-DBioRED

BioRED\[[28](https://arxiv.org/html/2607.01972#bib.bib65)\]is a document\-level biomedical relation\-extraction corpus of PubMed abstracts preprocessed into the same graph output as SciERC \(Section[IV\-C](https://arxiv.org/html/2607.01972#S4.SS3)\)—coreference\-clusterentitiesjoined by reference\-linkedrelations\. Its distinctive feature is that each gold entity id is an opaque normalized concept accession not derivable from the abstract, so relations must, in principle, be routed by referential alignment rather than literal\-id matching\. We draw a seeded50/50/10050/50/100train/val/test split, with train and val subsampled from BioRED’s train/dev pools, and the test split is the full BioRED test split\. See details in Appendix[B\-B](https://arxiv.org/html/2607.01972#A2.SS2)\.

### IV\-EBio AMR

The Bio AMR Corpus\[[53](https://arxiv.org/html/2607.01972#bib.bib55)\]annotates biomedical sentences \(cancer\-related PubMed text\) with Abstract Meaning Representation\[[4](https://arxiv.org/html/2607.01972#bib.bib54)\]graphs: each sentence maps to a rooted, directed graph of concept nodes and labeled edges\. Like BioRED \(Section[IV\-D](https://arxiv.org/html/2607.01972#S4.SS4)\), AMR’s variable names are arbitrary and hidden, so referential alignment alone can carry out relation routing\. Bio AMR is the purest case: the same concept recurs on many nodes, so content individuates nothing, and structure does all the work\. We convert each gold PENMAN graph into a JSON object ofnodesandrelations, dropping the source variable letters so that the model emits its own arbitrary node ids and only the graph structure is scored; a worked instance is shown in Fig\.[10](https://arxiv.org/html/2607.01972#A2.F10)\(Appendix[B\-C](https://arxiv.org/html/2607.01972#A2.SS3)\)\. The registered schema scores this referentially and pairs with literal\-id*strict*ablation \(Appendix[B\-C](https://arxiv.org/html/2607.01972#A2.SS3)\), which measures how much of the score referential alignment carries\. We draw a seeded, unstratified100/100/200100/100/200train/val/test subsample of the corpus\.

### IV\-FNATURAL PLAN \(Trip Planning\)

This dataset and the one that follows target OA’s*order*semantics\. NATURAL PLAN\[[65](https://arxiv.org/html/2607.01972#bib.bib66)\]is a natural\-language planning benchmark\. We use its*Trip Planning*task, which asks for a strictly*ordered*itinerary—a sequence of cities to visit, each for a given number of days, reachable only over a stated set of direct flights\. Unlike the synthetic Facts2Order \(Section[IV\-B](https://arxiv.org/html/2607.01972#S4.SS2)\), the gold order here is imposed by a genuine planning constraint rather than designed in, so a correct plan must obtain both the set of stays*and*their sequence right\. Each instance’s inputcontextis based on the day budget, the per\-city stays and meeting windows, and the direct\-flight list; the gold output is taken straight from the releasedcities/durationsfields \(Fig\.[11](https://arxiv.org/html/2607.01972#A2.F11), Appendix[B\-D](https://arxiv.org/html/2607.01972#A2.SS4)\)\. We draw a difficulty\-stratified100/100/200100/100/200train/val/test split keyed on the number of cities \(33–1010\)\.

### IV\-GROCStories

ROCStories\[[33](https://arxiv.org/html/2607.01972#bib.bib67)\]is a corpus of five\-sentence everyday commonsense stories\. The correct order is fixed by narrative coherence in real text rather than designed in by sortable keys, so recovering it demands commonsense reasoning\. We retain only stories that segment into exactlyN=5N=5sentences; the inputcontextpresents those sentences scrambled and labeled1\.\.N1\.\.N, and the gold output is the index permutation that restores the original reading order, emitted as a JSONindexlist \(Fig\.[12](https://arxiv.org/html/2607.01972#A2.F12), Appendix[B\-E](https://arxiv.org/html/2607.01972#A2.SS5)\)\. The data key is deliberately neutral \(indices, notorder\), so neither schema nor seed prompt hints that the task is reordering\. We draw a100/100/200100/100/200train/val/test split \(all instances areN=5N=5\)\. See more details in Appendix[B\-E](https://arxiv.org/html/2607.01972#A2.SS5)\.

## VResults

We evaluate theObject Alignerin two stages\. An*intrinsic*study \(Section[V\-A](https://arxiv.org/html/2607.01972#S5.SS1)\) validates its two novel mechanisms with no LLM in the loop, in the perturbation\-based design of STED\[[56](https://arxiv.org/html/2607.01972#bib.bib2)\]: the score must stay high under benign corruptions of the gold and fall, in proportion, under harmful ones\. Because STED already validates the machinery, the two metrics share \(Hungarian\-style order\-invariant matching above all\) our experiments target onlyObject Aligner’s additions: referential alignment \(RA, Section[III\-E](https://arxiv.org/html/2607.01972#S3.SS5)\) and the order\-sensitive*sequence*regime \(Section[III\-C](https://arxiv.org/html/2607.01972#S3.SS3)\)\. An*extrinsic*study \(Section[V\-B](https://arxiv.org/html/2607.01972#S5.SS2)–[V\-E](https://arxiv.org/html/2607.01972#S5.SS5)\) measures what these mechanisms buy as the reward and feedback signal driving prompt optimization \(GEPA\[[1](https://arxiv.org/html/2607.01972#bib.bib4)\]\)\.

### V\-AIntrinsic validation of the alignment mechanisms

The setup is simple: a deterministic generator produces gold outputs, controlled corruptions damage them, and we measure how the scores​\(g,p;S\)\\mathrm\{s\}\(g,p\\,;\\,S\)responds \(AUROC, rank correlations\)\. A good score must do two things at once: stay constant under changes that preserve meaning and drop in proportion to genuine damage\. RA must ignore identifier relabeling yet register every structural edit, and the sequence regime must grade reorderings yet degrade gracefully when items are dropped or inserted\.

#### Invariance to relabeling\.

Referential alignment promises that the score does not depend on how a candidate numbers its records: identifiers are matched through the inferred bijectionπ\\pi, not by value \(Section[III\-E](https://arxiv.org/html/2607.01972#S3.SS5)\)\. We generate100100gold Org2Graph graphs \(Section[IV\-A](https://arxiv.org/html/2607.01972#S4.SS1)\) in the plain\-narrow configuration, with sizes, edge densities, and twin densityt∼U​\[0,1\]t\\sim U\[0,1\]drawn at random \(Algorithm[3](https://arxiv.org/html/2607.01972#alg3), Appendix[C](https://arxiv.org/html/2607.01972#A3)\), and for each emit five exact copies under independent identifier relabelings \(500500pairs\)\. Each copy is graph\-isomorphic to its gold, so a relabel\-invariant score must give all five exactly11; the per\-gold variance is the diagnostic\. RA passes exactly \(score1\.0001\.000, variance0on all100100golds\) even on the nine where*every*record is a property\-twin and the bijection is pinned only by the reference structure \(see Fig\.[13](https://arxiv.org/html/2607.01972#A3.F13), Appendix[C](https://arxiv.org/html/2607.01972#A3)\)\. Theplainablation \(idScope/refdropped, identifiers compared by value—our stand\-in for any reference\-unaware metric\) averages0\.7380\.738with per\-gold variance up to5\.5×10−35\.5\\times 10^\{\-3\}: a flawless output forfeits a quarter of its credit, by an amount that depends on the arbitrary choice of identifiers—exactly the noise a score should not inject\.

#### Sensitivity to damage\.

Invariance alone is trivial; therefore, the second experiment checks that RA still falls, in proportion, under genuine damage\. On100100fresh golds we apply a*single*edit typek=0,…,8k\{=\}0,\\dots,8times on top of a relabeled copy \(k=0k\{=\}0is the relabel alone\): recoding a categorical value, rerouting a reference, deleting or inserting a record, or an edge—54005400pairs \(Algorithm[4](https://arxiv.org/html/2607.01972#alg4), Appendix[C](https://arxiv.org/html/2607.01972#A3)\)\. RA starts at1\.01\.0and decreases monotonically withkkunder every operation \(Fig\.[4](https://arxiv.org/html/2607.01972#S5.F4), left; per\-operation Spearman−0\.66\-0\.66to−0\.89\-0\.89, Table[VI](https://arxiv.org/html/2607.01972#A3.T6)in Appendix[C](https://arxiv.org/html/2607.01972#A3)\), and separates damaged from clean almost perfectly \(AUROC0\.9960\.996\); the residual is correct, not noise—4242of the48004800edited candidates, mostly reroutes between property\-twins, are graph\-isomorphic to their gold and correctly scored11\. Plain separates significantly worse \(AUROC0\.8350\.835\), and*reference rerouting*shows why \(Fig\.[4](https://arxiv.org/html/2607.01972#S5.F4), right\): the one edit that corrupts the graph without touching any record’s attributes moves only RA \(1\.00→0\.931\.00\\\!\\to\\\!0\.93over eight reroutes; Spearman−0\.660\-0\.660\), while plain remains flat \(0\.74→0\.730\.74\\\!\\to\\\!0\.73;−0\.011\-0\.011\)—the damage most specific to graph extraction is precisely the damage it cannot see\. A control sweep without the relabel confirms that this gap is specific to relabeled identifiers, not the edits themselves: with identifiers matching by value, RA and plain nearly coinciding except under reference rerouting and categorical relabel \(data omitted for brevity\)\.

![Refer to caption](https://arxiv.org/html/2607.01972v1/x7.png)Figure 4:Sensitivity to damage \(Org2Graph generator\), pooled over graph sizes and seeds: each of the six edit types applied*alone*,kktimes, on top of an identifier relabel \(k=0k\{=\}0is the relabel only\)\.*Left*: RA;*right*: the plain ablation\. RA starts at1\.01\.0and degrades monotonically under every operation; plain starts at0\.740\.74—the relabel alone already costs it—and is blind to reference rerouting \(flat orange curve\), the one edit that corrupts routing without touching any record’s attributes\.
#### Sequence regime\.

The sequence regime \(Section[III\-C](https://arxiv.org/html/2607.01972#S3.SS3)\) claims to*grade*order: a nearly\-sorted output must outscore a scrambled one\. OnFacts2Order\(Section[IV\-B](https://arxiv.org/html/2607.01972#S4.SS2)\)—configured with the item count drawn per instance asN∼U​\{3\.\.12\}N\\sim U\\\{3\.\.12\\\}, the key type drawn uniformly from \{integer, date, ordinal\},0–33irrelevant distractor sentences, and the*stated*\-key regime \(no numeric decoys\)\. OA scores the index permutations directly, so the surface sentences are ignored\. We corrupt300300gold orderings in two ways\. The first is pure reorderings: adjacent transpositions up to a target Kendall distance, block reversals, and moves\. The second is length\-changing edits of the next paragraph\. Together these yield46394639pairs \(generation: Algorithm[5](https://arxiv.org/html/2607.01972#alg5); worked instance: Fig\.[5](https://arxiv.org/html/2607.01972#S5.F5)\)\. Each pair is scored in four ways: thesequencealignment under test; the order\-agnosticsetalignment \(Hungarian matching of the label bag\), standing in for STED\-style matching;exactmatch \(Perfect\-Match Ratio\); and Kendallτ\\tau, the graded reference one would hand\-pick for this task \(0on any non\-permutation\)\. On pure reordering the sequence score falls smoothly with Kendall distance \(see Table[II](https://arxiv.org/html/2607.01972#S5.T2); Spearman−0\.706\-0\.706\), while separating sorted from corrupted perfectly \(AUROC1\.0001\.000\)\. The exact match also separates perfectly—it is0on every corrupted candidate by construction—but offers no gradient\. The set score is*exactly constant*at1\.01\.0for every pure reordering \(the label bag never changes\): zero order signal, with AUROC0\.7050\.705owing to incidental length changes alone\. Kendallτ\\tauis graded, as designed \(Spearman−0\.621\-0\.621\); OA’s sequence alignment tracks this task\-specific reference without being built for the task\. Fig\.[15](https://arxiv.org/html/2607.01972#A3.F15)\(Appendix[C](https://arxiv.org/html/2607.01972#A3)\) shows this behavior graphically\.

```
Here are the items:
Item 1: Saiph weighs 37 kilograms.
Item 2: Antares weighs 696 kilograms.
Item 3: Arcturus weighs 933 kilograms.
Item 4: Bellatrix weighs 883 kilograms.

gold: {"indices": [1, 2, 4, 3]}
      (mass order: 37 < 696 < 883 < 933)
```

Figure 5:AFacts2Orderinstance \(N=4N\{=\}4, integer key\) with its unique gold order, and four candidates scored four ways—the per\-instance view of Table[II](https://arxiv.org/html/2607.01972#S5.T2)\. Only the*sequence*score grades both the reorderings and the dropped item\.Models also drop or add new records; therefore, we further delete or insertk=1\.\.3k\{=\}1\.\.3items per candidate \(insertions duplicate existing ones\)\. A single dropped item sends both the exact match and Kendallτ\\tauto0: Kendallτ\\tauis undefined between orderings of different item sets \(and scores0\), and the exact match trivially fails on any length change\. Instead the OA’s sequence alignment loses credit in proportion to the edit count through its gap\-aligned denominator \(Section[III\-C](https://arxiv.org/html/2607.01972#S3.SS3)\)\. Under these pure length\-changing edits \(no accompanying reordering\), the set score coincides with the sequence\. Nevertheless, across both axes, the sequence is the only scorer that stays graded \(Table[II](https://arxiv.org/html/2607.01972#S5.T2)\), using the same generic, schema\-driven machinery that scores every other structure in theObject Aligner\.

TABLE II:Mean score by corruption family \(Facts2Ordergenerator,46394639pairs\)\. Adjacent transpositions at Kendall distanceddkeep the items and change only their order; deletions and insertions change the length\.

### V\-BExtrinsic study: setup

We study theObject Alignerin the role it was originally designed for: a deterministic reward \(and, optionally, a deterministic feedback signal\) inside a prompt\-optimization loop\.

#### Optimizer and models\.

We use a single optimizer, GEPA\[[1](https://arxiv.org/html/2607.01972#bib.bib4)\]: our grid of datasets, ablations, arms, and seeds makes a second optimizer computationally prohibitive, and GEPA is a strong representative—state of the art \(surpassing MIPROv2\[[39](https://arxiv.org/html/2607.01972#bib.bib10)\]\), sample\-efficient \(up to35×35\\timesfewer rollouts than GRPO\), and a standard optimizer of the popular DSPy framework\[[20](https://arxiv.org/html/2607.01972#bib.bib9)\]\. GEPA is a reflective prompt optimizer that iteratively rewrites a task prompt\. It maintains a Pareto frontier of candidate prompts over the individual training instances and proposes each mutation from an LLM reflection step that reads execution traces\. Candidates are screened on small reflection minibatches \(size three\) rather than full validation sweeps\. The reflection \(proposer\) LM is a*frozen*GPT\-5model \(gpt\-5\-2025\-08\-07\)\[[35](https://arxiv.org/html/2607.01972#bib.bib5)\]: pairing the reflector with a strong frontier model is the recommended practice for reflective optimization and the regime in which GEPA was developed\[[1](https://arxiv.org/html/2607.01972#bib.bib4)\]\. The prompt being optimized drives a separate, locally served*task*model that emits the structured JSON\. We report two task models\[[16](https://arxiv.org/html/2607.01972#bib.bib6)\]that bracket a wide capacity range—the large, strongerGemma\-4\-26B\-A4B\-itand the much smaller, consumer\-gradeGemma\-4\-E4B\-it—to check that the qualitative findings are not an artifact of one model’s capacity\.777As an additional robustness check, we ranQwen3\.5\-35B\-A3BandQwen3\.5\-9B\[[43](https://arxiv.org/html/2607.01972#bib.bib7)\]as task models on a subset of datasets\. The qualitative conclusions were unchanged\.

#### Protocol\.

The datasets are described in Section[IV](https://arxiv.org/html/2607.01972#S4)\. Each is split into three disjoint partitions: a*feedback*set, from which GEPA draws the size\-33reflection minibatches; a*Pareto*set, on which candidate prompts are scored to maintain the frontier; and a held\-out*test*set, touched only for the final evaluation\. For each \(dataset, model, arm, ablation\) cell, we run GEPA under a fixed rollout budget of800800metric calls for the synthetic Org2Graph and Facts2Order tasks and600600calls for the real datasets, repeating over1010random seeds\. We aggregate the mean±\\pmsample standard deviation over the seeds\. Every experiment is seeded from a single minimal system prompt that specifies only the required JSON output format and deliberately omits any hints for the input\-to\-prediction transform; GEPA must discover that mapping purely fromObject Aligner’s scalar score and, in the feedback arm, its textual feedback, isolating whether the metric supplies enough optimization gradient\. Verbatim seed prompts for all datasets are listed in Appendix[D](https://arxiv.org/html/2607.01972#A4)\. The task model decodes under JSON\-Schema–constrained decoding with an81928192\-token output cap, greedily on Org2Graph \(temperature0\) and at a temperature of0\.30\.3on every other dataset; a run that degenerates into repeated tokens and fails to complete is retried up to twice with the temperature raised by0\.20\.2per attempt, and output that still fails to complete counts as a failure \(scored0; see “Evaluation” below\)\.

#### Score and feedback arms\.

GEPA consumes whatever the metric returns\. We compare two*arms*\. In both arms, the scalarObject Alignerscore is GEPA’s fitness signal, which drives Pareto\-frontier maintenance and candidate selection; the arms differ only in what the reflection step additionally reads\. In thescorearm, the metric returns onlyObject Aligner’s scalar graded score; therefore, the reflection model sees nothing but a number\. In thefeedbackarm, it also returnsObject Aligner’s deterministic, decomposable feedback string \(Section[III\-F](https://arxiv.org/html/2607.01972#S3.SS6)\)—a ranked list of theK=5K=5most important repair operations with their exact score deltas—in the natural\-language reflection slot\. The two arms share everything else, so their difference isolates the value of the feedback\.

#### Ablation axes\.

Each dataset isolates one of the twoObject Alignerextensions, and we treat that extension as a control on GEPA’s*fitness function*\. The question is not whether theObject Aligner*can*measure a property \(which was already established\), but whether giving GEPA a reward that*perceives*that property yields a better optimized prompt than a reward that is blind to it\. On the graph\-extraction datasets, the axis isreferential alignment\(Section[III\-E](https://arxiv.org/html/2607.01972#S3.SS5)\): theRAschema marks the identifier and reference fields, so the score is invariant to identifier relabeling—it judges a graph up to isomorphism rather than penalizing a correct graph that merely names its nodes differently—while theplainschema drops that machinery and compares identifiers by value, so it cannot see relabeling at all\. On the ordered\-output datasets, the axis is the sequence regime \(Section[III\-C](https://arxiv.org/html/2607.01972#S3.SS3)\): thesequenceschema perceives the output order, whileset, the order\-agnostic Hungarian matching, rewards membership only, and is blind to order\. On each axis, one arm’s reward sees the target property and the other does not\.

#### Evaluation\.

We always evaluate on held\-out data with the*property\-sensitive*schema \(the relabel\-invariant RA schema for the referential axis, the*sequence*schema for the order axis\)\. Only these can detect the property in question \(graph isomorphism and correct order\), and they are the richest scale common to every dataset, including the synthetic graph and ordering tasks that have no native metric\. Holding this schema fixed, our quantity of interest is the seed\-paired contrastRA−\-plainandsequence−\-set: since only the reward changes between the two conditions, the contrast attributes any gain to the reward\.

We understand that this estimate may not be fully honest, because the evaluation schema coincides with the property\-aware arm’s own reward \(RA, sequence\), which can favor that arm\. Therefore, we cross\-check every contrast against a fully schema\-independent, task\-native metric wherever one exists \(defined per dataset in Appendix[E](https://arxiv.org/html/2607.01972#A5)\); on every real\-world dataset that carries one, the native metric agrees in the direction of the OA\-score conclusion\.

#### Statistical reporting\.

Every extrinsic contrast is seed\-paired \(the same1010seeds in both conditions\)\. We summarize its uncertainty with a95%95\\%bias\-corrected\-and\-accelerated \(BCa\) bootstrap confidence interval over the per\-seed differences \(20,00020\{,\}000resamples\), reported inline for the small\-effect cells where the direction is not otherwise evident\. We treat these intervals descriptively—as effect\-size estimates rather than a battery of hypothesis tests—and throughout call a contrast*detectable*when its interval excludes zero and*within noise*when it does not\.

#### Research questions\.

We organize the results around three questions, and at this stage, we only*enumerate*the cases in which each extension helps; the dataset properties that drive these patterns are discussed in Section[VI](https://arxiv.org/html/2607.01972#S6)\.

- •RQ1\.When does referential alignment \(RA\) improve the optimized prompt?
- •RQ2\.When does the order\-sensitive \(sequence\) regime help over the order\-agnostic \(set\) one?
- •RQ3\.When does the deterministic feedback help over the scalar score alone?

![Refer to caption](https://arxiv.org/html/2607.01972v1/x8.png)Figure 6:The three contrast effects drawn as*dumbbells*on the absoluteObject Alignerscore\. Each series joins its two contrast conditions \(filled and open\) with a connector whose length shows the seed\-paired effect\. The thin caps depict±1\\pm 1std over seeds\. Colour denotes the task model; in panels\(A,B\)marker shape denotes the reflection arm \(circle==feedback, triangle==score\)\.\(A\)RA vs\. plain on theObject Alignerscore under the RA schema\.\(B\)sequence vs\. set on the sequenceObject Alignerschema\.\(C\)feedback vs\. score on each dataset’sObject Alignerevaluation schema, pooled over both ablations and shown for every dataset on both axes\. Dataset codes: O2G==Org2Graph \(P/C==plain/coded values, W/N==wide/narrow vocabulary\), F2O==Facts2Order \(S/H==stated/hidden sort key\)\.

### V\-CReferential alignment

Table[III](https://arxiv.org/html/2607.01972#S5.T3)and Fig\.[6](https://arxiv.org/html/2607.01972#S5.F6)\(A\) compare the RA and plain rewards across seven datasets: four synthetic Org2Graph variants and three real\-world graph\-extraction tasks—SciERC, BioRED, and Bio AMR\. The effect is sharply dataset\-dependent\. RA pays off on the synthetic Org2Graph family, and there the gain grows with the difficulty of recovering the graph: from\+0\.03\+0\.03–0\.070\.07\(PW to PN, Gemma\-4\-26B\) on the*plain*variants \(readable categorical values\) to\+0\.10\+0\.10–0\.210\.21\(CW on Gemma\-4\-E4B to CN on Gemma\-4\-26B\) on the*coded*variants, where the gold values are opaque codes the optimizer must learn a legend for\. Crucially, this advantage appears almost entirely in the feedback arm—under the score arm the RA\-vs\-plain gap collapses to within noise \(Fig\.[6](https://arxiv.org/html/2607.01972#S5.F6)\(A\), open markers\), so RA and feedback are not independent levers but interact: the relabel\-invariant reward only becomes actionable once the feedback localizes which references are wrong\. Both task models exhibit the same pattern\.

On SciERC and BioRED, the effect is far less pronounced, bordering on noise: the seed\-paired RA−\-plain gap stays within noise in every cell but one \(Gemma\-4\-26B feedback/score−0\.001\-0\.001/\+0\.005\+0\.005on SciERC and\+0\.011\+0\.011/\+0\.026\+0\.026888The lone exception is the BioRED Gemma\-4\-26B score arm:\+0\.026\+0\.026\(CI\[\+0\.012,\+0\.048\]\[\+0\.012,\+0\.048\]\), a small gain at the edge of detectability\.on BioRED\)\. Although BioRED’s gold entity ids are opaque concept accessions not derivable from the text \(which in principle should force referential routing\), its entities stay well individuated by their surface mentions and its gold relations are sparse, therefore, exactly as in SciERC, the relabel\-invariant reward has little routing left to recover\. Bio AMR provides a different picture: here, the RA advantage approaches the coded Org2Graph variants in magnitude\. Its dense AMR graphs reuse the same concept symbol across many nodes, so node content individuates almost nothing, and the score turns entirely on reference routing\. Here, though, the pattern flips: the RA advantage lives in the*score*arm and all but disappears under feedback \(Gemma\-4\-26B feedback/score\+0\.026\+0\.026/\+0\.111\+0\.111;−0\.010\-0\.010/\+0\.093\+0\.093on Gemma\-4\-E4B\)—the mirror image of coded Org2Graph\. The two arms diverge over*where*the routing signal can travel\. A plain score is blind to routing on AMR: edges are keyed on gold node identifiers the model never sees, so the scalar rewards only node content and leaves routing untouched—and RA reward supplies exactly that missing edge credit, hence its large score\-arm gain\. Under feedback, the gap closes instead, because OA’s feedback*text*still spells out the mis\-routed edges even when the plain score ignores them; the reflection model uses that referential information in the prompt even though GEPA’s plain\-score selection never rewards it, so plain and RA feedback converge on the same graph\. In effect, plain feedback has already done the RA’s job\.

The contrast with coded Org2Graph turns on a single difference: AMR node content is genuine, text\-derivable semantics \(e\.g\.,express\-03,protein,cell\), whereas Org2Graph’s coded identifiers are arbitrary codes with no content handle\. That semantic content is what lets plain feedback leak the routing here: the fixes it lists are concrete content corrections the model can actually act on \(e\.g\., “add a node for this entity”\), and carrying them out repairs the graph structure as a by\-product—so under feedback plain already teaches routing and the RA advantage collapses there\. On coded Org2Graph plain feedback can only report unfollowable code mismatches, so relabeling is recoverable only through RA feedback and the advantage instead survives there, shifting the collapse to the score arm\.

TABLE III:Referential\-alignment axis\. Each ablation optimizes the prompt \(via GEPA\) against its ownObject Alignerreward as fitness \(using RA or plain schema, Section[V\-B](https://arxiv.org/html/2607.01972#S5.SS2)\) All configurations are then evaluated using the RA schema, measured on each dataset’s held\-out test split \(mean±\\pmseed std,1010seeds\); the larger of RA/plain within each arm isbold\. The final pair is the feedback−\-score improvement per ablation \(means only\)\. O2G==Org2Graph variants \(P/C==plain/coded values, W/N==wide/narrow vocabulary\)\.The Org2Graph sweep over four fixed variants spanning obfuscation×\\timeswidth \(*plain*/*coded*×\\times*wide*/*narrow*; Section[IV\-A](https://arxiv.org/html/2607.01972#S4.SS1)\) makes the difficulty scaling explicit; the per\-variant feedback\-arm numbers are the four O2G rows of Table[III](https://arxiv.org/html/2607.01972#S5.T3)and the top four series in Fig\.[6](https://arxiv.org/html/2607.01972#S5.F6)\(A\)\. The family varies on two axes: identifier obfuscation \(plain readable values vs\. opaque codes\) and vocabulary size \(wide vs\. narrow code pools\), with graph topology held fixed along the obfuscation axis so that plain and coded variants differ only in surface form\. Obfuscation is the dominant driver: switching from plain to coded values widens the RA advantage several\-fold on both models \(RA−\-plain rising from\+0\.03\+0\.03/\+0\.07\+0\.07\(PW/PN\) on the plain variants to\+0\.14\+0\.14/\+0\.21\+0\.21\(CW/CN\) on the coded variants for Gemma\-4\-26B, and from\+0\.05\+0\.05/\+0\.06\+0\.06to\+0\.10\+0\.10/\+0\.15\+0\.15for Gemma\-4\-E4B\), because once the surface tokens carry no signal the only way to credit a correct edge is through the inferred identifier bijection\. Narrowing the vocabulary, which forces more property\-identical records and thus leans harder on the structural tie\-break, adds a smaller further increment\. The advantage is monotone in our informal difficulty ranking\.

### V\-DOrder sensitivity

Table[IV](https://arxiv.org/html/2607.01972#S5.T4)and Fig\.[6](https://arxiv.org/html/2607.01972#S5.F6)\(B\) compare the sequence and set rewards, both scored on the sequenceObject Alignerschema\. Order sensitivity helps precisely when the position of each element is itself part of the answer\. Evaluated on the sequence schema, optimizing with the sequence reward drives the score from approximately0\.830\.83to0\.980\.98, against0\.390\.39to0\.580\.58for the set reward—a sequence−\-set gap between\+0\.33\+0\.33\(F2O S, Gemma\-4\-E4B\) and\+0\.48\+0\.48\(F2O H, Gemma\-4\-26B\) on every variant and both models\. The effect carries over to our real ordering tasks to a lesser extent: on ROCStories, the sequence reward increases the sequence\-schema score by approximately0\.060\.06\. On NATURAL PLAN the gain is marginal—detectable but practically negligible: the sequence−\-set gap stays≤\+0\.025\\leq\+0\.025across all four cells—about\+0\.02\+0\.02–0\.0250\.025on Gemma\-4\-26B \(both arms\) and≤\+0\.009\\leq\+0\.009on Gemma\-4\-E4B\.999Although small, the Gemma\-4\-26B gap is statistically detectable:\+0\.025\+0\.025\(95%95\\%CI\[\+0\.008,\+0\.045\]\[\+0\.008,\+0\.045\]\) under the score reward and\+0\.023\+0\.023\(95%95\\%CI\[\+0\.005,\+0\.046\]\[\+0\.005,\+0\.046\]\) under feedback, both excluding zero; on Gemma\-4\-E4B it is indistinguishable from zero\.

The contrast with Facts2Order explains this result\. The NATURAL PLAN itinerary is also strictly ordered \(Section[IV\-F](https://arxiv.org/html/2607.01972#S4.SS6)\), solving the task already entails finding the order: to satisfy the direct\-flight graph and the day budget, the model must determine which city follows which, and once it has, it emits the cities in that sequence as the natural way to present the answer—even though the order\-agnostic set reward never asks it to\. Therefore, ordering is a by\-product of the solution, not a separate thing to optimize, so rewarding it adds little\. Specifically, the set\-optimized prompt already scores0\.8550\.855/0\.6550\.655on the sequence schema, only marginally below the sequence\-optimized0\.8780\.878/0\.6640\.664\. Facts2Order is the mirror image: the items are handed to the model already solved, and only their arrangement by the sort key is in question, so the set reward leaves ordering unsolved \(on F2O H the set\-optimized prompt collapses to0\.3870\.387on the sequence schema against0\.8710\.871for sequence\-optimized\), and ordering is the entire task\.

TABLE IV:Order\-sensitivity axis\. Each ablation optimizes the prompt \(via GEPA\) against its ownObject Alignerreward as fitness \(using sequence or set schema, Section[V\-B](https://arxiv.org/html/2607.01972#S5.SS2)\), and all configurations are then evaluated using the sequence schema, measured on each dataset’s held\-out test split \(mean±\\pmseed std,1010seeds\); the larger of sequence/set within each arm isbold\. The final pair is the feedback−\-score improvement per ablation \(means only\)\. F2O==Facts2Order variants \(S/H==stated/hidden sort key\)\.The two Facts2Order variants contrast the sort\-key visibility \(stated→\\tohidden; Section[IV\-B](https://arxiv.org/html/2607.01972#S4.SS2)\)\. Both draw the item count fromN∈\{4,5,7,9\}N\\in\\\{4,5,7,9\\\}, the key type from \{integer, date, ordinal\}, and0or22irrelevant distractor sentences\. In the stated variant \(S\), each item carries a single key clause; in the hidden variant \(H\), each item additionally carries22–33numeric decoy clauses, shuffled per item so the sort field must be discovered fromObject Alignerfeedback \(Fig\.[14](https://arxiv.org/html/2607.01972#A3.F14), Appendix[C](https://arxiv.org/html/2607.01972#A3), shows a worked hidden\-key instance—the most complex configuration\)\. Their per\-variant numbers are the two F2O rows in Table[IV](https://arxiv.org/html/2607.01972#S5.T4)and the corresponding series in Fig\.[6](https://arxiv.org/html/2607.01972#S5.F6)\(B\)\. Unlike the RA sweep, where the advantage only emerges as extraction grows harder, the sequence−\-set gap is large on*every*variant \(\+0\.33\+0\.33–0\.480\.48\): when the answer is itself an ordering, the order\-agnostic set reward leaves the bulk of the signal on the table regardless of difficulty\. Hiding the sort key instead moves the*absolute*score and the stability of optimization: it lowers the sequence score and, on Gemma\-4\-26B, leaves it seed\-unstable \(0\.871±0\.1480\.871\\pm 0\.148\) under both rewards alike—a score\-vs\-feedback question taken up in Section[V\-E](https://arxiv.org/html/2607.01972#S5.SS5), not the sequence\-vs\-set effect\.

### V\-EFeedback

Across all 11 dataset×\\timesaxis cells, the feedback−\-score columns of Tables[III](https://arxiv.org/html/2607.01972#S5.T3)and[IV](https://arxiv.org/html/2607.01972#S5.T4)and the summary in Fig\.[6](https://arxiv.org/html/2607.01972#S5.F6)\(C\) indicate that feedback helps in proportion to how much structure the task hides from a scalar reward\. It is decisive where the optimizer must*discover*something a number cannot convey: a value\-to\-code legend, a relation schema, or a strict output format\. SciERC is the starkest case: the score arm collapses to within noise, whereas feedback recovers a working extractor\. The same mechanism drives the large gains in the coded Org2Graph variants and Bio AMR\. Where the format is simple and a scalar gradient already locates the error, the feedback adds nothing\. The order\-sensitivity axis is the cleanest negative\. Across all four ordered\-output datasets, the feedback−\-score gap remains within noise on both arms and both models \(Table[IV](https://arxiv.org/html/2607.01972#S5.T4)\); the two largest movements are a faint\+0\.022\+0\.022on NATURAL PLAN and\+0\.021\+0\.021on F2O H, both on the smaller Gemma\-4\-E4B under the sequence reward\.

![Refer to caption](https://arxiv.org/html/2607.01972v1/x9.png)Figure 7:Feedback\-breadth sweep\. The scalar reward is held*identical*across all arms; only the amount of ranked\-correction*items*the OA metric shows the GEPA reflection LM varies with the correction capKK\(Section[III\-F](https://arxiv.org/html/2607.01972#S3.SS6)\), from none \(score,K=0K\{=\}0\) throughK=1K\\,\{=\}\\,1,55\(the OA default\), and1010, to all uncapped corrections \(K=∞K\{=\}\\infty\)\. Thegoldcolumn is an additional baseline that appends the full serialized gold answer\. Each point is the mean GEPA\-optimized holdout OA score over seeds \(whiskers±1\\pm 1std\); the per\-datasetxxpositions are slightly offset to reduce overlap\. Both Gemma models are shown\.Fig\.[7](https://arxiv.org/html/2607.01972#S5.F7)resolves the contrast into a breadth curve: holding the scalar reward identical, it varies only the number of ranked correctionsKKtheObject Alignermetric shows the reflection LM\. We run the full sweep on the five cells that most sharply discriminate the arms—one plain and one coded Org2Graph variant rather than all four, the two real graph tasks where the structure is most hidden \(SciERC, Bio AMR\), and ROCStories as the order\-axis control\. The shape of the curve tracks difficulty: on SciERC and the plain O2G PN, it is steeply concave, a single ranked correction \(K=1K\{=\}1\) already captures most of the gain, whereas on the harder coded O2G CW and the dense Bio AMR, it climbs gradually and most of the gain arrives only byK=5K\{=\}5–1010\. On ROCStories it remains flat throughout, mirroring the null result above\.

Thegoldcolumn appends the full serialized gold answer in the reflection slot—an upper\-information control rather than a usable reward signal\. It is not a standard input: handed the answer, the reflection LM must itself work out which parts of its output were wrong, in effect performing LLM\-as\-judge, and on a more verbose prompt than any deterministic reward would emit\. It is also markedly less stable across the seeds on the hardest cells: on the coded O2G CW thegoldarm spans±0\.15\\pm 0\.15, against±0\.05\\pm 0\.05for the capped feedback arms\. Its purpose is to bound the value of feedback information from above, andObject Alignerapproaches that bound with little feedback\. The defaultK=5K\{=\}5arm already nearsgoldon most cells, and the uncapped arm meets or exceeds it on the harder ones \(Bio AMR, the coded Org2Graph variants, SciERC\)\. The fact that a short ranked list rivals showing the entire answer is direct evidence the gains come fromObject Aligner’s*prioritized, localized*corrections, not from answer leakage\. It also shows thatK=5K\{=\}5, the cap used throughout our main experiments, was conservative rather than optimal, whereas on the hardest cells, the curve is still climbing atK=5K\{=\}5, so a larger cap would have scored higher at the cost of a longer feedback—a trade we did not tune per dataset\.

## VIDiscussion

The intrinsic study \(Section[V\-A](https://arxiv.org/html/2607.01972#S5.SS1)\) shows that each new mechanism is the*uniquely correct*measurement of its target property: referential alignment \(RA\) separates damaged graphs from clean ones almost perfectly, where the reference\-unaware*plain*ablation, blind to reference rerouting, does not, and the*sequence*regime is the only scorer that grades order at all\. The extrinsic study asks whether a reward that*perceives*that property yields a better prompt, and there the answer is conditional\. The lesson is thata faithful measurement is not automatically a useful reward: a property\-aware reward helps the search only when the property is \(i\)*not already recovered as a by\-product*of solving the task, and \(ii\)*actionable*by the optimizer\. The first condition governs when RA \(RQ1\) and the sequence regime \(RQ2\) help, and the second condition connects both to deterministic feedback \(RQ3\)\.

RQ1\.RA helps precisely when node content fails to individuate records, so the score must turn on reference routing rather than on attributes: coded Org2Graph variants, whose identifiers are opaque codes, and Bio AMR, whose dense graphs reuse the same concept symbol across many nodes\. Where surface mentions already individuate entities and relations are sparse \(SciERC, BioRED\), routing is solved before RA is consulted, and the gap becomes small\. In the synthetic family, the advantage is monotone in extraction difficulty, widening as readable values become opaque codes, with obfuscation the dominant driver and vocabulary size the smaller one\. This is not “RA fails on real data”: RA is the only setting under which a rerouted edge that leaves attributes intact is visible to the score at all, and it pays off extrinsically on Bio AMR, where the same gain carries over in sign to AMR’s own Smatch metric \(Appendix[E](https://arxiv.org/html/2607.01972#A5)\)\. What we could not find is a naturally occurring corpus reproducing Org2Graph’s adversarial regime—many property\-twin records behind obfuscated ids—under which RA’s payoff is largest; whether such “realistic” data exists in the wild is an open question that normalized relational databases, whose surrogate keys carry no meaning, may answer \(Section[IX](https://arxiv.org/html/2607.01972#S9)\)\. Several of the datasets that we spot\-checked behaved consistently\. RA and feedback are also not independent levers: on coded Org2Graph, the RA advantage lives in the feedback arm and vanishes under the scalar score, whereas on Bio AMR, it is the mirror image\. The difference is that AMR concepts are genuine, text\-derivable semantics, so even plain feedback lists content fixes that repair structure as a by\-product, whereas coded identifiers carry no such handle and are recoverable only through RA feedback\.

RQ2\.The sequence reward is helpful only when the arrangement*is*the deliverable\. On Facts2Order, where items arrive already solved and only their ordering is in question, the sequence reward beats the order\-agnostic set reward by a wide margin for every variant\. NATURAL PLAN is the contrast: satisfying the flight graph and day budget already forces the city order, which the model then emits as the natural presentation even though the set reward never asks for it, so the sequence reward adds only a marginal gain; ROCStories sits between with a modest gain\. Hiding the sort key lowers the*absolute*score and its seed stability under*both*rewards alike: a difficulty effect, not a sequence\-vs\-set one\. As with RA, this neutrality is a property of the data, not the extension failing: where correct ordering is inherent to the task, the sequence reward has little extra to teach, yet it remains the only scorer that detects an incorrect order when one occurs\. Order sensitivity thus helps as a reward when the output order is itself part of the answer, and is neutral when a correct solution fixes it for free\.

RQ3\.Deterministic feedback is the most consistently valuable mechanism, and its value scales with how much structure the task hides from a scalar reward\. It is decisive where the optimizer must*discover*something a number cannot convey—a value\-to\-code legend, a relation schema, a strict output format: on SciERC the score arm collapses to within noise while feedback recovers a working extractor, and the same drives the gains on coded Org2Graph and Bio AMR\. The feedback\-breadth sweep \(Fig\.[7](https://arxiv.org/html/2607.01972#S5.F7)\) rules out the obvious confound:Object Aligner’s short ranked list of localized repairs*matches*thegoldbaseline \(which appends the entire answer\) on most cells and*exceeds*it on the hardest ones, while appending less text, direct evidence the gains come from prioritized, localized corrections rather than answer leakage\.

The two checks guard against artifacts\. Task\-model capacity: the two Gemma models bracket a wide range, and capacity shifts magnitudes and absolute scores but not the*sign or ordering*of any contrast whose95%95\\%CI excludes zero\. Because the reward GEPA optimized was always anObject Alignerschema, we cross\-check against a schema\-independent native metric on every real\-world dataset that carries one \(Appendix[E](https://arxiv.org/html/2607.01972#A5)\): the native verdict agrees in direction with the headline conclusions, diverging only in informative ways\.

For practitioners, the advice is compact and shares one shape: each extension is the correct measurement of its property, and as a reward, each only helped or stayed neutral in our experiments—never a meaningful loss on anyObject Aligner\-score contrast101010The one exception is negligible: on the ROCStories set arm, feedback trails score by−0\.004\-0\.004\(95%95\\%bootstrap CI\[−0\.006,−0\.000\]\[\-0\.006,\-0\.000\]\)\.—so the practical question is mainly where each*pays off most*\. Enable RA for any \(hyper\)graph\-structured output: it is correct whenever quality is judged up to isomorphism, degrades to plain comparison in the worst case, and pays off most when identifiers are arbitrary and content does not individuate records\. Enable the sequence regime whenever the output order is part of correctness—there it never hurts and pays off most when ordering is the deliverable\. Finally deterministic feedback is preferred by default: it is the broadest\-paying lever, costs no extra model call, and pays off most where the task hides structure from a scalar reward\. Unlike an LLM judge, it is reproducible and auditable across any number of rollouts \(Section[I](https://arxiv.org/html/2607.01972#S1)\)\.

## VIILimitations

The main limitations fall into two categories: intrinsic properties of the metric and the scope of our empirical evaluation\.

- •Ordinal, not calibrated\.The score is ordinal rather than calibrated:0and11are anchored and scores are comparable, but an intermediate value such as0\.510\.51is not a literal fraction of correctness—a propertyObject Alignershares with many similarity metrics\.
- •Approximate identifier bijection\.Referential alignment recovers the gold–candidate identifier bijection by Hungarian assignment on masked attributes, tie\-broken by11\-WL color refinement \(Section[III\-E](https://arxiv.org/html/2607.01972#S3.SS5)\), which is sound but incomplete approximation to graph isomorphism\. On graphs,11\-WL cannot separate, the inferred bijection—and thus the score—can be suboptimal; higher\-orderkk\-WL would tighten it \(Section[IX](https://arxiv.org/html/2607.01972#S9)\)\.
- •Self\-referential and mutually\-referential scopes\.Referential alignment resolves scopes in dependency order, so mutually referential scopes \(or scope referencing itself\) form a cycle with no valid order and fall back to property\-only alignment, discarding the reference\-value signal\. Resolving such cases consistently is a fixed\-point problem \(akin to collective entity resolution\[[6](https://arxiv.org/html/2607.01972#bib.bib61)\]\), which we leave for future work\. In practice, many such cycles can be sidestepped by*reifying*the offending references as separate junction \(link\) table—pure reference\-carrier records with noidScopeof their own, the relational model’s*relationship relation*\[[9](https://arxiv.org/html/2607.01972#bib.bib62)\]—which restores an acyclic scope order, at the cost of treating each edge as an independently matchable record rather than a per\-record attribute\.
- •Scope of the empirical study\.We use a single optimizer \(GEPA\), a single reflection LM \(a strong frontier model, which is best practice\), and two task models chosen partly by our available infrastructure; whether the extrinsic findings transfer to other prompt\-optimization frameworks or model families is untested, although our partial Qwen robustness check was consistent\.
- •Limited hyperparameter exploration\.The cost of the full grid prevented us from sweeping alternative schema\-construction strategies, GEPA hyperparameters, andObject Alignermetaparameters \(e\.g\., the feedback list lengthKK\)\.
- •Not a benchmark study\.Our aim was to demonstrateObject Aligner’s general properties in a prompt\-optimization setup, not to set state\-of\-the\-art baselines on the datasets; as a highly configurable metric,Object Alignermay need per\-task schema tuning for the best performance\.

## VIIIConclusion

We presentedObject Aligner, an open\-source Python library that scores structured LLM outputs against a gold reference deterministically by recursively aligning their JSON trees and awarding partial credit at the granularity a schema declares\. Because it is configured entirely through JSON Schema extensions, adapting theObject Alignerto a new task is a matter of annotating a schema rather than writing code, and it drops directly into existing prompt\-optimization frameworks such as DSPy, GEPA, and TextGrad as a reproducible, auditable, and model\-free alternative to an LLM judge\. Its central contribution,*referential alignment*, makes the score invariant to identifier relabeling, so that cross\-referenced records—graphs and hypergraphs—can be compared up to isomorphism, while a complementary order\-sensitive*sequence*regime targets ranking, and the same alignment emits ranked, localized repair suggestions at no extra model cost\. Used as the reward inside GEPA across synthetic and real\-world datasets,Object Alignerhelped or stayed neutral, never a meaningful loss: referential alignment pays off most when identifiers are arbitrary and content fails to individuate records, the sequence regime when the output order is itself the deliverable, and deterministic feedback proved the broadest\-paying lever\. These findings translate into compact, deployable guidance—enable referential alignment for any \(hyper\)graph output, the sequence regime whenever order is part of correctness, and prefer deterministic feedback by default—positioningObject Aligneras a practical, drop\-in measurement and feedback signal for building and optimizing the structured\-output pipelines on which LLM applications increasingly depend\.

## IXFuture Work

Three directions follow most directly from our findings and the limitations above:

- •Referential alignment for relational data\.The normalized relational databases reproduce Org2Graph’s adversarial regime \(Section[VI](https://arxiv.org/html/2607.01972#S6)\): surrogate keys that carry no content\-derived meaning, with foreign keys that are exactlyObject Aligner’sidScope/refstructure, with junction tables as the purest property\-twin case\. This suggests the application ofObject Alignerto text\-to\-database extraction and database\-state diffing, building on the record\-linkage lineage \(Section[II\-A](https://arxiv.org/html/2607.01972#S2.SS1)\), and collective entity resolution\[[6](https://arxiv.org/html/2607.01972#bib.bib61)\]\.
- •Differentiable surrogates\.Object Aligner’s score is derived from a discrete Hungarian assignment; therefore, it can only act as a black\-box scalar or textual reward \(Section[VI](https://arxiv.org/html/2607.01972#S6)\)\. Relaxing the assignment into a soft, entropy\-regularized \(Sinkhorn\-style\) matching may yield a differentiable surrogate, openingObject Alignerto gradient\-based optimization and end\-to\-end fine\-tuning rather than search alone\.
- •Stronger structural tie\-breaking\.Replace 1\-WL with higher\-orderkk\-dimensional Weisfeiler–Leman to resolve its known blind spots \(e\.g\., one66\-cycle vs\. two disjoint33\-cycles\), trading the near\-linear cost for the combinatorial\-in\-kkscaling ofkk\-WL\[[32](https://arxiv.org/html/2607.01972#bib.bib39)\]\.

## Appendix ASchema Keyword Reference

Table[V](https://arxiv.org/html/2607.01972#A1.T5)maps the mathematical notation used in Section[III](https://arxiv.org/html/2607.01972#S3)to the concrete schema keywords accepted by the implementation\. The schema reuses the JSON Schema surface syntax; standard validation keywords \(required,enum,minItems, …\) are honored when validating the candidate but do not affect the score\. Custom primitive comparators are registered on the aligner and referenced by name\. Table[V](https://arxiv.org/html/2607.01972#A1.T5)lists only theObject Alignerscoring extensions\.

TABLE V:Notation of Section[III](https://arxiv.org/html/2607.01972#S3)versus schema keywords\. Defaults in the last column\.Concept / symbolKeywordDefaultPrimitive leaf\(string / number / boolean\)comparatorσ\\sigma\(string\)scorejarocomparatorσ\\sigma\(number\)scoreinvdiffthresholdτ\\tauthreshold0empty\-value score \(asymmetric\)nullScore0Map\(object\)key comparatorκ\\kappakeyScorejarokey thresholdτK\\tau\_\{K\}keyThreshold0key weightwKw\_\{K\}keyImportance0value weightwVw\_\{V\}valueImportance11per\-property weightωi\\omega\_\{i\}valueWeight11Sequence\(array\)order regimeorderfixedorder\-agnostic"align"order\-sensitive"fixed"prefix weightsprefixWeights11penalize missing \(nmissn\_\{\\mathrm\{miss\}\}\)ignoreMissingpenalizepenalize excess \(nexcn\_\{\\mathrm\{exc\}\}\)ignoreExcesspenalizeprefix importancewpw\_\{p\}prefixImportance—tail importancewrw\_\{r\}restImportance—Referential\(primitive inside a sequence\)identifier fieldidScope—reference fieldref—TheObject Alignerscoring extensions that appear in the worked schema examples throughout the paper are glossed below: the standard JSON Schema keywords carry their usual meaning\. The authoritative specification is the official documentation at[https://github\.com/aic\-factcheck/object\_aligner](https://github.com/aic-factcheck/object_aligner)\.

- •score— primitive comparator \(exact,jaro,invdiff, …\)\.
- •threshold— minimum primitive similarity to count as a match; below it the pair scores0\.
- •order— array regime:"align"\(order\-agnostic Hungarian matching\) or"fixed"\(order\-sensitive\)\.
- •keyImportance/valueImportance— relative weight of an object’s keys vs\. its values in the node score\.
- •valueWeight— per\-property weight within an object\.
- •prefixWeights— per\-position weights for theprefixItemsentries\.
- •ignoreExcess— do not penalize extra, unmatched candidate elements\.
- •idScope— declares a primitive as an identifier within a named scope \(enabling referential alignment\)\.
- •ref— declares a primitive as a reference resolved against a namedidScope\.

## Appendix BDatasets

This appendix details the real\-world datasets used in this study: how each was preprocessed from its source corpus into the JSON output shape we score, theObject Alignerschemas, and the task\-native metric we report alongside theObject Alignerscore\. The synthetic probes \(Org2Graph andFacts2Order\) are generated rather than preprocessed, and carry no external metric\. Because our preprocessing reshapes the original data, the native metric we report sometimes diverges from the dataset’s official metric; we flag each such divergence in the relevant subsection\.

### B\-ASciERC

The raw corpus is thesciERC\_processedrelease of\[[27](https://arxiv.org/html/2607.01972#bib.bib64)\]\([http://nlp\.cs\.washington\.edu/sciIE/](http://nlp.cs.washington.edu/sciIE/)\)\. Each raw document carries per\-sentence token lists, NER spans \(token offsets\), mention\-pair relations, and coreference clusters\. Preprocessing turns each coreference cluster into one entity object \(Fig\.[8](https://arxiv.org/html/2607.01972#A2.F8), middle\) whosementionslist holds its de\-duplicated surface forms, with the clustertypedecided by the majority vote across its spans; NER spans in no cluster become singleton entities\. Mention\-pair relations are projected to cluster\-level\(subject, predicate, object\)triples and de\-duplicated, and any relation with an endpoint outside a cluster is dropped\. Token and character offsets are discarded\. One training document fails to parse, resulting in the349/50/100349/50/100native split\. The schema scored in the reported run \(Fig\.[8](https://arxiv.org/html/2607.01972#A2.F8), bottom\) applies Hungarian alignment at two levels: first, to match entities and then to match the mentions within each matched entity\. A coreference cluster that is wrongly split or merged, and therefore costs score\.

RecognitionofpropernounsinJapanesetexthasbeenstudiedasapartofthemoregeneralproblemof

morphologicalanalysis…Ourapproach…istoconsiderthegiventaskasamorphologicalanalysisproblem…

\{"entities":\[

\{"id":"e1","type":"Task","mentions":\[\{"text":"Recognitionofpropernouns"\},\{"text":"It"\}\]\},

\{"id":"e3","type":"Task","mentions":\[\{"text":"morphologicalanalysis"\},\{"text":"morphologicalanalysisproblem"\}\]\},

\{"id":"e4","type":"Method","mentions":\[\{"text":"‘‘Amorph’’"\},\{"text":"Amorph"\},\{"text":"analyzer"\},\{"text":"it"\}\]\}\],

"relations":\[\{"subject":"e1","predicate":"PART\-OF","object":"e3"\}\]\}

\{"type":"object","properties":\{

"entities":\{"type":"array","order":"align","items":\{"type":"object","properties":\{

"id":\{"type":"string", "idScope":"entity"\},

"type":\{"type":"string","score":"exact"\},

"mentions":\{"type":"array","order":"align","items":\{"type":"object","properties":\{

"text":\{"type":"string","score":"exact"\}\}\}\}\}\}\},

"relations":\{"type":"array","order":"align","items":\{"type":"object","properties":\{

"subject":\{"type":"string", "ref":"entity"\},

"predicate":\{"type":"string","score":"exact"\},

"object":\{"type":"string", "ref":"entity"\}\}\}\}\}\}

Figure 8:SciERC\.*Top*: the input*context*\(title and abstract\)\.*Middle*: the gold output—each entity is a coreference cluster, relations reference entities by id\.*Bottom*: the schema as run; thehighlightedfragments are exactly what referential alignment adds—idScopeon entity ids andrefon relation endpoints; theplainablation simply omits them, leaving plain string leaves compared by literal equality\.As the native metric, we use SciERC’s coref\-aware*entity\-level*relation F1: predicted entity clusters are aligned to gold, and then relations are scored at the cluster level and micro\-averaged, with the symmetric relations \(COMPARE,CONJUNCTION\) matching in either argument order\. Because preprocessing discards the token offsets and aligns clusters by their surface mention text, this entity\-level F1 is not the official span\-based SciERC relation F1, which matches entity boundaries by offset\. The same reshaping that defines our task also redefines the metric\.

### B\-BBioRED

The raw corpus is the BioRED release of\[[28](https://arxiv.org/html/2607.01972#bib.bib65)\]\([https://ftp\.ncbi\.nlm\.nih\.gov/pub/lu/BioRED/BIORED\.zip](https://ftp.ncbi.nlm.nih.gov/pub/lu/BioRED/BIORED.zip)\):600600PubMed titles and abstracts with document\-level entity and relation annotations\. Preprocessing closely follows SciERC’s \(Appendix[B\-A](https://arxiv.org/html/2607.01972#A2.SS1)\): mentions collapse into typed, coreference\-merged entities \(typeby majority vote across the mentions; Fig\.[9](https://arxiv.org/html/2607.01972#A2.F9), middle\), relations become de\-duplicated\(subject, predicate, object\)triples with any out\-of\-entity endpoint dropped, offsets are discarded, and title and abstract form the inputcontext\(Fig\.[9](https://arxiv.org/html/2607.01972#A2.F9), top\)\. Two differences matter\. First, an entity is keyed by its*normalized concept id*—an Entrez or MeSH accession fixed by the source databases and not derivable from the abstract, whereas SciERC’s entity ids are arbitrary; the model must invent its own, so relations are routed by referential alignment rather than literal\-id matching \(Fig\.[9](https://arxiv.org/html/2607.01972#A2.F9), bottom\), and a mention tagged with several ids joins every matching entity\. Second, the schema scores mentiontextbyjaro\_winklerrather than exact match and up\-weights entitytype\. We draw a seeded50/50/10050/50/100train/val/test subsample\.

Debrisoquine phenotype and the pharmacokinetics … of metoprolol and its enantiomers\.

Themetabolismofthecardioselectivebeta\-blockermetoprololisundergeneticcontrolofthe

debrisoquine/sparteinetype…

\{"entities":\[

\{"id":"D003647","type":"ChemicalEntity","mentions":\[\{"text":"Debrisoquine"\},\{"text":"debrisoquine"\}\]\},

\{"id":"154","type":"GeneOrGeneProduct","mentions":\[\{"text":"beta\-2receptor"\},\{"text":"beta\-2adrenoceptor"\}\]\},

\{"id":"D008790","type":"ChemicalEntity","mentions":\[\{"text":"metoprolol"\}\]\},

\{"id":"153","type":"GeneOrGeneProduct","mentions":\[\{"text":"beta\-1adrenoceptor"\}\]\}\],

"relations":\[

\{"subject":"153","predicate":"Negative\_Correlation","object":"D008790"\},

\{"subject":"D008790","predicate":"Association","object":"D003647"\}\]\}

\{"type":"object","properties":\{

"entities":\{"type":"array","order":"align","items":\{"type":"object","properties":\{

"id":\{"type":"string", "idScope":"entity"\},

"type":\{"type":"string","score":"exact","valueWeight":1\.5\},

"mentions":\{"type":"array","order":"align","items":\{"type":"object","properties":\{

"text":\{"type":"string","score":"jaro\_winkler"\}\}\}\}\}\}\},

"relations":\{"type":"array","order":"align","items":\{"type":"object","properties":\{

"subject":\{"type":"string", "ref":"entity"\},

"predicate":\{"type":"string","score":"exact"\},

"object":\{"type":"string", "ref":"entity"\}\}\}\}\}\}

Figure 9:BioRED\.*Top*: the input*context*—title \(bold\) and abstract\.*Middle*: the gold output—entities are coreference clusters keyed by normalized concept ids \(the model invents its own\), relations reference them by id\.*Bottom*: the schema as run; thehighlightedidScope/reffragments are what referential alignment adds over theplainablation\. Mentiontextusesjaro\_winkler;typeis up\-weighted\.The native metric mirrors SciERC’s coreference\-aware entity\-level relation, F1 \(Appendix[B\-A](https://arxiv.org/html/2607.01972#A2.SS1)\), but differs in two ways\. BioRED relations are directed, so there is no either\-order matching \(SciERC allows it for its symmetric predicates\)\. The alignment pairs entities by their surface mentions, not their gold concept ids, so the score is not comparable to the official id\-keyed metric\.

### B\-CBio AMR

The raw corpus is Release 3\.0 of the Bio AMR Corpus\[[53](https://arxiv.org/html/2607.01972#bib.bib55)\]\([https://amr\.isi\.edu/download/2018\-01\-25/amr\-release\-bio\-v3\.0\.txt](https://amr.isi.edu/download/2018-01-25/amr-release-bio-v3.0.txt)\):∼6,900\{\\sim\}6\{,\}900sentences from cancer\-related PubMed articles \(each the inputcontext; Fig\.[10](https://arxiv.org/html/2607.01972#A2.F10), top\) annotated with Abstract Meaning Representation\[[4](https://arxiv.org/html/2607.01972#bib.bib54)\]graphs in PENMAN notation\. Preprocessing parses each gold graph into the JSON output shape in Fig\.[10](https://arxiv.org/html/2607.01972#A2.F10)\(middle\): arootreference, a list ofnodes\(each an instance carrying itsconceptsymbol, with constant\-valuedattributesnested under their node\), and a flat list ofrelations\(source/role/target\)\. The PENMAN variable letters are dropped \(they survive only as opaque node ids\) so the schema routes relations by referential alignment over those ids \(Fig\.[10](https://arxiv.org/html/2607.01972#A2.F10), bottom\)\. The conversion is lossless: every gold round\-trips back to its triple set\. A single seeded shuffle with no stratification partitions the corpus into disjoint100/100/200100/100/200train/val/test splits\.

ExpressionandlocalizationofPHBinpancreaticcancercellsandtissue…

\{"root":"a",

"nodes":\[

\{"id":"a","concept":"and","attributes":\[\]\},

\{"id":"e","concept":"express\-03","attributes":\[\]\},

\{"id":"p2","concept":"protein","attributes":\[\]\},

\{"id":"n","concept":"name","attributes":\[\{"role":":op1","value":"PHB"\}\]\},

\{"id":"b","concept":"be\-located\-at\-91","attributes":\[\]\},

\{"id":"c","concept":"cell","attributes":\[\]\},

\{"id":"c2","concept":"cancer","attributes":\[\]\},\.\.\.\],

"relations":\[

\{"source":"a","role":":op1","target":"e"\},

\{"source":"e","role":":ARG2","target":"p2"\},

\{"source":"p2","role":":name","target":"n"\},

\{"source":"b","role":":ARG1","target":"p2"\},

\{"source":"c","role":":mod","target":"c2"\},\.\.\.\]\}

\{"type":"object","properties":\{

"root":\{"type":"string", "ref":"node"\},

"nodes":\{"type":"array","order":"align","items":\{"type":"object","properties":\{

"id":\{"type":"string", "idScope":"node"\},

"concept":\{"type":"string","score":"exact"\},

"attributes":\{"type":"array","order":"align","items":\{"type":"object","properties":\{

"role":\{"type":"string","score":"exact"\},

"value":\{"type":"string","score":"exact"\}\}\}\}\}\}\},

"relations":\{"type":"array","order":"align","items":\{"type":"object","properties":\{

"source":\{"type":"string", "ref":"node"\},

"role":\{"type":"string","score":"exact"\},

"target":\{"type":"string", "ref":"node"\}\}\}\}\}\}

Figure 10:Bio AMR\.*Top*: the input*context*—a biomedical sentence\.*Middle*: the gold output—a rooted meaning graph ofconceptnodes, with constant attributes nested under their node andrelationsreferencing nodes by id; the variable letters are arbitrary, so the model picks its own\.*Bottom*: the schema as run; thehighlightedidScope/reffragments are exactly what referential alignment adds—thestrictablation omits them, comparing the arbitrary variable letters by literal equality so relation routing collapses\. All other leaves areexactand every array is order\-agnostic \(AMR is a set of triples\)\.As the native metric, we use AMR’s standard metric, Smatch\[[7](https://arxiv.org/html/2607.01972#bib.bib53)\]: the best node bijection between the predicted and gold triple sets, with F1 micro\-averaged over the corpus\. That bijection search is preciselyObject Aligner’s referential alignment, so the registered referential schema tracks Smatch by construction\.

### B\-DNATURAL PLAN

The raw data are the Trip Planning split of NATURAL PLAN\[[65](https://arxiv.org/html/2607.01972#bib.bib66)\]\([https://github\.com/google\-deepmind/natural\-plan](https://github.com/google-deepmind/natural-plan)\)\. Each raw example ships a zero\-shot prompt, the target cities and their stay durations as\*\*\-delimited fields and a free\-text reference plan\. Preprocessing keeps the prompt verbatim as the inputcontext\(Fig\.[11](https://arxiv.org/html/2607.01972#A2.F11), top\) and builds the gold directly from thecities/durationsfields into anitinerarylist of\{city, days\}objects in the visiting order \(Fig\.[11](https://arxiv.org/html/2607.01972#A2.F11), middle\)\. The NATURAL PLAN is evaluation\-only \(no native train split\), so we keep train and val small \(100100each\) and a200200\-exampletestsplit, all difficulty\-stratified by the number of cities \(33–1010\)\. The schema used in the reported run \(Fig\.[11](https://arxiv.org/html/2607.01972#A2.F11), bottom\) aligns the itinerary by position, so producing the correct stays in the wrong order loses score\.

Youplantovisit3Europeancitiesfor14daysintotal\.Youonlytakedirectflightstocommutebetweencities\.

YouwouldliketovisitFlorencefor6days\.YouwanttomeetafriendinFlorencebetweenday9andday14\.You

wouldliketovisitBarcelonafor5days\.YouwouldliketovisitHelsinkifor5days\.

Herearethecitiesthathavedirectflights:

BarcelonaandFlorence,HelsinkiandBarcelona\.

Findatripplanofvisitingthecitiesfor14daysbytakingdirectflightstocommutebetweenthem\.

\{"itinerary":\[

\{"city":"Helsinki","days":5\},

\{"city":"Barcelona","days":5\},

\{"city":"Florence","days":6\}\]\}

\{"type":"object","properties":\{

"itinerary":\{"type":"array","order":"fixed","items":\{"type":"object","properties":\{

"city":\{"type":"string","score":"exact"\},

"days":\{"type":"integer","score":"invdiff"\}\}\}\}\}\}

Figure 11:NATURAL PLAN \(Trip Planning\)\.*Top*: the input*context*—the zero\-shot trip problem, stating the day budget, the per\-city stays and meeting windows, and the available direct flights\.*Middle*: the gold output—an ordereditineraryof\{city, days\}stays, taken from the released target fields\.*Bottom*: the schema as run; thehighlightedvalue is the*sequence*arm—orderset to"fixed", which aligns the itinerary positionally so a scrambled sequence is penalized—while the*set*ablation replaces it with"align"\(order\-blind Hungarian matching\)\.cityis matched exactly \(it is copied from the prompt\) anddaysby the graded inverse\-difference comparator\.As the native metric we use NATURAL PLAN’s official metric, the binary per\-example exact\-match solve rate: a plan solves the instance iff its\(city, days\)itinerary reproduces the gold stay for stay, in order, averaged over the split\.

### B\-EROCStories

The raw data are the ROCStories corpus\[[33](https://arxiv.org/html/2607.01972#bib.bib67)\], taken from[https://huggingface\.co/datasets/mintujupally/ROCStories](https://huggingface.co/datasets/mintujupally/ROCStories)\. Each story is split into sentences on sentence\-ending punctuation, and only stories that are segmented into exactly five sentences are retained\. For each story, we draw a random non\-identity permutation and present the sentences in that scrambled order—labeled1\.\.N1\.\.Nin the inputcontext\(Fig\.[12](https://arxiv.org/html/2607.01972#A2.F12), top\)—and recorded as gold the permutation of those labels that restores the original reading order \(Fig\.[12](https://arxiv.org/html/2607.01972#A2.F12), middle\)\. The splits are mutually disjoint:100/100100/100train/val and a200200\-exampletest\. The schema scored in the reported run \(Fig\.[12](https://arxiv.org/html/2607.01972#A2.F12), bottom\) aligns theindiceslist positionally\.

Herearethesentencesinscrambledorder:

Sentence1:GottoAustin,buthadtotakeaplane\.

Sentence2:JimwasdrivingfromAtlantatoAustinwhenhiscarbrokedown\.

Sentence3:Itseemedlikethemiddleofnowheresohebegantowalk\.

Sentence4:Whenhereturned,hiscarcouldn’tbefound\.

Sentence5:Hecameacrossagasstationtenmilesawayandgotarideback\.

\{"indices":\[2,3,5,4,1\]\}

\{"type":"object","properties":\{

"indices":\{"type":"array","order":"fixed","items":\{"type":"integer","score":"exact"\}\}\}\}

Figure 12:ROCStories \(sentence ordering\)\.*Top*: the input*context*—the five story sentences presented in scrambled order and labelled1\.\.N1\.\.N\.*Middle*: the gold output—theindicespermutation of those labels that restores the original reading order\.*Bottom*: the schema as run; thehighlightedvalue is the*sequence*arm—orderset to"fixed", which aligns theindiceslist positionally so a scrambled order is penalized—while the*set*ablation replaces it with"align"\(order\-blind Hungarian matching\)\. The data key is the neutralindices\(notorder\), so neither schema nor prompt reveals that the task is a reordering\.As the native metric, we use the standard sentence\-ordering Perfect\-Match Ratio \(PMR\)\[[5](https://arxiv.org/html/2607.01972#bib.bib68)\], the fraction of stories whose predicted order reproduces the gold permutation exactly, alongside Kendall’sτ\\tauas a graded view of partial order\[[5](https://arxiv.org/html/2607.01972#bib.bib68)\]\. Because the predicted bag of labels is always exactly the gold bag, the order\-blind*set*arm of theObject Alignerreward scores≈1\.0\\approx\\\!1\.0regardless of sequence, so the entire sequence\-vs\-set gap is attributable to order alone\.

## Appendix CIntrinsic Validation

This appendix collects additional material for the intrinsic validation of Section[V\-A](https://arxiv.org/html/2607.01972#S5.SS1)\.

#### A worked referential\-alignment instance\.

Fig\.[13](https://arxiv.org/html/2607.01972#A3.F13)shows the unit of the invariance probe of Section[V\-A](https://arxiv.org/html/2607.01972#S5.SS1): a small Org2Graph instance and its relabel\-only candidate\. Every identifier differs, yet referential alignment recovers the bijectionπ\\piand scores1\.01\.0, while the plain ablation compares the renumbered identifiers by value and matches no reference\. The twoMilo/scientistrecords show why this is nontrivial: tied in every attribute, they are separable only by structure—twin A is the*source*of thereports\_toandknowsedges, twin B only a target—which the structural tie\-break of Section[III\-E](https://arxiv.org/html/2607.01972#S3.SS5)uses to pin the bijection\.

gold:

\{"people":\[

\{"id":"474ee2","name":"Milo","title":"scientist"\},<\-twinA

\{"id":"904d0c","name":"Milo","title":"scientist"\},<\-twinB

\{"id":"478cc2","name":"Theo","title":"scientist"\}\],

"companies":\[

\{"id":"308da8","name":"Linton","industry":"retail"\},

\{"id":"80425d","name":"Oscorp","industry":"media"\}\],

"employment":\[

\{"person":"474ee2","company":"308da8","role":"contractor"\},

\{"person":"904d0c","company":"308da8","role":"associate"\},

\{"person":"478cc2","company":"308da8","role":"associate"\}\],

"acquaintance":\[

\{"source":"474ee2","target":"904d0c","relation":"reports\_to"\},

\{"source":"474ee2","target":"478cc2","relation":"knows"\}\],

"partnership":\[

\{"source":"308da8","target":"80425d","relation":"competes\_with"\}\]\}

candidate\(relabelonly\):

thesamegraph,everyid

renumberedandallreferences

rewrittenthroughthemap:

474ee2\-\>a5cd68

904d0c\-\>4d3c1a

478cc2\-\>ca264e

308da8\-\>18b8ff

80425d\-\>25165e

Figure 13:A small Org2Graph instance \(readable values, narrow vocabulary\) and its relabel\-only candidate\.*Every*identifier differs between gold and candidate\. The twoMilo/scientistrecords marked twin A and twin B are property\-twins: identical in every attribute\.
#### RA Data generation\.

Algorithms[3](https://arxiv.org/html/2607.01972#alg3)and[4](https://arxiv.org/html/2607.01972#alg4)provide the exact procedures for the two RA experiments described in Section[V\-A](https://arxiv.org/html/2607.01972#S5.SS1)\. In this appendix, theemitappends one gold/candidate pair, with its family and magnitude tags, to the generated dataset\. Every gold draws its parameters independently viaSampleParams\(Algorithm[3](https://arxiv.org/html/2607.01972#alg3), lines 2–4\), so both experiments cover the same distribution of graphs\. They share three additional primitives\.SampleGolddraws an Org2Graph instance with readable \(un\-obfuscated\) categorical values and the narrow6/6/6/3/36/6/6/3/3vocabulary \(Section[IV\-A](https://arxiv.org/html/2607.01972#S4.SS1)\) at the requested sizes and edge densities;MakeTwinsturns a fractionttof each scope’s records into property\-twins;Relabelreassigns every record a fresh identifier, disjoint from the gold’s, and rewrites all references accordingly\. In Algorithm[4](https://arxiv.org/html/2607.01972#alg4), each\(o,k\)\(o,k\)candidate is built independently from the gold\. The six edit operations𝒪\\mathcal\{O\}are as follows:

- •*categorical relabel*— pick a uniformly random categorical leaf \(a person’s title, a company’s industry, an employment role, or an edge relation\) and set it to a*different*code from the same codebook \(Section[IV\-A](https://arxiv.org/html/2607.01972#S4.SS1)\);
- •*reference rerouting*— pick a random edge endpoint whose scope holds≥2\{\\geq\}2records and rewire it to a different record’s id;
- •*record deletion*— pick a random scope still holding≥2\{\\geq\}2records, delete a random record together with every incident edge;
- •*edge deletion*— delete a uniformly random edge;
- •*record insertion*— insert a fresh record with a new id and sampled attributes \(a new person also receives an employment edge to a random company\);
- •*edge insertion*— add a new acquaintance \(two distinct random people\) or partnership \(two distinct random companies\) edge with a sampled relation\.

Algorithm 3RA invariance \(identifier relabelings\): dataset generation\.0:golds

G=100G\{=\}100; relabelings per gold

K=5K\{=\}5
0:

100×5=500100\\times 5=500gold/candidate pairs

1:foreachof the

GGgoldsdo

2:SampleParams: people

np∼U​\{3\.\.10\}n\_\{p\}\\sim U\\\{3\.\.10\\\}, companies

nc∼U\{2\.\.⌈np/2⌉\}n\_\{c\}\\sim U\\\{2\.\.\\lceil n\_\{p\}/2\\rceil\\\}
3:twin density

t∼U​\[0,1\]t\\sim U\[0,1\]
4:edge densities

ρa∼U​\[0\.2,0\.5\]\\rho\_\{a\}\\sim U\[0\.2,0\.5\],

ρp∼U​\[0\.3,0\.7\]\\rho\_\{p\}\\sim U\[0\.3,0\.7\]
5:

g←SampleGold​\(np,nc,ρa,ρp\)g\\leftarrow\\textsc\{SampleGold\}\(n\_\{p\},n\_\{c\},\\rho\_\{a\},\\rho\_\{p\}\);

MakeTwins​\(g,t\)\\textsc\{MakeTwins\}\(g,t\)
6:emit

KKpairs

\(g,Relabel​\(g\)\)\\bigl\(g,\\;\\textsc\{Relabel\}\(g\)\\bigr\), each relabeling drawn independently

7:endfor

Algorithm 4RA per\-operation sweep: dataset generation\.0:golds

G=100G\{=\}100, parameters drawn per gold bySampleParamsas in Algorithm[3](https://arxiv.org/html/2607.01972#alg3); edit operations

𝒪\\mathcal\{O\}\(

66ops, see text\)

0:

100×2×6×9=10800100\\times 2\\times 6\\times 9=10800gold/candidate pairs \(

54005400with the relabel, scored in Section[V\-A](https://arxiv.org/html/2607.01972#S5.SS1);

54005400without, for the control sweep\)

1:foreachof the

GGgoldsdo

2:

g←SampleGold​\(np,nc,ρa,ρp\)g\\leftarrow\\textsc\{SampleGold\}\(n\_\{p\},n\_\{c\},\\rho\_\{a\},\\rho\_\{p\}\);

MakeTwins​\(g,t\)\\textsc\{MakeTwins\}\(g,t\)
3:for

r∈\{relabel,no\-relabel\}r\\in\\\{\\text\{relabel\},\\text\{no\-relabel\}\\\},

o∈𝒪o\\in\\mathcal\{O\},

k=0,…,8k=0,\\dots,8do

4:

c←Relabel​\(g\)c\\leftarrow\\textsc\{Relabel\}\(g\)if

r=r\{=\}relabel, else a copy of

gg
5:for

kktimesdo

6:if

oohas no feasible target in

ccthen break

7:apply

ooonce to

ccat a random feasible target

8:endfor

9:emit

\(g,c,o,k,r\)\(g,\\,c,\\,o,\\,k,\\,r\)
10:endfor

11:endfor

#### RA per\-operation effects\.

Table[VI](https://arxiv.org/html/2607.01972#A3.T6)quantifies the per\-operation sensitivity behind Fig\.[4](https://arxiv.org/html/2607.01972#S5.F4): the Spearman correlation of score with the number of applicationskkof a single edit\.

TABLE VI:RA per\-operation sensitivity \(Org2Graph generator\): Spearman correlation of score with the number of applicationskkof a single edit, over the sweep of Section[V\-A](https://arxiv.org/html/2607.01972#S5.SS1)\(54005400pairs, every edit applied on top of an identifier relabel; Algorithm[4](https://arxiv.org/html/2607.01972#alg4)\)\. More negative means more sensitive\. In the*reference rerouting*row plain is blind to the one edit that corrupts routing alone\.
#### A worked Facts2Order instance\.

Fig\.[14](https://arxiv.org/html/2607.01972#A3.F14)shows a hidden\-keyFacts2Orderinstance \(N=4N\{=\}4, integer key\): the designated sort key \(weight\) is buried among per\-item numeric decoy clauses on other attributes \(price, length\) whose values are unconstrained, their order shuffled per item, so the key is never identifiable as the lone number\. Only the weights are drawn distinct, fixing a unique gold order; the optimizer must discover fromObject Alignerfeedback that weight—not price or length—is the sort field\. \(The intrinsic study described in Section[V\-A](https://arxiv.org/html/2607.01972#S5.SS1)ignores these surface sentences and scores the index permutations directly; cf\. Fig\.[5](https://arxiv.org/html/2607.01972#S5.F5)\.\)

```
Here are the items:

Item 1: It is priced at 512 dollars.
        Saiph weighs 37 kilograms.
        It measures 268 centimetres.
Item 2: Antares weighs 696 kilograms.
        It is priced at 144 dollars.
        It measures 91 centimetres.
Item 3: It measures 730 centimetres.
        Arcturus weighs 933 kilograms.
        It is priced at 305 dollars.
Item 4: Bellatrix weighs 883 kilograms.
        It measures 410 centimetres.
        It is priced at 77 dollars.

gold: {"indices": [1, 2, 4, 3]}
```

Figure 14:The most complexFacts2Orderconfiguration: a hidden\-key instance \(the F2O\-H variant of the extrinsic study, Section[V\-D](https://arxiv.org/html/2607.01972#S5.SS4)\)\. Each item states its weight \(the sort key\) among numeric decoy clauses on price and length\. The gold permutation sorts the items by weight ascending\.
#### Order data generation\.

Algorithm[5](https://arxiv.org/html/2607.01972#alg5)provides the exact procedure for theFacts2Ordercorruption pairs described in Section[V\-A](https://arxiv.org/html/2607.01972#S5.SS1), which yields46394639pairs at the sampled sizes \(the per\-gold count increases withNN\)\. Every gold sample draws its parameters viaSampleParams\(Section[IV\-B](https://arxiv.org/html/2607.01972#S4.SS2)\), andSampleGoldreturns the unique permutation of1\.\.N1\.\.Nthat sorts the items, which the study scores directly \(ignoring surface sentences\)\. Each gold is corrupted once per family into a candidate, tagged by its realized Kendall distancedd, and all draws use per\-gold seeds for reproducibility\. Each family targets a distinct failure mode:

- •*adjacent transpositions*\(k=1\.\.N−1k\{=\}1\.\.N\{\-\}1swaps\) — walkingddup in approximately unit steps to trace the partial\-credit curve;ddis recomputed per candidate, so a fixedkkspreads over a small range as shown in Table[II](https://arxiv.org/html/2607.01972#S5.T2);
- •*block reversal*/*block move*— reverse, or cut and reinsert, one contiguous block \(length≥2\{\\geq\}2\):*non\-local*disorder — large per\-item displacement that a single local swap cannot produce;
- •*deletion*/*insertion*\(drop, or inject duplicate, labels\) — change the candidate length, exercising the gap\-aligned DP denominator and excess penalty\. Unlike the reorderings, these alter the label bag, so the order\-agnostic baseline reacts to them but not to pure order\.

The uncorrupted gold paired with itself is thenone\(d=0d\{=\}0\) reference row in Table[II](https://arxiv.org/html/2607.01972#S5.T2)\. Corruptions that reproduce the gold are resampled, becauseAdjTransposeandBlockMovecan compose to the identity\. Across the generated pairs,ddis concentrated near the gold: of the28642864length\-preserving pairs, it ranges0–5555with a median of22\(the dense low\-ddregion comes from the adjacent\-transposition dose, the sparse tail from the block families\), while the17751775length\-changing*deletion*/*insertion*pairs leaveddundefined\.

Algorithm 5Order\-sensitivity experiment \(Facts2Order\): dataset generation\.0:golds

G=300G\{=\}300; ascending sort

0:gold/candidate pairs, each tagged by the corruption that produced it \(nonefor the reference\)

1:foreachof the

GGgoldsdo

2:draw parameters viaSampleParams\(Sec\.[IV\-B](https://arxiv.org/html/2607.01972#S4.SS2)\)

3:

g←SampleGoldg\\leftarrow\\textsc\{SampleGold\}\{gold permutation of

1\.\.N1\.\.N\}

4:emit

\(g,g\)\(g,\\,g\)\{none:

d=0d\{=\}0reference\}

5:for

k=1,…,N−1k=1,\\dots,N\{\-\}1do

6:emit

\(g,AdjTranspose​\(g,k\)\)\(g,\\,\\textsc\{AdjTranspose\}\(g,k\)\)
7:endfor

8:emit

\(g,BlockReverse​\(g\)\)\(g,\\,\\textsc\{BlockReverse\}\(g\)\)
9:emit

\(g,BlockMove​\(g\)\)\(g,\\,\\textsc\{BlockMove\}\(g\)\)
10:for

k=1,…,min⁡\(3,N−1\)k=1,\\dots,\\min\(3,N\{\-\}1\)do

11:emit

\(g,Delete​\(g,k\)\)\(g,\\,\\textsc\{Delete\}\(g,k\)\)
12:endfor

13:for

k=1,…,3k=1,\\dots,3do

14:emit

\(g,Insert​\(g,k\)\)\(g,\\,\\textsc\{Insert\}\(g,k\)\)
15:endfor

16:endfor

17:resampleany candidate equal to its gold \{discard identity corruptions\}

#### Order\-sensitivity curves\.

Fig\.[15](https://arxiv.org/html/2607.01972#A3.F15)shows the curve view of Table[II](https://arxiv.org/html/2607.01972#S5.T2), showing the full trend of the table samples at a few points\. One feature deserves comment: on pure reorderings Kendallτ\\tausits*above*the sequence score at every distance\. This is a difference of scale, not of sensitivity—τ\\taurescales the inversion*count*, while the sequence alignment pays per displaced*item*—so the two are comparable in trend, but not in level\.

![Refer to caption](https://arxiv.org/html/2607.01972v1/x10.png)Figure 15:Partial\-credit curve \(Facts2Ordergenerator\): mean score vs\. Kendall distance on the adjacent\-transposition family\. The order\-sensitive*sequence*alignment decays smoothly; the order\-agnostic*set*alignment is flat at1\.01\.0\(no order signal\);*exact*match is all\-or\-nothing; Kendallτ\\tauis graded here but task\-specific \(and0on any length change\)\.

## Appendix DSystem Prompts

This appendix lists the initial candidate system prompt used to seed GEPA for every dataset\. As discussed in the protocol of Section[V](https://arxiv.org/html/2607.01972#S5)\(paragraph “Protocol”\), each seed prompt is deliberately minimal: beyond a one\-line statement of the task and the required JSON*output*shape, it gives no guidance on*how*to produce the prediction\. The task model is told what kind of object to emit, but not the rules, edge cases, or step\-by\-step procedure for deriving it from the input, so GEPA must recover that mapping purely from theObject Aligner’s scalar score and—in the feedback arm—its textual feedback\. This isolates the question of whether the metric supplies sufficient optimization “gradient” to drive the search from a near\-empty starting point\. The prompts below are reproduced verbatim\.

#### Facts2Order\.

OutputaSINGLEJSONobjectwithexactlyonetop\-levelkey:

"indices":alistofintegers—apermutationof1\.\.N,whereNisthenumber

ofinputitems\.Example:\{"indices":\[3,1,4,2\]\}\.

EmitJSONonly—nocommentary,nomarkdownfences\.

#### SciERC\.

YouextractascientificknowledgegraphfromanAI\-paperabstract\.

OutputaSINGLEJSONobjectwithexactlythesetwotop\-levelkeys:

"entities":alistof\{"id":<string\>,"type":<string\>,"mentions":\[\{"text":<string\>\},\.\.\.\]\}objects\.

"relations":alistof\{"subject":<entity\-id\>,"predicate":<string\>,"object":<entity\-id\>\}objects\.

Useshort,stableidsofyourownchoosing\(e\.g\."e0","e1"\);theexactid

stringsdonotmatteraslongaseachentityhasoneconsistentidandevery

relation’s"subject"/"object"referencesidsdefinedin"entities"\.Each

entitycarriesallofitssurfacementionsinits"mentions"list\.

EmitJSONonly—nocommentary,nomarkdownfences\.

#### BioRED\.

YouextractabiomedicalrelationgraphfromaPubMedtitleandabstract\.

OutputaSINGLEJSONobjectwithexactlythesetwotop\-levelkeys:

"entities":alistof\{"id":<string\>,"type":<string\>,"mentions":\[\{"text":<string\>\},\.\.\.\]\}objects\.

"relations":alistof\{"subject":<entity\-id\>,"predicate":<string\>,"object":<entity\-id\>\}objects\.

Useshort,stableidsofyourownchoosing\(e\.g\."e0","e1"\);theexactid

stringsdonotmatteraslongaseachentityhasoneconsistentidandevery

relation’s"subject"/"object"referencesidsdefinedin"entities"\.Each

entitycarriesallofitssurfacementionsinits"mentions"list\.

EmitJSONonly—nocommentary,nomarkdownfences\.

#### Bio AMR\.

YouconvertanEnglishsentenceintoanAbstractMeaningRepresentation\(AMR\)graph\.

OutputaSINGLEJSONobjectwithexactlythesetop\-levelkeys:

"root":theidofthetopnode\.

"nodes":alistof\{"id":<string\>,"concept":<string\>,"attributes":\[\{"role":<string\>,"value":<string\>\},\.\.\.\]\}objects\.

"relations":alistof\{"source":<node\-id\>,"role":<string\>,"target":<node\-id\>\}objects\.

Useshort,stableidsofyourownchoosing\(e\.g\."n0","n1"\);theexactidstrings

donotmatteraslongaseveryrelationandtherootreferenceidsdefinedin"nodes"\.

EmitJSONonly—nocommentary,nomarkdownfences\.

#### NATURAL PLAN \(Trip Planning\)\.

Youareanexperttripplanner\.Theusermessagestateshowmanydaysthetrip

lasts,whichcitiestovisitandforhowlong,anymeeting/eventday\-window

constraints,andtheavailabledirectflightsbetweencities\.Findanitinerary

thatsatisfieseveryconstraint,takingonlythelisteddirectflights\.

OutputaSINGLEJSONobjectwithexactlyonetop\-levelkey:

"itinerary":alistof\{"city":<string\>,"days":<integer\>\}objects,inthe

orderthecitiesarevisited\.

EmitJSONonly—nocommentary,nomarkdownfences\.

#### ROCStories\.

OutputaSINGLEJSONobjectwithexactlyonetop\-levelkey:

"indices":alistofintegers—apermutationof1\.\.N,whereNisthenumber

ofinputsentences\.Example:\{"indices":\[3,1,4,2,5\]\}\.

EmitJSONonly—nocommentary,nomarkdownfences\.

## Appendix ENative\-Metric Results

![Refer to caption](https://arxiv.org/html/2607.01972v1/x11.png)Figure 16:Native\-metric companion to Fig\.[6](https://arxiv.org/html/2607.01972#S5.F6): the same three contrasts and dumbbell encoding, but on each dataset’s task\-native metric \(annotated per row\) and restricted to the real\-world datasets, since the synthetic probes carry no native metric\.\(A\)RA vs\. plain on relation F1 / Smatch;\(B\)sequence vs\. set on EM \(NATURAL PLAN\) and PMR plus Kendall’sτ\\tau\(ROCStories\);\(C\)feedback vs\. score\.Where a real\-world task carries a standard metric of its own we report it here as a fully schema\-independent cross\-check on the OA\-score conclusions of Section[V](https://arxiv.org/html/2607.01972#S5)\. Because GEPA always optimized anObject Alignerschema, a metric that lies outside that schema \(and so is unaffected by any ablation\) provides an independent external check\. On all three axes, the native verdict agrees with the headline directions of Fig\.[6](https://arxiv.org/html/2607.01972#S5.F6)\(Fig\.[16](https://arxiv.org/html/2607.01972#A5.F16)\): sequence beats set wherever order matters, feedback beats score everywhere, and on Bio AMR, the gain from referential alignment over the plain schema, seen in theObject Alignerscore under the score reward, carries over in*sign*to Smatch \(\+0\.030\+0\.030, Gemma\-4\-26B\), though within seed noise \(95%95\\%bootstrap CI\[−0\.03,\+0\.09\]\[\-0\.03,\+0\.09\]\)\. One qualitative difference is worth flagging: on SciERC and BioRED the score arm collapses to*exactly*0\.0000\.000relation F1 \(both RA and plain\), even though itsObject Alignergraded score does not: relation F1 credits only relations whose subject, predicate, and object all match the gold; a near\-miss earns nothing\. Labelling the right pair of entitiesUSED\-FOR, where the gold saysPART\-OF, for instance, scores0even though two of its three fields are correct\. The model become accurate enough to register any F1 only under the feedback reward\. Elsewhere the native andObject Aligner\-score gaps differ only in magnitude, agreeing in sign and ordering\.

These native metrics are not all standard leaderboard tasks\. For SciERC and BioRED, there are no published number lines up with our setup: the relation F1 we report is simply the conventional metric for relation extraction on data of that kind, adopted here as a schema\-independent alternative to theObject Alignerscore rather than to match an external baseline\. The remaining three—Bio AMR, NATURAL PLAN, and ROCStories—do have published reference points, which we quote below; even there the comparison is loose \(splits, models, and task framing all differ\), so we cite them only to locate the scale of each metric, not as head\-to\-head results\.

On Bio AMR, the prompted model sits far below the supervised state of the art\. Our best \(Gemma\-4\-26B, feedback RA\) reaches≈0\.40\\approx\\\!0\.40Smatch F1, whereas purpose\-built AMR parsers report low\-to\-mid\-0\.800\.80s on the Bio test set—StructBART with Maximum\-Bayes\-Smatch ensemble distillation\[[23](https://arxiv.org/html/2607.01972#bib.bib56)\]at≈0\.81\\approx\\\!0\.81\. These parsers are fine\-tuned on AMR \(including the Bio training portion\) and emit PENMAN on the corpus’s standard split\. We score a general\-purpose open model emitting a JSON graph under a GEPA\-optimized prompt on a seeded200200\-example subsample\. The gap reflects supervised, in\-domain training versus none, not a weakness of theObject Alignerreward\.

On NATURAL PLAN \(Trip Planning\), whose official metric is the exact\-match \(EM\) solve rate, our best Gemma\-4\-26B result reaches≈0\.63\\approx\\\!0\.63EM—above the benchmark’s figures \(GPT\-431\.1%31\.1\\%, the strongest model Gemini 1\.5 Pro34\.8%34\.8\\%\)\[[65](https://arxiv.org/html/2607.01972#bib.bib66)\]\. The lead is not evidence of better planning: those are 2024\-era models \(although frontier at that time\) evaluated55\-shot over the full16001600\-example set, whereas we score a newer\-generation open model on a200200\-example, city\-count\-stratified split with a GEPA\-optimized prompt, so the gap reflects the changed evaluation setup \(a different\-generation model and an optimized prompt\) not planning ability\.

On ROCStories, the prompted model again sits somewhat below the supervised state\-of\-the\-art\. Our Gemma\-4\-26B sequence arm scores≈0\.65\\approx\\\!0\.65PMR and≈0\.87\\approx\\\!0\.87Kendall’sτ\\tau, whereas Re\-BART\[[5](https://arxiv.org/html/2607.01972#bib.bib68)\], fine\-tuned end\-to\-end to order the sentences, reports0\.820\.82PMR and0\.940\.94τ\\tau\(full corpus,80:10:1080\{:\}10\{:\}10split,≈9\.8\\approx\\\!9\.8K test stories\)\. The gap is smaller on the gradedτ\\tauthan on exact\-match PMR, as expected when a prompted model recovers the rough order but rarely the exact permutation\.

## Acknowledgment

This work was supported by the Ministry of Education, Youth and Sports of the Czech Republic through the e\-INFRA CZ \(ID:90254\)\. The author thanks Herbert Ullrich for proof\-reading the manuscript and for his valuable suggestions\.

During the preparation of this work, the author used generative AI tools, predominantly the Anthropic Claude Code \(mainly Opus models\) and secondarily ChatGPT \(OpenAI GPT\-5\)\. The core of the Object Aligner library was written manually and later refactored with AI assistance\. Several extensions, the test and experimental\-suite code, and supporting utilities were drafted with AI, which was also used for code review to identify errors\. For the manuscript, AI was used for grammar correction and extensively for rephrasing\. Some passages, mainly in the related\-work and dataset\-description sections, were drafted by AI from detailed bullet outlines written by the author, and all figures and tables were produced by AI\-generated scripts that construct them from the actual computed results\. AI was also used to assist in researching related topics\. The author reviewed, verified, and edited all AI\-assisted content and takes full responsibility for the content of this publication\.

## References

- \[1\]L\. A\. Agrawal, S\. Tan, D\. Soylu, N\. Ziems, R\. Khare, K\. Opsahl\-Ong, A\. Singhvi, H\. Shandilya, M\. J\. Ryan, M\. Jiang, C\. Potts, K\. Sen, A\. Dimakis, I\. Stoica, D\. Klein, M\. Zaharia, and O\. Khattab\(2026\)GEPA: reflective prompt evolution can outperform reinforcement learning\.InThe Fourteenth International Conference on Learning Representations,Cited by:[1st item](https://arxiv.org/html/2607.01972#S1.I1.i1.p1.1),[§II\-C](https://arxiv.org/html/2607.01972#S2.SS3.p2.1),[§III\-F](https://arxiv.org/html/2607.01972#S3.SS6.p1.1),[§V\-B](https://arxiv.org/html/2607.01972#S5.SS2.SSS0.Px1.p1.1),[§V](https://arxiv.org/html/2607.01972#S5.p1.1)\.
- \[2\]\(2025\)Stickler: a library for evaluating structured data and AI outputs with weighted field comparison and custom comparators\.Note:[https://github\.com/awslabs/stickler](https://github.com/awslabs/stickler)Software, no accompanying publication; first commit 2025\-10\-07, version 0\.4\.0 \(May 2026\) compared hereCited by:[§I](https://arxiv.org/html/2607.01972#S1.p6.1),[§II\-B](https://arxiv.org/html/2607.01972#S2.SS2.p1.1),[TABLE I](https://arxiv.org/html/2607.01972#S2.T1.51.50.1.3.1.1)\.
- \[3\]A\. Bagga and B\. Baldwin\(1998\)Algorithms for scoring coreference chains\.InProceedings of the First International Conference on Language Resources and Evaluation \(LREC\) Workshop on Linguistic Coreference,Granada, Spain,pp\. 563–566\.Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px2.p1.1)\.
- \[4\]L\. Banarescu, C\. Bonial, S\. Cai, M\. Georgescu, K\. Griffitt, U\. Hermjakob, K\. Knight, P\. Koehn, M\. Palmer, and N\. Schneider\(2013\-08\)Abstract Meaning Representation for sembanking\.InProceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse,A\. Pareja\-Lora, M\. Liakata, and S\. Dipper \(Eds\.\),Sofia, Bulgaria,pp\. 178–186\.Cited by:[§B\-C](https://arxiv.org/html/2607.01972#A2.SS3.p1.2),[§IV\-E](https://arxiv.org/html/2607.01972#S4.SS5.p1.1)\.
- \[5\]S\. Basu Roy Chowdhury, F\. Brahman, and S\. Chaturvedi\(2021\-11\)Is everything in order? a simple way to order sentences\.InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing,M\. Moens, X\. Huang, L\. Specia, and S\. W\. Yih \(Eds\.\),Online and Punta Cana, Dominican Republic,pp\. 10769–10779\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.emnlp-main.841)Cited by:[§B\-E](https://arxiv.org/html/2607.01972#A2.SS5.p2.2),[Appendix E](https://arxiv.org/html/2607.01972#A5.p5.9)\.
- \[6\]I\. Bhattacharya and L\. Getoor\(2007\-03\)Collective entity resolution in relational data\.ACM Trans\. Knowl\. Discov\. Data1\(1\),pp\. 5–es\.External Links:ISSN 1556\-4681,[Document](https://dx.doi.org/10.1145/1217299.1217304)Cited by:[3rd item](https://arxiv.org/html/2607.01972#S7.I1.i3.p1.1),[1st item](https://arxiv.org/html/2607.01972#S9.I1.i1.p1.1)\.
- \[7\]S\. Cai and K\. Knight\(2013\-08\)Smatch: an evaluation metric for semantic feature structures\.InProceedings of the 51st Annual Meeting of the Association for Computational Linguistics \(Volume 2: Short Papers\),H\. Schuetze, P\. Fung, and M\. Poesio \(Eds\.\),Sofia, Bulgaria,pp\. 748–752\.Cited by:[§B\-C](https://arxiv.org/html/2607.01972#A2.SS3.p2.1),[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px3.p1.1)\.
- \[8\]N\. Carion, F\. Massa, G\. Synnaeve, N\. Usunier, A\. Kirillov, and S\. Zagoruyko\(2020\)End\-to\-end object detection with transformers\.InComputer Vision – ECCV 2020,Cham,pp\. 213–229\.External Links:ISBN 978\-3\-030\-58452\-8Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px2.p1.1)\.
- \[9\]P\. P\. Chen\(1976\-03\)The entity\-relationship model—toward a unified view of data\.ACM Trans\. Database Syst\.1\(1\),pp\. 9–36\.External Links:ISSN 0362\-5915,[Document](https://dx.doi.org/10.1145/320434.320440)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px4.p1.1),[3rd item](https://arxiv.org/html/2607.01972#S7.I1.i3.p1.1)\.
- \[10\]Y\. Chen, W\. Gantt, W\. Gu, T\. Chen, A\. S\. White, and B\. Van Durme\(2023\-05\)Iterative document\-level information extraction via imitation learning\.InProceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics,A\. Vlachos and I\. Augenstein \(Eds\.\),Dubrovnik, Croatia,pp\. 1858–1874\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.eacl-main.136)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px2.p1.1)\.
- \[11\]S\. Dehpour\(2026\)DeepDiff: deep difference and search of any Python object/data\.Note:[https://github\.com/seperman/deepdiff](https://github.com/seperman/deepdiff)Version 9\.0\.0Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px4.p1.1)\.
- \[12\]X\. Du, A\. Rush, and C\. Cardie\(2021\-04\)GRIT: generative role\-filler transformers for document\-level event entity extraction\.InProceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume,P\. Merlo, J\. Tiedemann, and R\. Tsarfaty \(Eds\.\),Online,pp\. 634–644\.External Links:[Document](https://dx.doi.org/10.18653/v1/2021.eacl-main.52)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px2.p1.1)\.
- \[13\]I\. P\. Fellegi and A\. B\. Sunter\(1969\)A theory for record linkage\.Journal of the American Statistical Association64\(328\),pp\. 1183–1210\.External Links:[Document](https://dx.doi.org/10.1080/01621459.1969.10501049)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px4.p1.1)\.
- \[14\]N\. Ferguson, J\. Pennington, N\. Beghian, A\. Mohan, D\. Kiela, S\. Agrawal, and T\. H\. Nguyen\(2026\)ExtractBench: a benchmark and evaluation methodology for complex structured extraction\.External Links:2602\.12247,[Link](https://arxiv.org/abs/2602.12247)Cited by:[§I](https://arxiv.org/html/2607.01972#S1.p6.1),[§II\-B](https://arxiv.org/html/2607.01972#S2.SS2.p1.1),[TABLE I](https://arxiv.org/html/2607.01972#S2.T1.51.50.1.5.1.1)\.
- \[15\]C\. Fernando, D\. Banarse, H\. Michalewski, S\. Osindero, and T\. Rocktäschel\(2024\)Promptbreeder: self\-referential self\-improvement via prompt evolution\.InProceedings of the 41st International Conference on Machine Learning,ICML’24\.Cited by:[§II\-C](https://arxiv.org/html/2607.01972#S2.SS3.p1.1)\.
- \[16\]Gemma Team, Google DeepMind\(2026\)Gemma 4 model card\.Note:[https://ai\.google\.dev/gemma/docs/core/model\_card\_4](https://ai.google.dev/gemma/docs/core/model_card_4)Open\-weights model family; task models are thegemma\-4\-26B\-A4B\-itandgemma\-4\-E4B\-itinstruction\-tuned variants\. Accessed: 2026\-06\-28Cited by:[§V\-B](https://arxiv.org/html/2607.01972#S5.SS2.SSS0.Px1.p1.1)\.
- \[17\]Q\. Guo, R\. Wang, J\. Guo, B\. Li, K\. Song, X\. Tan, G\. Liu, J\. Bian, and Y\. Yang\(2024\)Connecting large language models with evolutionary algorithms yields powerful prompt optimizers\.InThe Twelfth International Conference on Learning Representations,Cited by:[§II\-C](https://arxiv.org/html/2607.01972#S2.SS3.p1.1)\.
- \[18\]M\. A\. Jaro\(1989\)Advances in record\-linkage methodology as applied to matching the 1985 census of Tampa, Florida\.Journal of the American Statistical Association84\(406\),pp\. 414–420\.Cited by:[§III\-B](https://arxiv.org/html/2607.01972#S3.SS2.p1.5)\.
- \[19\]M\. G\. Kendall\(1938\)A new measure of rank correlation\.Biometrika30\(1/2\),pp\. 81–93\.External Links:[Document](https://dx.doi.org/10.1093/biomet/30.1-2.81),ISSN 00063444Cited by:[§IV\-B](https://arxiv.org/html/2607.01972#S4.SS2.p1.5)\.
- \[20\]O\. Khattab, A\. Singhvi, P\. Maheshwari, Z\. Zhang, K\. Santhanam, S\. V\. A, S\. Haq, A\. Sharma, T\. T\. Joshi, H\. Moazam, H\. Miller, M\. Zaharia, and C\. Potts\(2024\)DSPy: compiling declarative language model calls into state\-of\-the\-art pipelines\.InThe Twelfth International Conference on Learning Representations,Cited by:[1st item](https://arxiv.org/html/2607.01972#S1.I1.i1.p1.1),[§II\-C](https://arxiv.org/html/2607.01972#S2.SS3.p1.1),[§III\-F](https://arxiv.org/html/2607.01972#S3.SS6.p1.1),[§V\-B](https://arxiv.org/html/2607.01972#S5.SS2.SSS0.Px1.p1.1)\.
- \[21\]N\. M\. Kriege, P\. Giscard, and R\. C\. Wilson\(2016\)On valid optimal assignment kernels and applications to graph classification\.InProceedings of the 30th International Conference on Neural Information Processing Systems,NIPS’16,Red Hook, NY, USA,pp\. 1623–1631\.External Links:ISBN 9781510838819Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px3.p1.1),[§III\-E](https://arxiv.org/html/2607.01972#S3.SS5.SSS0.Px3.p3.1)\.
- \[22\]H\. W\. Kuhn\(1955\)The Hungarian method for the assignment problem\.Naval Research Logistics Quarterly2\(1–2\),pp\. 83–97\.External Links:[Document](https://dx.doi.org/10.1002/nav.3800020109)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px2.p1.1),[§III\-C](https://arxiv.org/html/2607.01972#S3.SS3.SSS0.Px1.p1.4)\.
- \[23\]Y\. Lee, R\. Astudillo, H\. Thanh Lam, T\. Naseem, R\. Florian, and S\. Roukos\(2022\-07\)Maximum Bayes Smatch ensemble distillation for AMR parsing\.InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,Seattle, United States,pp\. 5379–5392\.External Links:[Document](https://dx.doi.org/10.18653/v1/2022.naacl-main.393)Cited by:[Appendix E](https://arxiv.org/html/2607.01972#A5.p3.4)\.
- \[24\]V\. I\. Levenshtein\(1966\)Binary codes capable of correcting deletions, insertions, and reversals\.Soviet Physics Doklady10\(8\),pp\. 707–710\.Cited by:[§III\-B](https://arxiv.org/html/2607.01972#S3.SS2.p1.5)\.
- \[25\]R\. Linacre, S\. Lindsay, T\. Manassis, Z\. Slade, T\. Hepworth, R\. Kennedy, and A\. Bond\(2022\)Splink: free software for probabilistic record linkage at scale\.International Journal of Population Data Science7\(3\)\.External Links:[Document](https://dx.doi.org/10.23889/ijpds.v7i3.1794)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px4.p1.1)\.
- \[26\]Y\. Liu, D\. Iter, Y\. Xu, S\. Wang, R\. Xu, and C\. Zhu\(2023\-12\)G\-eval: NLG evaluation using GPT\-4 with better human alignment\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 2511–2522\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.153)Cited by:[§I](https://arxiv.org/html/2607.01972#S1.p2.1),[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px4.p1.1),[§III\-F](https://arxiv.org/html/2607.01972#S3.SS6.p1.1)\.
- \[27\]Y\. Luan, L\. He, M\. Ostendorf, and H\. Hajishirzi\(2018\-October\-November\)Multi\-task identification of entities, relations, and coreference for scientific knowledge graph construction\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,E\. Riloff, D\. Chiang, J\. Hockenmaier, and J\. Tsujii \(Eds\.\),Brussels, Belgium,pp\. 3219–3232\.External Links:[Document](https://dx.doi.org/10.18653/v1/D18-1360)Cited by:[§B\-A](https://arxiv.org/html/2607.01972#A2.SS1.p1.1),[§IV\-C](https://arxiv.org/html/2607.01972#S4.SS3.p1.2)\.
- \[28\]L\. Luo, P\. Lai, C\. Wei, C\. N\. Arighi, and Z\. Lu\(2022\)BioRED: a rich biomedical relation extraction dataset\.Briefings in Bioinformatics23\(5\),pp\. bbac282\.External Links:[Document](https://dx.doi.org/10.1093/bib/bbac282)Cited by:[§B\-B](https://arxiv.org/html/2607.01972#A2.SS2.p1.2),[§IV\-D](https://arxiv.org/html/2607.01972#S4.SS4.p1.1)\.
- \[29\]X\. Luo\(2005\-10\)On coreference resolution performance metrics\.InProceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing,R\. Mooney, C\. Brew, L\. Chien, and K\. Kirchhoff \(Eds\.\),Vancouver, British Columbia, Canada,pp\. 25–32\.Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px2.p1.1)\.
- \[30\]N\. Mihindukulasooriya, N\. S\. D’Souza, Md\. F\. M\. Chowdhury, and H\. Samulowitz\(2025\)Automatic prompt optimization for knowledge graph construction: insights from an empirical study\.External Links:2506\.19773,[Link](https://arxiv.org/abs/2506.19773)Cited by:[§II\-C](https://arxiv.org/html/2607.01972#S2.SS3.p1.1)\.
- \[31\]N\. S\. Moosavi and M\. Strube\(2016\-08\)Which coreference evaluation metric do you trust? A proposal for a link\-based entity aware metric\.InProceedings of the 54th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),K\. Erk and N\. A\. Smith \(Eds\.\),Berlin, Germany,pp\. 632–642\.External Links:[Document](https://dx.doi.org/10.18653/v1/P16-1060)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px2.p1.1)\.
- \[32\]C\. Morris, M\. Ritzert, M\. Fey, W\. L\. Hamilton, J\. E\. Lenssen, G\. Rattan, and M\. Grohe\(2019\)Weisfeiler and Leman go neural: higher\-order graph neural networks\.InProceedings of the Thirty\-Third AAAI Conference on Artificial Intelligence and Thirty\-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence,AAAI’19/IAAI’19/EAAI’19\.External Links:ISBN 978\-1\-57735\-809\-1,[Document](https://dx.doi.org/10.1609/aaai.v33i01.33014602)Cited by:[3rd item](https://arxiv.org/html/2607.01972#S9.I1.i3.p1.5)\.
- \[33\]N\. Mostafazadeh, N\. Chambers, X\. He, D\. Parikh, D\. Batra, L\. Vanderwende, P\. Kohli, and J\. Allen\(2016\-06\)A corpus and cloze evaluation for deeper understanding of commonsense stories\.InProceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,K\. Knight, A\. Nenkova, and O\. Rambow \(Eds\.\),San Diego, CA, USA,pp\. 839–849\.External Links:[Document](https://dx.doi.org/10.18653/v1/N16-1098)Cited by:[§B\-E](https://arxiv.org/html/2607.01972#A2.SS5.p1.3),[§IV\-G](https://arxiv.org/html/2607.01972#S4.SS7.p1.4)\.
- \[34\]T\. Naseem, A\. Shah, H\. Wan, R\. Florian, S\. Roukos, and M\. Ballesteros\(2019\-07\)Rewarding Smatch: transition\-based AMR parsing with reinforcement learning\.InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics,A\. Korhonen, D\. Traum, and L\. Màrquez \(Eds\.\),Florence, Italy,pp\. 4586–4592\.External Links:[Document](https://dx.doi.org/10.18653/v1/P19-1451)Cited by:[§II\-C](https://arxiv.org/html/2607.01972#S2.SS3.p1.1)\.
- \[35\]OpenAI\(2025\)GPT\-5 system card\.Note:[https://cdn\.openai\.com/gpt\-5\-system\-card\.pdf](https://cdn.openai.com/gpt-5-system-card.pdf)Published 2025\-08\-07; reflection model uses thegpt\-5\-2025\-08\-07snapshot\. Accessed: 2026\-06\-28Cited by:[§V\-B](https://arxiv.org/html/2607.01972#S5.SS2.SSS0.Px1.p1.1)\.
- \[36\]J\. Opitz, A\. Daza, and A\. Frank\(2021\)Weisfeiler\-Leman in the Bamboo: novel AMR graph metrics and a benchmark for AMR graph similarity\.Transactions of the Association for Computational Linguistics9,pp\. 1425–1441\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00435)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px3.p1.1)\.
- \[37\]J\. Opitz, L\. Parcalabescu, and A\. Frank\(2020\)AMR similarity metrics from principles\.Transactions of the Association for Computational Linguistics8,pp\. 522–538\.External Links:[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00329)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px3.p1.1)\.
- \[38\]J\. Opitz\(2023\-05\)SMATCH\+\+: standardized and extended evaluation of semantic graphs\.InFindings of the Association for Computational Linguistics: EACL 2023,A\. Vlachos and I\. Augenstein \(Eds\.\),Dubrovnik, Croatia,pp\. 1595–1607\.External Links:[Document](https://dx.doi.org/10.18653/v1/2023.findings-eacl.118)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px3.p1.1),[TABLE I](https://arxiv.org/html/2607.01972#S2.T1.51.50.1.6.1.1)\.
- \[39\]K\. Opsahl\-Ong, M\. J\. Ryan, J\. Purtell, D\. Broman, C\. Potts, M\. Zaharia, and O\. Khattab\(2024\-11\)Optimizing instructions and demonstrations for multi\-stage language model programs\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, FL, USA,pp\. 9340–9366\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.525)Cited by:[§II\-C](https://arxiv.org/html/2607.01972#S2.SS3.p1.1),[§V\-B](https://arxiv.org/html/2607.01972#S5.SS2.SSS0.Px1.p1.1)\.
- \[40\]Á\. Pándy, R\. Lakatos, and A\. Hajdu\(2026\)Error\-driven prompt optimization for arithmetic reasoning: a code generation approach using on\-premises small language models on tabular data\.IEEE Access14,pp\. 62570–62583\.External Links:[Document](https://dx.doi.org/10.1109/ACCESS.2026.3685125)Cited by:[§II\-C](https://arxiv.org/html/2607.01972#S2.SS3.p1.1)\.
- \[41\]S\. G\. Patil, H\. Mao, F\. Yan, C\. C\. Ji, V\. Suresh, I\. Stoica, and J\. E\. Gonzalez\(2025\)The Berkeley function calling leaderboard \(BFCL\): from tool use to agentic evaluation of large language models\.InProceedings of the 42nd International Conference on Machine Learning \(ICML\),Proceedings of Machine Learning Research, Vol\.267,pp\. 48371–48392\.Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px4.p1.1)\.
- \[42\]M\. Pawlik and N\. Augsten\(2016\)Tree edit distance: robust and memory\-efficient\.Information Systems56,pp\. 157–173\.External Links:ISSN 0306\-4379,[Document](https://dx.doi.org/10.1016/j.is.2015.08.004)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px1.p1.1)\.
- \[43\]Qwen Team\(2026\)Qwen3\.5: towards native multimodal agents\.Note:[https://qwen\.ai/blog?id=qwen3\.5](https://qwen.ai/blog?id=qwen3.5)Blog post; published February 2026\. Accessed: 2026\-06\-28Cited by:[footnote 7](https://arxiv.org/html/2607.01972#footnote7)\.
- \[44\]K\. Ramnath, K\. Zhou, S\. Guan, S\. S\. Mishra, X\. Qi, Z\. Shen, S\. Wang, S\. Woo, S\. Jeoung, Y\. Wang, H\. Wang, H\. Ding, Y\. Lu, Z\. Xu, Y\. Zhou, B\. Srinivasan, Q\. Yan, Y\. Chen, H\. Ding, P\. Xu, and L\. L\. Cheong\(2025\-11\)A systematic survey of automatic prompt optimization techniques\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 33078–33110\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1681),ISBN 979\-8\-89176\-332\-6Cited by:[§I](https://arxiv.org/html/2607.01972#S1.p4.1)\.
- \[45\]R\. Rei, C\. Stewart, A\. C\. Farinha, and A\. Lavie\(2020\-11\)COMET: a neural framework for MT evaluation\.InProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),B\. Webber, T\. Cohn, Y\. He, and Y\. Liu \(Eds\.\),Online,pp\. 2685–2702\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.213)Cited by:[§I](https://arxiv.org/html/2607.01972#S1.p2.1)\.
- \[46\]K\. Riesen and H\. Bunke\(2009\-06\)Approximate graph edit distance computation by means of bipartite graph matching\.Image Vision Comput\.27\(7\),pp\. 950–959\.External Links:ISSN 0262\-8856,[Document](https://dx.doi.org/10.1016/j.imavis.2008.04.004)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px2.p1.1)\.
- \[47\]T\. Schnabel and J\. Neville\(2024\-11\)Symbolic prompt program search: a structure\-aware approach to efficient compile\-time prompt optimization\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, FL, USA,pp\. 670–686\.External Links:[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.37)Cited by:[§II\-C](https://arxiv.org/html/2607.01972#S2.SS3.p1.1)\.
- \[48\]T\. Sellam, D\. Das, and A\. Parikh\(2020\-07\)BLEURT: learning robust metrics for text generation\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,D\. Jurafsky, J\. Chai, N\. Schluter, and J\. Tetreault \(Eds\.\),Online,pp\. 7881–7892\.External Links:[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.704)Cited by:[§I](https://arxiv.org/html/2607.01972#S1.p2.1)\.
- \[49\]S\. Sheikhi, L\. Lovén, and P\. Kostakos\(2026\)Beyond the leaderboard: a survey of the science of evaluation, benchmarking, and methodologies for large language models\.IEEE Access14\(\),pp\. 66493–66515\.External Links:[Document](https://dx.doi.org/10.1109/ACCESS.2026.3686088)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px4.p1.1)\.
- \[50\]N\. Shervashidze, P\. Schweitzer, E\. J\. van Leeuwen, K\. Mehlhorn, and K\. M\. Borgwardt\(2011\)Weisfeiler\-Lehman graph kernels\.Journal of Machine Learning Research12\(77\),pp\. 2539–2561\.Cited by:[§III\-E](https://arxiv.org/html/2607.01972#S3.SS5.SSS0.Px3.p1.2),[§III\-E](https://arxiv.org/html/2607.01972#S3.SS5.SSS0.Px3.p3.1)\.
- \[51\]J\. L\. Stoisser, M\. Boubnovski Martell, L\. Phillips, C\. Hansen, and J\. Fauqueur\(2025\)STRuCT\-LLM: unifying tabular and graph reasoning with reinforcement learning for semantic parsing\.External Links:2506\.21575,[Link](https://arxiv.org/abs/2506.21575)Cited by:[§II\-C](https://arxiv.org/html/2607.01972#S2.SS3.p1.1)\.
- \[52\]K\. Tai\(1979\-07\)The tree\-to\-tree correction problem\.J\. ACM26\(3\),pp\. 422–433\.External Links:ISSN 0004\-5411,[Document](https://dx.doi.org/10.1145/322139.322143)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px1.p1.1)\.
- \[53\]USC Information Sciences Institute\(2018\)Bio AMR corpus, release 3\.0\.Note:[https://amr\.isi\.edu/download\.html](https://amr.isi.edu/download.html)amr\-release\-bio\-v3\.0;∼\\sim6,900 biomedical sentencesCited by:[§B\-C](https://arxiv.org/html/2607.01972#A2.SS3.p1.2),[§IV\-E](https://arxiv.org/html/2607.01972#S4.SS5.p1.1)\.
- \[54\]S\. Vassileva, I\. Koychev, and S\. Boytcheva\(2025\)FMI@SU ToxHabits: evaluating LLMs performance on toxic habit extraction in Spanish clinical texts\.InBioCreative IX Workshop at IJCAI 2025,External Links:2604\.06403,[Link](https://arxiv.org/abs/2604.06403)Cited by:[§II\-C](https://arxiv.org/html/2607.01972#S2.SS3.p1.1)\.
- \[55\]M\. Vilain, J\. Burger, J\. Aberdeen, D\. Connolly, and L\. Hirschman\(1995\)A model\-theoretic coreference scoring scheme\.InSixth Message Understanding Conference \(MUC\-6\),Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px2.p1.1)\.
- \[56\]G\. Wang, J\. Yu, X\. Zhang, D\. jiang, Y\. Song, P\. He, X\. Liu, and T\. Deb\(2025\)STED and consistency scoring: a framework for evaluating LLM structured output reliability\.InNeurIPS 2025 Workshop on Structured Probabilistic Inference & Generative Modeling,Cited by:[§I](https://arxiv.org/html/2607.01972#S1.p6.1),[§II\-B](https://arxiv.org/html/2607.01972#S2.SS2.p1.1),[TABLE I](https://arxiv.org/html/2607.01972#S2.T1.51.50.1.4.1.1),[§V](https://arxiv.org/html/2607.01972#S5.p1.1)\.
- \[57\]Y\. Wang, D\. J\. DeWitt, and J\. Cai\(2003\)X\-diff: an effective change detection algorithm for XML documents\.InProceedings 19th International Conference on Data Engineering \(Cat\. No\.03CH37405\),Bangalore, India,pp\. 519–530\.External Links:[Document](https://dx.doi.org/10.1109/ICDE.2003.1260818)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px1.p1.1)\.
- \[58\]B\. Weisfeiler and A\. A\. Leman\(1968\)The reduction of a graph to canonical form and the algebra which appears therein\.Nauchno\-Technicheskaya Informatsia2\(9\),pp\. 12–16\.Note:English translation by G\. Ryabov available onlineCited by:[3rd item](https://arxiv.org/html/2607.01972#S1.I1.i3.p1.1),[§III\-E](https://arxiv.org/html/2607.01972#S3.SS5.SSS0.Px3.p1.2)\.
- \[59\]C\. Yang, X\. Wang, Y\. Lu, H\. Liu, Q\. V\. Le, D\. Zhou, and X\. Chen\(2024\)Large language models as optimizers\.InThe Twelfth International Conference on Learning Representations,Cited by:[§II\-C](https://arxiv.org/html/2607.01972#S2.SS3.p1.1)\.
- \[60\]M\. Yüksekgönül, F\. Bianchi, J\. Boen, S\. Liu, P\. Lu, Z\. Huang, C\. Guestrin, and J\. Zou\(2025\-03\)Optimizing generative AI by backpropagating language model feedback\.Nature639\(8055\),pp\. 609–616\.Cited by:[1st item](https://arxiv.org/html/2607.01972#S1.I1.i1.p1.1),[§II\-C](https://arxiv.org/html/2607.01972#S2.SS3.p1.1),[§III\-F](https://arxiv.org/html/2607.01972#S3.SS6.p1.1)\.
- \[61\]Q\. Zhan, Y\. Wang, and H\. Huang\(2026\)Assessment of generative named entity recognition in the era of large language models\.External Links:2601\.17898,[Link](https://arxiv.org/abs/2601.17898)Cited by:[§IV\-C](https://arxiv.org/html/2607.01972#S4.SS3.p1.2)\.
- \[62\]K\. Zhang and D\. Shasha\(1989\)Simple fast algorithms for the editing distance between trees and related problems\.SIAM Journal on Computing18\(6\),pp\. 1245–1262\.External Links:[Document](https://dx.doi.org/10.1137/0218082)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px1.p1.1)\.
- \[63\]K\. Zhang\(1996\-03\)A constrained edit distance between unordered labeled trees\.Algorithmica15\(3\),pp\. 205–222\.External Links:ISSN 0178\-4617,[Document](https://dx.doi.org/10.1007/BF01975866)Cited by:[§II\-A](https://arxiv.org/html/2607.01972#S2.SS1.SSS0.Px1.p1.1)\.
- \[64\]T\. Zhang, V\. Kishore, F\. Wu, K\. Q\. Weinberger, and Y\. Artzi\(2020\)BERTScore: evaluating text generation with BERT\.InInternational Conference on Learning Representations,Cited by:[§I](https://arxiv.org/html/2607.01972#S1.p2.1)\.
- \[65\]H\. S\. Zheng, S\. Mishra, H\. Zhang, X\. Chen, M\. Chen, A\. Nova, L\. Hou, H\. Cheng, Q\. V\. Le, E\. H\. Chi, and D\. Zhou\(2024\)NATURAL PLAN: benchmarking LLMs on natural language planning\.External Links:2406\.04520,[Link](https://arxiv.org/abs/2406.04520)Cited by:[§B\-D](https://arxiv.org/html/2607.01972#A2.SS4.p1.4),[Appendix E](https://arxiv.org/html/2607.01972#A5.p4.6),[§IV\-F](https://arxiv.org/html/2607.01972#S4.SS6.p1.3)\.
- \[66\]Y\. Zhou, A\. I\. Muresanu, Z\. Han, K\. Paster, S\. Pitis, H\. Chan, and J\. Ba\(2023\)Large language models are human\-level prompt engineers\.InThe Eleventh International Conference on Learning Representations,Cited by:[§II\-C](https://arxiv.org/html/2607.01972#S2.SS3.p1.1)\.

![[Uncaptioned image]](https://arxiv.org/html/2607.01972v1/figures/jan_drchal.jpg)Jan Drchalis an assistant professor at the Faculty of Electrical Engineering \(FEE\), Czech Technical University in Prague \(CTU\)\. He received the Ing\. \(2006\) and Ph\.D\. \(2013\) degrees in electronics and computer science from CTU\. His doctoral dissertation focused on evolutionary optimization of artificial neural networks\. He is a member of the Artificial Intelligence Center \([http://aic\.fel\.cvut\.cz](http://aic.fel.cvut.cz/)\)\.Following his doctoral work, he worked mostly on machine learning applications in transportation and robotics\. Since 2021, his research has centered on natural language processing, with application domains including AI\-assisted journalism, automated fact\-checking, information extraction, and prompt optimization\.