ModularSQL：Text-to-SQL系统多重性盲点的运行时防护措施

# ModularSQL: A Runtime Guardrail for the Multiplicity Blind Spot in Text-to-SQL
Source: [https://arxiv.org/html/2609.29573](https://arxiv.org/html/2609.29573)
###### Abstract

Text\-to\-SQL systems are increasingly deployed atop production\-style databases, where queries that pass benchmark evaluation can still produce results that distort downstream workflows\. A query missingDISTINCTcan inflate downstream aggregates such asSUMand report incorrect totals; duplicated rows can pollute analytics dashboards; a Cartesian\-style join explosion can trigger query timeouts and cost overruns\. Standard set\-based execution accuracy \(Set\-EX\) implicitly collapses duplicate rows in execution outputs and therefore fails to surface these deployment risks\.

We call this theMultiplicity Blind Spot\(MBS\)\. To quantify it, we operationalizeMultiset\-EX, a multiplicity\-preserving criterion that retains duplicate row counts\. Across released DeepEye\-SQL prediction artifacts from three open\-source backbones: Qwen2\.5\-Coder\-32B, Qwen3\-Coder\-30B\-A3B, and Gemma\-3\-27B, on the executable BIRD\-Dev subset \(N=1,532N=1\{,\}532\), we observe a stable Set\-EX vs\. Multiset\-EX gap of 5\.81–6\.79 pp, suggesting that production\-relevant multiplicity errors recur even in released Text\-to\-SQL artifacts\. The gap persists beyond this pipeline family: released DAIL\-SQL\+GPT\-4 predictions show a 5\.22 pp gap and the official BIRD GPT\-3\.5\-turbo baseline a 3\.39 pp gap on the same subset\.

We then introduceModularSQL, a post\-selection runtime guardrail\. ModularSQL probes each executed result for multiplicity anomalies and applies deterministic patches or low\-cost LLM rescues only on flagged queries, leaving unaffected queries unchanged\. Integrated atop the DeepEye\-SQL reproduction with Qwen3\-Coder, ModularSQL preserves the Set\-EX baseline \(72\.06%\), improves Multiset\-EX from 65\.86% to 67\.75% \(\+1\.89 pp\), and flags 77 high\-risk anomalies, predominantly cardinality explosions, while adding only $0\.0076 in total LLM cost and 120 ms amortized latency per query in our run\. These results suggest that benchmark execution accuracy does not necessarily imply execution\-safe SQL and that lightweight multiplicity\-aware guardrails can narrow this gap with low deployment overhead\.

Code will be available at[https://github\.com/Ruixi1313/ModularSQL](https://github.com/Ruixi1313/ModularSQL)

## 1Introduction

Text\-to\-SQL is increasingly being used as an interface to databases in production settings for business intelligence, analytics, and decision support\. However, SQL that is correct under benchmark evaluation does not necessarily guarantee execution safety\. When duplicate tuples are incorrectly introduced or removed, the resulting query can distort aggregates, dashboards, and downstream analytical workflows\. This gap between success and execution safety motivates our study\.

LLM\-based Text\-to\-SQL systems increasingly rely on multi\-stage pipelines involving schema linking, value grounding, execution feedback, revision, and candidate selection\([Li et al\., 2026](https://arxiv.org/html/2609.29573#bib.bib7)\), and are commonly evaluated on benchmarks such as BIRD\([Li et al\., 2023](https://arxiv.org/html/2609.29573#bib.bib8)\)and Spider\([Yu et al\., 2018](https://arxiv.org/html/2609.29573#bib.bib12)\)\. Standard execution accuracy considers a predicted query correct if its result agrees with the gold query on a given database\. Evaluation protocols differ in how matches are defined: BIRD’s official evaluator treats results as*sets*, collapsing duplicate rows, whereas Spider’s test\-suite evaluator treats them as multisets\([Zhong et al\., 2020](https://arxiv.org/html/2609.29573#bib.bib14)\)\. We therefore follow the set\-based protocol used for all BIRD leaderboard results\. This evaluation tolerates surface\-form variation, but, as previous work shows, only approximates semantic correctness\([Zhong et al\., 2020](https://arxiv.org/html/2609.29573#bib.bib14)\)and can produce false positives or negatives\([Kim et al\., 2025](https://arxiv.org/html/2609.29573#bib.bib6)\)\.

Existing critiques primarily focus on semantic equivalence: whether two SQL programs express the same logical intent\. We identify a complementary gap: execution correctness under set\-based comparison, which we denote asSet\-EX, does not imply execution safety\. Classical relational algebra treats relations as sets\([Codd, 1970](https://arxiv.org/html/2609.29573#bib.bib2)\), and many Text\-to\-SQL evaluators similarly compare output after collapsing duplicate tuples\. This collapse discards multiplicity information that remains observable in actual SQL execution, where duplicates persist unless explicitly removed byDISTINCTor aggregation\([Dayal et al\., 1982](https://arxiv.org/html/2609.29573#bib.bib3)\)\. A predicted query may therefore receive full execution credit while producing duplicated rows through a missingDISTINCT, an erroneous join, or a Cartesian\-style explosion, the same failure modes that distort production analytics\.

We call this phenomenon theMultiplicity Blind Spot\(MBS\): the evaluator discards row multiplicity before comparison, so a Text\-to\-SQL system can appear correct under conventional execution accuracy while remaining unsafe for realistic deployment\. To measure this gap, we useMultiset\-EXas a complementary diagnostic criterion that preserves tuple multiplicities during result comparison\. We do not propose it as a replacement for official benchmark metrics\. Instead, it allows us to quantify failures that Set\-EX cannot observe\. On the executable BIRD\-Dev subset \(N=1,532N=1\{,\}532\), released DeepEye\-SQL artifacts from Qwen2\.5\-Coder\-32B, Qwen3\-Coder\-30B\-A3B, and Gemma\-3\-27B show a stable 5\.81–6\.79 pp Set\-EX versus Multiset\-EX gap\. The gap is not unique to this pipeline family: released DAIL\-SQL\+GPT\-4 predictions show a 5\.22,pp gap, while the official BIRD GPT\-3\.5\-turbo baseline shows a 3\.39,pp gap on the same query set\.

We then introduceModularSQL, a post\-selection runtime guardrail that operates atop existing pipelines\. ModularSQL probes each executed result for multiplicity anomalies \(e\.g\., duplication\-ratio thresholds\) and applies deterministic patches or low\-cost LLM rescues only on flagged queries, leaving unaffected queries unchanged\. Integrated atop our DeepEye\-SQL reproduction with Qwen3\-Coder, ModularSQL preserves the Set\-EX baseline \(72\.06%\)\. Under the stricter Multiset\-EX metric, the same reproduced baseline scores 65\.86%, and ModularSQL improves it to 67\.75% \(\+1\.89 pp\)\. It also flags 77 high\-risk anomalies, predominantly cardinality explosions, while adding only $0\.0076 to the total LLM cost in our run\. The candidate\-free guardrail components transfer unchanged to two additional pipelines’ released predictions, improving Multiset\-EX by 10 and 8 queries, respectively, while leaving Set\-EX unchanged\. These results suggest that benchmark execution accuracy does not necessarily imply execution\-safe SQL\.

Our contributions are:

- •We identify theMultiplicity Blind Spot\(MBS\), a structural limitation of set\-based execution evaluation that discards SQL multiplicity semantics and can mask multiplicity\-sensitive execution risks\.
- •We operationalizeMultiset\-EXas a complementary multiplicity\-aware diagnostic criterion to measure failures hidden by Set\-EX\.
- •We provide cross\-backbone*and cross\-pipeline*empirical evidence: three DeepEye\-SQL backbone artifacts show a stable 5\.81–6\.79 pp gap, and the gap persists on released DAIL\-SQL\+GPT\-4 \(5\.22 pp\) and BIRD GPT\-3\.5\-turbo \(3\.39 pp\) predictions\.
- •We introduceModularSQL, a post\-selection runtime guardrail that flags 77 high\-risk multiplicity anomalies and improves Multiset\-EX from 65\.86% to 67\.75% \(\+1\.89 pp\) while preserving the reproduced Set\-EX baseline at 72\.06%, with candidate\-free components that transfer unchanged to two additional pipelines\.
- •We characterize the guardrail through detector precision/recall, threshold cross\-validation, wall\-clock latency, and a non\-LLM selection baseline that isolates the multiplicity\-aware signal from generic LLM repair ability\.

## 2Background and Related Work

### 2\.1Text\-to\-SQL Pipelines and Execution Feedback

Modern Text\-to\-SQL systems have evolved from direct SQL generation into multi\-stage orchestration pipelines that decompose, regenerate, and select among candidate SQL queries based on intermediate execution feedback\([Pourreza and Rafiei, 2023](https://arxiv.org/html/2609.29573#bib.bib9);[Gao et al\., 2024](https://arxiv.org/html/2609.29573#bib.bib4);[Li et al\., 2026](https://arxiv.org/html/2609.29573#bib.bib7)\)\. DeepEye\-SQL\([Li et al\., 2026](https://arxiv.org/html/2609.29573#bib.bib7)\), the system we build on, follows a seven\-stage workflow; we focus on its final three:\(S5\)candidate generation, where the backbone LLM produces 12 candidate SQL queries per question;\(S6\)revision, where candidates are iteratively repaired through LLM\-driven rewrites guided by syntactic checks and execution errors; and\(S7\)selection, where surviving candidates are adjudicated through a tournament procedure to produce the final SQL\.

The execution feedback consulted in these stages is primarily*validity\-oriented*: it triggers on parse failures, invalid schema references, and runtime execution errors\. Related LLM self\-debugging work extends this feedback channel for general code generation\([Chen et al\., 2024](https://arxiv.org/html/2609.29573#bib.bib1)\), but the multiplicity structure of executed output, including whether duplicates are sparse or dense and whether a join has amplified the row count, is not explicitly monitored by these feedback signals\. This leaves a diagnostic gap that validity\-oriented feedback alone does not close\.

#### Relation to repair and refinement methods\.

Execution\-guided repair\([Chen et al\., 2024](https://arxiv.org/html/2609.29573#bib.bib1)\)and multi\-candidate refinement\([Pourreza and Rafiei, 2023](https://arxiv.org/html/2609.29573#bib.bib9);[Gao et al\., 2024](https://arxiv.org/html/2609.29573#bib.bib4);[Li et al\., 2026](https://arxiv.org/html/2609.29573#bib.bib7)\)regenerate or revise SQL using execution feedback\. ModularSQL is complementary: our main setting measures the 6\.20,pp Multiset\-EX gap*after*DeepEye\-SQL’s revision \(S6\) and tournament selection \(S7\)\. The guardrail closes 30\.5% of this residual gap post hoc while preserving Set\-EX\. We do not evaluate an additional full repair round; this limitation is stated in Section[Limitations](https://arxiv.org/html/2609.29573#Sx1)\.

### 2\.2From Set\-EX to Multiplicity\-Preserving Evaluation

Letqpq\_\{p\}denote a predicted SQL query,qgq\_\{g\}the gold SQL query, andDDthe target database\. LetR⁡\(q,D\)R\(q,D\)be the*multiset*of result rows produced by executing queryqqonDD\.Set\-EXtreats a prediction as correct when the predicted and gold result rows match as sets\. Let:

Sp\\displaystyle S\_\{p\}=set⁡\(R⁡\(qp,D\)\),\\displaystyle=\\mathrm\{set\}\(R\(q\_\{p\},D\)\),Sg\\displaystyle S\_\{g\}=set⁡\(R⁡\(qg,D\)\)\.\\displaystyle=\\mathrm\{set\}\(R\(q\_\{g\},D\)\)\.SetEX\(qp,qg,D\)=𝟏\[Sp=Sg\]\.\\mathrm\{SetEX\}\(q\_\{p\},q\_\{g\},D\)=\\mathbf\{1\}\[S\_\{p\}=S\_\{g\}\]\.This robustly ignores row\-order differences, but also collapses duplicate multiplicities\. This set\-collapsing behavior is not a simplification we introduce but follows the official BIRD protocol: both the main BIRD evaluator \(execute\_sqlinevaluation\.py\)111[https://github\.com/AlibabaResearch/DAMO\-ConvAI/blob/main/bird/llm/src/evaluation\.py](https://github.com/AlibabaResearch/DAMO-ConvAI/blob/main/bird/llm/src/evaluation.py)and the BIRD Mini\-Dev evaluator \(evaluation\_ex\.py\)222[https://github\.com/bird\-bench/mini\_dev/blob/main/evaluation/evaluation\_ex\.py](https://github.com/bird-bench/mini_dev/blob/main/evaluation/evaluation_ex.py)compareset\(predicted\_res\) == set\(ground\_truth\_res\)\. In contrast, Spider’s test\-suite evaluator uses multiset comparison \(multiset\_eqinexec\_eval\.py\)\([Zhong et al\., 2020](https://arxiv.org/html/2609.29573#bib.bib14)\)\. Thus, MBS is a property of set\-collapsing protocols such as BIRD’s official evaluation, not of Text\-to\-SQL evaluation in general\. Previous critiques of execution\-based evaluation focus on*semantic equivalence*\([Zhong et al\., 2020](https://arxiv.org/html/2609.29573#bib.bib14);[Kim et al\., 2025](https://arxiv.org/html/2609.29573#bib.bib6)\), a direction orthogonal to ours: even when Set\-EX certifies a prediction as answer\-correct, its execution may still contain unsafe duplicate structure\.

Classical relational algebra treats relations as sets\([Codd, 1970](https://arxiv.org/html/2609.29573#bib.bib2)\), whereas actual SQL execution follows bag semantics: aSELECTwithoutDISTINCTpreserves duplicates that affect aggregation operators such asCOUNTandSUM\.[Dayal et al\. \(1982\)](https://arxiv.org/html/2609.29573#bib.bib3)formalize this through an extended algebra with explicit duplicate\-elimination control\.

Aligned with this perspective, we operationalize a multiplicity\-preserving variant of execution accuracy:

MultisetEX\(qp,qg,D\)=\[R\(qp,D\)=R\(qg,D\)\],\\mathrm\{MultisetEX\}\(q\_\{p\},q\_\{g\},D\)=\\mathbf\{1\}\\\!\\left\[\\,R\(q\_\{p\},D\)=R\(q\_\{g\},D\)\\,\\right\],Here the comparison is order\-insensitive but multiplicity\-preserving: duplicate tuple counts are retained, while row order is ignored\. Multiset\-EX is at least as strict as Set\-EX:MultisetEX=1⇒SetEX=1\\mathrm\{MultisetEX\}=1\\Rightarrow\\mathrm\{SetEX\}=1, but the converse does not necessarily hold\.

Queries that pass Set\-EX while failing Multiset\-EX realize theMultiplicity Blind Spot\(MBS\)\. In such cases, theset⁡\(⋅\)\\mathrm\{set\}\(\\cdot\)projection removes precisely the duplicate information that distinguishes the predicted and gold outputs, so Set\-EX cannot detect the discrepancy by construction\. This blindness is a property of the evaluator itself; its empirical incidence on any specific artifact depends on the producing pipeline\. We use Multiset\-EX as a complementary diagnostic lens, not as a replacement for Set\-EX\.

## 3ModularSQL Methodology

ModularSQL is a post\-selection runtime guardrail layer that complements the DeepEye\-SQL output rather than replacing the underlying pipeline\. It probes the executed result of each selected SQL query for multiplicity\-related anomalies and applies targeted interventions, including deterministic patches and low\-cost LLM rescue only when an anomaly is detected\. Figure[1](https://arxiv.org/html/2609.29573#S3.F1)summarizes the architecture\.

Question\+\+DB SchemaDeepEye\-SQLS5 Candidate Gen\.→\\rightarrowS6 Revision→\\rightarrowS7 SelectionSelected SQLModularSQL Runtime Probedup\_ratio/ empty result / execution errorNormalreturn unchangedAnomalouspatch or LLM rescueFinal SQLModularSQL post\-selection guardrail

Figure 1:Overview of ModularSQL\. ModularSQL operates as a post\-selection runtime guardrail layer atop DeepEye\-SQL\. After S7 selects an executable SQL query, ModularSQL probes the execution behavior for multiplicity\-related anomalies such as high duplication ratio, empty results, or execution errors\. Unaffected queries pass through unchanged, while anomalous queries are routed through targeted interventions, including deterministic patches or a low\-cost LLM rescue\.### 3\.1Runtime Probe Design

Given an executable SQL queryqpq\_\{p\}produced by S7, the runtime probe attempts to executeqpq\_\{p\}on the target databaseDDand, when execution succeeds, obtains the result row multisetRp=R⁡\(qp,D\)R\_\{p\}=R\(q\_\{p\},D\)\. The probe then inspects three health conditions: \(i\) execution error, \(ii\) empty result, and \(iii\) excessive duplicate multiplicity\. Leterr⁡\(qp,D\)∈\{0,1\}\\mathrm\{err\}\(q\_\{p\},D\)\\in\\\{0,1\\\}be the indicator that executingqpq\_\{p\}onDDraises a database error\. Whenerr⁡\(qp,D\)=0\\mathrm\{err\}\(q\_\{p\},D\)=0and\|Rp\|\>0\|R\_\{p\}\|\>0, the duplication ratio

dup​\_​ratio​\(Rp\)=\|Rp\|−\|unique⁡\(Rp\)\|\|Rp\|∈\[0,1\)\\mathrm\{dup\\\_ratio\}\(R\_\{p\}\)\\;=\\;\\frac\{\|R\_\{p\}\|\-\|\\mathrm\{unique\}\(R\_\{p\}\)\|\}\{\|R\_\{p\}\|\}\\;\\in\\;\[0,1\)\(1\)measures the fraction of result rows that are redundant duplicate copies, i\.e\., rows beyond the first occurrence of each distinct row\. For compactness, leterr=err⁡\(qp,D\)\\mathrm\{err\}=\\mathrm\{err\}\(q\_\{p\},D\)andρ⁡\(Rp\)=dup​\_​ratio​\(Rp\)\\rho\(R\_\{p\}\)=\\mathrm\{dup\\\_ratio\}\(R\_\{p\}\)\. A query is marked*anomalous*by the following ordered rule:

anomalous⁡\(qp\)=\{1,err=11,err=0,\|Rp\|=0,1,err=0,\|Rp\|\>0,ρ⁡\(Rp\)≥τ,0otherwise\.\\mathrm\{anomalous\}\(q\_\{p\}\)=\\begin\{cases\}1,&\\mathrm\{err\}=1\\\\ 1,&\\mathrm\{err\}=0,\\ \|R\_\{p\}\|=0,\\\\ 1,&\\begin\{aligned\} &\\mathrm\{err\}=0,\\ \|R\_\{p\}\|\>0,\\\\ &\\rho\(R\_\{p\}\)\\geq\\tau,\\end\{aligned\}\\\\ 0&\\text\{otherwise\}\.\\end\{cases\}
We deployτ=0\.80\\tau=0\.80as the operating point; the full sensitivity sweep and cross validation are reported in Section[4](https://arxiv.org/html/2609.29573#S4)\.

The three conditions target different failure regimes\. Condition \(i\) catches queries that are demonstrably broken at the database level\. Condition \(iii\) targets the multiplicity\-sensitive failures characterized in Section[2\.2](https://arxiv.org/html/2609.29573#S2.SS2): missingDISTINCT, join amplification, and Cartesian\-style row explosion, all of which may remain hidden under set\-based evaluation\. Condition \(ii\) is a conservative trigger rather than a correctness claim: an empty result is sometimes legitimate \(the gold query may also return no rows\), but in deployment we flag it as a probe signal that triggers downstream interventions \(Sections[3\.2](https://arxiv.org/html/2609.29573#S3.SS2)and[3\.3](https://arxiv.org/html/2609.29573#S3.SS3)\)\. The multiplicity check requires a linear scan ofRpR\_\{p\}; conditions \(i\) and \(ii\) are read directly from the database response\. No condition invokes an LLM\. The probe targets*runtime\-observable physical anomalies*; semantic errors with normal\-looking outputs are out of scope\. The operating point prioritizes precision over recall \(Section[4\.6](https://arxiv.org/html/2609.29573#S4.SS6)\)\. Among the three conditions, condition \(i\) overlaps with the validity\-oriented feedback described in Section[2\.1](https://arxiv.org/html/2609.29573#S2.SS1), whereas conditions \(ii\) and \(iii\) introduce additional runtime signals: empty outputs and duplicate multiplicity anomalies\.

The probe therefore induces an asymmetric intervention policy\. Queries that do not trigger any anomaly are returned unchanged, while flagged queries are routed to targeted interventions\. These interventions comprise a low\-cost LLM rescue followed by deterministic patches \(Sections[3\.3](https://arxiv.org/html/2609.29573#S3.SS3)and[3\.2](https://arxiv.org/html/2609.29573#S3.SS2)\); the ordering is empirically necessary and is examined in Section[4](https://arxiv.org/html/2609.29573#S4)\. This design preserves the behavior of the underlying Text\-to\-SQL pipeline on unflagged queries and keeps additional inference cost proportional to the number of anomalous cases rather than the total workload\. In this sense, ModularSQL acts as a post\-selection guardrail: it requires only the selected SQL and its executed result, rather than modifying candidate generation, revision, or selection\.

### 3\.2Deterministic Patches

Following the LLM rescue layer \(Section[3\.3](https://arxiv.org/html/2609.29573#S3.SS3)\), ModularSQL applies two deterministic rewrites that target the most common multiplicity\-sensitive failures\. Both rewrites are syntactic and invoke no LLM\.

P1 \(DISTINCT injection\)\.If the duplication ratio of the predicted query’s executed result satisfiesdup​\_​ratio​\(Rp\)≥0\.80\\mathrm\{dup\\\_ratio\}\(R\_\{p\}\)\\geq 0\.80, ModularSQL inserts aDISTINCTclause into the outermostSELECT\. When the query contains aLIMITclause,DISTINCTonly deduplicates before theLIMITcut, so injection can alter even set\-semantics results\. We identified this guard through cross\-pipeline evaluation \(Section[4\.3](https://arxiv.org/html/2609.29573#S4.SS3)\); none of the patch\-eligible queries in our DeepEye\-SQL reproduction containsLIMIT, so the reported results are unaffected\. The threshold is equivalent to total rows being at least five times the number of distinct rows, the characteristic signature of join\-induced cartesian explosion; below this regime, duplicates are more likely to be semantically meaningful \(e\.g\., two distinct entities sharing a display name\)\.

P2 \(DISTINCT removal\)\.For predicted queries that already contain aDISTINCTclause, ModularSQL tentatively removes it, re\-executes the modified query, and checks whether the resulting row multiset has a very low duplication ratio \(dup​\_​ratio≤0\.10\\mathrm\{dup\\\_ratio\}\\leq 0\.10\)\. If so, the originalDISTINCTis treated as a candidate over\-application by the backbone model, and the rewritten query is committed\.

The two thresholds are not symmetric: although a single split point between "too few" and "too many" duplicates might seem natural, empirically the duplication ratio is a strong signal for addingDISTINCTbut only an actionable signal for removal in the extreme tail \(dup​\_​ratio≤0\.10\\mathrm\{dup\\\_ratio\}\\leq 0\.10\)\. This asymmetry suggests that unnecessaryDISTINCTs introduced by LLMs reflect query\-intent misclassifications rather than mechanical row amplification, and motivates treating the two patches as independent rules rather than as symmetric counterparts\. Sensitivity to both thresholds is reported in Section[4](https://arxiv.org/html/2609.29573#S4)\.

### 3\.3Budget\-Aware LLM Rescue

For queries that the runtime probe flags as anomalous, ModularSQL invokes a low\-cost LLM rescue step before the deterministic patches of Section[3\.2](https://arxiv.org/html/2609.29573#S3.SS2)\. The rescue treats the LLM as a judge that selects among the candidate set produced by the upstream pipeline rather than generating new SQL from scratch\([Zheng et al\., 2023](https://arxiv.org/html/2609.29573#bib.bib13)\)\. Algorithm[1](https://arxiv.org/html/2609.29573#alg1)summarizes the procedure\.

Algorithm 1Budget\-Aware LLM Rescue1:Predicted SQL

qpq\_\{p\}, candidate set

CCfrom upstream stages, database

DD, previews

PP
2:

Rp←Execute​\(qp,D\)R\_\{p\}\\leftarrow\\textsc\{Execute\}\(q\_\{p\},D\)⊳\\trianglerightSection[3\.1](https://arxiv.org/html/2609.29573#S3.SS1)

3:if

anomalous⁡\(qp\)=0\\mathrm\{anomalous\}\(q\_\{p\}\)=0then

4:return

qpq\_\{p\}⊳\\trianglerighttrust S7 selection

5:endif

6:

prompt←BuildPrompt​\(q​u​e​s​t​i​o​n,s​c​h​e​m​a,qp,C,P\)\\textit\{prompt\}\\leftarrow\\textsc\{BuildPrompt\}\(question,schema,q\_\{p\},C,P\)
7:

idx←LLMJudge​\(prompt\)\\textit\{idx\}\\leftarrow\\textsc\{LLMJudge\}\(\\textit\{prompt\}\)
8:if

idx≥0\\textit\{idx\}\\geq 0then

9:return

C⁡\[idx\]C\[\\textit\{idx\}\]⊳\\trianglerightcandidate replaces S7 output before patches

10:else

11:return

qpq\_\{p\}⊳\\trianglerightLLM declined; fall back to S7

12:endif

Input\.The LLM receives the natural\-language question, a schema summary, the original S7 selection, the full set of revised candidates from S6 \(twelve in our setting\), and a short execution preview of each candidate \(the first few result rows, an empty\-result flag, or an execution error\)\. It is also told which probe condition triggered the rescue\.

Selection\.The LLM returns an index into the candidate set, or a special token indicating that no candidate looks safe\. If a candidate is selected, ModularSQL replaces the S7 prediction with that candidate; if no selection is made, the S7 prediction is retained unchanged\.

Cost\.Because the rescue is gated by the probe, its LLM budget scales with the number of flagged queries rather than the total workload\. On our BIRD\-Dev reproduction the rescue fires on 77 of 1,532 queries and adds approximately $0\.0076 in total LLM cost in our run \(under $0\.0001 per affected query under our provider pricing\)\. End\-to\-end accuracy and per\-condition trigger statistics are reported in Section[4](https://arxiv.org/html/2609.29573#S4)\.

## 4Experiments

### 4\.1Experimental Setup

We evaluate ModularSQL on the BIRD\-Dev split\([Li et al\., 2023](https://arxiv.org/html/2609.29573#bib.bib8)\), comprising 1,534 question\-SQL\-database triples\. Gold and predicted SQL are executed under a 5 s timeout, excluding two queries \(qids 518, 701\) whose gold SQL exceeds this limit and giving an evaluable subset ofN=1,532N=1\{,\}532\. We reproduce the seven\-stage DeepEye\-SQL pipeline\([Li et al\., 2026](https://arxiv.org/html/2609.29573#bib.bib7)\)with Qwen3\-Coder\-30B\-A3B\([Qwen Team, 2025](https://arxiv.org/html/2609.29573#bib.bib11)\)as the backbone LLM\. ModularSQL operates atop this reproduction with operating pointsτ=0\.80\\tau=0\.80for P1 \(DISTINCTinjection\) andτ=0\.10\\tau=0\.10for P2 \(DISTINCTremoval\)\. All predictions are scored under both Set\-EX and Multiset\-EX, defined in Section[2\.2](https://arxiv.org/html/2609.29573#S2.SS2)\. For cross\-pipeline evaluation, we additionally use two released BIRD\-Dev prediction sets: DAIL\-SQL\+GPT\-4’s main configuration\([Gao et al\., 2024](https://arxiv.org/html/2609.29573#bib.bib4)\)and the official BIRD GPT\-3\.5\-turbo \(knowledge\) baseline\([Li et al\., 2023](https://arxiv.org/html/2609.29573#bib.bib8)\)\. As each provides a single final SQL per question, only the candidate\-free detector and deterministic patches apply\.

### 4\.2Cross\-Backbone Evidence of MBS

We first establish that MBS is not specific to our reproduction by evaluating three released DeepEye\-SQL prediction artifacts on the same evaluable BIRD\-Dev subset \(N=1,532N=1\{,\}532\)\. The artifacts are generated by Qwen2\.5\-Coder\-32B\([Qwen Team, 2024](https://arxiv.org/html/2609.29573#bib.bib10)\), Qwen3\-Coder\-30B\-A3B\([Qwen Team, 2025](https://arxiv.org/html/2609.29573#bib.bib11)\), and Gemma\-3\-27B\([Gemma Team, 2025](https://arxiv.org/html/2609.29573#bib.bib5)\)\. Table[1](https://arxiv.org/html/2609.29573#S4.T1)reports Set\-EX and Multiset\-EX scores for each artifact and for our own DeepEye\-SQL reproduction with Qwen3\-Coder, computed with the evaluator implementations described in Section[2\.2](https://arxiv.org/html/2609.29573#S2.SS2)\.

All four measurements exhibit a stable Set\-EX versus Multiset\-EX gap between 5\.81 and 6\.79 percentage points across different backbone families and released artifacts\. Our reproduction \(6\.20 pp\) falls inside this range\. This is consistent with our framing that the blindness is induced by set\-based evaluation, while its empirical incidence recurs across artifacts, including those produced by multi\-stage Text\-to\-SQL pipelines that already incorporate execution feedback and tournament selection\.

Table 1:Set\-EX and Multiset\-EX on executable BIRD\-Dev \(N=1,532N=1\{,\}532\)\. Scores are percentages; gaps are percentage points\.
### 4\.3Cross\-Pipeline Transfer

To assess whether MBS and the candidate\-free guardrail components transfer beyond DeepEye\-SQL, we apply the detector and deterministic patches unchanged \(τ=0\.80\\tau=0\.80\) to two released BIRD\-Dev prediction sets \(Section[4\.1](https://arxiv.org/html/2609.29573#S4.SS1)\)\. As shown in Table[2](https://arxiv.org/html/2609.29573#S4.T2), the MBS gap persists \(5\.22 pp and 3\.39 pp\), while weaker pipelines trigger substantially more anomalies \(270 and 555 vs\. 77 for DeepEye\-SQL\)\. With theLIMITguard \(Section[3\.2](https://arxiv.org/html/2609.29573#S3.SS2)\), the deterministic patch improves Multiset\-EX by\+10\+10and\+8\+8queries, respectively, with no change in Set\-EX\. We do not evaluate the candidate\-based LLM rescue because these releases provide only one SQL per question and thus no candidate pool\.

Table 2:Candidate\-free guardrail components applied unchanged to two released BIRD\-Dev prediction sets \(N=1,532N\{=\}1\{,\}532\)\. Set\-EX is unchanged by the patch on both systems\.
### 4\.4Main Results and Ablations

Table 3:Main results on the executable BIRD\-Dev subset \(N=1,532N=1\{,\}532\)\.Δ\\Deltais measured relative to the DeepEye reproduction\. Rows 1 to 3 give the cumulative effect of the deterministic patches; rows 4 to 5 contrast the two intervention orderings\. The highlight v2 row is the canonical ModularSQL configuration used throughout the paper\. The dup\-ratio selector replaces the LLM rescue with a zero\-cost rule that selects the healthy candidate with the lowest duplication ratio; unlike the LLM rescue, it does not fully preserve Set\-EX, yielding a net gain of 2 Set\-EX queries\.Table[3](https://arxiv.org/html/2609.29573#S4.T3)quantifies the end\-to\-end effect of each ModularSQL component on our DeepEye\-SQL reproduction with Qwen3\-Coder\-30B\-A3B\. ModularSQL v2 preserves the Set\-EX baseline \(1,104 of 1,532; 72\.06%\) and improves Multiset\-EX from 65\.86% to 67\.75%, a net gain of 29 queries \(\+1\.89 pp\)\. Set\-EX is invariant across all five configurations while Multiset\-EX improves over the baseline under every intervention, confirming that the guardrail changes execution outcomes specifically along the multiplicity dimension and leaves the underlying pipeline’s set\-level behavior intact\.

The two deterministic patches together account for 24 of the 29 net Multiset\-EX rescues: P1 \(DISTINCT injection atτ=0\.80\\tau=0\.80\) contributes 21 queries \(\+1\.37 pp\), and P2 \(DISTINCT removal atτ=0\.10\\tau=0\.10\) adds 3 more on top of P1 \(\+0\.20 pp\)\. The remaining \+5 queries in ModularSQL v2 come from the LLM rescue layer, which substitutes alternative S6 candidates before the patches refine the result\.

Rows 4 and 5 of Table[3](https://arxiv.org/html/2609.29573#S4.T3)compare the two intervention orderings introduced in Section[3\.1](https://arxiv.org/html/2609.29573#S3.SS1)\. Applying the deterministic patches before the LLM rescue \(v1\) yields a Multiset\-EX net gain of only \+20 queries, 9 fewer than v2 \(\+29\)\. The 0\.58 pp gap quantifies the cost of letting the LLM rescue overwrite already\-patched SQL, and motivates the rescue\-first ordering used throughout the rest of the paper\.

#### Non\-LLM rescue baseline\.

Replacing the LLM rescue with a zero\-cost selector that chooses the healthy candidate with the lowest duplication ratio yields 1,031/1,532 \(67\.30%\) Multiset\-EX, versus 1,038 with the LLM rescue, recovering 22 of 29 net corrections \(75\.9%\)\. Thus, most gains come from the multiplicity\-aware signal rather than generic LLM repair\. The remaining difference isolates the LLM’s role: among 76 evaluable flagged queries, 30 have no healthy candidate, causing the ratio rule to fall back to the base SQL; 12 of the LLM’s 13 additional fixes occur in these cases\.

### 4\.5Analysis of Runtime Interventions

#### Trigger reasons\.

Of the 77 queries the probe flags as anomalous, 70 \(91%\) trigger condition \(iii\) \(duplication ratio aboveτ\\tau\), 6 \(8%\) trigger condition \(ii\) \(empty result\), and 1 triggers condition \(i\) \(execution error\)\. The duplication ratio signal therefore dominates trigger volume, consistent with our characterization of MBS as a multiplicity\-amplification phenomenon\. In our run, the LLM returned a valid candidate index for all 77 triggered cases; the fallback path defined in Algorithm[1](https://arxiv.org/html/2609.29573#alg1)was not invoked\.

#### Cost\.

The intervention adds approximately $0\.0076 in LLM cost in our run \(under $0\.0001 per affected query\); the deterministic patches incur no inference cost\. Because the LLM rescue is gated by the probe, end\-to\-end cost scales with the number of flagged queries rather than the total workload\. Only 5\.0% \(77/1,532\) of evaluated queries trigger the rescue stage\.

#### Rescue effect on flagged queries\.

Within the 77 cases the probe flags, the LLM rescue improves Multiset\-EX from 12/77 \(15\.6%\) to 29/77 \(37\.7%\) relative to the raw S7 selection, with 25 multiplicity\-level fixes against 8 regressions\. This local gain does not translate directly into end\-to\-end improvement, because the deterministic patches also modify some of the same queries\. Consequently, the rescue layer contributes a net\+5\+5queries to end\-to\-end Multiset\-EX in v2\.

#### By\-difficulty gain\.

Table[4](https://arxiv.org/html/2609.29573#S4.T4)breaks down the Multiset\-EX gain by BIRD difficulty stratum\. The largest gain is observed in the challenging stratum \(\+4\.17 pp\), consistent with the intuition that complex multi\-join queries expose more multiplicity\-sensitive failure modes\. Together, these results suggest that multiplicity\-sensitive failures are relatively rare but disproportionately concentrated in complex queries, making them well suited for selective runtime intervention\.

Table 4:Multiset\-EX gain by BIRD difficulty stratum \(N=1,532N=1\{,\}532\)\. Scores are percentages;Δ\\Deltais the ModularSQL v2 gain over the DeepEye reproduction\. The challenging stratum shows the largest relative gain\.

### 4\.6Detector Diagnostics

With Multiset\-EX failure as ground truth, the deployed trigger \(τ=0\.80\\tau=0\.80\) is precision\-first: precision is 0\.844 on DeepEye\-SQL, 0\.945 on DAIL\-SQL\+GPT\-4, and 0\.991 on GPT\-3\.5\-turbo\. The exec\-error and empty\-result triggers achieve 1\.000 precision, while the duplication trigger achieves 0\.797–0\.878 across systems\. For MBS failures \(Set\-EX\-correct but Multiset\-EX\-wrong\), recall is 0\.35–0\.43\. Recall over all SQL errors is necessarily partial because semantic errors with normal\-looking outputs are invisible to runtime probing \(Section[3\.1](https://arxiv.org/html/2609.29573#S3.SS1)\)\. Notably, the fixedτ\\tautransfers to unseen DAIL\-SQL and GPT\-3\.5\-turbo pipelines, with precision improving rather than degrading\. Leave\-one\-database\-out cross\-validation over nine databases with P1\-fire cases yields a held\-out net gain of\+17\+17, non\-negative in 8/9 folds \(worst:−2\-2\); removing the threshold reduces the held\-out net to\+5\+5\. Full results are in Appendix[I](https://arxiv.org/html/2609.29573#A9)\.

#### Latency\.

On a commodity laptop with local SQLite databases, detection reuses fetched result rows and costs 3μ\\mus median per query\. TheDISTINCTpatch adds one execution on 56 queries \(2 ms median, 82 ms p90\)\. The LLM rescue, triggered on 77/1,532 queries, takes 1\.48 s median end\-to\-end \(1\.36 s for the API call, 47 ms for candidate re\-execution\)\. Overall, the guardrail adds 120 ms per query amortized across the workload\.

## 5Discussion

ModularSQL is designed as a deployment\-friendly intervention layer that complements rather than replaces an existing Text\-to\-SQL pipeline\. The deployment profile of this design appears clearly in our reproduction: the Set\-EX score reported by BIRD\-style evaluators is preserved exactly, the probe fires on only 5% of queries, and the additional LLM cost over a 1,532\-query workload is under one cent in our run, and the amortized latency overhead is 120 ms per query\. Deployment teams that already trust their selection\-stage output can adopt the guardrail incrementally without retraining the backbone or modifying the upstream stages\. Because the 6\.20,pp gap remains after DeepEye\-SQL’s own revision and selection stages, the guardrail complements rather than replaces existing repair and refinement methods\.

The observed 30\.5% reduction in the MBS gap also highlights a broader evaluation consideration\. Set\-EX, while operationally useful for benchmarking, is structurally blind to multiplicity\-level errors that real downstream applications, such as analytics dashboards, aggregate reports, and materialized views, cannot tolerate\. Pairing benchmark Set\-EX with a complementary multiplicity\-aware diagnostic such as Multiset\-EX can help surface this gap during evaluation rather than at deployment\. The residual 4\.31 pp gap suggests that multiplicity\-sensitive failures are not exhausted by duplicate\-elimination patterns alone\.

## 6Conclusion

We introduced ModularSQL, a post\-selection runtime guardrail that exposes and partially mitigates the Multiplicity Blind Spot in benchmark\-correct Text\-to\-SQL pipelines\. On our DeepEye\-SQL reproduction with Qwen3\-Coder, ModularSQL preserves the Set\-EX baseline \(72\.06%\) and closes 30\.5% of the reproduced MBS gap, with candidate\-free components that transfer unchanged to two additional pipelines’ released predictions, adding less than one cent of the LLM cost and 120 ms amortized latency per 1,532\-query workload\. These results suggest that benchmark execution accuracy does not necessarily imply execution\-safe SQL, and that lightweight multiplicity\-aware guardrails are a practical complement to existing Text\-to\-SQL pipelines and evaluation practices\.

## Limitations

#### Pipeline scope\.

The full ModularSQL system, including the candidate\-based LLM rescue, is evaluated only on our DeepEye\-SQL reproduction\. The candidate\-free detector and patches transfer to released DAIL\-SQL\+GPT\-4 and BIRD GPT\-3\.5\-turbo predictions \(Section[4\.3](https://arxiv.org/html/2609.29573#S4.SS3)\), but these provide only one SQL per question, leaving the rescue stage untested beyond DeepEye\-SQL\. We found no publicly released BIRD\-Dev predictions with candidate pools from other pipelines\.

#### Benchmark and engine\.

We evaluate on BIRD\-Dev only, on SQLite databases\. Spider, BIRD\-Test, and production database engines such as PostgreSQL or BigQuery are not yet covered\.

#### Patch coverage\.

The deterministic patch family currently covers onlyDISTINCT\-level rewrites \(P1: injection; P2: removal\)\. More complex multiplicity\-sensitive operators, includingGROUP BYaggregation semantics, nestedCOUNT/SUMexpressions, and JOIN topology rewrites, remain out of scope\. Closing the residual 4\.31 pp gap likely requires extending the patch family to address these operators\.

#### LLM rescue prompt\.

The rescue prompt does not yet instruct the LLM to prefer candidates that minimize spurious duplicates\. Aligning the prompt with the multiplicity\-aware probe \(e\.g\., providing each candidate’s duplication ratio as part of the context\) is a natural low\-cost extension\.

#### Threshold calibration\.

The P1 duplication\-ratio thresholdτ=0\.80\\tau=0\.80is calibrated against the P1\-fire cases in our DeepEye\-SQL reproduction\. Other artifact families or query distributions may require re\-tuning, although our sensitivity sweep \(Appendix[A](https://arxiv.org/html/2609.29573#A1)\) shows a plateau overτ∈\[0\.55,0\.80\]\\tau\\in\[0\.55,0\.80\]\. Cross\-pipeline transfer \(Section[4\.3](https://arxiv.org/html/2609.29573#S4.SS3)\) and fold\-level cross\-validation \(Section[4\.6](https://arxiv.org/html/2609.29573#S4.SS6)\) provide additional out\-of\-distribution evidence at the same operating point\.

#### No direct repair comparison\.

We do not evaluate an additional repair or regeneration round on flagged queries\. The non\-LLM selector \(Section[4\.4](https://arxiv.org/html/2609.29573#S4.SS4)\) isolates the multiplicity\-aware signal from generic LLM repair, while direct comparison with execution\-guided repair is left to future work\.

## References

- Chen et al\. \(2024\)Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou\. 2024\.Teaching large language models to self\-debug\.In*International Conference on Learning Representations*\.
- Codd \(1970\)E\. F\. Codd\. 1970\.[A relational model of data for large shared data banks](https://doi.org/10.1145/362384.362685)\.*Communications of the ACM*, 13\(6\):377–387\.
- Dayal et al\. \(1982\)Umeshwar Dayal, Nathan Goodman, and Randy H\. Katz\. 1982\.[An extended relational algebra with control over duplicate elimination](https://doi.org/10.1145/588111.588132)\.In*Proceedings of the 1st ACM SIGACT\-SIGMOD Symposium on Principles of Database Systems*, pages 117–123\. ACM\.
- Gao et al\. \(2024\)Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou\. 2024\.[Text\-to\-SQL empowered by large language models: A benchmark evaluation](https://doi.org/10.14778/3641204.3641221)\.*Proceedings of the VLDB Endowment*, 17\(5\):1132–1145\.
- Gemma Team \(2025\)Gemma Team\. 2025\.[Gemma 3 technical report](https://arxiv.org/abs/2503.19786)\.*Preprint*, arXiv:2503\.19786\.
- Kim et al\. \(2025\)Heegyu Kim, Jeon Taeyang, SeungHwan Choi, Seungtaek Choi, and Hyunsouk Cho\. 2025\.[FLEX: Expert\-level false\-less EXecution metric for text\-to\-SQL benchmark](https://doi.org/10.18653/v1/2025.naacl-long.228)\.In*Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies*, pages 4448–4475\. Association for Computational Linguistics\.
- Li et al\. \(2026\)Boyan Li, Chong Chen, Zhujun Xue, Yinan Mei, and Yuyu Luo\. 2026\.[DeepEye\-SQL: A software\-engineering\-inspired text\-to\-SQL framework](https://doi.org/10.1145/3802035)\.*Proceedings of the ACM on Management of Data*, 4\(3\)\.
- Li et al\. \(2023\)Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, Xuanhe Zhou, Chenhao Ma, Guoliang Li, Kevin C\.\-C\. Chang, Fei Huang, Reynold Cheng, and Yongbin Li\. 2023\.[Can LLM already serve as a database interface? a BIg bench for large\-scale database grounded text\-to\-SQLs](https://proceedings.neurips.cc/paper_files/paper/2023/file/83fc8fab1710363050bbd1d4b8cc0021-Paper-Datasets_and_Benchmarks.pdf)\.In*Advances in Neural Information Processing Systems*, volume 36\. Curran Associates, Inc\.
- Pourreza and Rafiei \(2023\)Mohammadreza Pourreza and Davood Rafiei\. 2023\.DIN\-SQL: Decomposed in\-context learning of Text\-to\-SQL with self\-correction\.In*Advances in Neural Information Processing Systems*, volume 36\.
- Qwen Team \(2024\)Qwen Team\. 2024\.[Qwen2\.5\-Coder technical report](https://arxiv.org/abs/2409.12186)\.*Preprint*, arXiv:2409\.12186\.
- Qwen Team \(2025\)Qwen Team\. 2025\.[Qwen3 technical report](https://arxiv.org/abs/2505.09388)\.*Preprint*, arXiv:2505\.09388\.
- Yu et al\. \(2018\)Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev\. 2018\.[Spider: A large\-scale human\-labeled dataset for complex and cross\-domain semantic parsing and text\-to\-SQL task](https://doi.org/10.18653/v1/D18-1425)\.In*Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing*, pages 3911–3921\. Association for Computational Linguistics\.
- Zheng et al\. \(2023\)Lianmin Zheng, Wei\-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph Gonzalez, and Ion Stoica\. 2023\.Judging LLM\-as\-a\-judge with MT\-bench and chatbot arena\.In*Advances in Neural Information Processing Systems*, volume 36\.
- Zhong et al\. \(2020\)Ruiqi Zhong, Tao Yu, and Dan Klein\. 2020\.[Semantic evaluation for text\-to\-SQL with distilled test suites](https://doi.org/10.18653/v1/2020.emnlp-main.29)\.In*Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing \(EMNLP\)*, pages 396–411\. Association for Computational Linguistics\.

## Appendix AP1 Threshold Sensitivity

The deployed operating pointτ=0\.80\\tau=0\.80for P1 \(DISTINCT injection\) was selected by a sweep on the 87 P1\-fire cases in our DeepEye\-SQL reproduction \(46 fixed \+ 41 broken atτ=0\\tau=0\)\. Table[5](https://arxiv.org/html/2609.29573#A1.T5)reports fix\-break counts at representative thresholds\. The deployed operating pointτ=0\.80\\tau=0\.80sits at the top of a plateau spanningτ∈\[0\.55,0\.80\]\\tau\\in\[0\.55,0\.80\], with Net at least\+20\+20throughout this range\. Leave\-one\-database\-out cross\-validation over the 9 databases with P1\-fire cases yields a held\-out net of\+17\+17at the same operating point, indicating stability of the threshold choice beyond the fit in the sample\. The protocol difference is worth noting: Table[5](https://arxiv.org/html/2609.29573#A1.T5)reports a\+22\+22gain on the 87\-case threshold\-selection subset, whereas the end\-to\-end gain is\+21\+21\(Table[3](https://arxiv.org/html/2609.29573#S4.T3)\), because the refined rule also triggers on one case outside the calibration subset, causing a PASS\-to\-FAIL regression\.

Table 5:P1 \(DISTINCT injection\) threshold sensitivity in our DeepEye\-SQL reproduction\. Fix and Break count the rule’s effect on the 87 P1\-fire cases at eachτ\\tau; Net isFix−Break\\text\{Fix\}\-\\text\{Break\}\. The deployed operating pointτ=0\.80\\tau=0\.80at the top of the plateau is highlighted\.
## Appendix BP2 Threshold Sensitivity and Asymmetric Signal

We deploy P2 \(DISTINCT removal\) at the conservative operating pointτ=0\.10\\tau=0\.10, where the rule fires 5 times in our reproduction with 3 fixes and 0 regressions\. Table[6](https://arxiv.org/html/2609.29573#A2.T6)reports a sweep over four additional thresholds\.

Table 6:P2 \(DISTINCT removal\) threshold sensitivity in our DeepEye\-SQL reproduction\. Higherτ\\tauraises the firing rate but does not improve net rescue; the fix\-to\-break ratio collapses to1:11\{:\}1atτ=0\.30\\tau=0\.30and inverts aboveτ=0\.50\\tau=0\.50\.#### Asymmetric signal\.

The duplication ratio thresholdτ\\taubehaves asymmetrically across the two patches\. P1 \(injection\) remains a strong signal up toτ≈0\.85\\tau\\approx 0\.85before its yield collapses; P2 \(removal\) is actionable only in the extreme tailτ≤0\.20\\tau\\leq 0\.20, with the fix\-break tradeoff degrading sharply above this point\. This asymmetry suggests that unnecessaryDISTINCTs introduced by an LLM\-driven pipeline reflect query\-intent misclassification rather than mechanical duplicate amplification, and that "add DISTINCT" and "remove DISTINCT" should be treated as independent rules rather than mirror images of one another\. This motivates designing the patch family asymmetrically rather than as paired add/remove rules calibrated to a single split point on the duplication ratio\.

## Appendix CLLM Rescue Prompt

The LLM rescue stage of Algorithm[1](https://arxiv.org/html/2609.29573#alg1)is implemented as a single judge\-style query to the backbone model\. The exact prompt template used in our experiments is shown below\. Placeholders such as\{question\},\{schema\_summary\}, and\{cand\_text\}are instantiated on a per\-query basis\. The\{cand\_text\}placeholder expands to twelve candidate entries of the form "\[i\]SQL: <candidate SQL\> Result: <execution preview\>", one per candidate\.

> You are selecting the best SQL from candidate solutions for a BIRD benchmark question\. Question: \{question\} Hint: \{hint\} Schema \(relevant tables/columns\): \{schema\_summary\} The currently selected SQL has a physical issue \(\{trigger\_reason\}\): \{base\_sql\} Result: \{base\_preview\} Here are 12 alternative candidates with their execution results: \{cand\_text\} Pick the candidate that best answers the question\. Respond with ONLY the candidate number \(0\-11\), nothing else\. Your answer:

#### Parameters\.

We call the model withtemperature=0\.0andmax\_tokens=64\. The response is parsed by extracting the first integer in\[0,11\]\[0,11\]; if no valid integer is found, the rescue invokes the fallback branch of Algorithm[1](https://arxiv.org/html/2609.29573#alg1)and returns the original S7 selection\. Each candidate’s SQL is truncated to 600 characters and the schema summary to 1,500 characters to keep the prompt within typical context budgets\. The prompt contains no multiplicity\-aware instructions and does not expose duplication\-ratio statistics to the model\. Consistent with prior LLM\-as\-a\-judge work\([Zheng et al\., 2023](https://arxiv.org/html/2609.29573#bib.bib13)\), the rescue stage usestemperature=0\.0to maximize selection stability and reproducibility\.

#### Trigger reason placeholder\.

The placeholder is set to one of three values aligned with the probe condition in Section[3\.1](https://arxiv.org/html/2609.29573#S3.SS1):cartesian\_explosion\(condition iii: duplication ratio aboveτ\\tau\),empty\_result\(condition ii\), orexec\_error\(condition i\)\.

## Appendix DP2 Implementation Safeguards

The P2 rewrite is applied only when all of the following safeguards are satisfied:

- •The query contains no aggregate function \(COUNT,SUM,AVG, etc\.\)\.
- •The query contains noGROUP BYclause\.
- •RemovingDISTINCTstrictly increases the row count of the executed result\.

## Appendix EIntervention\-Order Ablation: v1 vs v2

The two intervention orderings introduced in Section[3\.1](https://arxiv.org/html/2609.29573#S3.SS1)differ by 9 queries in net Multiset\-EX gain \(\+20\+20for v1,\+29\+29for v2\)\. This appendix isolates where the 9\-query gap originates\. Rows 4 and 5 of Table[7](https://arxiv.org/html/2609.29573#A5.T7)correspond to the two orderings reported in Table[3](https://arxiv.org/html/2609.29573#S4.T3)\.

Table 7:Per\-stage Multiset\-EX counts \(N=1,532\)\. The marginalΔ\\Deltacolumn reports the gain or loss relative to the \+P1\+P2 configuration\. In v1 the rescue acts on patched SQL and results in a net loss of 4 Multiset\-EX\-correct queries; in v2 the rescue acts on the original S7 selection before patches refine the result, yielding 5 additional queries beyond patches alone\.#### Overwrite versus refinement\.

In v1, P1 and P2 have already corrected the multiplicity behavior of some flagged queries; the LLM rescue replaces these corrected SQLs with its own selection and in 4 cases, overwrites a patched fix with a worse candidate\. In v2, the rescue acts on the original S7 selection, and P1 and P2 then operate on the rescued SQL\. The 9\-query swing between v1 and v2 isolates the cost of letting the LLM rescue overwrite already\-patched SQL\.

## Appendix FBy\-Difficulty Per\-Stage Breakdown

We extend the by\-difficulty summary of Table[4](https://arxiv.org/html/2609.29573#S4.T4)to show the Multiset\-EX count at each intermediate pipeline stage\. The deterministic patches and the two intervention orderings can then be inspected stratum by stratum\.

Table 8:Multiset\-EX counts per BIRD\-Dev difficulty stratum at each pipeline stage\. "v1" denotes the P1\+P2→\\rightarrowRescue ordering and "v2" the canonical Rescue→\\rightarrowP1\+P2 ordering, matching Table[3](https://arxiv.org/html/2609.29573#S4.T3)\.#### Stratum\-level v1 vs v2 differences\.

The 9\-query swing between v1 and v2 \(1038−10291038\-1029\) splits as\+6\+6in the simple stratum,\+2\+2in moderate, and\+1\+1in challenging\. P2 contributes at most two additional fixes within any individual difficulty stratum, consistent with its conservative operating point \(τ=0\.10\\tau=0\.10\) and the asymmetric\-signal finding in Appendix[B](https://arxiv.org/html/2609.29573#A2)\.

## Appendix GMultiset Equality Implementation Check

Multiset equality between predicted and gold result rows admits two natural implementations: comparison ofcollections\.Counterobjects, and string\-keyed sorted\-tuple equality \(sorted\(rows, key=str\)\)\. On the released Qwen3\-Coder DeepEye\-SQL prediction artifact, both implementations agree on allN=1,532N=1\{,\}532evaluable samples \(1019 passes; 0 discrepancies\), confirming that the reported Multiset\-EX scores are invariant across the two implementations considered here\.

## Appendix HCross\-Pipeline Details

Table[9](https://arxiv.org/html/2609.29573#A8.T9)extends the cross\-pipeline summary in Section[4\.3](https://arxiv.org/html/2609.29573#S4.SS3)with per\-trigger\-reason flagged counts and the P1 patch fix/break breakdown for each system\.

Table 9:Per\-system flagged counts by trigger reason and P1 patch outcomes on BIRD\-Dev \(N=1,532N=1\{,\}532\)\. “Exec” = execution error, “Empty” = empty result set, “Cart” = cartesian explosion \(dup\-ratio≥τ\\geq\\tau\)\. P1 fires only on cartesian\-flagged queries that do not contain aLIMITclause\. Fix and Brk count Multiset\-EX transitions \(→10\\\!\\to\\\!1and→01\\\!\\to\\\!0, respectively\) among the P1\-fired subset\.#### Trigger\-reason distribution\.

The trigger distribution varies markedly across systems\. DeepEye\-SQL, the strongest baseline, has few execution errors or empty results \(7 combined\), whereas GPT\-3\.5\-turbo has 514\. Cartesian explosions are more stable \(41–74\), suggesting that duplicate\-generating joins reflect benchmark schema structure rather than model\-specific failures\.

#### P1 firing rate\.

P1 fires on 56 of DeepEye\-SQL’s 70 cartesian\-flagged queries; the 14 excluded cases contain aLIMITclause\. On weaker systems, fewer cartesian queries and the LIMIT guard further reduce the P1\-eligible pool \(49 for DAIL\-SQL, 11 for GPT\-3\.5\-turbo\)\. Despite this smaller pool, the patch maintains a non\-negative Multiset\-EX net across all three systems, with no Set\-EX regressions on DAIL\-SQL or GPT\-3\.5\-turbo\.

Table 10:Set\-EX and Multiset\-EX accuracy before and after the candidate\-free guardrail \(P1 only\) on the two cross\-pipeline systems\. Set\-EX is preserved exactly; Multiset\-EX improves by 0\.65 pp and 0\.52 pp, respectively\.

## Appendix IDetector Diagnostics Tables

This appendix expands the detector diagnostics in Section[4\.6](https://arxiv.org/html/2609.29573#S4.SS6)with full precision, recall, and per\-trigger breakdowns across all three evaluated systems\.

#### Overall precision and MBS recall\.

Table[11](https://arxiv.org/html/2609.29573#A9.T11)reports overall detector precision \(the fraction of flagged queries that are true Multiset\-EX failures\) and MBS recall \(the fraction of multiplicity\-blind\-spot failures that are flagged\)\. The detector flags queries with an execution error \(condition i\), an empty result set \(condition ii\), or a duplication ratio≥τ\\geq\\tau\(condition iii\)\.

Table 11:Detector precision \(PP\) and MBS recall \(RR\) on BIRD\-Dev \(N=1,532N=1\{,\}532\)\. TP denotes flagged queries that are Multiset\-EX failures; FP denotes flagged queries that are Multiset\-EX correct\. MBS failures pass Set\-EX but fail Multiset\-EX; the detector prioritizes precision rather than maximizing MBS recall\.
#### Per\-reason precision\.

Table[12](https://arxiv.org/html/2609.29573#A9.T12)reports precision by trigger reason\. Execution errors and empty results achieve perfect precision \(1\.000\) across all three systems, as both are definitive failure signals\. Cartesian explosion is the only source of false positives, with precision ranging from 0\.797 to 0\.878\.

Table 12:Per\-trigger\-reason precision\. Each cell shows precision followed by \(TP/total\) for that trigger reason\. All false positives originate from the cartesian\-explosion trigger\.
#### Design rationale\.

The detector prioritizes precision over recall: a false positive incurs one LLM rescue call and added latency, but typically leaves the output unchanged if the original candidate is re\-selected; a false negative is simply a missed opportunity\. The resulting MBS recall \(0\.35–0\.43\) reflects this trade\-off\. The detector catches MBS failures only when they coincide with a runtime anomaly; “silent” MBS failures with normal, non\-empty, non\-exploded outputs are invisible by construction\. Higher MBS recall would require semantic query analysis \(e\.g\., determining whetherDISTINCTis warranted\), which we leave to future work\.