DPC: Training-Free Text-to-SQL Candidate Selection via Dual-Paradigm Consistency Boyan Li, Ou Ocean Kun Hei, Yue Yu, Yuyu Luo† The Hong Kong University of Science and Technology (Guangzhou) † Corresponding author Code: https://github.com/HKUSTDial/DPC
Abstract
arXiv:2604.15163v2 [cs.DB] 17 Apr 2026
While Large Language Models (LLMs) demonstrate impressive proficiency in generating SQL queries, they fundamentally lack the capability to self-evaluate correctness without an execution oracle. This limitation creates a stark Generation-Selection Gap, where high potential accuracy (Pass@K) fails to translate into execution accuracy (Pass@1). Although supervised verifiers offer mitigation, they incur prohibitive annotation costs and suffer from domain fragility. Consequently, recent research has pivoted to the training-free setting. However, existing methods—such as SelfConsistency or LLM-as-a-Judge—remain hampered by systematic bias (consensus on hallucinations) and symbolic blindness (inability to simulate execution states). We introduce DPC (Dual-Paradigm Consistency), a multiagent framework that reformulates SQL selection from a probabilistic guessing task on hidden data into a deterministic verification task on visible data. Specifically, DPC employs a S LICER and a T ESTER agent to collaboratively construct a Minimal Distinguishing Database (MDD)—an adversarial, fully observable micro-environment engineered to expose logical discrepancies between candidates. To break the self-correction bias, a S OLVER agent then verifies the SQL candidates by crossreferencing their execution against a parallel Python/Pandas solution. By validating execution consistency between declarative (SQL) and imperative (Python) paradigms, DPC robustly discriminates correct logic from systematic hallucinations. Experiments on BIRD and Spider across multiple LLMs demonstrate that our method consistently outperforms existing selection baselines, achieving absolute accuracy improvements of up to 2.2% over strong competitors like Self-Consistency.
1
Introduction
Recent advances in Large Language Models (LLMs) have revolutionized the Text-to-SQL task,
enabling systems to generate highly plausible SQL queries for complex questions (Li et al., 2023b; Yu et al., 2018; Zhu et al., 2025a; Li et al., 2024). However, due to the inherent stochasticity of LLMs, a single generation is often unreliable. Consequently, state-of-the-art approaches adopt a “generate-thenselect” paradigm: generating K candidate queries to cover the correct logic (Li et al., 2025b,a). In this work, we focus on the critical downstream task: SQL Candidate Selection—the process of autonomously identifying the single correct executable query from a pool of K candidates. Despite remarkable generative capabilities, a significant Generation-Selection Gap persists. As shown in Table 2, while the potential Pass@K is high (e.g., 58.8% on BIRD), real-world users demand a single correct answer; yet, the actual Pass@1 significantly lags behind (∼50%). Although training-based verifiers (Liu et al., 2025d) attempt to bridge this via fine-tuning, prohibitive annotation costs and poor generalization limit their utility. Consequently, focus has shifted to the training-free setting, yet current mechanisms remain inadequate. Heuristic approaches like SelfConsistency (Xie et al., 2025) fail under systematic bias, where models consistently converge on errors (Figure 1, Top-Left). Similarly, LLM-as-aJudge (Lee et al., 2025) is hindered by symbolic blindness: constrained by limited data context and unable to mentally simulate execution, it relies on internal priors and exhibits intrinsic judgement bias (Figure 1, Top-Right). We argue that the failure of current selection methods stems from three intrinsic challenges in the Text-to-SQL paradigm. First, (C1) Partial Observability: Real-world databases are typically too massive to fit within the context window. This forces the model to verify logic against a “hidden” data distribution (Dhidden ) based on probabilistic assumptions rather than concrete evidence, creating an epistemic gap that precludes rigorous verifica-
tion. Second, (C2) Symbolic Blindness: Even if the data were visible, LLMs lack an internal interpreter to reliably simulate the state changes of complex SQL operations (e.g., nested JOINs) merely by inspection. Finally, (C3) Intrinsic Confirmation Bias: When acting as a selector, the model inherently favors candidates that align with its internal priors—even if flawed. This results in a “conflict of interest” where the model fails to objectively distinguish between its own hallucinations and correct logic. To address these challenges, we propose DPC (Dual-Paradigm Consistency), a multi-agent framework that shifts SQL selection from a probabilistic guessing task on hidden data to a deterministic reasoning task on visible data. DPC follows a progressive verification pipeline. First, to overcome Partial Observability (C1), we introduce the concept of Adversarial Environment Synthesis. Instead of reasoning about the massive, invisible database, DPC employs a S LICER and a T ESTER agent to construct a Minimal Distinguishing Database (MDD). The MDD is not merely a data sample, but a context-fitting, adversarial micro-environment specifically engineered to yield divergent execution results for conflicting SQLs. This transforms the verification environment from partially observable to Fully Observable, enabling the model to ground its decisions in explicit execution evidence. Second, to mitigate Symbolic Blindness (C2) and Intrinsic Confirmation Bias (C3), we introduce Dual-Paradigm Verification. Rather than relying solely on SQL generation, we leverage the model’s inherent competency disparity across programming languages. Extensive research indicates that LLMs exhibit superior reasoning capabilities in widespread imperative languages like Python, attributed to their dominance in pre-training corpora relative to domain-specific query languages (Twist et al., 2025; Lozhkov et al., 2024). Furthermore, translating a user’s intent into imperative Python code forces the model to explicitly plan the data manipulation steps, offering a distinct reasoning path from declarative SQL formulation (Gao et al., 2023; Yang et al., 2025). Therefore, DPC employs a S OLVER agent to generate a parallel Python solution on the MDD. By treating the higher-confidence Python solution as a proxy ground truth, DPC pinpoints the correct SQL candidate that aligns with the imperative logic. Our contributions are summarized as follows: • We identify Partial Observability and Systematic
Heuristic Selection (Systematic Bias)
LLM-as-a-Judge (Restrict Perception & Bias) tri Res
Consensus on errors
Judger
ta Da ct
Judg emen t Bias
Our Dual-Paradigm Consistency (DPC) BS-F1 Calc
Declarative Paradigm Imperative
Exec Result
Paradigm Minimal Distinguish Database
SQL with Higher Score
Figure 1: Comparison of Selection Paradigms. (TopLeft) Heuristic Selection relies on majority voting but fails when models exhibit systematic bias, reaching a consensus on errors. (Top-Right) LLM-as-a-Judge suffers from restricted perception due to the lack of execution feedback, relying solely on internal priors. (Bottom) Our DPC framework introduces a Dual-Paradigm approach (SQL & Python). It synthesizes a Minimal Distinguishing Database to execute both paradigms, determining correctness through deterministic result consistency rather than probability.
Bias as the core constraints in SQL selection and propose the Minimal Distinguishing Database (MDD) to facilitate rigorous, fully observable, and adversarial verification. • We introduce DPC, a training-free framework that leverages Dual-Paradigm Consistency to identify semantic correctness via complementary reasoning paths. • We design a Bipartite Soft-F1 (BS-F1) metric to handle formatting heterogeneities and rowordering ambiguity between SQL and Python results, ensuring rigorous alignment. • Extensive experiments on BIRD and Spider benchmarks demonstrate that DPC consistently outperforms state-of-the-art training-free selection methods, establishing new standards in execution accuracy.
2
Problem Formulation
2.1
The SQL Selection Task
Let Q denote a natural language question and S = (T , C, R) denote a relational database schema, where T is a set of tables, C is a set of columns, and R represents foreign key relations. Given Q and S, an LLM generates a set of K candidate SQL queries, denoted as Y = {y1 , y2 , . . . , yK }. The objective of SQL Selection is to identify
the optimal candidate y ∗ ∈ Y that is semantically equivalent to the ground truth query ygt . Semantic equivalence is defined by execution correctness on the database instance: E XEC(y ∗ , D) = E XEC(ygt , D)
(1)
where D = (S, V) represents the database instance, consisting of the schema S and the actual data values (records) V. In the training-free setting, we seek a selection function f that maximizes the likelihood of identifying y ∗ under fixed model parameters θ: f ∗ = argmax P f (Q, S, Y; θ) ≡ ygt (2)
Discriminative Validity, where VM DD is adversarially engineered to expose semantic discrepancies. Formally, for any pair of semantically distinct candidates yi , yj ∈ Y (where yi ̸≡ yj ), their execution on the MDD must diverge: yi ̸≡ yj =⇒ E XEC(yi , DM DD ) ̸= E XEC(yj , DM DD )
(4)
Under these constraints, DPC transforms the task from probabilistic guessing on hidden data to deterministic consistency checking on visible data.
3
The DPC Framework
3.1
Framework Overview
f
subject to the constraint that f operates without access to the ground truth ygt or gradient updates. 2.2
The Observability Gap
A critical constraint in real-world deployment is that the data instance is partially observable. Let W denote the context window capacity of the LLM. Typically, the full data values V far exceed this limit (|V| ≫ W ). While standard prompting strategies may provide a small set of sample values Vexample ⊂ V (e.g., 3-shot row samples) to the model, the vast majority of the data distribution remains unseen, denoted as Vunseen = V \ Vexample . This partial observability creates a verification gap. The provided Vexample is typically static and sparse, lacking the boundary cases necessary to verify complex logic. For instance, distinguishing between INNER JOIN and LEFT JOIN requires specific records (e.g., unmatched keys) that are likely absent in a random sample. Consequently, execution consistency on the example subset is a necessary but insufficient condition for correctness: E XEC(yi , Dexample ) = E XEC(ygt , Dexample ) ̸⇒ yi ≡ ygt
(3)
where Dexample = (S, Vexample ). The model is thus forced to select y ∗ based on probabilistic priors rather than definitive execution evidence. 2.3
Objective Reformulation
Since verification on partial Dexample is unreliable, we reformulate the selection task by constructing a Minimal Distinguishing Database (MDD), denoted as DM DD = (S ′ , VM DD ). To serve as a valid verification ground, the MDD must satisfy two core properties: (i) Contextual Feasibility, requiring the synthetic instance to fit within the LLM’s context window (|DM DD | ≤ W ); and (ii)
DPC is a multi-agent system bridging the epistemic gap in SQL selection. As shown in Figure 2, it reformulates the task into a progressive verification pipeline, prioritizing external execution evidence over internal priors. It coordinates three agents (S LICER, T ESTER, S OLVER) across four phases (see Appendix A for detailed prompts): Stage 1: Candidate Clustering & Pairing. Instead of evaluating all candidates individually, DPC first condenses the selection space by clustering semantically identical queries. It then samples a Champion-Challenger pair to represent the most significant logical conflict, maximizing the efficiency of subsequent verification. Stage 2: Adversarial Environment Synthesis. To resolve the conflict, the S LICER and T ESTER agents collaborate to synthesize the Minimal Distinguishing Database (MDD). Unlike random sampling, the MDD is adversarially generated to be context-fitting (small enough for the context window) yet discriminative (guaranteeing divergent execution results for conflicting logic). Stage 3: Dual-Paradigm Execution. With the environment fully observable, DPC introduces a complementary reasoning path. The S OLVER agent generates an imperative Python script to solve the question on the MDD. This Python execution serves as a high-confidence reference anchor for verifying the declarative SQL candidates. Stage 4: Consistency Verification. Finally, DPC determines the winner by comparing the SQL execution results against the Python reference. We employ a novel Bipartite Soft-F1 metric to robustly quantify semantic equivalence, overcoming both formatting heterogeneities and row-ordering ambiguity inherent to cross-paradigm verification.
STAGE 1: Candidate Clustering & Pairing
STAGE 2: Adversarial Environment Synthesis
STAGE 3: Dual-Paradigm Execution
Schema Sclicing
Declarative Paradigm (SQL)
SQL Candidates
Full Schema (S)
Full Database (D)
Rchamp Slice & Dry-run
Ychamp Execution & Clustering
Ychal
STAGE 4: Consistency Verification
Slicer
Feedback
Ychamp Sliced Schema (S’)
Ychal
Exec
Rchamp
Decimal/Float Datetime/Timestap
MDD Rchal
Rchal
Rpy
Atomic Value Normalization
None/NaN/Null
Float (4 decimals) YYYY-MM-DD NULL Object
Global Optimal Matching Group A
Group B
Minimal Distinguishing Database (MDD) Generation
Group C S’
Challenger (Ychal)
Ychal
Tester
Optimal Matching
Score Aggregation
Ychamp Champion (Ychamp)
Python Exec
Hungarian Algorithm
Question
Construct & Run SQLs Feedback
Imperative Paradigm (Python)
MDD
MDD
Solver
Rpy
Compare
BS-F1 (TP/FP/FN)
Select Ychamp OR
Ychal
Figure 2: Overview of the DPC framework. The pipeline transforms selection into verification by identifying conflicting candidates (Champion vs. Challenger) and synthesizing an adversarial Minimal Distinguishing Database (MDD). By executing candidates alongside a generated Python reference on this fully observable environment, DPC utilizes the BS-F1 to deterministically identify the correct logic via cross-paradigm consistency.
3.2
Stage 1: Candidates Clustering & Pairing
The primary objective of this stage is to distill the noisy candidate set into two representative SQLs for verification. Since LLMs generate SQL queries probabilistically, the raw output often contains syntactic variations that are logically equivalent (e.g., varying alias names or keyword capitalization). Execution-Consistency Clustering. We adopt an execution-based strategy to simplify the candidate set. Let D denote the original database instance available for execution. We execute every candidate yk ∈ Y on D to obtain its execution result Ek . Candidates are then grouped into clusters C1 , C2 , ..., Cm such that all queries within a cluster produce identical execution results: ya , yb ∈ Ci ⇐⇒ E XEC(ya , D) = E XEC(yb , D) (5) Champion-Challenger Selection. To maximize the efficiency of the subsequent verification, we select the two most dominant SQLs for a pairwise duel (Li et al., 2025a; Sheng and Xu, 2025). We sort the clusters by size |Ci | in descending order to identify two candidates: the Champion (ychamp ), selected from the largest cluster Cmax , which represents the model’s “Majority Vote” choice and serves as a strong baseline; and the Challenger (ychal ), from the second-largest cluster, representing the most probable alternative hypothesis that conflicts with the consensus. 3.3
Stage 2: Adversarial Environment Synthesis
To resolve the conflict between the Champion (ychamp ) and the Challenger (ychal ), DPC constructs
a Minimal Distinguishing Database (MDD). This synthetic environment is engineered to satisfy the contextual feasibility and discriminative validity defined in Section 2.3. See Appendix B for a detailed running example. The S LICER Agent. The full database schema S often contains numerous tables and columns irrelevant to the current query, introducing noise and consuming context window space. The S LICER Agent iteratively distills a focused schema subgraph S ′ ⊆ S that contains only the tables and columns necessary for the candidate SQLs. To ensure the structural integrity of the sliced schema (e.g., ensuring no referenced columns or foreign keys are missing), we implement a DryRun Validation loop. In each iteration t, the agent predicts a schema candidate St′ . We then attempt to execute (or EXPLAIN) both ychamp and ychal on an empty database instance structured with St′ . If execution fails, the database error message et is fed back to the agent for self-correction: ′ St+1 = S LICER(Q, S, {ychamp , ychal }, et ) (6) The process terminates upon dry-run success or reaching the maximum iteration limit Tmax . The T ESTER Agent. Based on the validated slice S ′ , the T ESTER Agent constructs the synthetic data values VM DD to populate the MDD. Unlike random data generation, this process is adversarial: the goal is to generate specific records that expose the semantic discrepancy between the two candidates. We employ a Discriminative Feedback loop to guarantee effectiveness. In iteration t, the agent generates a data sample Vt . We execute both candi-
dates on Dt = (S ′ , Vt ) and compare their outputs. If the outputs are identical, it implies the data is insufficient to distinguish the logic (e.g., lacking a specific boundary case for a JOIN). The agent receives a penalty signal and regenerates the data: Vt+1 = T ESTER(Q, S ′ , ychamp , ychal , Iequal ) (7) where Iequal is the feedback indicator. The loop continues until the condition is met or the maximum iteration limit Tmax is reached: E XEC(ychamp , DM DD ) ̸= E XEC(ychal , DM DD ) (8)
3.4
Stage 3: Dual-Paradigm Execution
With the fully observable MDD established, DPC performs dual-paradigm execution. We execute the candidate SQLs (ychamp and ychal ) on the DM DD to obtain their deterministic outputs, denoted as Echamp and Echal respectively. Simultaneously, to verify these results against an independent reasoning path, this stage employs a S OLVER Agent to solve the user question Q using Python (Pandas). The Solver Agent. To establish a reliable verification standard, we leverage the model’s competency disparity between paradigms. Imperative Python code benefits from superior pre-training coverage (Twist et al., 2025; Lozhkov et al., 2024) and explicit step-by-step logic derivation (Gao et al., 2023; Yang et al., 2025), offering a higherconfidence reasoning path than declarative SQL. Consequently, we treat the Python execution result as a robust reference anchor (proxy ground truth) to validate the SQL candidates. To guarantee the executability of the generated logic, we implement a Runtime Self-Correction loop. The agent initially generates a Python script ρ0 based on the schema and data in DM DD . In each iteration t, we execute ρt within a sandboxed Python environment. If the execution raises an exception (e.g., KeyError, SyntaxError), the traceback message et is captured and fed back to the agent to guide the debugging process: ρt+1 = S OLVER(Q, DM DD , ρt , et )
(9)
This iterative refinement continues until the script executes successfully or the retry limit Tmax is reached, yielding the final execution result Epy . 3.5
Stage 4: Consistency Verification
The final stage is to determine which candidate— the Champion or the Challenger—aligns with the verified Python reference anchor Epy .
Algorithm 1 Bipartite Soft-F1 (BS-F1) Calculation Input: SQL Result Esql , Python Proxy Epy Output: Consistency Score s ∈ [0, 1] 1: Preprocessing: 2: Rsql ← N ORMALIZE(Esql ) 3: Rpy ← N ORMALIZE(Epy ) 4: Global Optimal Matching: 5: Let N = |Rsql |, M = |Rpy | 6: Initialize Cost Matrix C of size N × M 7: for i ∈ 0 . . . N − 1, j ∈ 0 . . . M − 1 do 8: mij ← column overlap ratio between Rsql [i] and Rpy [j] 9: Ci,j ← 1.0 − mij ▷ Minimize cost ≡ Maximize overlap 10: end for 11: ▷ Solve Assignment Problem via Hungarian Algorithm 12: A ← H UNGARIANA LGORITHM(C) 13: Initialize T P, F P, F N ← 0, 0, 0 14: Aggregation: 15: for assigned pair (i, j) in A do Accumulate T P, F P, F N based on match of 16: (Rsql [i], Rpy [j]) 17: end for 18: ▷ Handle Unmatched Rows (Penalties) 19: F P += (N − |A|) ▷ Unmatched SQL rows 20: F N += (M − |A|) ▷ Unmatched Python rows 21: Final Calculation: P P , R ← T PT+F 22: P ← T PT+F P N P ·R 23: return 2 · P +R
The Challenge of Heterogeneity. In standard Text-to-SQL evaluation, Execution Accuracy (EX) checks for strict identity between result sets. However, this rigid metric fails in our crossparadigm setting (Esql vs. Epy ) due to: (i) Type Incompatibility: SQL and Python/Pandas utilize different internal representations for semantically identical values (e.g., SQL DECIMAL vs. Python float, SQL NULL vs. Python NaN), causing standard EX to reject valid matches. (ii) Ordering Ambiguity: SQL query results without ORDER BY are unordered sets, whereas Pandas operations often produce implicitly ordered indices, leading to false negatives despite identical content. To bridge this gap without introducing human intervention, we propose the Bipartite Soft-F1 (BSF1), a robust deterministic metric grounded in combinatorial optimization. BS-F1 Calculation. As detailed in Algorithm 1, the BS-F1 metric addresses heterogeneity through a three-step pipeline: (i) Atomic Value Normalization: We apply type-agnostic mapping (Table 1) to unify diverse data types into a canonical format, resolving type incompatibilities. (ii) Global Optimal Matching: To handle ordering ambiguity, we model result alignment as a Maximum Weight Bipartite Matching problem. By constructing a
Table 1: Atomic Value Normalization used in DPC. Original Type/Value
Normalized Target
Decimal, float datetime, Timestamp None, NaN, "null" String w/ whitespace
float (rounded to 4 decimals) ISO String (YYYY-MM-DD) None (Null Object) Stripped String
cost matrix of semantic distances between SQL and Python rows, we utilize the Hungarian Algorithm (Kuhn, 2010; Munkres, 1957; Crouse, 2016) to find the global optimal assignment maximizing column-level overlap. (iii) Score Aggregation: Based on the optimal assignment, we aggregate True Positives (TP) while unmatched rows (extra or missing data) strictly penalize the score as FP or FN. The harmonic mean of the resulting Precision and Recall yields the final BS-F1 score. The candidate with the highest BS-F1 score is selected as the final prediction y ∗ : y∗ = argmax BS-F1(Ey , Epy ) (10) y∈{ychamp ,ychal }
By grounding verification in a metric that is robust to format heterogeneity yet sensitive to semantic logic, DPC effectively mitigates the intrinsic bias of LLM self-correction.
4
Experiments
We conduct comprehensive experiments to verify the effectiveness and efficiency of DPC, benchmarking it against both general-purpose base LLMs and state-of-the-art NL2SQL specialized solutions. 4.1
Experimental Settings
Datasets. We evaluate DPC on two benchmarks: BIRD (Li et al., 2023b), a large-scale dataset focusing on real-world database complexity and external knowledge, and Spider (Yu et al., 2018), the standard for evaluating SQL structural generalization. We use the official BIRD Mini-Dev split (500 queries) for efficient agentic reasoning evaluation and the standard Spider Test split (2,147 quries). Base LLMs. To assess adaptability, we employ three backbones: GPT-5 (OpenAI, 2025) as the frontier proprietary model, and DeepSeekV3.2 (Liu et al., 2025a) alongside Qwen2.5-Coder7B-Instruct (Hui et al., 2024) as state-of-the-art open-source representatives. Selection Baselines. We compare DPC against two categories of selection strategies: (1) Heuristic-based: Random selection; ExecutionGuided Selection (Li et al., 2023a), which selects
the first candidate that executes without errors; and Self-Consistency (SC) (Li et al., 2025b), which selects the SQL yielding the most frequent execution result. (2) LLM-based: Multiple-Choice Selection (MCS) (Lee et al., 2025), where an LLM is prompted to select the best candidate from the pool. Integration with Text-to-SQL Systems. To demonstrate plug-and-play capability, we integrate DPC into: (1) Prompting-based systems, including DAIL-SQL (Gao et al., 2024) and CHESS (Talaei et al., 2024); and (2) Fine-tuning-based models, including OmniSQL-7B (Li et al., 2025c) and XiYanCoder-7B (Liu et al., 2025d). Evaluation Metrics. We adopt a multidimensional protocol focusing on Effectiveness and Efficiency. We report Execution Accuracy (EX) (Li et al., 2023b) as the primary metric, alongside Upper Bound (Pass@N) (Pourreza et al., 2025) which represents the theoretical maximum performance within the candidate pool. For practical overhead, we measure the average Token Cost and Latency per query to assess the efficiency of the selection process. Implementation Details. For local inference, we deploy Qwen2.5-Coder-7B-Instruct, XiYanCoder-7B, and OmniSQL-7B using the vLLM library (Kwon et al., 2023) on a single NVIDIA A800 (80GB) GPU. Meanwhile, DeepSeek-V3.2 and GPT-5 are accessed via their official APIs. Throughout the DPC process, we set the sampling temperature to 0.7 and the self-correction limit to T = 3. 4.2
Main Results and Analysis
DPC Consistently Outperforms Baselines. As shown in Table 2, DPC achieves state-of-the-art performance across all base LLMs on both BIRD and Spider. On the challenging BIRD benchmark, DPC significantly boosts the performance of opensource models; for instance, it improves the EX of DeepSeek-V3.2 from 51.2% (Self-Consistency) to 53.4%. Even for the frontier GPT-5 model, which already exhibits high baseline accuracy, DPC manages to extract further gains, pushing its EX from 50.4% (Multiple-Choice Selection) to 51.2%. Furthermore, compared to heuristic-based methods like Execution-Guided Selection, DPC demonstrates a superior ability to identify semantically correct SQLs that might otherwise be overlooked by simple execution checks. We provide a detailed case
Table 2: Comparison of different SQL selection methods using multiple base LLMs on BIRD and Spider datasets. Upper Bound (Pass@N) indicates the theoretical maximum accuracy within the candidate pool (N = 5). The best results among selection methods are bolded, and the second-best results are underlined. BIRD EX (%)
Selection Methods
Spider EX (%)
Qwen
DeepSeek
GPT
Qwen
DeepSeek
GPT
Upper Bound (Pass@5)
57.6
58.8
56.4
84.8
77.6
76.6
Heuristic-based Random Selection (Baseline) Execution-Guided Selection (Li et al., 2023a) Self-Consistency (Li et al., 2025b)
37.0 44.4 46.4
48.2 49.8 51.2
46.0 50.2 49.4
72.3 75.0 76.5
70.9 72.6 71.6
71.9 72.4 72.2
LLM-based Multiple-Choice Selection (Lee et al., 2025) Dual-Paradigm Consistency (Ours)
43.6 47.6
51.0 53.4
50.4 51.2
74.8 77.5
72.8 73.2
72.9 73.3
Table 3: Performance enhancement of DPC integrated into SOTA Text-to-SQL systems on BIRD (N = 5).
Set Category
Execution Accuracy (%) Sim.
Mod.
Prompting-based DAIL-SQL (Gao et al., 2024) + SC 64.2 46.0 + DPC (Ours) 62.9 49.2 CHESS (Talaei et al., 2024) + SC 78.4 + DPC (Ours) 78.4
Cha.
32.3 35.3
All
48.6 50.4
63.2 64.8
55.9 56.9
66.2 67.2
Fine-tuning-based OmniSQL-7B (Li et al., 2025c) + SC 64.8 43.6 + DPC (Ours) 68.9 44.0
40.2 41.2
49.2 50.8
XiYanCoder-7B (Liu et al., 2025d) + SC 70.9 49.2 + DPC (Ours) 71.6 52.0
36.3 40.2
53.0 55.4
Execution Accuracy (%)
Method / Selection
Table 4: Robustness analysis against systematic bias on BIRD using Qwen2.5-Coder-7B-Instruct.
56 54 52 50 48 46 44 42
Versatility in System Integration. Table 3 demonstrates DPC integrated into existing SOTA Text-to-SQL systems. To ensure a rigorous and cost-effective evaluation, we employ Qwen2.5Coder-7B-Instruct as the uniform backbone for both the candidate generation in prompting-based systems and the logic execution in the DPC verification agent. We observe that DPC serves as a powerful “plug-and-play” enhancement layer across all tested systems. For prompting-based frameworks, it boosts DAIL-SQL and CHESS by +1.8% and +1.0% respectively. More importantly, for finetuning-based models like XiYanCoder-7B, DPC yields a significant improvement of +2.4%.
Overall (Full Set)
46.4
47.6
+1.2
Majority-Correct Majority-Incorrect
100.0 0.0
97.0 21.4
-3.0 +21.4
Self-Consistency (SC) DPC (Ours)
Increasing Performance Gap = 2.4% = 1.2%
3
5
7
9
Candidate Pool Size (N)
11
Figure 3: Impact of candidate pool size N on BIRD.
4.3 study comparing DPC with the Self-Consistency baseline in Appendix B.
SC (%) DPC (%) Gain
Robustness Analysis
Resilience to Systematic Bias. Traditional SC fails completely (0% accuracy) on the MajorityIncorrect Set, where model-internal biases lead to consistent but erroneous outputs (Table 4). In contrast, DPC recovers 21.4% of these hard cases by leveraging cross-paradigm verification. Although a minor regression (-3.0%) occurs in the MajorityCorrect set, the significant recovery in biased scenarios proves that DPC serves as a critical safety net when the majority consensus is flawed (see Appendix C for detailed error analysis). Scaling with Candidate Size. As shown in Figure 3, increasing the candidate pool size (N ∈ {3, . . . , 11}) widens DPC’s lead over SC from 1.2% to 2.4%. This trend validates DPC’s superior scalability and proficiency in filtering high-confidence noise within dense search spaces.
Table 5: Efficiency and cost-benefit analysis on BIRD using Qwen2.5-Coder-7B-Instruct. Token cost and latency are reported as average values per question. Selection Methods
EX (%)
Tokens (K)
Latency (s)
Heuristic-based Random Selection Execution-Guided Self-Consistency
37.0 44.4 46.4
-
∼0.0 ∼0.3 ∼0.8
LLM-based Multiple-Choice DPC (Ours)
43.6 47.6
∼4.2 ∼3.8
∼5.1 ∼4.2
Table 6: Ablation study of DPC components on the BIRD dataset. The baseline is the full DPC framework equipped with Qwen2.5-Coder-7B-Instruct.
4.4
Components
EX (%)
∆ EX (%)
DPC (Full)
47.6
–
w/o Tester Agent w/o Slicer Agent w/o Python Agent w/o BS-F1 Metric w/o Self-Correction
46.2 46.8 47.0 47.0 47.2
-1.4 -0.8 -0.6 -0.6 -0.4
Efficiency and Cost-Benefit Analysis
Efficiency vs. LLM Baselines. Compared to MCS, DPC achieves higher accuracy (+4.0% EX) with lower overhead, reducing token consumption (4.2k→3.8k) and latency (5.1s→4.2s), as shown in Table 5. This efficiency stems from the S LICER Agent, which proactively prunes irrelevant schema, avoiding the redundancy inherent in full-schema selection methods. Accuracy-Efficiency Trade-off. While heuristic baselines (e.g., SC) are near-instantaneous, they compromise precision. DPC outperforms SC by 1.2% on BIRD. In high-stakes domains (e.g., finance) where data integrity is paramount, the marginal latency increase (∼3.6s) is a justifiable investment for the substantial gain in reliability. 4.5
Ablation Study
Table 6 dissects component-wise contributions on the BIRD set. The T ESTER Agent proves most critical (-1.4%), validating the necessity of adversarial feedback for effective MDD construction, followed by the S LICER Agent (-0.8%) which mitigates schema noise. The remaining modules (S OLVER Agent, BS-F1, and Self-Correction) collectively contribute to the robustness, confirming the holistic efficacy of the dual-paradigm design.
5
Related Works
Text-to-SQL Generation and Selection. Recent prompting (Gao et al., 2024; Talaei et al., 2024; Shuai et al., 2025; Luo et al., 2025) and fine-tuning (Liu et al., 2025d; Li et al., 2025c) methods achieve strong Pass@K performance on benchmarks such as BIRD and Spider (Liu et al., 2025b; Zhu et al., 2025b; Liu et al., 2025c; Li et al., 2026). Yet a critical Generation-Selection Gap remains: models can generate correct candidates but often fail to identify them (Wang et al., 2025). Training-based verifiers (Liu et al., 2025d; Pourreza et al., 2025) improve selection but require costly annotations and generalize poorly across domains, while training-free methods such as Self-Consistency (Xie et al., 2025) and LLM-asa-Judge (Lee et al., 2025) are limited by systematic bias and Symbolic Blindness. In contrast, DPC casts selection as deterministic verification in an adversarially synthesized environment. Data Synthesis for Text-to-SQL. To address data scarcity (Hu et al., 2023; Zhu et al., 2024; Wu et al., 2024, 2025, 2026; Xie et al., 2026), recent work uses LLMs to synthesize Text-to-SQL corpora. Frameworks such as OmniSQL (Li et al., 2025c) and SQL-Factory (Li et al., 2025d) emphasize scale through multi-agent generation, while SQLForge (Guo et al., 2025), SQLord (Cheng et al., 2025), and SING-SQL (Caferoglu et al., 2025) improve reliability and domain adaptation via reversegeneration pipelines. However, these methods are designed for offline training augmentation and produce static datasets that lack the discriminative validity needed for inference-time verification. In contrast, DPC uses synthesis for online verification by dynamically constructing a query-specific Minimal Distinguishing Database (MDD).
6
Conclusion
In this paper, we introduce Dual-Paradigm Consistency (DPC), a training-free framework that bridges the generation-selection gap in Text-to-SQL. By synthesizing an adversarial Minimal Distinguishing Database (MDD) and leveraging the complementary reasoning of SQL and Python, DPC transforms selection from probabilistic guessing into deterministic verification. Experimental results on BIRD and Spider demonstrate that DPC significantly mitigates systematic bias and achieves stateof-the-art execution accuracy.
Limitations Despite significant gains, DPC incurs higher inference latency than direct generation due to its multiagent synthesis and execution pipeline. Future work will explore adaptive verification—triggering DPC only under high uncertainty—to optimize efficiency. Additionally, under complex implicit constraints, synthesized environments may structurally diverge from real distributions; thus, improving the semantic fidelity of adversarial synthesis remains a critical direction for future exploration.
Acknowledgments This paper was supported by the NSF of China (62402409); Youth S&T Talent Support Programme of Guangdong Provincial Association for Science and Technology (SKXRC2025461); the Young Talent Support Project of Guangzhou Association for Science and Technology (QT-2025-001); Guangzhou Basic and Applied Basic Research Foundation (2026A1515010269, 2025A04J3935, 2023A1515110545); and Guangzhou-HKUST(GZ) Joint Funding Program (2025A03J3714).
References Hasan Alp Caferoglu, Mehmet Serhat Çelik, and Özgür Ulusoy. 2025. SING-SQL: A synthetic data generation framework for in-domain text-to-sql translation. CoRR, abs/2509.25672. Song Cheng, Qiannan Cheng, Linbo Jin, Lei Yi, and Guannan Zhang. 2025. Sqlord: A robust enterprise text-to-sql solution via reverse data generation and workflow decomposition. In Companion Proceedings of the ACM on Web Conference 2025, WWW 2025, Sydney, NSW, Australia, 28 April 2025 - 2 May 2025, pages 919–923. ACM. David Frederic Crouse. 2016. On implementing 2d rectangular assignment algorithms. IEEE Trans. Aerosp. Electron. Syst., 52(4):1679–1696. Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. 2024. Text-to-sql empowered by large language models: A benchmark evaluation. Proc. VLDB Endow., 17(5):1132–1145. Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. PAL: program-aided language models. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 10764–10799. PMLR.
Yu Guo, Dong Jin, Shenghao Ye, Shuangwu Chen, Jianyang Jianyang, and Xiaobin Tan. 2025. Sqlforge: Synthesizing reliable and diverse data to enhance text-to-sql reasoning in llms. In Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 - August 1, 2025, pages 8441– 8452. Association for Computational Linguistics. Yiqun Hu, Yiyun Zhao, Jiarong Jiang, Wuwei Lan, Henghui Zhu, Anuj Chauhan, Alexander Hanbo Li, Lin Pan, Jun Wang, Chung-Wei Hang, Sheng Zhang, Jiang Guo, Mingwen Dong, Joseph Lilien, Patrick Ng, Zhiguo Wang, Vittorio Castelli, and Bing Xiang. 2023. Importance of synthesizing high-quality data for text-to-sql parsing. In Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023, pages 1327–1343. Association for Computational Linguistics. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, An Yang, Rui Men, Fei Huang, Xingzhang Ren, Xuancheng Ren, Jingren Zhou, and Junyang Lin. 2024. Qwen2.5-coder technical report. CoRR, abs/2409.12186. Harold W. Kuhn. 2010. The hungarian method for the assignment problem. In Michael Jünger, Thomas M. Liebling, Denis Naddef, George L. Nemhauser, William R. Pulleyblank, Gerhard Reinelt, Giovanni Rinaldi, and Laurence A. Wolsey, editors, 50 Years of Integer Programming 1958-2008 - From the Early Years to the State-of-the-Art, pages 29–47. Springer. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP 2023, Koblenz, Germany, October 23-26, 2023, pages 611– 626. ACM. Dongjun Lee, Choongwon Park, Jaehyuk Kim, and Heesoo Park. 2025. MCS-SQL: leveraging multiple prompts and multiple-choice selection for text-to-sql generation. In Proceedings of the 31st International Conference on Computational Linguistics, COLING 2025, Abu Dhabi, UAE, January 19-24, 2025, pages 337–353. Association for Computational Linguistics. Boyan Li, Chong Chen, Zhujun Xue, Yinan Mei, and Yuyu Luo. 2025a. Deepeye-sql: A softwareengineering-inspired text-to-sql framework. CoRR, abs/2510.17586. Boyan Li, Yuyu Luo, Chengliang Chai, Guoliang Li, and Nan Tang. 2024. The dawn of natural language to SQL: are we fully ready? [experiment, analysis & benchmark ]. Proc. VLDB Endow., 17(11):3318– 3331. Boyan Li, Yiran Peng, Yupeng Xie, Sirong Lu, Yizhang Zhu, Xing Mu, Xinyu Liu, and Yuyu Luo. 2026. Deepeye: A steerable self-driving data agent system. arXiv preprint arXiv:2603.28889.
Boyan Li, Jiayi Zhang, Ju Fan, Yanwei Xu, Chong Chen, Nan Tang, and Yuyu Luo. 2025b. Alpha-sql: Zeroshot text-to-sql using monte carlo tree search. In Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025. OpenReview.net. Haoyang Li, Shang Wu, Xiaokang Zhang, Xinmei Huang, Jing Zhang, Fuxin Jiang, Shuai Wang, Tieying Zhang, Jianjun Chen, Rui Shi, Hong Chen, and Cuiping Li. 2025c. Omnisql: Synthesizing highquality text-to-sql data at scale. Proc. VLDB Endow., 18(11):4695–4709. Haoyang Li, Jing Zhang, Cuiping Li, and Hong Chen. 2023a. RESDSQL: decoupling schema linking and skeleton parsing for text-to-sql. In Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14, 2023, pages 13067–13075. AAAI Press. Jiahui Li, Tongwang Wu, Yuren Mao, Yunjun Gao, Yajie Feng, and Huaizhong Liu. 2025d. Sql-factory: A multi-agent framework for high-quality and largescale SQL generation. CoRR, abs/2504.14837. Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, Xuanhe Zhou, Chenhao Ma, Guoliang Li, Kevin Chen-Chuan Chang, Fei Huang, Reynold Cheng, and Yongbin Li. 2023b. Can LLM already serve as A database interface? A big bench for largescale database grounded text-to-sqls. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023.
Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, Tianyang Liu, Max Tian, Denis Kocetkov, Arthur Zucker, Younes Belkada, Zijian Wang, Qian Liu, Dmitry Abulkhanov, Indraneil Paul, and 38 others. 2024. Starcoder 2 and the stack v2: The next generation. CoRR, abs/2402.19173. Tianqi Luo, Chuhan Huang, Leixian Shen, Boyan Li, Shuyu Shen, Wei Zeng, Nan Tang, and Yuyu Luo. 2025. nvbench 2.0: A benchmark for natural language to visualization under ambiguity. CoRR, abs/2503.12880. James Munkres. 1957. Algorithms for the assignment and transportation problems. Journal of the Society for Industrial and Applied Mathematics, 5(1):32–38. OpenAI. 2025. GPT-5 System Card. Technical report, OpenAI. Technical report. Mohammadreza Pourreza, Hailong Li, Ruoxi Sun, Yeounoh Chung, Shayan Talaei, Gaurav Tarlok Kakkar, Yu Gan, Amin Saberi, Fatma Ozcan, and Sercan Ö. Arik. 2025. CHASE-SQL: multi-path reasoning and preference optimized candidate selection in text-to-sql. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net. Lei Sheng and Shuai-Shuai Xu. 2025. CSC-SQL: corrective self-consistency in text-to-sql via reinforcement learning. CoRR, abs/2505.13271. Zhihao Shuai, Boyan Li, Siyu Yan, Yuyu Luo, and Weikai Yang. 2025. Deepvis: Bridging natural language and data visualization through step-wise reasoning. CoRR, abs/2508.01700.
Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, and 1 others. 2025a. Deepseek-v3. 2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556.
Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. 2024. CHESS: contextual harnessing for efficient SQL synthesis. CoRR, abs/2405.16755.
Xinyu Liu, Shuyu Shen, Boyan Li, Peixian Ma, Runzhi Jiang, Yuxin Zhang, Ju Fan, Guoliang Li, Nan Tang, and Yuyu Luo. 2025b. A survey of text-to-sql in the era of llms: Where are we, and where are we going? IEEE Trans. Knowl. Data Eng., 37(10):5735–5754.
Lukas Twist, Jie M. Zhang, Mark Harman, Don Syme, Joost Noppen, and Detlef D. Nauck. 2025. Llms love python: A study of llms’ bias for programming languages and libraries. CoRR, abs/2503.17181.
Xinyu Liu, Shuyu Shen, Boyan Li, Nan Tang, and Yuyu Luo. 2025c. Nl2sql-bugs: A benchmark for detecting semantic errors in NL2SQL translation. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, V.2, KDD 2025, Toronto ON, Canada, August 3-7, 2025, pages 5662–5673. ACM.
Pengfei Wang, Baolin Sun, Xuemei Dong, Yaxun Dai, Hongwei Yuan, Mengdie Chu, Yingqi Gao, Xiang Qi, Peng Zhang, and Ying Yan. 2025. Agentar-scale-sql: Advancing text-to-sql through orchestrated test-time scaling. CoRR, abs/2509.24403.
Yifu Liu, Yin Zhu, Yingqi Gao, Zhiling Luo, Xiaoxia Li, Xiaorong Shi, Yuntao Hong, Jinyang Gao, Yu Li, Bolin Ding, and Jingren Zhou. 2025d. Xiyan-sql: A novel multi-generator framework for text-to-sql. CoRR, abs/2507.04701.
Yifan Wu, Yiran Peng, Yiyu Chen, Jianhao Ruan, Zijie Zhuang, Cheng Yang, Jiayi Zhang, Man Chen, Yenchi Tseng, Zhaoyang Yu, Liang Chen, Yuyao Zhai, Bang Liu, Chenglin Wu, and Yuyu Luo. 2026. Autowebworld: Synthesizing infinite verifiable web environments via finite state machines. Preprint, arXiv:2602.14296.
Yifan Wu, Jingze Shi, Bingheng Wu, Jiayi Zhang, Xiaotian Lin, Nan Tang, and Yuyu Luo. 2025. Concise reasoning, big gains: Pruning long reasoning trace with difficulty-aware prompting. Preprint, arXiv:2505.19716. Yifan Wu, Lutao Yan, Leixian Shen, Yunhai Wang, Nan Tang, and Yuyu Luo. 2024. Chartinsights: Evaluating multimodal large language models for low-level chart question answering. Preprint, arXiv:2405.07001. Xiangjin Xie, Guangwei Xu, Lingyan Zhao, and Ruijie Guo. 2025. Opensearch-sql: Enhancing text-to-sql with dynamic few-shot and consistency alignment. Proc. ACM Manag. Data, 3(3):194:1–194:24. Yupeng Xie, Zhiyang Zhang, Yifan Wu, Sirong Lu, Jiayi Zhang, Zhaoyang Yu, Jinlin Wang, Sirui Hong, Bang Liu, Chenglin Wu, and Yuyu Luo. 2026. Visjudgebench: Aesthetics and quality assessment of visualizations. Preprint, arXiv:2510.22373. Dayu Yang, Tianyang Liu, Daoan Zhang, Antoine Simoulin, Xiaoyi Liu, Yuwei Cao, Zhaopu Teng, Xin Qian, Grey Yang, Jiebo Luo, and Julian J. McAuley. 2025. Code to think, think to code: A survey on code-enhanced reasoning and reasoning-driven code intelligence in llms. CoRR, abs/2502.19411. Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir R. Radev. 2018. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 3911–3921. Association for Computational Linguistics. Yizhang Zhu, Shiyin Du, Boyan Li, Yuyu Luo, and Nan Tang. 2024. Are large language models good statisticians? In NeurIPS. Yizhang Zhu, Runzhi Jiang, Boyan Li, Nan Tang, and Yuyu Luo. 2025a. Elliesql: Cost-efficient text-to-sql with complexity-aware routing. CoRR, abs/2503.22402. Yizhang Zhu, Liangwei Wang, Chenyu Yang, Xiaotian Lin, Boyan Li, Wei Zhou, Xinyu Liu, Zhangyang Peng, Tianqi Luo, Yu Li, Chengliang Chai, Chong Chen, Shimin Di, Ju Fan, Ji Sun, Nan Tang, Fugee Tsung, Jiannan Wang, Chenglin Wu, and 6 others. 2025b. A survey of data agents: Emerging paradigm or overstated hype? CoRR, abs/2510.23587.
A
Prompts for DPC Agents
To facilitate reproducibility and provide transparency into the DPC framework, we present the detailed prompt templates used for the three core agents: S LICER, T ESTER, and S OLVER. Each agent operates with a distinct system prompt that defines its specific role, reasoning instructions, and strict output formats (e.g., JSON or Python code), ensuring robust communication and error handling within the multi-agent pipeline.
Retry Prompt Template: S LICER Agent The previously identified Schema Slice has issues . Error details : --{error_message} --Please analyze the error ( it could be a format issue , missing tables / columns , or incorrect names ) and provide the corrected , complete Schema Slice . Ensure your response strictly follows the output format defined in the system prompt ( including < thinking > and < result > tags ) . The content inside < result > MUST be a valid , standard JSON string without any comments .
A.2 A.1
Prompts for S LICER Agent
The S LICER Agent performs schema pruning to extract a Minimal Schema Slice, identifying only the tables and columns strictly necessary for the subsequent verification pipeline. System Prompt: S LICER Agent You are a database expert . Your task is to identify the minimum , non - duplicate set of tables and columns required to execute the provided Candidate SQL Queries . You should : 1. Analyze the provided Candidate SQL Queries . 2. Identify all tables and columns from the Full Database Schema that are actually used in these SQLs ( SELECT , JOIN , WHERE , GROUP BY , etc .) . 3. Ensure you only use table and column names exactly as they appear in the Full Database Schema . 4. Provide a concise thinking process before the final result . Your output MUST follow this format : < thinking > [ Your step -by - step analysis here ] </ thinking > < result > { " relevant_schema ": [ { " table ": " table_name " , " columns ": [" column1 " , " column2 "] }, ... ] } </ result > IMPORTANT : The content inside < result > MUST be a valid , standard JSON string that can be parsed by ` json . loads () `. DO NOT include any comments ( like // or /* */) or extra text inside the JSON block .
User Prompt Template: S LICER Agent Full Database Schema : {full_schema} Candidate SQL Queries : {candidate_sqls} Please identify the relevant tables and columns used in the Candidate SQL Queries , following the output format defined in the system prompt .
Prompts for T ESTER Agent
The T ESTER Agent is responsible for generating adversarial data (Minimal Distinguishing Database) to expose logical discrepancies. System Prompt: T ESTER Agent You are a database QA engineer . Your task is to generate a minimal set of synthetic test data ( rows ) that will cause two different SQL queries to return DIFFERENT results . You should : 1. Analyze the Natural Language Question and the Candidate SQLs . 2. Identify the logical difference between SQL 1 and SQL 2 ( e . g . , a filter condition , a join type , or an aggregation ) . 3. Generate a sufficient but minimal set of data that specifically triggers this logical difference . 4. Ensure the data adheres to the Sliced Database Schema ( correct table / column names , types , and foreign key relationships ) . 5. Leverage metadata in the Schema : Use column descriptions , value descriptions , and example values provided in the schema to ensure the generated test data is realistic and follows the expected data distribution / format of the original database . 6. Provide a concise thinking process before the final result . Your output MUST follow this format : < thinking > [ Your analysis of why the SQLs differ and how your data will expose that ] </ thinking > < result > { " test_data ": { " table_name1 ": [ {" column1 ": value1 , " column2 ": value2 }, ... ], " table_name2 ": [...] } } </ result > IMPORTANT : The content inside < result > MUST be a valid , standard JSON string that can be parsed by ` json . loads () `. DO NOT include any comments ( like // or /* */) or extra text inside the JSON block .
User Prompt Template: T ESTER Agent Sliced Database Schema : {sliced_schema} Natural Language Question : {question} {evidence_str}
Candidate SQL 1 ( Champion ) : {sql_1} Candidate SQL 2 ( Challenger ) : {sql_2} Please generate the test data that makes SQL 1 and SQL 2 yield different results , following the output format defined in the system prompt .
Retry Prompt Template: T ESTER Agent The previously generated test data has issues or is INEFFECTIVE . Error details : --{error_message} --Please analyze the logic difference between SQL 1 and SQL 2 again , and provide a corrected set of test data that yields DIFFERENT results . Ensure your response strictly follows the output format defined in the system prompt ( including < thinking > and < result > tags ) . The content inside < result > MUST be a valid , standard JSON string without any comments .
A.3
### Available DataFrames ( Pandas Variables ) : {df_names} ### Natural Language Question : {question} {evidence_str} Please write the Pandas code to solve the question . Remember : - Only return columns explicitly asked for in the question . - The column order must match the question ' s order . - Ensure the final answer is stored in the ` result ` variable as a DataFrame .
Retry Prompt Template: S OLVER Agent The previously generated code has issues . Error details : --{error_message} --Please analyze the error and provide the corrected Python code . Ensure your response strictly follows the output format defined in the system prompt ( including < thinking > and < result > tags ) .
Prompts for S OLVER Agent
The S OLVER Agent functions as the reference anchor, executing Python logic on the synthesized MDD to determine the correct result. System Prompt: S OLVER Agent You are a Python data scientist expert in Pandas . Your task is to write a Python script to answer a natural language question based on provided database tables ( loaded as Pandas DataFrames ) . You should : 1. Analyze the schema and the provided test data . 2. Use the provided DataFrames ( already available in the namespace with their table names ) . 3. Write clean , efficient Pandas code to compute the answer . 4. IMPORTANT : Store the final result in a variable named ' result '. 5. The ' result ' MUST ALWAYS be a pandas DataFrame . Even for single values or lists , wrap them in a DataFrame . 6. COLUMN SELECTION : The final DataFrame MUST ONLY contain columns that are explicitly asked for in the question . Do not include extra or redundant columns . 7. COLUMN ORDERING : The order of columns in the final DataFrame MUST strictly follow the order mentioned in the natural language question . Your output MUST follow this format : < thinking > [ Your step -by - step logic for solving the problem using Pandas ] </ thinking > < result > [ Your Python code here ] </ result >
User Prompt Template: S OLVER Agent {test_data_with_types} ### Database Relationships ( PK / FK ) : {relationships}
B
Case Study: Identifying Missing Constraints
We present a detailed case study (QID: 1500, Source: BIRD Mini-Dev Split) to demonstrate how DPC leverages dual-paradigm execution to detect missing logical constraints (e.g., missing JOINs) that heuristic methods often overlook. B.1
The Conflict: Missing Table Constraint
Question: “Please list the product description of the products consumed in September, 2013.” The baseline SC method selects a Champion SQL that relies solely on the transactions_1k table, assuming the transaction date is sufficient. In contrast, DPC selects a Challenger SQL that introduces an additional JOIN with the yearmonth table, implying that a valid consumption record requires a corresponding monthly entry. Step 1: Logical Divergence Champion SQL (Incorrect):
Challenger SQL (Correct):
SELECT p . Description FROM transactions_1k t JOIN products p ON ... WHERE t . Date LIKE ' 201309% '
SELECT p . Description FROM products p JOIN transactions_1k t ... JOIN yearmonth ym ON ... WHERE ym . Date LIKE ' 201309% '
→ MISSING constraint: ‘yearmonth’ table.
→ ENFORCES constraint: ‘yearmonth’ table.
B.2
Adversarial Synthesis (MDD)
The T ESTER agent identifies that the Champion SQL is a superset of the Challenger. To expose the error, it constructs a “trap” scenario in the Minimal Distinguishing Database (MDD): 1. It creates a valid transaction for Customer 100 in September 2013 (Product: ‘LPG’). 2. Crucially, it omits Customer 100 from the yearmonth table for that period. This data distribution creates a decisive split: • Champion Result: Returns [‘LPG’] (Matches transaction date). • Challenger Result: Returns Empty (Fails INNER JOIN with yearmonth). Table 7: Synthesized MDD. The agent creates a “Ghost Transaction” (Row 1) that lacks a parent record in the YearMonth table (Bottom). Table: transactions_1k TxID
CustID
Date
1
100
20130915
Product LPG (ID:2)
Table: yearmonth CustID
C
Error Analysis
To deliver a thorough analysis, we present a systematic error analysis comparing our DPC framework against the Self-Consistency (SC) baseline, based on execution results from the BIRD Mini-Dev split. We first introduce our error taxonomy used throughout the analysis, then provide a statistical overview of error disagreements, followed by detailed case studies illustrating how DPC ’s dual-paradigm verification resolves or fails to resolve specific error types. C.1
Error Taxonomy
We adopt a two-level error taxonomy to categorize discrepancies between DPC and SC, adapted from prior studies on Text-to-SQL errors. Errors are classified into two primary categories:
Date
200 201309 300 201309 – Cust 100 is Missing! –
B.3
Selection Result: The Python execution returns an Empty DataFrame, perfectly aligning with the Challenger SQL’s result. DPC determines that the Champion SQL hallucinated a simplification (ignoring the yearmonth table) that contradicts the cross-paradigm consensus. Thus, DPC correctly identifies the missing constraint and selects the Challenger SQL.
Dual-Paradigm Verification
The S OLVER agent executes the verification logic using Pandas. As shown below, the Python agent explicitly merges the yearmonth dataframe, validating the structural necessity of this table. Step 3: Python Logic Alignment # Solver Agent 's Logic : # 1. Filter yearmonth table first . # Note : Python explicitly uses the yearmonth constraint . mask = yearmonth [ ' Date ' ]. str . startswith ( ' 201309 ') filtered_ym = yearmonth [ mask ] # 2. explicit MERGE acts as a filter ( Inner Join ) # This line kills the result because Customer 100 is NOT in filtered_ym merged_tx = pd . merge ( filtered_ym , transactions_1k , on = ' CustomerID ') result = pd . merge ( merged_tx , products , ...) [[ ' Description ' ]]
1. Semantic-Level Errors (A) Arise from misunderstandings of query semantics, such as incorrect filtering, aggregation, ordering, or result representation. These errors often stem from natural language ambiguity or misinterpretation of query intent. 2. Structural/Schema-Level Errors (B) Involve incorrect schema linking, join path selection, or table/column references. These errors reflect failures in mapping the question to the underlying database structure. A detailed breakdown of error subcategories, along with representative examples, is provided in Appendix Table 8. Based on our validation on the BIRD Mini-Dev split, among the cases where DPC succeeds while SC fails, three error subtypes dominate the distribution, as follows: Result Representation Error (A5) constitutes 41.7% of the observed failures, followed by Schema Linking Error (B3) at 25%, and Predicate Misalignment Filter (A1) accounting for 16.7%.
Table 8: Error Taxonomy for DPC vs. SC Comparison Category
Semantic (A)
Structural (B)
C.2 C.2.1
Code
Error Type
Description
A1
Predicate Misalignment
A2 A3 A4
Aggregation Misuse Grouping Error Ordering & Superlative
A5
Result Format Error
Incorrect filtering logic, value hallucination, misuse of operators (e.g., LIKE, IN, BETWEEN), or logical connectors (AND/OR). Misapplication of aggregate functions (COUNT, SUM, AVG, etc.). Missing or incorrect GROUP BY columns. Incorrect ORDER BY or LIMIT usage, especially in queries involving “most,” “least,” or ranking. Issues in the SELECT clause, including wrong data types, missing or extra columns, and formatting errors (e.g., rounding, casting).
B1 B2 B3
Wrong Join Path Join Condition Error Schema Linking Error
Incorrect table joins, including missing, extra, or incorrect tables. Incorrect ON conditions in join operations. Correct semantic understanding but failure in composing multitable logic, often due to incorrect column or table mapping.
Case Studies: Semantic-Level Errors Predicate Misalignment
(QID: 1464 from BIRD Mini-Dev Split) SC SQL (Incorrect):
DPC SQL (Correct):
SELECT m. first_name , m. last_name , i . amount FROM member AS m JOIN income AS i ON m . member_id = i . link_to_member WHERE i. date_received = ' 9/9/2019 ';
SELECT T2 . first_name , T2 . last_name , T1 . amount FROM income AS T1 JOIN member AS T2 ON T1 . link_to_member = T2 . member_id WHERE STRFTIME ( '%Y -% m -% d ' , T1 . date_received ) = ' 2019 -09 -09 ';
→ INCORRECT predicate value: uses colloquial date format not supported by the database schema.
→ CORRECT predicate grounding: enforces ISO date format consistent with schema constraints.
Predicate Misalignment (Value Hallucination). Case ID 1464 illustrates a representative predicatelevel error where SC relies on natural language priors to generate a colloquial date literal (’9/9/2019’) that does not conform to the database’s expected format, resulting in execution failure despite correct intent detection. Schema-Grounded Correction. DPC successfully aligns the predicate value with schema constraints by explicitly normalizing the date representation to the ISO format (’2019-09-09’), ensuring compatibility with the underlying database. Robust Literal Alignment. This case demonstrates that effective text-to-SQL generation requires grounding literal value predictions in database-specific formatting rules rather than surface-level textual patterns, highlighting DPC ’s advantage over SC in handling A1 Predicate Misalignment errors.
C.2.2
Result Representation Error
(QID: 227 from BIRD Mini-Dev Split) SC SQL (Incorrect):
DPC SQL (Correct):
SELECT CAST ( SUM ( CASE WHEN label = '+ ' THEN 1 ELSE 0 END ) AS REAL ) * 100 / COUNT (*) AS percent FROM molecule ;
SELECT ROUND ( CAST ( SUM ( CASE WHEN label = '+ ' THEN 1 ELSE 0 END ) AS REAL ) * 100 / COUNT ( molecule_id ) , 3 ) AS percent FROM molecule ;
→ INCORRECT result representation: missing ROUND(..., 3), does not satisfy the required precision.
→ CORRECT result representation: explicit casting and rounding ensure alignment with the question.
Result Representation Awareness. In this case, SC correctly captures the underlying computation for estimating the proportion of carcinogenic molecules, but fails to align with the question’s output specification by omitting explicit precision control (i.e., ROUND(..., 3)). Question-Aligned Refinement. DPC refines the SELECT clause by enforcing both numeric type casting and result formatting, ensuring that the returned value strictly adheres to the requirement of three decimal places. Consistency Selection. By favoring SQL candidates whose execution results match the questiondefined output format, DPC successfully corrects result-level mismatches on top of a logically correct foundation. This demonstrates its advantage over SC in handling A5 Result Representation Errors.
C.3 C.3.1
Case Studies: Structural/Schema-Level Errors Wrong Join Path
(QID: 1340 from BIRD Mini-Dev Split) SC SQL (Incorrect):
Challenger SQL (Correct):
SELECT SUM ( CASE WHEN SUBSTR ( T1 . event_date , 1, 4) = ' 2019 ' THEN T2 . spent ELSE 0 END ) SUM ( CASE WHEN SUBSTR ( T1 . event_date , 1, 4) = ' 2020 ' THEN T2 . spent ELSE 0 END ) AS difference FROM event AS T1 INNER JOIN budget AS T2 ON T1 . event_id = T2 . link_to_event INNER JOIN expense AS T3 ON T2 . budget_id = T3 . link_to_budget WHERE T1 . status = ' Student_Club '
SELECT SUM ( CASE WHEN SUBSTR ( T1 . event_date , 1 , 4) = ' 2019 ' THEN T2 . spent ELSE 0 END ) - SUM ( CASE WHEN SUBSTR ( T1 . event_date , 1 , 4) = ' 2020 ' THEN T2 . spent ELSE 0 END ) AS difference FROM event AS T1 JOIN budget AS T2 ON T1 . event_id = T2 . link_to_event
→ CORRECT Join Path: Understand the natural language query and identifying the appropriate tables.
→Incorrect Join Path: Utilized unnecessary extra tables for redundant filtering.
Adversarial Environment Synthesis DPC constructs a Minimal Distinguishing Database (MDD) tailored to expose logical discrepancies between candidate queries. The MDD is engineered to be small enough to fit within the model’s context window yet rich enough to differentiate between semantically distinct SQLs. This allows DPC to execute both candidates in a fully observable, controlled environment. Dual-Paradigm Verification Instead of relying solely on SQL execution or internal model priors, DPC leverages a Python/Pandas reference generated by the Solver agent. The Python solution, derived from the same MDD, serves as a high-confidence proxy ground truth. By comparing the execution results of the SQL candidates against this independent imperative reference, DPC can objectively determine which SQL aligns with the correct logic. Avoidance of Systematic Bias Unlike SC, which tends to reinforce the model’s internal biases, DPC ’s cross-paradigm verification breaks the cycle of self-confirmation. By grounding the decision in explicit execution evidence from an adversarial environment and an independent reasoning path, DPC avoids falling into consensus traps on erroneous
logic. In this case, DPC correctly selects the simpler and accurate query that joins only event and budget tables, avoiding the unnecessary and erroneous inclusion of the expense table and the extraneous status filter. C.3.2
Schema Linking Error
(QID: 1134 from BIRD Mini-Dev Split) SC SQL (Incorrect):
Challenger SQL (Correct):
SELECT ( SELECT jumping FROM Player_Attributes
SELECT ( SELECT jumping FROM Player_Attributes
WHERE player_api_id = 6) ( SELECT jumping FROM Player_Attributes
WHERE id = 6) ( SELECT jumping FROM Player_Attributes
WHERE player_api_id = 23) AS difference ;
→ INCORRECT column mapping: uses player_api_id instead of id.
WHERE id = 23) AS difference ;
→ CORRECT column mapping: uses id as player identifier.
Constructing Minimum Differentiating Database (MDD): The S LICER and T ESTER of DPC collaborate to synthesize a minimal execution environment in which the two candidate column mappings (player_api_id vs. id) produce different results, thereby exposing the underlying semantic discrepancy. Dual Paradigm Verification: DPC reimplements the SQL logic using Python scripts on the MDD, producing a high-confidence reference output that serves as an execution-level oracle. Consistency Alignment: By comparing SQL execution results against the Python reference using the BS-F1 metric, DPC selects the semantically consistent and correct SQL formulation (i.e., using id instead of player_api_id). Therefore, DPC successfully overcomes the symbolic blind spot and system bias inherent in SC, achieving precise alignment with user intent. C.4
Discussion: How DPC Addresses These Errors
DPC ’s dual-paradigm design systematically targets two fundamental error classes prevalent in SC and other training-free methods: 1. Schema Mapping Errors (Category B3) LLMs suffer from partial observability—they
only see a schema snippet during inference. DPC ’s S LICER agent extracts a relevant schema subgraph and validates it via dry-run execution on an empty database. If joins or column references fail, the error feedback iteratively corrects the schema linking before any data synthesis. This acts as schema-level unit testing, eliminating B3 errors before execution. 2. Result Representation Errors (Category A5) LLMs struggle to infer exact output format (columns, types, rounding). DPC ’s S OLVER generates a parallel Python/Pandas script on the same micro-database, providing a high-confidence reference output Epy . The BS-F1 metric then compares SQL results against Epy , penalizing formatting mismatches, extra/missing columns, or incorrect data types. This cross-paradigm consistency check enforces output alignment. SC’s failures often stem from systematic bias and lack of execution grounding. DPC mitigates these by enforcing schema validation and crossparadigm consistency, exposing errors through adversarial database construction and deterministic matching.