SQLoom: Set-Contrastive Learning of Syntactic, Semantic, and Logical Dependencies for Text-to-SQL

arXiv:2611.03298v1 [cs.DB] 2 Nov 2026

SQLoom: Set-Contrastive Learning of Syntactic, Semantic, and Logical Dependencies for Text-to-SQL Hao Fu

Degang Sun

Wan Wei

haofu@vt.edu Virginia Tech

dgsun@cnic.cn Computer Network Information Center, Chinese Academy of Sciences

wanwei@cnic.cn Computer Network Information Center, Chinese Academy of Sciences

Jinxia Wei

Chun Long

Peng Gao

weijinxia@cnic.cn Computer Network Information Center, Chinese Academy of Sciences

longchun@cnic.cn Computer Network Information Center, Chinese Academy of Sciences

penggao@vt.edu Virginia Tech

Abstract Text-to-SQL translates natural language questions into SQL queries, enabling database access without writing SQL. Fine-tuning openweight large language models (LLMs) is cost-effective, but existing methods treat SQL as ordinary text and optimize token-level likelihood without explicitly modeling its syntactic, semantic, and logical requirements. Thus, a single incorrect alias or comparison operator can invalidate a query while contributing little to training loss. We propose SQLoom, a constraint-aware fine-tuning framework with supervision at three levels: syntactic validity under SQL grammar and database schema, semantic alignment between query decisions and the question, and logical consistency across dependent clause decisions. SQLoom extends supervised fine-tuning with a unified set-contrastive objective favoring sets of correct outputs over minimally different incorrect ones at each level. LLM agents propose this supervision, which a SQL parser and execution on counterexample databases verify. We also release Spider-Refined, a training dataset correcting 378 flawed Spider annotations and enriching database schemas. Experiments on Spider, BIRD, and three perturbed Spider variants show that SQLoom improves execution accuracy for Qwen3-1.7B from 62.9% zero-shot to 88.7% on Spider development, reaching 89.2% on Spider test and 73.4% on BIRD development. It outperforms fine-tuning, reinforcement learning, and agent-based baselines with up to 70 billion parameters. On question perturbations, SQLoom loses at most 0.6 accuracy points versus up to 12.1 for agent-based methods. It generates SQL in a single pass at 0.42 s per query, 23× faster than a GPT-4-based agent method. These results demonstrate the effectiveness of set-contrastive supervision over token-level training.

CCS Concepts • Information systems → Question answering; Structured Query Language.

Keywords Text-to-SQL, Large Language Models, Constrained Fine-Tuning, Preference Optimization, Natural Language Interfaces to Databases

1

Introduction

Text-to-SQL translates natural language questions into executable SQL queries [15, 38], enabling users without SQL expertise to query

relational databases through natural language interfaces. In recent years, large language models (LLMs) have been widely adopted in text-to-SQL research, owing to their strong natural language understanding [6, 7]. However, general-purpose LLMs are not specifically trained for this task, and they often generate queries that reference non-existent columns, misinterpret the question, or combine inconsistent clauses. To mitigate these errors, prompt-based agent methods equip LLMs with extensive error-correction mechanisms, such as self-correction [1, 20], multi-agent refinement [2, 31, 39], and candidate selection [19]. These mechanisms require multiple calls to proprietary LLMs per question, which increases latency and cost [41] and hinders deployment in Web services with high query volumes. Efficient fine-tuning methods that equip general-purpose LLMs with SQL-specific competence are therefore needed. Existing fine-tuning methods for text-to-SQL fall into three main categories [7]. Data-centric methods enlarge the training corpus through SQL-centric pre-training [13] and data synthesis [11, 36]. Each sample still teaches a single reference string, so the model has to infer the rules of SQL from millions of samples, e.g., 2.5M for OmniSQL [11]. Task-decomposition methods add auxiliary tasks or stages such as schema linking and skeleton parsing [12, 21, 23, 28]. They help the model select tables and columns, but the query itself is still learned by imitating one reference token by token. Feedbackdriven methods optimize execution-based preferences or rewards with DPO or reinforcement learning [17, 22, 37, 40]. A reward judges the whole query, so it cannot tell which part caused an error, and it still rewards a wrong query when its error does not affect the result on the given database. Most of these methods treat SQL as ordinary text, and even structure-aware rewards [9, 18] score a complete query instead of contrasting the tokens, decisions, and dependent clauses where errors occur. Without supervision from this structure, the current fine-tuning paradigm faces three challenges, illustrated in Figure 1. C1 Syntactic validity. A query must use only defined tables, columns, and aliases. The first query in Figure 1 selects T3.name, but no table is named T3. PICARD [26] filters such tokens with a parser during decoding, but the model still prefers them and needs the parser at deployment. C2 Semantic alignment. Each phrase of the question must map to the correct SQL fragment. “Different courses” requires COUNT(DISTINCT course_id) and “at least two” requires >= 2, whereas the second query writes COUNT(*) > 2. Schema-linking objectives [23, 28] teach the model which tables and columns to

WWW ’27, May 10–14, 2027, Dublin, Ireland

Hao Fu, Degang Sun, Wan Wei, Jinxia Wei, Chun Long, and Peng Gao

learns the grammar in training and needs no parser to correct its output at inference. • We design a semantic constraint that teaches the model to map each question phrase to the correct SQL fragment rather than a confusable one. It grounds question phrases in the AST nodes that realize them and contrasts each node with confusions mined from the model’s own errors. • We design a logical constraint that keeps dependent clauses consistent when one of them changes. It derives these dependencies from a dependency graph over clauses and contrasts consistent and inconsistent queries on counterexample databases where their results differ. Figure 1: Three challenges of SQL generation on the running example. Each row shows a query with one type of error, and highlighted spans mark the wrong decisions.

use, but not which aggregates and operators. C3 Logical consistency. Dependent clauses must change together. The third query joins section, which yields one row per course section, but keeps COUNT(*), so a course is counted several times. Each clause looks reasonable alone, and the query often returns the correct result on the test database, so execution rewards miss it. To tackle these challenges, we propose SQLoom, a constraintaware fine-tuning framework. Our key insight is that SQL generation is better supervised by contrasting sets of outputs than by imitating a single reference. Owing to the grammar and the schema, a question often has several correct queries, and at many positions the correct output is a class of tokens, e.g., the columns of the joined tables after SELECT, rather than one token. Token-level SFT rewards one member and pushes down the others, including correct ones. SQLoom therefore augments SFT with a set-contrastive objective that favors a set of correct outputs over a set of wrong outputs differing from them at a single place, at three granularities. Syntactic constraint for C1. At each position, the correct set contains the tokens allowed by the grammar and the schema, and the wrong set contains the rest. Semantic constraint for C2. For each question phrase, the correct set contains its reference SQL fragment, e.g., >= 2 for “at least two”, and the wrong set contains fragments the model confuses with it, e.g., > 2. Logical constraint for C3. For each pair of dependent clauses, we edit the question so that one clause must change, e.g., counting only courses with a lab section. The correct set contains the query that also updates the dependent clause, e.g., the count, and the wrong set contains the query that changes only the edited clause. The three constraints thus address the three challenges by teaching which tokens are legal, which fragments match the question, and which clauses must change together. Our technical contributions are summarized below. • We propose SQLoom, which replaces token-by-token imitation of one reference query with set-contrastive supervision labeled by a parser and query execution, instantiated as three constraints on syntax, semantics, and logic. • We design a syntactic constraint that uses the parse state as a training signal. An incremental parser with schema scope derives the legal tokens at each position, and a trie over schema identifiers aligns grammar terminals with subword tokens, so the model

Constructing these constraints requires correct and wrong sets for every training example, which are absent from the original annotations. The syntactic sets are derived directly by the parser, whereas the semantic and logical sets are built by a constraint synthesizer with four LLM agents. For the semantic constraint, an error-mining agent finds the model’s wrong decisions and an alignment agent maps question phrases to SQL fragments with their confusions. For the logical constraint, an editing agent rewrites the question along a dependency edge with the two queries, and a verification agent checks the edited query’s intent. Each output must pass the parser and execution checks. While synthesizing constraints in our experiments, the error-mining agent also flagged 378 Spider annotations that mislabel correct predictions or teach incorrect queries. After manual review, we correct these annotations and release Spider-Refined, a training dataset that extends Spider with the corrected annotations, schema descriptions generated by the synthesizer, and the verified constraint samples. Spider-Refined serves only as training data, and all evaluations use the official labels. Experiments lead to four findings. • The gains come from the objective. With the same training data, replacing the constraints with plain SFT lowers EX by 8.8 points on Spider Test and 10.3 points on BIRD Dev. • A 1.7B model outperforms much larger systems. SQLoom reaches 88.7% EX on Spider Dev (62.9% zero-shot), 89.2% on Spider Test, and 73.4% on BIRD Dev, above all evaluated baselines and zero-shot Qwen3-235B and GPT-4. • SQLoom is robust to perturbed questions. It loses at most 0.6 points on the three Spider variants, whereas agent-based methods lose 8.8–12.1 points on Spider-DK. • SQLoom is efficient. It generates SQL in one pass (0.42 s per query), 23× faster than CogSQL with GPT-4 in our setup. Overall, these results show that with constraint-based supervision, even a small LLM can generate SQL accurately and robustly.

2 Background 2.1 Problem Formulation Given a natural language question 𝑞 and a database D with schema S (tables, columns, primary and foreign keys), text-to-SQL generates a query 𝑦 = (𝑦1, . . . , 𝑦𝑇 ) whose execution result on D answers 𝑞. Let 𝑥 = (𝑞, S) denote the model input. Fine-tuning methods train an LLM 𝑝𝜃 by minimizing the negative log-likelihood of a reference

SQLoom : Set-Contrastive Learning of Syntactic, Semantic, and Logical Dependencies for Text-to-SQL

Table 1: Supervision signals of representative fine-tuning methods. Syn., Sem., and Log. denote explicit supervision of syntactic validity, semantic alignment beyond tables and columns, and cross-clause consistency.

WWW ’27, May 10–14, 2027, Dublin, Ireland

Wrong column

Wrong ordering or LIMIT

Other model error

(a) Errors per difficulty level Easy (470)

Method CodeS [13], OmniSQL [11] ROUTE [23], JOLT-SQL [28] ExCoT [40] SQL-R1 [17], Arctic [37] Ma et al. [18] EXPO-SQL [9] SQLoom

Training signal Syn. Sem. Log. token likelihood ✗ ✗ ✗ + schema linking ✗ ✗ ✗ query-level preference ✗ ✗ ✗ query-level reward ✗ ✗ ✗ AST-matching reward ✓ ✗ ✗ clause-level reward ✓ ✗ ✗ unified set-contrastive objective ✓ ✓ ✓

(b) Share by error type (%)

Wrong column

7

60.7 20.4

Semantic Medium (857)

61 Wrong table

Hard (463)

17.5

58 Ordering/LIMIT

Extra (357)

85

0

25

50

75

100

0.9 0.5

Other

0

20

40

60

Number of errors

Figure 2: Model errors of LoRA fine-tuning on Spider Dev (a) per difficulty level, with question counts, and (b) by type.

query 𝑦 ∗ , LSFT = −

𝑇 ∑︁

∗
log 𝑝𝜃 𝑦𝑡∗ | 𝑥, 𝑦 <𝑡 .

(1)

𝑡 =1

2.2

Semantic error

Wrong table

Related Work

Prompt-based Agent Methods. Early neural parsers encode the schema with task-specific architectures [14, 32, 43]. LLM-based methods instead improve a general model at inference, through example selection, decomposition, and schema pruning [6, 10, 20, 24, 30, 35], or through agents that plan, refine, and select among candidates [1, 2, 19, 27, 31, 39]. Other work checks SQL with tools rather than with a model. Constrained decoding runs an incremental SQL parser during generation and discards tokens that violate the grammar or the schema [26], and test-suite evaluation executes a predicted query on several generated databases to expose queries that are correct only by coincidence on the original one [42]. In essence, these methods repair or detect errors after generation instead of preventing them. The repair is judged by the same LLM or by execution failures, so a query that executes but answers another question often passes, and each question pays for several LLM calls [41]. The tools also act outside training, so the model does not learn from them and the parser must stay in the serving loop. Data-centric Fine-tuning. This direction scales the training corpus through SQL-centric pre-training and data synthesis [11, 13, 29, 36]. Its limitation lies in the form of supervision rather than its amount (Table 1). Each sample still provides one reference string, so the model must infer grammar rules, question-to-SQL bindings, and clause dependencies statistically, and equivalent queries for one question even provide conflicting targets. Task-decomposition Fine-tuning. This direction splits generation into schema linking, skeleton parsing, and SQL stages, or adds them as auxiliary tasks [12, 21, 23, 28]. It strengthens the recognition of the input, namely which tables and columns to use (Table 1), but leaves output-side decisions, such as aggregates, operators, and their consistency across clauses, to token-level imitation. Feedback-driven Fine-tuning. This direction optimizes executionbased preferences or rewards with DPO or reinforcement learning [16, 17, 22, 25, 37, 40], and recent work refines the reward with graph, AST, or clause-level matching [9, 18, 33]. The core problem is the signal itself. An outcome-level reward spreads the same credit over all tokens, and single-database execution wrongly rewards inconsistent queries with correct results (Table 1).

2.3

Motivation

Problem measurement. To measure which errors remain under standard fine-tuning, we fine-tune Qwen3-1.7B on the Spider training set with LoRA and token-level SFT, and classify its 211 model errors on the Spider development set (Figure 2; categories in Appendix D). Wrong columns dominate from medium to extra-hard questions, followed by semantic errors. These queries are valid but bind a question element to a wrong decision, which typically spans only one or a few tokens. These model errors trace back to Eq. 1, which raises the likelihood of the reference without explicitly penalizing incorrect outputs and ignores three properties of SQL. 1) Validity is decidable. Given the grammar and the schema, an incremental parser determines which tokens are legal at each position, yet SFT does not explicitly lower the probability of illegal ones. 2) Errors are local but decisive. A wrong decision, such as COUNT(*) instead of COUNT(DISTINCT ·), occupies a small share of the SFT loss but changes the answer; conversely, since SQL labels are not unique, an equivalent rewriting of 𝑦 ∗ is penalized as much as a wrong query. 3) Decisions are dependent. A one-to-many join may require DISTINCT, and GROUP BY must follow the target entity, but a query that violates such a dependency can still return the correct result on the given database. Requirements. A fine-tuning objective for SQL should therefore meet three requirements. First, it contrasts the reference with outputs certified to be wrong by a parser or by execution beyond the given database. Second, it confines the contrast to the structural unit in which the outputs differ. Third, it applies at the level of tokens, decisions, and dependent decisions.

3 Methodology 3.1 Overview Token-level SFT maximizes the likelihood of the reference without penalizing incorrect outputs, so wrong tokens, wrong decisions, and inconsistent clauses receive little training signal. SQLoom supervises these errors explicitly while keeping single-pass inference. As shown in Figure 3, the constraint synthesizer (Section 4.1) builds sets of correct outputs and minimally different wrong outputs at the syntactic, semantic, and logical levels. A set-contrastive objective favors the correct set over the wrong set at each of the three levels. A three-stage procedure trains the model with this objective.

WWW ’27, May 10–14, 2027, Dublin, Ireland

Hao Fu, Degang Sun, Wan Wei, Jinxia Wei, Chun Long, and Peng Gao

Figure 3: Overview of SQLoom. The constraint synthesizer builds positive and negative sets, a set-contrastive loss favors positives (colored) over minimally different negatives (red) at three levels, and training proceeds in three stages. Three-level correctness hierarchy. The three challenges occur at different granularities, so one loss on the whole query cannot tell where an error lies. We therefore define one unit per challenge, namely a token for C1, a decision for C2, and a pair of dependent decisions for C3. A decision is the SQL fragment that realizes one element of the question, e.g., >= 2 for “at least two”, and corresponds to a node of the abstract syntax tree (AST) of the query. A logical unit links two decisions in different clauses, e.g., a join and the count that depends on it. Oracle-certified constraints. Since one question can have several correct queries, comparing an output with the reference cannot tell whether the output is wrong or merely written differently. For each unit 𝑢 we therefore build a constraint 𝑐 = (P𝑐 , N𝑐 , 𝑀𝑐 ), where P𝑐 is a set of correct (admissible) outputs, N𝑐 is a set of wrong (inadmissible) outputs that differ from them only in 𝑢, and 𝑀𝑐 (𝑦) marks the token positions of 𝑦 that belong to 𝑢. Wrong outputs are certified by an oracle, i.e., a program with a definite answer rather than a learned model, whereas correct outputs are references or pass additional checks. The oracle is a schema-aware SQL parser at the syntactic level. At the other two levels it executes the queries on the original database D and on counterexample databases {D 𝑗′ }, i.e., small generated databases on which similar queries return different results. Appendix C details both oracles. Set-contrastive objective. We compare outputs only at the poÍ 𝑝 (𝑦 |𝑥,𝑦 ) sitions of their unit. Let 𝑟𝜃 (𝑦, 𝑀) = 𝑡 ∈𝑀 log 𝑝 𝜃 (𝑦𝑡𝑡 |𝑥,𝑦<𝑡 be how <𝑡 ) ref much more likely the model makes 𝑦 at positions 𝑀 than a frozen reference model 𝑝 ref , here the model after warm-up. Each constraint

Table 2: Instantiations of the set-contrastive constraint (Eq. 2) at the three levels. The last row lists each level’s negatives.

Unit 𝑢

P𝑐 N𝑐 𝑀𝑐 Oracle Negatives

Syntactic Semantic Logical decision position 𝑡 AST node 𝑑𝑖 aligned to dependent decision pair element 𝑒𝑖 (edge of 𝐺 ) valid next tokens reference decision propagated 𝑦 ′ invalid next verified confusions partially propagated 𝑦˜ ′ tokens position 𝑡 tokens of 𝑑𝑖 tokens of affected clauses ′ parser execution on D, D execution on D, D ′ invalid valid, misaligned valid, aligned, inconsistent

is enforced by Í ℓ (𝑐) = − log Í

𝑦 ∈ P𝑐 exp

𝛽 𝑟𝜃 (𝑦, 𝑀𝑐 (𝑦))

𝑦 ∈ P𝑐 ∪N𝑐 exp

𝛽 𝑟𝜃 (𝑦, 𝑀𝑐 (𝑦))

,

(2)

which raises the total probability of the correct set over the wrong set at the positions of the unit, while the rest of the query keeps the plain SFT signal. The temperature 𝛽 controls how sharply the two sets are separated. The full objective is ∑︁ ∑︁ L = LSFT + 𝜆𝑘 L𝑘 , L𝑘 = ℓ (𝑐), (3) 𝑘 ∈ {syn,sem,log}

𝑐 ∈ C𝑘

where C𝑘 is the set of constraints at level 𝑘, and LSFT covers the references and all admissible outputs. Table 2 lists the instantiations. Semantic negatives are valid queries, and logical negatives also keep each decision consistent with the question (last row of Table 2), so each level covers errors that the previous levels miss.

SQLoom : Set-Contrastive Learning of Syntactic, Semantic, and Logical Dependencies for Text-to-SQL

WWW ’27, May 10–14, 2027, Dublin, Ireland

The loss never penalizes equivalent outputs and updates only the constrained unit (Appendix A).

3.2

Syntactic Constraint

The syntactic constraint teaches the model which tokens may come next (C1). Legality depends on the grammar and the schema, e.g., an alias defined inside a subquery is not visible outside it, and SQL keywords and names do not align with the LLM’s subword tokens. We feed 𝑦 ∗ token by token into an incremental parser, i.e., a SQL parser that can analyze an unfinished query, extended with ∗ , the parser returns a state 𝑠 the schema. After each prefix 𝑦 <𝑡 𝑡 that records what may come next and which tables, aliases, and columns are visible. The legal set Vvalid (𝑠𝑡 ) contains each token 𝑣 after which the query can still be completed into a valid one, e.g., only table names of S after FROM. Since a name may span several subword tokens, legality is checked on the decoded text with a prefix tree (trie) of keywords and schema names, so the partial name stu of student counts as legal. The constraint is applied at decision positions T𝑑 , i.e., the first token of each keyword, name, operator, or parenthesis, and the legal sets are cached per database. The syntactic instance of Eq. 2 is ∑︁ ∑︁ ∗
Lsyn = − log 𝑝𝜃 𝑣 | 𝑥, 𝑦 <𝑡 , (4) 𝑡 ∈ T𝑑

𝑣 ∈ Vvalid (𝑠𝑡 )

which moves probability from illegal tokens to the whole legal set rather than to the reference token alone, so legal alternatives are not pushed down. The model thus learns validity during training and needs no parser at inference.

3.3

Semantic Constraint

The semantic constraint binds each element of the question to the SQL decision that realizes it (C2). Datasets do not annotate which phrase maps to which part of the query, and a seemingly wrong alternative may be an equivalent rewrite. For each training example, we extract alignments A = {(𝑒𝑖 , 𝑑𝑖 )}, where 𝑒𝑖 is an element of 𝑞, e.g., “at least two”, and 𝑑𝑖 is the AST node of 𝑦 ∗ that realizes it, e.g., >= 2 (Figure 9). Elements fall into six types, namely entity, attribute, value, aggregation, comparison/quantifier, and ordering. An alignment is kept only if replacing 𝑑𝑖 by another option of the same type changes the execution result, which confirms that 𝑑𝑖 depends on 𝑒𝑖 . As an auxiliary task, the model writes the alignment list before the SQL, supervised by LSFT . For each alignment, P𝑐 = {𝑦 ∗ }, and N𝑐 contains confusions 𝑦˜ that replace 𝑑𝑖 with a similar but wrong option, e.g., a similar column, COUNT(*) for COUNT(DISTINCT ·), or > for >=. Candidates are ranked by how often the current model makes them, so training targets its actual mistakes. Only valid confusions that return a different result from 𝑦 ∗ are kept, so equivalent rewrites are not penalized by design. With one confusion per unit, Eq. 2 reduces to the pairwise loss of direct preference optimization (DPO) [25] restricted to the tokens of 𝑑𝑖 , which corrects the wrong decision without changing the rest of the query.

3.4

Logical Constraint

The logical constraint teaches the model to update dependent clauses together (C3). A single training pair rarely shows how

Figure 4: Component dependency graph 𝐺 on an example query. Each labeled edge marks a component that must be checked when the source component changes.

one clause should change when another changes, and an inconsistent query often returns the correct result on D, so executing it there does not reveal the error. We split a query into components (FROM/JOIN, WHERE, GROUP BY, aggregation, HAVING, SELECT, ORDER BY/LIMIT, and sub-queries) and build a graph 𝐺 in which an edge states that a certain change to one component requires checking another (Figure 4; all edges in Table 4). For example, adding a one-to-many join requires checking whether the count still counts each entity once. Each edge is a unit. Edges mark what may change, and execution decides how. Starting from (𝑞, 𝑦 ∗ ), we change one aligned element of the question and generate an edited question 𝑞 ′ , a propagated query 𝑦 ′ in which the edited component and the components that depend on it are updated, and a partially propagated query 𝑦˜ ′ in which only the edited component changes. Then P𝑐 = {𝑦 ′ } and N𝑐 = {𝑦˜ ′ } for input (𝑞 ′, S). For example, restricting the running example to courses with a lab section adds a join with section; 𝑦 ′ counts DISTINCT T2.course_id, whereas 𝑦˜ ′ keeps COUNT(*), which counts a course per lab section. The pair isolates the dependency and is kept only if the two queries differ on a counterexample database, 𝑦 ′ runs on all databases, and the verification agent judges 𝑦 ′ consistent with 𝑞 ′ . The mask 𝑀𝑐 covers the components that differ between 𝑦 ′ and ′ 𝑦˜ . Since both queries contain the edit, the loss focuses on the components that must change because of it. Eq. 2 again reduces to a DPO-style loss on these positions, and 𝑦 ′ is also added to LSFT . Appendices B, C, and E show examples of all levels.

3.5

Training Procedure

The training procedure combines the three constraints in one model. Preference losses are unstable before the model can write reasonable SQL, and later fine-tuning stages tend to overwrite earlier skills. SQLoom therefore trains in three stages (Algorithm 1). Stage 1 is a short SFT warm-up whose result also serves as 𝑝 ref . Stage 2 trains with Eq. 3. Stage 3 prepends to each input the 𝑘 most similar verified question–query pairs of the same database, since Web applications answer many questions over one database. After each

WWW ’27, May 10–14, 2027, Dublin, Ireland

Algorithm 1: Three-stage training procedure of SQLoom. Input: training set T, pre-trained LLM 𝑀𝑝 𝑀 ← Correct(SFT of 𝑀𝑝 on T for one epoch) ; // Stage 1 𝑝 ref ← 𝑀; build Csyn , Csem , Clog with the constraint synthesizer; foreach mini-batch B of T ∪ Csyn ∪ Csem ∪ Clog do update 𝑀 by minimizing L (Eq. 3) on B ; // Stage 2 𝑀 ←Correct(𝑀); 𝑀 ← Correct(SFT with top-𝑘 in-database examples) ; // Stage 3 return 𝑀; Function Correct(𝑀): train 𝑀 with LSFT on verified corrections of 𝐴𝑒 (𝑀, T ) and a replay of 𝑝 sample earlier examples; merge LoRA adapters; return 𝛾𝑀 + (1 − 𝛾 )𝑀𝑝 ;

Figure 5: Agent-based constraint synthesizer. Agents or a parser propose candidates in three lanes, LLM-free oracles accept or reject them (dashed), and counterexample databases D ′ serve the semantic and logical lanes.

stage, the model is further fine-tuned on queries corrected by the error-mining agent 𝐴𝑒 (Section 4.1), mixed with a share of earlier examples (replay), and its weights are averaged with the pre-trained ones as in WiSE-FT [34] to reduce forgetting.

4 Verified Constraint Synthesis 4.1 Agent-based Constraint Synthesizer The constraint synthesizer instantiates the constraints of Section 3.1. As shown in Figure 5, four LLM agents propose candidates, and the grammar and execution oracles decide their membership in P𝑐 or N𝑐 . The parser alone yields the syntactic constraint, 𝐴𝑒 and 𝐴𝑎 the semantic one, and 𝐴𝑑 and 𝐴𝑣 the logical one. • Error-mining agent 𝐴𝑒 . After each stage, 𝐴𝑒 finds the wrong decision in each failed training prediction and writes a close correction, repaired by execution for at most 𝑖 𝑐max rounds. Corrections serve as SFT examples, the wrong decisions select the units

Hao Fu, Degang Sun, Wan Wei, Jinxia Wei, Chun Long, and Peng Gao

for semantic and logical constraints, and suspicious references are flagged for manual review. • Alignment agent 𝐴𝑎 proposes semantic units (alignments) and up to three confusable alternatives per decision. • Editing agent 𝐴𝑑 samples a logical unit (an edge of 𝐺) and produces 𝑞 ′ , 𝑦 ′ , and 𝑦˜ ′ . • Verification agent 𝐴𝑣 back-translates 𝑦 ′ into a question and checks whether it matches 𝑞 ′ . Counterexample database generation. Given a database D and two queries 𝑦1 and 𝑦2 with the same result on D, the generator searches for a database D ′ that satisfies the constraints of D and on which 𝑦1 and 𝑦2 return different results. It (1) samples a sub-database containing the tables referenced by the queries, following foreign keys to keep referential integrity; (2) mutates it by duplicating rows with fresh primary keys, perturbing values at the boundaries of query constants (e.g., 2 in >= 2), and inserting additional child rows into one-to-many relationships; and (3) keeps a mutated instance if it satisfies the primary-key, foreign-key, NOT NULL, and type constraints and separates the two queries. Duplicated child rows expose join-induced double counting, boundary values expose inclusive versus exclusive comparisons, and ties expose superlative and LIMIT errors. The search stops when a separating instance is found or a mutation budget is exhausted. The same procedure is used for the non-equivalence check of confusions and the substitution test of alignments. Each training example contributes at most one pair per edge type in Table 4. Because each candidate must pass checks that do not depend on an LLM, agent errors reduce the amount of data rather than its correctness. For example, an incorrect alignment fails the substitution test, an incorrect propagated query is not separated from its partial counterpart, and an equivalent confusion is discarded. The agent prompts are listed in Appendix F.

4.2

The Spider-Refined Dataset

During error mining, many apparent model errors turned out to be Spider annotation problems. We manually reviewed all cases flagged by 𝐴𝑒 and introduce Spider-Refined, a training dataset built from Spider that corrects these annotations, enriches the database schemas, and adds the verified alignments, confusions, and counterfactual pairs produced by the constraint synthesizer. Spider-Refined is used only for training; evaluation uses the official sets. Among the confirmed annotation issues, the most frequent is a correct prediction that is rejected by execution-based comparison (163 cases), followed by under-specified questions whose references return extra (71) or missing (37) columns, and references whose logic contradicts the question (31). Figure 14 lists all issue types. For each confirmed issue, the reference query or the question is minimally revised; when several outputs are equally valid, the alternatives are recorded. All revisions are verified by execution. Spider schemas often use terse or ambiguous identifiers. For each database, the synthesizer agents generate descriptions of (i) table semantics, e.g., player stores player records; (ii) value types, e.g., REAL for continuous values such as income and INTEGER for counts; (iii) relationships, e.g., one club has many players; (iv) potential pitfalls, e.g., a floating-point primary key that makes equality comparison unreliable; and (v) intended use cases, e.g., retrieving the

SQLoom : Set-Contrastive Learning of Syntactic, Semantic, and Logical Dependencies for Text-to-SQL

GPT-4

TA-SQL

DEA-SQL

88.6

88.1

SQLoom

(b) EX on Spider Dev vs. Spider-DK (%)

(a) EX on Spider variants (%)

90

CogSQL

WWW ’27, May 10–14, 2027, Dublin, Ireland

(d) EX by training set (%)

(c) Cross-dataset EX (%)

82.1

88.5 SQLoom

76.3

80

90

89.2 88.7

CogSQL

80

88

60 TA-SQL

87.8

87.5

48.6

70

86

39.8

Dev

DK

40

GPT-4

60

84 DK

Syn

Realistic

65

70

75

80

85

90

S→B SFT

B→S

Dev

SQLoom

Spider

Test Spider-Refined

Figure 6: Robustness and transfer. (a) EX on the Spider variants. (b) EX on Spider Dev (hollow) and Spider-DK (filled); a shorter segment means higher robustness. (c) Cross-dataset EX (S = Spider, B = BIRD). (d) EX when training on Spider vs. Spider-Refined. players of a club or aggregating income per club. The descriptions are checked manually and added to the schema input S and the agent prompts.

5

Experiments

We evaluate how SQLoom compares with fine-tuning, reinforcement learning, and agent-based methods and with general-purpose LLMs, its robustness to perturbed questions and unseen datasets, how each component contributes, and its cost.

5.1

Experimental Settings

Datasets. We use Spider [38] (10,181 questions, 200 databases, 138 domains), its variants Spider-DK [5], Spider-Syn [4], and SpiderRealistic [3], and BIRD [15] (12,751 pairs, 95 databases, 33.4 GB). We train two models, one on the Spider-Refined training set, evaluated on Spider and its variants, and one on the BIRD training set, evaluated on BIRD. Evaluation databases are disjoint from training databases, so all evaluations are cross-database. Spider difficulty levels follow the official script, which counts SQL components such as joins, aggregations, and nested queries. Baselines. We compare with fine-tuning methods (T5-3B+PICARD [26], RESDSQL [12], SQL-PaLM [29], CodeS [13], SENSE [36], ROUTE [23], OmniSQL [11], and JOLT-SQL [28]), preference and RL methods (ExCoT [40], SQL-R1 [17], and Arctic-Text2SQL-R1 [37]), and agentbased methods (SQLFixAgent [2], DEA-SQL [35], SuperSQL [10], CogSQL [39], TA-SQL [24], and AskData [27]). Metrics and implementation. We report execution accuracy (EX). SQLoom fine-tunes Qwen3-1.7B with LoRA [8] on one RTX 4090 GPU and decodes greedily without constrained decoding. The agents use GPT-4. We set 𝜆syn = 0.5, 𝜆sem = 𝜆log = 1.0, 𝛽 = 0.1, and 𝑘 = 3, selected by grid search on a held-out 10% of the training set (Appendix C). Results of SQLoom are averaged over 3 seeds.

5.2

Table 3: EX (%) on Spider and BIRD. The best results are in bold red and the second best are underlined, and † denotes self-consistency. SQLoom uses no BIRD evidence and reports mean ± std over three seeds. Method Fine-tuning T5-3B + PICARD RESDSQL-3B + NatSQL SQL-PaLM CodeS-7B SENSE-7B ROUTE + Qwen2.5-7B OmniSQL-7B JOLT-SQL + Qwen2.5-Coder-7B Preference / reinforcement learning ExCoT + LLaMA-3.1-70B SQL-R1 + Qwen2.5-Coder-7B† Arctic-Text2SQL-R1-7B Agent-based CodeS-7B + SQLFixAgent DEA-SQL + GPT-4 SuperSQL + GPT-4 CogSQL + GPT-4 TA-SQL + GPT-4 AskData + GPT-4o SQLoom + Qwen3-1.7B

Dev

Spider Test

BIRD Dev

79.3 84.1 82.8 85.4 83.2 83.6 81.2 87.0

75.1 79.9 – 83.5 83.5 83.7 87.9 86.8

– – – 57.2 51.8 55.9 63.9 60.4

– 87.6 –

86.6 88.7 88.8

68.5 66.6 68.9

86.2 85.4 87.0 85.4 85.0 – 88.7±0.3

83.9 87.1 – 86.4 – – 89.2±0.2

– 58.9 58.5 59.6 56.2 67.4 73.4±0.5

SQLoom uses 1.7B. Its margin is largest on BIRD, whose questions involve more clauses and more chances for inconsistency. Unlike most baselines, SQLoom does not use the external knowledge evidence of BIRD; adding it raises EX to 76.8%. The gains are also not explained by imitating the teacher, since oracles vet agent outputs and GPT-4 is weaker on the Spider variants (Figure 6a). Takeaway. Explicit SQL supervision can matter more than scale: a small model trained to reject near misses beats much larger ones.

Main Results

As shown in Table 3, SQLoom achieves the best EX on all three sets with the smallest model, exceeding the strongest baselines by 1.1, 0.4, and 4.5 points. It outperforms JOLT-SQL, the best fine-tuning method, by 13.0 points on BIRD, and AskData, the best agent-based method, by 6.0 points, without proprietary LLM calls. As Table 3 and Figure 13 show, all baselines above 85% on the Spider test set or 60% on BIRD use at least 7B parameters, whereas

5.3

Comparison with General-Purpose LLMs

Figure 8 shows that model size does not determine zero-shot accuracy. Qwen3-8B matches Qwen3-235B, and the open-weight DeepSeek-V3 is 14.4 points lower. Reasoning and code-specialized models score lowest; most of their failures lack a final SQL statement within the generation budget, which is why the longer-reasoning DeepSeek-R1-671B scores below DeepSeek-R1-8B. SQLoom lifts

WWW ’27, May 10–14, 2027, Dublin, Ireland

Hao Fu, Degang Sun, Wan Wei, Jinxia Wei, Chun Long, and Peng Gao

(a) EX drop of ablations (pts)

(b) Spider Test EX by difficulty (%)

(c) Training time (GPU hours)

(d) Inference latency (s/query)

100 w/o context aug. w/o constraint training w/o error-mined corr. w/o SFT warm-up w/o Lsyn w/o Lsem w/o Llog Stage 2 = SFT only single-stage mix

12 80 Easy Medium Hard Extra All

60 40 Spider Test BIRD Dev

20

S1 / SFT S2 S3

10.8

8

SQLoom

0.42

SFT

0.40

4.6 SQL-R1

2.9

4 CogSQL

9.8

0 0

10

20

30

40

Base

+S1

+S2

+S3

SFT

SQLoom

0.1

1

10

Figure 7: Ablation and cost. (a) EX drop of ablated stages (above the dashed line) and constraint levels (below). (b) Spider Test EX by difficulty after Stages 1 to 3 (S1 to S3). (c) Training time. (d) Per-query latency (log scale). Qwen3-1.7B + SQLoom Qwen3-235B Qwen3-8B GPT-4 Qwen3-1.7B DeepSeek-V3 Gemma3-4B Qwen3-0.6B Spark-X1 Qwen2.5-Coder-7B DeepSeek-R1-8B DeepSeek-R1-671B Seed-Coder-8B-R Gemini-2.0 Gemma3-27B Gemma3-1B

88.7 75.3 75 72.5

Takeaway. Contrasting references with near-miss errors gives a signal that more SFT data cannot replace, given a warmed-up reference.

+25.8

62.9 60.8 42 40.3 35.2

5.6

Open-weight Proprietary

2.9 0.1 0 0 0 0

Code-specialized Fine-tuned

25

50

75

100

EX on Spider Dev (%)

Figure 8: Zero-shot EX of general-purpose LLMs on Spider Dev, compared with Qwen3-1.7B fine-tuned by SQLoom (top bar).

In Figure 7b, the SFT warm-up mainly helps hard and extra-hard questions (+15.0 and +21.7 points), where the base model often fails to produce well-formed SQL. Constraint training brings the largest gains at each level, especially for medium to extra-hard questions (+31.6 to +43.0 points). As a result, the gap between easy and extra-hard questions shrinks from 53.7 to 9.6 points. Context augmentation adds a further 0.7–2.1 points. Finally, EX varies by at most 1.4 points over the tested values of 𝛽 and 𝜆log (Figure 10). SQLoom moves the agent cost to training (Figure 7c,d). Training takes 10.8 GPU hours (2.3× SFT) and 21.3 GB of memory, plus 38K GPT-4 calls. Inference is as fast as SFT (0.42 s per query) and 23× faster than CogSQL with GPT-4.

Qwen3-1.7B from 62.9% to 88.7%, i.e., 13.4 points above the best zero-shot model, which is 138× larger.

5.4

Takeaway. Decision-level supervision helps most on multi-clause queries, and building it once with LLM agents is cheaper than calling agents for every query.

Robustness and Transfer

On Spider-DK, Spider-Syn, and Spider-Realistic (Figure 6a,b), SQLoom loses at most 0.6 points relative to Spider Dev, whereas agent-based methods lose 8.8–12.1 points on Spider-DK and 8.9 points on SpiderSyn. Lexical matching between question and schema breaks under domain knowledge and synonyms, whereas SQLoom grounds question elements in verified decisions. In the cross-dataset setting (Figure 6c), SQLoom improves over SFT trained on the same data by 8.8 points from Spider to BIRD and 5.8 points from BIRD to Spider. Training on Spider-Refined instead of Spider improves EX on the official sets by 1.2 and 1.4 points (Figure 6d), so the corrections help beyond evaluation. Takeaway. Supervising properties of SQL itself, rather than patterns of one benchmark, carries over to new phrasings and datasets.

5.5

Analysis and Cost

22.6 9.6

Ablation Study

All training stages contribute. Removing constraint training costs 12.5 and 9.7 points, and removing error-mined correction costs 11.1 and 17.2 points (Figure 7a). Removing the SFT warm-up hurts most. Each constraint level helps; the semantic and logical constraints contribute most, especially on BIRD. Replacing all constraints with SFT on the same data loses 8.8 and 10.3 points. Single-stage mixing loses 3.6 and 5.3 points, as its reference model lacks the warm-up.

6

Conclusion and Future Work

We propose SQLoom, which enforces the syntactic validity, semantic alignment, and logical consistency of SQL with oracle-verified set-contrastive supervision, and release the Spider-Refined dataset. With 1.7B parameters, SQLoom outperforms larger fine-tuned, RL, and agent-based systems in a single pass. In future work, we will fine-tune larger open-weight LLMs with SQLoom to study how constraint-based supervision scales with model size, and construct more constraints, e.g., for nested queries and other SQL dialects.

References [1] Arian Askari, Christian Poelitz, and Xinye Tang. 2025. Magic: Generating selfcorrection guideline for in-context text-to-sql. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 23433–23441. [2] Jipeng Cen, Jiaxin Liu, Zhixu Li, and Jingjing Wang. 2025. SQLFixAgent: Towards Semantic-Accurate Text-to-SQL Parsing via Consistency-Enhanced Multi-Agent Collaboration. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 49–57. [3] Xiang Deng, Ahmed Hassan Awadallah, Christopher Meek, Oleksandr Polozov, Huan Sun, and Matthew Richardson. 2020. Structure-grounded pretraining for text-to-sql. arXiv preprint arXiv:2010.12773 (2020). [4] Yujian Gan, Xinyun Chen, Qiuping Huang, Matthew Purver, John R Woodward, Jinxia Xie, and Pengsheng Huang. 2021. Towards robustness of text-to-SQL models against synonym substitution. arXiv preprint arXiv:2106.01065 (2021).

SQLoom : Set-Contrastive Learning of Syntactic, Semantic, and Logical Dependencies for Text-to-SQL

[5] Yujian Gan, Xinyun Chen, and Matthew Purver. 2021. Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalization. arXiv:2109.05157 [cs.CL] [6] Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jingren Zhou. 2024. Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation. Proceedings of the VLDB Endowment 17, 5, 1132–1145. [7] Zijin Hong, Zheng Yuan, Qinggang Zhang, Hao Chen, Junnan Dong, Feiran Huang, and Xiao Huang. 2024. Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL. arXiv preprint arXiv:2406.08426 (2024). [8] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. Lora: Low-rank adaptation of large language models. ICLR 1, 2 (2022), 3. [9] Jaehoon Lee, CheolWon Na, Suyoung Bae, Jin-Seop Lee, Jihyung Lee, YunSeok Choi, and Jee-Hyong Lee. 2026. EXPO-SQL: Execution-based Clause-level Policy Optimization for Text-to-SQL. In Findings of the Association for Computational Linguistics: ACL 2026. 22000–22019. [10] Boyan Li, Yuyu Luo, Chengliang Chai, Guoliang Li, and Nan Tang. 2024. The dawn of natural language to sql: Are we fully ready? arXiv preprint arXiv:2406.01265 (2024). [11] Haoyang Li, Shang Wu, Xiaokang Zhang, Xinmei Huang, Jing Zhang, Fuxin Jiang, Shuai Wang, Tieying Zhang, Jianjun Chen, Rui Shi, Hong Chen, and Cuiping Li. 2025. OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scale. Proceedings of the VLDB Endowment 18, 11 (2025), 4695–4709. [12] Haoyang Li, Jing Zhang, Cuiping Li, and Hong Chen. 2023. Resdsql: Decoupling schema linking and skeleton parsing for text-to-sql. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 13067–13075. [13] Haoyang Li, Jing Zhang, Hanbing Liu, Ju Fan, Xiaokang Zhang, Jun Zhu, Renjie Wei, Hongyan Pan, Cuiping Li, and Hong Chen. 2024. Codes: Towards building open-source language models for text-to-sql. Proceedings of the ACM on Management of Data 2, 3 (2024), 1–28. [14] Jinyang Li, Binyuan Hui, Reynold Cheng, Bowen Qin, Chenhao Ma, Nan Huo, Fei Huang, Wenyu Du, Luo Si, and Yongbin Li. 2023. Graphix-t5: Mixing pre-trained transformers with graph-aware layers for text-to-sql parsing. In Proceedings of the AAAI conference on artificial intelligence, Vol. 37. 13076–13084. [15] Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, et al. 2023. Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls. Advances in Neural Information Processing Systems 36 (2023), 42330–42357. [16] Hanbing Liu, Haoyang Li, Xiaokang Zhang, Ruotong Chen, Haiyong Xu, Tian Tian, Qi Qi, and Jing Zhang. 2025. Uncovering the Impact of Chain-of-Thought Reasoning for Direct Preference Optimization: Lessons from Text-to-SQL. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 21223–21261. [17] Peixian Ma, Xialie Zhuang, Chengjin Xu, Xuhui Jiang, Ran Chen, and Jian Guo. 2025. SQL-R1: Training Natural Language to SQL Reasoning Model By Reinforcement Learning. In Advances in Neural Information Processing Systems. [18] Xingyu Ma, Xin Tian, Lingxiang Wu, Xuepeng Wang, Zhilin Zhang, Xueming Tang, and Jinqiao Wang. 2026. Addressing Semantic Blind Spots in Text-to-SQL via Component Pre-generation and AST Matching Rewards. In Proceedings of the 43rd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 306). 84969–84988. [19] Mohammadreza Pourreza, Hailong Li, Ruoxi Sun, Yeounoh Chung, Shayan Talaei, Gaurav Tarlok Kakkar, Yu Gan, Amin Saberi, Fatma Ozcan, and Sercan O. Arik. 2025. CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL. In The Thirteenth International Conference on Learning Representations. [20] Mohammadreza Pourreza and Davood Rafiei. 2023. DIN-SQL: Decomposed InContext Learning of Text-to-SQL with Self-Correction. In Advances in Neural Information Processing Systems. [21] Mohammadreza Pourreza and Davood Rafiei. 2024. DTS-SQL: Decomposed Textto-SQL with Small Large Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2024. [22] Mohammadreza Pourreza, Shayan Talaei, Ruoxi Sun, Xingchen Wan, Hailong Li, Azalia Mirhoseini, Amin Saberi, and Sercan Ö. Arik. 2025. Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQL. arXiv preprint arXiv:2503.23157 (2025). [23] Yang Qin, Chao Chen, Zhihang Fu, Ze Chen, Dezhong Peng, Peng Hu, and Jieping Ye. 2025. ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL. In The Thirteenth International Conference on Learning Representations. [24] Ge Qu, Jinyang Li, Bowen Li, Bowen Qin, Nan Huo, Chenhao Ma, and Reynold Cheng. 2024. Before generation, align it! a novel and effective strategy for mitigating hallucinations in text-to-sql generation. arXiv preprint arXiv:2405.15307 (2024). [25] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. In Advances in Neural Information Processing Systems.

WWW ’27, May 10–14, 2027, Dublin, Ireland

[26] Torsten Scholak, Nathan Schucher, and Dzmitry Bahdanau. 2021. PICARD: Parsing incrementally for constrained auto-regressive decoding from language models. arXiv preprint arXiv:2109.05093 (2021). [27] Vladislav Shkapenyuk, Divesh Srivastava, Theodore Johnson, and Parisa Ghane. 2025. Automatic Metadata Extraction for Text-to-SQL. arXiv preprint arXiv:2505.19988 (2025). [28] Jinwang Song, Hongying Zan, Kunli Zhang, Lingling Mu, Yingjie Han, Haobo Hua, and Min Peng. 2025. JOLT-SQL: Joint Loss Tuning of Text-to-SQL with Confusion-aware Noisy Schema Sampling. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 6040–6053. [29] Ruoxi Sun, Sercan Ö Arik, Alex Muzio, Lesly Miculicich, Satya Gundabathula, Pengcheng Yin, Hanjun Dai, Hootan Nakhost, Rajarishi Sinha, Zifeng Wang, et al. 2023. Sql-palm: Improved large language model adaptation for text-to-sql (extended). arXiv preprint arXiv:2306.00739 (2023). [30] Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. 2024. CHESS: Contextual Harnessing for Efficient SQL Synthesis. arXiv preprint arXiv:2405.16755 (2024). [31] Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, Linzheng Chai, Zhao Yan, Qian-Wen Zhang, Di Yin, Xing Sun, and Zhoujun Li. 2025. MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL. In Proceedings of the 31st International Conference on Computational Linguistics. [32] Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, and Matthew Richardson. 2020. RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 7567–7578. [33] Han Weng, Puzhen Wu, Cui Longjie, Yi Zhan, Boyi Liu, Yuanfeng Song, Dun Zeng, Yingxiang Yang, Qianru Zhang, Dong Huang, Xiaoming Yin, Yang Sun, and Xing Chen. 2025. Graph-Reward-SQL: Execution-Free Reinforcement Learning for Text-to-SQL via Graph Matching and Stepwise Reward. In Findings of the Association for Computational Linguistics: EMNLP 2025. 12917–12943. [34] Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Raphael Gontijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, and Ludwig Schmidt. 2022. Robust Fine-Tuning of Zero-Shot Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7959–7971. [35] Yuanzhen Xie, Xinzhou Jin, Tao Xie, MingXiong Lin, Liang Chen, Chenyun Yu, Lei Cheng, ChengXiang Zhuo, Bo Hu, and Zang Li. 2024. Decomposition for enhancing attention: Improving llm-based Text-to-SQL through workflow paradigm. arXiv preprint arXiv:2402.10671 (2024). [36] Jiaxi Yang, Binyuan Hui, Min Yang, Jian Yang, Junyang Lin, and Chang Zhou. 2024. Synthesizing Text-to-SQL Data from Weak and Strong LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 7864–7875. [37] Zhewei Yao, Guoheng Sun, Łukasz Borchmann, Zheyu Shen, Minghang Deng, Bohan Zhai, Hao Zhang, Ang Li, and Yuxiong He. 2026. Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL. In Findings of the Association for Computational Linguistics: ACL 2026. 26966–26995. [38] Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. 2018. Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium. [39] Hongwei Yuan, Xiu Tang, Ke Chen, Lidan Shou, Gang Chen, and Huan Li. 2025. Cogsql: A cognitive framework for enhancing large language models in text-tosql translation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 25778–25786. [40] Bohan Zhai, Canwen Xu, Yuxiong He, and Zhewei Yao. 2025. Optimizing Reasoning for Text-to-SQL with Execution Feedback. In Findings of the Association for Computational Linguistics: ACL 2025. 19206–19218. [41] Qizheng Zhang, Michael Wornow, and Kunle Olukotun. 2025. Cost-Efficient Serving of LLM Agents via Test-Time Plan Caching. arXiv preprint arXiv:2506.14852 (2025). [42] Ruiqi Zhong, Tao Yu, and Dan Klein. 2020. Semantic Evaluation for Text-to-SQL with Distilled Test Suites. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 396–411. [43] Victor Zhong, Caiming Xiong, and Richard Socher. 2017. Seq2sql: Generating structured queries from natural language using reinforcement learning. arXiv preprint arXiv:1709.00103 (2017).

A

Properties of the Set-Contrastive Objective

This appendix explains why Eq. 2 is a suitable training objective for all three levels. We show two properties. First, Eq. 2 unifies existing losses: the syntactic loss of Eq. 4 and the DPO loss are its special cases (Proposition A.1), so one objective serves all levels. Second,

WWW ’27, May 10–14, 2027, Dublin, Ireland

Hao Fu, Degang Sun, Wan Wei, Jinxia Wei, Chun Long, and Peng Gao

its gradient pushes down only oracle-certified negatives and fades once a constraint is satisfied (Proposition A.2), so equivalent queries are never penalized and training focuses on unsatisfied constraints. Notation. We analyze a single constraint 𝑐 = (P𝑐 , N𝑐 , 𝑀𝑐 ), write 𝑟 𝑦 = 𝑟𝜃 (𝑦, 𝑀𝑐 (𝑦)), and define ∑︁ 𝑒 𝛽𝑟 𝑦 𝜋 (𝑦) = Í , 𝜋 (N𝑐 ) = 𝜋 (𝑦), 𝛽𝑟 𝑦 ′ 𝑦 ′ ∈ P𝑐 ∪N𝑐 𝑒 𝑦 ∈ N𝑐 𝜋 P (𝑦) = Í

𝑒 𝛽𝑟 𝑦 𝑦 ′ ∈ P𝑐 𝑒

𝛽𝑟 𝑦 ′

(𝑦 ∈ P𝑐 ).

Intuitively, 𝜋 (𝑦) is the share of candidate 𝑦 among all candidates, 𝜋 P (𝑦) is its share among the positives, and 𝜋 (N𝑐 ) is the total share of the negatives, so ℓ (𝑐) is small when 𝜋 (N𝑐 ) is small. The first proposition recovers existing losses from Eq. 2.

Figure 9: Semantic units of the running example. Each question element is mapped to its SQL decision and contrasted with a confusion (red) that changes the result on D (bottom row) and is pushed down by the semantic constraint.

Proposition A.1 (Special cases). (a) At the syntactic level, with ∗ ⊕ 𝑣 for 𝑣 ∈ V, 𝛽 = 1, and 𝑝 candidates 𝑦 <𝑡 ref ≡ 1, Eq. 2 equals the syntactic loss of Eq. 4 at position 𝑡.
(b) If P𝑐 = {𝑦 + } and N𝑐 = {𝑦 − }, Eq. 2 equals − log 𝜎 𝛽 (𝑟 𝑦 + − 𝑟 𝑦 − ) , a DPO loss [25] whose reward is restricted to the positions 𝑀𝑐 .

Table 4: Dependency edges of 𝐺, the components each edit affects, and the check the propagated query must pass.

∗ ). Proof. (a) With 𝛽 = 1 and 𝑝 ref ≡ 1, exp(𝑟 𝑦 ) = 𝑝𝜃 (𝑣 | 𝑥, 𝑦 <𝑡 Í Since P𝑐 ∪ N𝑐 covers V, the denominator equals 𝑣 ∈ V 𝑝𝜃 (𝑣 | ∗ ) = 1, and the numerator is the probability mass of V 𝑥, 𝑦 <𝑡 valid (𝑠𝑡 ). (b) Dividing the numerator and the denominator by exp(𝛽𝑟 𝑦 + ) gives

ℓ (𝑐) = log 1 + exp(−𝛽 (𝑟 𝑦 + − 𝑟 𝑦 − )) = − log 𝜎 𝛽 (𝑟 𝑦 + − 𝑟 𝑦 − ) . □

Edit 𝛿 on join path (1:N added)

Dependent aggregation, SELECT

target entity

GROUP BY, SELECT, join HAVING, sub-query WHERE, HAVING ORDER BY, LIMIT, aggregation WHERE vs. HAVING

quantifier boundary / comparison superlative / top-𝑘 filter scope

The second proposition characterizes how Eq. 2 updates the masked reward 𝑟 𝑦 of each candidate.
Proposition A.2 (Gradient). ℓ (𝑐) = − log 1 −𝜋 (N𝑐 ) ≥ 0, and ( 𝛽 𝜋 (𝑦) ≥ 0, 𝑦 ∈ N𝑐 , 𝜕ℓ (𝑐) = (5)
𝜕𝑟 𝑦 𝛽 𝜋 (𝑦) − 𝜋 P (𝑦) ≤ 0, 𝑦 ∈ P𝑐 . The total gradient magnitudes over N𝑐 and P𝑐 both equal 𝛽 𝜋 (N𝑐 ). Proof. Write ℓ (𝑐) = − log

∑︁ 𝑦 ∈ P𝑐

𝑒 𝛽𝑟 𝑦 + log

∑︁

𝑒 𝛽𝑟 𝑦 .

B

Required check duplicate counting; DISTINCT or key grouping granularity “at least” / “all” / “none” inclusive vs. exclusive ties; MAX vs. sort pre- vs. post-aggregation

Additional Method Details

Semantic units. Figure 9 shows the semantic units of the running example. The six element types are entity (table or join path), attribute (column), value (literal and its column), aggregation (aggregate function and DISTINCT), comparison/quantifier (operator and boundary), and ordering (ORDER BY direction and LIMIT). Dependency edges. Table 4 lists the edges of the component dependency graph 𝐺. They are derived from SQL semantics and the key constraints of the schema.

𝑦 ∈ P𝑐 ∪N𝑐

The first claim follows from the definition of 𝜋. Differentiating the second term gives 𝛽𝜋 (𝑦) for each candidate, and differentiating the first gives −𝛽𝜋 P (𝑦) for 𝑦 ∈ P𝑐 . Since 𝜋 (𝑦) = 𝜋 P (𝑦) (1 − 𝜋 (N𝑐 )) ≤ 𝜋 P (𝑦), the sign for 𝑦 ∈ P𝑐
follows. Summing gives 𝛽𝜋 (N𝑐 ) over N𝑐 and 𝛽 1 − (1 − 𝜋 (N𝑐 )) = 𝛽𝜋 (N𝑐 ) over P𝑐 . □ Implications. Proposition A.2 has three consequences, which justify the claims in Section 3.1. • Equivalent outputs are not penalized. Only members of N𝑐 , which an oracle certifies as invalid or result-changing, are pushed down, while every positive is pushed up. An equivalent rewriting of 𝑦 ∗ is therefore never penalized, unlike in token-level SFT. • Bounded, self-annealing updates. The gradient magnitude is at most 𝛽 and vanishes as 𝜋 (N𝑐 ) → 0, so satisfied constraints stop contributing. • Localized credit. Since 𝑟𝜃 (𝑦, 𝑀𝑐 (𝑦)) depends only on 𝑀𝑐 (𝑦), other positions are supervised by LSFT alone.

C

Implementation Details

Training. Table 5 lists the hyperparameters and Algorithm 1 (Section 3.5) summarizes the training procedure. LoRA adapters are applied to all attention and feed-forward projections. Hyperparameter selection. We hold out 10% of the training set and grid-search 𝜆syn, 𝜆sem, 𝜆log ∈ {0.1, 0.5, 1.0, 2.0} and 𝛽 ∈ {0.05, 0.1, 0.2, 0.5} with the other values fixed. Figure 10 shows the sensitivity to 𝛽 and 𝜆log on Spider Dev. Oracle implementation. The grammar oracle is an incremental parser for the SQLite dialect, implemented from scratch. It keeps one scope table per nested query, which records the tables, aliases, and visible columns, and covers nested queries, set operations, and CTEs. Queries outside this grammar are discarded. The execution oracle runs queries in SQLite 3.42.0 with a timeout of 5 seconds. Two queries return the same result if their outputs are equal as multisets of rows, where row order matters only when the reference contains ORDER BY, floating-point values are compared up to 2 decimal places, and NULL values are equal to each other. A query that fails

SQLoom : Set-Contrastive Learning of Syntactic, Semantic, and Logical Dependencies for Text-to-SQL

Table 5: Hyperparameters of SQLoom, selected on a held-out 10% of the training set. Name Base LLM Agent LLM Fine-tuning method LoRA target modules Learning rate / batch size Epochs (Stage 1 / 2 / 3) Correction iterations 𝑖 𝑐max Replay ratio 𝑝 sample Interpolation rate 𝛾 Constraint weights 𝜆syn , 𝜆sem , 𝜆log Temperature 𝛽 Counterexample search per pair Seeds Retrieved examples 𝑘 Retrieval embedding / store

89

88

88

87

87

86 0.2

takes course_id

s1

c1

s1

c2

section sec_id course_id

inserted: a second, key-consistent lab section of c1 (k3) section sec_id course_id

type mutate

type

k1

c1

lab

k1

c1

lab

k2

c2

lab

k2

c2

lab

k3

c1

lab

Result for s1 under 𝑞 ′ : at least three lab courses

𝑦 ′ : COUNT(DISTINCT course_id) >= 3

𝑦˜ ′ : COUNT(*) >= 3

on D

on D ′

𝑦 ′ (propagated)

count 2, ✓

count 2, ✓

𝑦˜ ′ (partial)

count 2, ✓

count 3, ✗

Figure 12: A counterexample database for the join-path edit. The inserted lab section (highlighted) makes 𝑦˜ ′ wrongly return s1 (✗), while 𝑦 ′ stays correct (✓).

86 0.1

id

(b) Logical weight 𝜆log

89

0.05

Counterexample D ′

Original database D

Value Qwen3-1.7B GPT-4 LoRA, rank 16, 𝛼 32, dropout 0.05 q, k, v, o, gate, up, down projections 5e-5 / 1 1/2/2 5 0.1 0.5 0.5, 1.0, 1.0 0.1 10 sub-databases × 200 mutations 3 3 nomic-embed-text / Chroma

(a) Temperature 𝛽

EX (%)

WWW ’27, May 10–14, 2027, Dublin, Ireland

0.5

0

0.5

1

Spider Test EX (%)

2 90

Figure 10: Sensitivity of Spider Dev EX to the temperature 𝛽 and the logical weight 𝜆log .

SQLoom

85

BIRD Dev EX (%)

Arctic, SQL-R1 OmniSQL JOLT-SQLExCoT CodeS, ROUTE, SENSE

SENSE

PICARD

75 2B

Arctic ExCoT SQL-R1 OmniSQL

60

RESDSQL

80

70 SQLoom

50

7B

70B

2B

70B

Parameters (log scale)

Parameters (log scale) Fine-tuning

7B

Preference / RL

SQLoom

Figure 13: EX versus model size. Agent-based methods are omitted because the size of GPT-4 is undisclosed.

Figure 11: Legal and illegal next tokens derived by the grammar oracle at two decision positions of the running example.

General-purpose LLMs. All general-purpose LLMs in Section 5.3 receive the same prompt, consisting of the schema in DDL form with sample rows, the question, and an instruction to output a single SQLite query. The first SQL code block of the output is evaluated.

D or times out is treated as invalid, and its candidate is discarded. For each query pair, the counterexample generator tries up to 10 sub-databases with at most 200 mutations each. A different result on a database that satisfies the key and type constraints proves that two queries are not equivalent, so the negatives of the semantic and logical constraints are wrong by construction. Positives rest on other evidence. Semantic positives are the references, and a logical positive 𝑦 ′ must execute on all databases and pass verification. Valid-token sets. Figure 11 shows the legal and illegal tokens that the grammar oracle derives at two decision positions. Counterexample databases. Figure 12 shows a counterexample database for the join-path edit. The original data cannot distinguish the propagated and partially propagated queries; inserting a second, key-consistent lab section of the same course separates them.

Additional Results

Figure 14 lists the Spider annotation issues confirmed while building Spider-Refined (Section 4.2). Figure 13 plots the EX of fine-tuning and preference or RL methods against model size (Section 5.2). Table 7 defines the error categories of Section 2.3.

E

Examples of Auxiliary Supervision

Table 8 lists the confusion types used by the semantic constraint, and Table 6 gives one counterfactual pair for each dependency edge of Table 4. All examples use the college schema with tables student(id, name, dept_id), takes(id, course_id, grade), course(course_id, title, dept_id, credits), section(sec_id, course_id, type), and department(dept_id, dept_name).

WWW ’27, May 10–14, 2027, Dublin, Ireland

Hao Fu, Degang Sun, Wan Wei, Jinxia Wei, Chun Long, and Peng Gao

Table 6: Counterfactual pairs per edge type. 𝑞 ′ is the edited question, 𝑦 ′ the propagated query, and 𝑦˜ ′ the partially propagated query that updates only the edited clause. Edited question 𝑞 ′ Courses with a lab section per student? Students per department? Students who passed all courses. Courses with at least 4 credits. Top student in each course. Departments with over 100 enrollments in total.

Edge join path target entity quantifier boundary top-𝑘 filter scope

Correct but flagged Extra columns Missing columns Logical error Syntax error Wrong column Semantic mismatch Over-generic query Spurious ordering

Propagated 𝑦 ′ . . . JOIN section . . . COUNT(DISTINCT T.course_id) . . . COUNT(DISTINCT T.id) . . . GROUP BY D.dept_id . . . NOT EXISTS (. . . T.grade = ’F’) WHERE credits >= 4 . . . grade = (SELECT MAX(grade) . . . ) . . . GROUP BY dept_id HAVING COUNT(*) > 100

163 71 37 31 18 17 15 14 12

0

50

100

150

Figure 14: Annotation issues in Spider flagged by the errormining agent and confirmed manually. Table 7: Error categories used in Figure 2.

Wrong column Semantic error Wrong ordering or LIMIT Other model error

Definition wrong or misunderstood table mismatched, wrong-key, or non-existent column misread question, wrong alias or aggregate, or missing condition wrong LIMIT or sort direction

Example takes instead of student

e.g., hard-coded outputs

returns a constant

course_id when the title is asked AVG for “total”

ASC for “highest”

Table 8: Confusion types of the semantic constraint. Each confusion replaces one decision of the reference query. Element entity attribute value aggregation comparison quantifier ordering

F

asked to translate the question into one SQLite query and output only the SQL. Its input has the schema with descriptions, the question, and, in Stage 3, retrieved pairs from the same database. Prompt 1: Templates of the error-mining agent 𝐴𝑒

# cases

Type Wrong table

Partially propagated 𝑦˜ ′ same join, keeps COUNT(*) keeps GROUP BY T.id keeps WHERE T.grade <> ’F’ (any passed) keeps WHERE credits > 4 keeps ORDER BY grade DESC LIMIT 1 keeps the condition in WHERE

Reference decision FROM student course.title dept_name = ’Physics’ COUNT(DISTINCT course_id) credits >= 4 NOT EXISTS (. . . ) ORDER BY grade DESC LIMIT 1

Confusion FROM takes course.course_id dept_name = ’physics’ COUNT(*) credits > 4 NOT IN with nullable key ORDER BY grade ASC LIMIT 1

Prompt Templates

Each agent prompt starts with the schema and sample rows of the database as DDL and DML statements, together with the schema descriptions of Spider-Refined, and then lists the inputs below. Prompts 1–4 show the agent templates. The fine-tuned model is

Evaluation. You are a database expert. Determine whether the labeled SQL or the predicted SQL better aligns with the given question, based on the provided database schema and content. Correction. Given the question and an erroneous SQL query, correct the SQL while preserving its structure as closely as possible. Input. Question: {question}. Labeled SQL: {label_SQL}. Predicted SQL: {predict_SQL}.

Prompt 2: Template of the alignment agent 𝐴𝑎 System. You are a database expert. Given a question and its SQL query, list each semantic element of the question (entity, attribute, value, aggregation, comparison or quantifier, ordering) and the SQL fragment that realizes it. For each fragment, propose up to three alternatives of the same type that a model may confuse with it. Output one line per element in the form element -> fragment | alternatives. Input. Question: {question}. SQL: {label_SQL}. Frequent errors of the current model: {error_decisions}.

Prompt 3: Template of the editing agent 𝐴𝑑 System. You are a database expert. Given a question, its SQL query, and an edit type, rewrite the question so that it expresses the edited intent. Output (1) the edited question, (2) the correct SQL in which all clauses affected by the edit are updated, and (3) a SQL that applies the edit only to the targeted clause and copies all other clauses from the original SQL. Input. Question: {question}. SQL: {label_SQL}. Edit type and dependent clauses: {edge}.

Prompt 4: Template of the verification agent 𝐴𝑣 System. You are a database expert. Given a SQL query, write one natural language question that the query answers. Then decide whether your question has the same meaning as the given question. Answer yes or no with a short reason. Input. SQL: {edited_SQL}. Question: {edited_question}.

Related documents

• Towards Optimizing SQL Generation via LLM Routing#1108911
• Closing the Cross-Dialect Gap: Query Plans as a Portable Interface in Text-to-SQL#1118743
• Knapsack Optimization-based Schema Linking for LLM-based Text-to-SQL Generation#158668
• Tracing Recurrent and Emergent Hallucination Patterns in Decomposed Text-to-SQL Generation#1403516
• text2ql: Multi-Target Natural Language Querying via a Language-Agnostic Intermediate Representation#822516