LeGo-Code: Can Modular Curriculum Learning Advance Complex Code Generation? Insights from Text-to-SQL Salmane Chafik
Saad Ezzini
Ismail Berrada
Mohammed VI Polytechnic University Morocco
King Fahd University of Petroleum and Minerals Saudi Arabia
Mohammed VI Polytechnic University Morocco
arXiv:2604.18254v1 [cs.AI] 20 Apr 2026
Abstract Recently, code-oriented large language models (LLMs) have demonstrated strong capabilities in translating natural language into executable code. Text-to-SQL is a significant application of this ability, enabling non-technical users to interact with relational databases using natural language. However, state-of-the-art models continue to struggle with highly complex logic, particularly deeply nested statements involving multiple joins and conditions, as well as with real-world database schemas that are noisy or poorly structured. In this paper, we investigate whether curriculum learning can improve the performance of code-based LLMs on Text-to-SQL tasks. Employing benchmarks including Spider and BIRD, we fine-tune models under different curriculum strategies. Our experiments show that naïve curriculum, simply ordering training samples by complexity in a single epoch, fails to surpass standard fine-tuning due to catastrophic forgetting. To overcome this, we propose a Modular Adapter Composition (MAC) strategy. By sequentially training tier-specific adapters on incremental complexity levels (Easy to Extra-Hard), we create a scaffolded learning environment that improves performance on complex queries. Our approach not only produces measurable performance gains on the Spider and BIRD benchmarks but also provides a flexible, "Lego-like" architecture, allowing models to be composed and deployed based on specific schema difficulty requirements. These findings demonstrate that structured, modular learning is a superior alternative to monolithic fine-tuning for mastering the syntax and logic of complex code generation.
CCS Concepts • Software and its engineering → Software development techniques; • Computing methodologies → Natural language processing; • Information systems → Structured Query Language.
Keywords Text-to-SQL, Curriculum Learning, Code Generation, Large Language Models Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. EASE 2026, Glasgow, Scotland, United Kingdom © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-x-xxxx-xxxx-x/YYYY/MM https://doi.org/10.1145/nnnnnnn.nnnnnnn
ACM Reference Format: Salmane Chafik, Saad Ezzini, and Ismail Berrada. 2026. LeGo-Code: Can Modular Curriculum Learning Advance Complex Code Generation? Insights from Text-to-SQL. In Proceedings of The 30th International Conference on Evaluation and Assessment in Software Engineering (EASE 2026). ACM, New York, NY, USA, 7 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn
1
Introduction
During the last few years, the software engineering domain has benefited greatly from the introduction of large language models (LLMs) for code. Software developers now rely heavily on these tools in their daily activities, including generating ideas, designing application architectures, writing code snippets, testing code, maintaining projects, debugging and securing software. Today, tools such as Claude Code [1] and Cursor [2] can generate and manage even large repositories on GitHub. Since the introduction of the Transformer architecture [23], software engineering tasks leaderboards have been consistently dominated by code-focused LLMs. Structured Query Language (SQL) is one domain that has been highly influenced by these advances. Today, non-technical users can interact with relational databases using natural language rather than writing formal SQL queries. Users can ask questions in everyday language, often in multiple languages, not limited to English, without prior knowledge of SQL syntax, schema design, or database constraints. In response, AI systems translate these requests into executable SQL statements that retrieve the desired information. This domain, commonly referred to as Text-to-SQL, enables more accessible data exploration and democratizes database usage across organizations. Instead of relying solely on database specialists or analysts, business users, researchers, and decision-makers can directly query data sources through conversational interfaces. Modern systems can also incorporate schema awareness, contextual understanding, and ambiguity resolution to produce accurate and efficient queries [20]. With the introduction of multiple datasets and benchmarks, such as WikiSQL [26], SPIDER [25], and BIRD [16], the Text-to-SQL field has witnessed remarkable progress over time. Early approaches relied primarily on rule-based systems, template matching, or sketchbased methods [9]. These techniques relied on predefined query templates or structures with placeholders that were automatically filled based on the user’s input. However, their reliance on rigid templates made them inflexible and poorly suited to handle linguistic variability, ambiguous phrasing, or complex query requirements. To address these limitations, subsequent research explored graphbased approaches [24] and the use of intermediate representations and languages[13]. These methods aimed to better capture the structural relationships between natural language questions and database schemas, improving robustness and generalization beyond
EASE 2026, 9–12 June, 2026, Glasgow, Scotland, United Kingdom
fixed templates. More recently, modern approaches have leveraged Transformer-based architectures[15, 17], which excel at modeling long-range dependencies and deep semantic relationships in both natural language and code. Such models can generalize across domains, handle diverse linguistic expressions, and adapt to previously unseen database schemas. Consequently, Transformer-based methods have achieved state-of-the-art performance on benchmark datasets and enabled the practical deployment of natural-language interfaces to databases in real-world applications. However, even with these advances, state-of-the-art models still struggle with highly complex SQL queries [16]. This is particularly evident for deeply nested queries involving multiple joins, aggregations, and conditional clauses. The challenge becomes even greater when models must operate on very large databases containing noisy or inconsistent data, poorly normalized schemas, or non-descriptive attribute and key names. These real-world conditions introduce ambiguity and reasoning difficulties that current systems have yet to fully overcome. Motivated by these limitations, this work investigates whether curriculum learning can enhance the ability of code-oriented LLMs to handle complex Text-to-SQL cases. We focus particularly on queries that involve sophisticated logical reasoning, such as deeply nested subqueries, multiple table joins, and intricate filtering conditions, as well as scenarios involving large, noisy, or poorly structured real-world schemas, exactly as the samples present in the BIRD benchmark. Our primary contributions are the following: • A comparative study of direct sorting, single-stage finetuning, and multi-stage curriculum learning approaches. • LeGo-Code, A modular training approach using multiple adapters specialized for different query complexities. • An empirical evaluation defining the contexts in which curriculum learning yields improvements for Text-to-SQL tasks, alongside an analysis of its limitations. The remainder of this paper is organized as follows: Section 2 reviews related work relevant to text-to-SQL. Section 3 details the different curriculum learning strategies we employed for textto-SQL. Section 4 presents the experimental setup and comparative analyses against direct finetuning. Section 5 discusses the key findings, and finally, Section 6 concludes the paper and outlines potential directions for future research.
2
Related Work
Translating natural language questions into their correct executable SQL queries, while respecting the database schema and capturing the true intent of the request, requires a deep understanding of natural language to resolve ambiguities, as well as strong knowledge of code, SQL syntax, data types, constraints, and database structures. Prior work has explored sketch-based slot-filling methods [9, 11]. These approaches convert SQL generation into multiple classification tasks by utilizing predefined SQL query templates and predicting their components sequentially. While decomposing the text-to-SQL task into subtasks facilitates the generation of syntactically correct SQL queries, it also introduces several limitations, particularly reduced flexibility in handling diverse SQL query structures due to the rigid nature of the predefined sketches. Others
Salmane Chafik, Saad Ezzini, and Ismail Berrada
focused on Graph based methods [4, 5], employing either Graph Neural Networks (GNNs) or modified transformer architectures. These methods represent the database schema as a graph, capturing relationships between tables, their columns, and inter-table dependencies. By incorporating structural information through graph representations, researchers achieved modest performance improvements, though these gains often fell short of practical requirements. In the same context, other works have explored the use of intermediate languages [13, 19], either pre-existing ones or new ones, specifically designed for the text-to-SQL task. These intermediate representations are typically closer to natural language, providing a bridge between the input question and the target SQL query. By translating natural language into an intermediate form first, and then converting it into SQL, these methods aim to simplify the generation process and improve semantic alignment. The effectiveness of these methods depends heavily on the expressiveness of the intermediate language and the accuracy of the mapping from intermediate forms to executable SQL. With the introduction of several large-scale datasets, the field of text-to-SQL has progressed significantly. Notable benchmarks include WikiSQL [26], which provides NLQ–SQL pairs over single tables; Spider [25], the first handcrafted dataset featuring substantially more complex queries than WikiSQL, including multi-table joins and nested SQL statements; and BIRD [16], a challenging benchmark that introduces very large database schemas, dirty values, and real-world scenarios. BIRD contains highly complex queries, often involving up to eight joins, deeply nested structures, sophisticated aggregation functions, and statistical computations, making it the most demanding benchmark to date. Together with numerous multilingual and synthetic datasets [3, 6–8, 10, 12], these resources have provided the foundation for code-oriented LLMs [15, 17, 18] to achieve state-of-the-art performance and dominate text-to-SQL leaderboards. Recent research [21, 22] has demonstrated that prompt engineering can significantly improve the performance of LLMs on the text-to-SQL task by better guiding the model’s reasoning process. These approaches leverage in-context learning, where carefully selected demonstrations are provided at inference time, as well as chain-of-thought (CoT) prompting strategies that encourage stepby-step reasoning before generating the final SQL query. Similarly, other researchers have sought to enhance LLM performance without modifying model architectures, particularly through curriculum learning. This paradigm trains models progressively, starting with simpler text-to-SQL examples and gradually increasing difficulty to medium and highly complex queries. Recent work [27] explored curriculum learning for text-to-SQL using a synthetic dataset generated by both closed-source and open-source LLMs. Their experiments reported performance improvements when adopting a curriculum-based training strategy. However, the reliance on synthetically generated data, produced through prompting heuristics and LLM outputs, may introduce distributional biases and may not fully capture the diversity and realism of user-generated queries. Furthermore, their study employed a single-stage curriculum learning strategy, leaving open the question of how different curriculum designs might influence model generalization and robustness.
LeGo-Code: Can Modular Curriculum Learning Advance Complex Code Generation? Insights from Text-to-SQLEASE 2026, 9–12 June, 2026, Glasgow, Scotland, United Kingdom Dataset SPIDER BIRD
Subset Train Dev Train Dev
N° Ex 7 000 1 034 6 604 1 534
N° DB 140 20 69 11
N° Tab/DB 5.26 4.05 5.68 6.81
N° Col/TB 5.22 5.44 7.15 10.64
N° FK/DB 4.9 3.2 8.16 9.27
Table 1: Schema Complexity Comparison of SPIDER and BIRD Datasets. Overall, prior work has improved text-to-SQL through structured decoding, graph modeling, intermediate languages, and prompting strategies. However, the relative effectiveness of different training paradigms for handling varying query complexities remains underexplored. In particular, there is limited systematic comparison between direct fine-tuning, and curriculum learning strategies, and little investigation into modular training methods tailored to query difficulty. To address these gaps, we conduct a comparative study of these training approaches, introduce LeGo-Code, a modular adapterbased framework specialized for different query complexities, and provide an empirical analysis of when curriculum learning benefits text-to-SQL.
3
Approach
This section provides a detailed description of the datasets used, the sorting algorithm applied to organize samples from easy to complex, and the training strategies employed.
3.1
Datasets
In this work, we employ two cross-domain text-to-SQL benchmarks, SPIDER and BIRD, which differ totally in complexity, domain coverage, database scale, and real-world applicability. Together, they provide a comprehensive evaluation setting across varying levels of difficulty and practical use cases. SPIDER: A large-scale cross-domain text-to-SQL dataset containing 10,181 NLQ-SQL pairs corresponding to 5,693 unique SQL queries. The dataset spans 138 distinct domains and covers more than 200 databases, each consisting of multiple interrelated tables with complex schemas and numerous foreign-key relationships. SPIDER is the first large-scale hand-crafted benchmark for crossdomain semantic parsing, created by 11 computer science students to ensure high annotation quality and diversity. Its design requires models to generalize to unseen databases at test time, making it a challenging and widely adopted benchmark for both training and evaluation. The name “SPIDER” reflects its broad coverage across many domains, analogous to a spider traversing multiple interconnected nests (databases). BIRD: is a more recent text-to-SQL benchmark designed to reflect higher complexity and closer alignment with real-world database scenarios. It contains 12,751 unique NLQ–SQL pairs over 95 large-scale databases with a total size of approximately 33.4 GB. The dataset spans more than 37 professional domains, including blockchain, sports (e.g., hockey), healthcare, education, and finance. Unlike earlier benchmarks, BIRD emphasizes real-world characteristics: databases and queries are derived from practical applications and therefore include noisy inputs, incomplete or inconsistent values, abbreviations, domain-specific terminology, and complex schema structures. This makes BIRD particularly suitable for evaluating robustness, scalability, and practical deployment readiness of text-to-SQL systems.these use cases.
Table 1 presents schema-level statistics for the training and development subsets of SPIDER and BIRD, including dataset size and average structural properties of the underlying databases. Metrics such as tables per database, columns per table, and foreign keys per database provide insight into the relational complexity faced by text-to-SQL models. These statistics indicate that BIRD database schemas are generally more complex than those in SPIDER. For example, the BIRD development set contains databases with nine relationships on average, approximately seven tables, each with around eleven columns. In contrast, the SPIDER development set contains databases with only about three relationships, four tables, each with roughly five columns. This highlights the higher structural complexity and real-world difficulty of the BIRD benchmark compared to SPIDER. Additional complexity statistics are presented in the following subsection.
3.2
Curriculum Learning
Curriculum Learning (CL) is a method used in Machine Learning (ML) where training data is presented to the model in a specific order, typically from easy to complex examples. This approach is inspired by how humans learn, starting with simpler concepts and gradually progressing to more challenging ones. In standard machine learning, models usually receive training data in a random order. With curriculum learning, the model begins with simpler examples and progressively moves to harder ones, which can improve learning efficiency and lead to more accurate predictions. One major challenge in Curriculum Learning is defining what makes a training example “easy” or “hard.” However, in text-to-SQL tasks, it is relatively straightforward to assess the complexity of SQL queries based on their components. For example, queries that join multiple tables are more complex than those that access only a single table. To quantify this, we define a complexity score function 𝐶 (𝑄) that takes a query 𝑄 as input and outputs a real-valued score computed as a weighted sum over its key components. We extended the component weights from the work of [27] to include condition operators, aggregate functions, and, most importantly, the database size. Queries operating over larger database schemas are significantly more complex than those over smaller, simpler schemas. The detailed weights assigned to SQL keywords and functions are summarized in Table 2. Nested SQL queries are detected using an SQL parser named SQLGlot1 , and the database size score 𝐷𝐵_𝑠𝑐𝑜𝑟𝑒 is computed as (𝑑𝑏_𝑠𝑖𝑧𝑒 ∗ 2)/𝑚𝑎𝑥_𝑠𝑖𝑧𝑒 , where 𝑑𝑏_𝑠𝑖𝑧𝑒 is the number of characters in the database and 𝑚𝑎𝑥_𝑠𝑖𝑧𝑒 is the maximum database size in the dataset, resulting in a score between 0 and 2. The final complexity score is defined as follows: Í 𝐶 (𝑄) = 𝑘 ∈𝑄 𝑛𝑘 ∗ 𝑤𝑘 + 𝐷𝐵_𝑠𝑐𝑜𝑟𝑒 + 𝑁 𝑒𝑠𝑡𝑒𝑑_𝑠𝑐𝑜𝑟𝑒 (𝑄) Where 𝑛𝑘 , 𝑤𝑘 are, respectively, the number of occurrences and the weight of the SQL keyword or function 𝑘, and 𝑁 𝑒𝑠𝑡𝑒𝑑_𝑠𝑐𝑜𝑟𝑒 (𝑄) = 2 if the query contains a nested SQL subquery (0 otherwise). Following this, we merge the training sets of SPIDER and BIRD and sort the combined dataset using our scoring function, from easy to extra-hard samples. We denote the resulting dataset as SB-CL. For the subsequent training strategies, we require different subsets 1 https://github.com/tobymao/sqlglot
EASE 2026, 9–12 June, 2026, Glasgow, Scotland, United Kingdom
Keyword / Function WHERE AND/OR/NOR MAX/MIN/AVG/SUM COUNT/CAST/DISTINCT JOIN GROUP BY/HAVING ORDER BY/LIMIT UNION/INTERSECT/EXCEPT Nested SQL Queries Database Size
Salmane Chafik, Saad Ezzini, and Ismail Berrada
Weight 0.5 0.1 0.15 0.15 1.5 0.75 0.5 2 2 (db_size*2)/max_size
Table 2: Weights assigned to SQL keywords and functions Dataset SPIDER BIRD
Subset Train Dev Train Dev
Easy 2 561 411 840 188
Medium 1 957 263 1 441 269
Hard 1 113 163 2 288 709
Extra 1 369 197 2 032 368
Avg_Score 1.982 1.930 2.780 2.594
Table 3: Complexity Levels Comparison of SPIDER and BIRD. of this dataset; therefore, we partition SB-CL into four complexity sets of equal size (EASY, MEDIUM, HARD, and EXTRA). Summary statistics for these sets are presented in Table 3. These statistics highlight the complexity gap between SPIDER and BIRD, most SPIDER samples are concentrated in the EASY and MEDIUM sets, while BIRD samples are predominantly found in the HARD and EXTRA sets.
3.3
Training Strategies
Once the dataset is prepared, merged, and sorted, we evaluate several training strategies to study the effect of sample complexity and curriculum design on model performance. 3.3.1 LoRA Finetuning. In this approach, we perform standard finetuning without considering the complexity order of the samples. The dataset is shuffled, and the model is trained for multiple epochs over the entire dataset. To improve efficiency, we adopt ParameterEfficient Transfer Learning (PEFT) [14], specifically LoRA (LowRank Adaptation). In LoRA-based fine-tuning, the original model parameters are frozen, and only a small set of trainable adapter parameters is introduced. This significantly reduces computational cost and memory requirements compared to full fine-tuning, which is typically expensive for large language models. This setup serves as the baseline for comparison with the curriculumbased methods. 3.3.2 Single Stage Curriculum finetuning. In this method, we apply a single-stage curriculum learning strategy. The model is fine-tuned on SB-CL, our pre-ordered dataset, where samples are arranged from easy to complex. Unlike the baseline, the dataset is not shuffled. Samples are presented sequentially to ensure that the model encounters simpler examples first and gradually progresses to more difficult ones. This ordering is intended to stabilize training and facilitate incremental knowledge acquisition. As in the baseline, we employ PEFT with LoRA adapters, keeping the base model frozen and training only the adapter parameters. 3.3.3 Multi-Adapter based Curriculum finetuning. This approach implements a multi-stage curriculum using multiple adapters. The
SB-CL dataset is divided into four complexity groups: EASY, MEDIUM, HARD, and EXTRA. Training proceeds sequentially across stages: (1) Train Adapter 1 on the EASY subset while freezing the base model parameters. (2) Freeze the base model and Adapter 1, then train Adapter 2 on the MEDIUM subset. (3) Freeze the base model and the first two adapters, then train Adapter 3 on the HARD subset. (4) Freeze the base model and the first three adapters, then train Adapter 4 on the EXTRA subset. At the end of this process, the final system consists of the base model augmented with four adapters, each specialized for a specific complexity level. Because each adapter is trained on top of the previously learned representations, later stages benefit from knowledge acquired in earlier ones. This design offers substantial flexibility at inference time. Different adapters can be enabled, disabled, or combined depending on the desired performance characteristics, allowing the system to adapt to varying input complexity without retraining the base model. This finetuning pipeline is illustrated in Figure 1.
4
Evaluation
In this section, we evaluate the predefined methods described in Section 3.
4.1
Research Questions
We aim to address the following research questions: (1) RQ1: Does Curriculum Learning perform better than LoRA fine-tuning? In this research question, we investigate whether curriculum learning strategies outperform LoRA fine-tuning. (2) RQ2: What additional benefits does multi-adapter–based curriculum fine-tuning provide beyond overall performance improvement? This research question examines whether multi-adapter–based curriculum fine-tuning offers advantages beyond overall performance gains.
4.2
Implementation Details
To address the above research questions, we conducted a series of experiments on a computer equipped with one NVIDIA GeForce RTX 5090 with 32 GiB of memory. All experiments were implemented using the HuggingFace ecosystem2 , an open-source platform for natural language processing research. Specifically, we used: (1) the Transformers library, which supports inference, training, and fine-tuning of state-of-the-art pretrained models; (2) the Datasets library, which provides a unified interface for accessing and processing widely used NLP datasets; and (3) the Evaluate library, which offers standardized tools for computing evaluation metrics across machine learning tasks.
4.3
Experiments
To answer our research questions, we conducted the following experiments: 2 https://huggingface.co/
LeGo-Code: Can Modular Curriculum Learning Advance Complex Code Generation? Insights from Text-to-SQLEASE 2026, 9–12 June, 2026, Glasgow, Scotland, United Kingdom
Adapter 4 Adapter 3 Adapter 2 Adapter 1 Base Model Frozen Step 1: EASY
Step 2: MEDIUM
Step 3: HARD
Trainable
Step 4: EXTRA
Figure 1: Multi-adapter Curriculum finetuning EX1: In this experiment, we fine-tune Qwen2.5-Coder-1.5B [15], a recent coding LLM that demonstrates strong performance across multiple programming tasks, particularly text-to-SQL, using the three fine-tuning strategies detailed in Section 3. First, we employ LoRA fine-tuning, allowing the dataset to be shuffled without considering the order of samples, as in standard fine-tuning. Second, we apply single-stage curriculum fine-tuning, where we disable shuffling and ensure that the model processes the samples in increasing order of complexity (one epoch). Finally, we employ multi-adapter curriculum fine-tuning with multiple stages, dividing SB-CL into four subsets, EASY, MEDIUM, HARD, and EXTRA, and fine-tuning consecutively on each subset. The choice of a small model is intentional, it enables faster experimentation cycles and lower inference latency, while allowing us to focus on evaluating the proposed method independently of large model scale.
W2
W3 W4
W1
Adapter 1
Adapter 3
Adapter 2
Adapter 4
stages can provide improved performance compared to using a single adapter. This experiment is designed to demonstrate the flexibility and effectiveness of LeGo-Code, highlighting its ability to dynamically assemble specialized components to address tasks of different complexity levels. The results provide insight into how modular adapter composition can enable more efficient SQL query generation at each complexity level.
4.4
Answers to RQs
4.4.1 RQ1. Table 4 illustrates the execution accuracy of different training strategies detailed in Section 3. Qwen2.5-Coder-1.5BInstruct shows relatively low performance, achieving 44.3% on the SPIDER development set and 9.91% on the BIRD development set. LoRA fine-tuning using SB-CL substantially improves performance, yielding gains of +12.8% on SPIDER and +6.71% on BIRD. However, unexpectedly, single-stage curriculum learning, where the model is exposed to all SB-CL samples strictly in increasing order of complexity, results in a significant drop in performance on SPIDER (down to 34.1%), particularly on simple SQL queries, while producing only a modest improvement of +3.13% on BIRD. This performance remains considerably worse than that of LoRA fine-tuning. LeGo-Code, our approach based on multi-adapter fine-tuning applied sequentially across four SB-CL subsets from easy to extra hard, achieves the best results. It outperforms LoRA fine-tuning by +2.0% on SPIDER and +2.28% on BIRD, reaching 59.1% and 18.90% execution accuracy, respectively. Overall, the results demonstrate that staged multi-adapter training with progressively harder curricula is more effective than both standard fine-tuning and single-stage curriculum learning for improving execution accuracy on both benchmarks. 4.4.2 RQ2. To investigate whether LeGo-Code provides benefits beyond overall performance improvement, Figure 3 presents the
Base Model
Figure 2: LeGo-Code during Inference EX2: In this experiment, we evaluate multiple combinations of the adapters obtained from the previous experiment to investigate how LeGo-Code can be applied across different levels of task complexity and to determine which adapter combinations perform best for each level as depicted in Figure 2. Specifically, we analyze whether composing adapters trained on varying difficulty
Model SPIDER BIRD Qwen2.5-Coder-1.5B 44.3 9.91 LoRA finetuning 57.1 16.62 Single Stage CL 34.1 13.04 LeGo-Code 59.1 18.90 Table 4: Different training strategies Execution Accuracy on SPIDER and BIRD development sets
EASE 2026, 9–12 June, 2026, Glasgow, Scotland, United Kingdom
Salmane Chafik, Saad Ezzini, and Ismail Berrada
EASY
execution accuracy obtained when activating a single adapter at a time for each difficulty level of the BIRD development set. The results reveal strong specialization effects: each adapter performs best on the difficulty level it was trained on, while performance degrades on other levels. For example, Adapter 1, trained on the EASY subset, achieves 32.45% on EASY queries, substantially outperforming the baseline (25.00%) by +7.45%. However, its performance drops sharply on harder subsets (e.g., 2.72% on EXTRA). Similarly, Adapter 2 performs best on MEDIUM queries (22.30%), Adapter 3 on HARD queries (16.50%), and Adapter 4 shows comparatively better results on EXTRA queries (8.70%). These findings indicate that multi-adapter curriculum fine-tuning enables the model to learn difficulty-specific skills, with each adapter capturing knowledge tailored to a particular complexity level. In contrast, the single baseline model exhibits more uniform but generally lower performance across levels. Overall LeGo-Code provides modular specialization across difficulty levels. Rather than learning a single compromise solution, the model develops distinct capabilities for easy, medium, hard, and extra-hard queries. This enables more targeted reasoning, reduces interference between tasks of different complexity, and suggests the potential for adaptive inference strategies that select the most appropriate adapter based on query difficulty. Adapter 1
Adapter 2
Adapter 3
Adapter 4
Baseline
32.45
18.09
14.89
12.77
25.00
30
MEDIUM
14.13
22.30
13.38
13.38
18.96
HARD
8.18
19.32
16.50
14.25
17.95
EXTRA
25
2.72
8.15
7.34
8.70
8.15
20 15 10 5
Figure 3: Execution accuracy comparison of various adapters across difficulty levels on BIRD dev.
5
Discussion
Our evaluation of different training strategies shows that singlestage curriculum learning underperforms LoRA fine-tuning and even degrades performance on SPIDER. This might be due to catastrophic forgetting of simpler SQL patterns as training progresses toward harder examples, as well as insufficient revisiting of earlier samples, an effect mitigated in standard fine-tuning. These results indicate that data ordering alone does not guarantee improved learning. In contrast, LeGo-Code, which applies sequential multiadapter fine-tuning across difficulty levels, achieves the best overall performance. By allocating separate adapters to different complexity stages, the approach preserves knowledge acquired at each stage
while enabling later adapters to focus on more complex reasoning. Adapter-level analysis on the BIRD development set reveals strong specialization: each adapter performs best on queries matching its training difficulty and worse on others, suggesting that the method decomposes the task into difficulty-specific competencies.
6
Conclusion
The text-to-SQL field has seen significant progress in recent years with the introduction of new architectures, datasets, and benchmarks. Despite these advances, state-of-the-art models still struggle with complex, real-world queries over noisy and heterogeneous databases. In this paper, we examined curriculum learning strategies by comparing standard fine-tuning, single-stage curriculum learning, and a proposed multi-adapter approach, LeGo-Code. Our results show that naïve curriculum learning based solely on data ordering can be ineffective and may even degrade performance, likely due to the forgetting of simpler patterns. In contrast, LeGo-Code consistently achieves the best performance across benchmarks by sequentially fine-tuning specialized adapters on progressively more difficult subsets. These findings suggest that curriculum learning becomes substantially more effective when combined with modular adaptation mechanisms rather than relying on ordering strategies alone.
References [1] Anthropic. 2026. Claude Code. https://claude.com/product/claude-code [2] Anysphere. 2026. Cursor: The best way to code with AI. https://cursor.com [3] Daria Bakshandaeva, Oleg Somov, Ekaterina Dmitrieva, Vera Davydova, and Elena Tutubalina. 2022. PAUQ: Text-to-SQL in Russian. In Findings of the Association for Computational Linguistics: EMNLP 2022. Association for Computational Linguistics, 2355–2376. [4] Ben Bogin, Jonathan Berant, and Matt Gardner. 2019. Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy, 4560–4565. [5] Ruichu Cai, Jinjie Yuan, Boyan Xu, and Zhifeng Hao. 2021. SADGA: StructureAware Dual Graph Aggregation Network for Text-to-SQL. In Advances in Neural Information Processing Systems, Vol. 34. 7664–7676. [6] Salmane Chafik, Saad Ezzini, and Ismail Berrada. 2025. Dialect2SQL: A Novel Text-to-SQL Dataset for Arabic Dialects with a Focus on Moroccan Darija. In Proceedings of the 4th Workshop on Arabic Corpus Linguistics (WACL-4). 86–92. [7] Salmane Chafik, Saad Ezzini, and Ismail Berrada. 2025. Towards Automating Domain-Specific Data Generation for Text-to-SQL: A Comprehensive Approach. ACM Transactions on Software Engineering and Methodology (2025). [8] Salmane Chafik, Saad Ezzini, and Ismail Berrada. 2026. DarijaDB: Unlocking Text-to-SQL for Arabic Dialects. ACM Transactions on Asian and Low-Resource Language Information Processing (2026). [9] Li Dong and Mirella Lapata. 2018. Coarse-to-Fine Decoding for Neural Semantic Parsing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 731–742. [10] Longxu Dou, Yan Gao, Mingyang Pan, Dingzirui Wang, Wanxiang Che, Dechen Zhan, and Jian-Guang Lou. 2022. MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsing. arXiv:2212.13492 (Dec. 2022). [11] Catherine Finegan-Dollak, Jonathan K. Kummerfeld, Li Zhang, Karthik Ramanathan, Sesh Sadasivam, Rui Zhang, and Dragomir Radev. 2018. Improving Text-to-SQL Evaluation Methodology. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 351–360. [12] Jiaqi Guo, Ziliang Si, Yu Wang, Qian Liu, Ming Fan, Jian-Guang Lou, Zijiang Yang, and Ting Liu. 2021. Chase: A Large-Scale and Pragmatic Chinese Dataset for Cross-Database Context-Dependent Text-to-SQL. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2316–2331. [13] Jiaqi Guo, Zecheng Zhan, Yan Gao, Yan Xiao, Jian-Guang Lou, Ting Liu, and Dongmei Zhang. 2019. Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Florence, Italy, 4524–4535.
LeGo-Code: Can Modular Curriculum Learning Advance Complex Code Generation? Insights from Text-to-SQLEASE 2026, 9–12 June, 2026, Glasgow, Scotland, United Kingdom
[14] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In International conference on machine learning. PMLR, 2790–2799. [15] Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186 (2024). [16] Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Rongyu Cao, Ruiying Geng, Nan Huo, Xuanhe Zhou, Chenhao Ma, Guoliang Li, Kevin C. C. Chang, Fei Huang, Reynold Cheng, and Yongbin Li. 2023. Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs. arXiv:2305.03111 (2023). [17] Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel LamyPoirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, et al. 2024. Starcoder 2 and the stack v2: The next generation. arXiv preprint arXiv:2402.19173 (2024). [18] Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, et al. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950 (2023). [19] Ohad Rubin and Jonathan Berant. 2021. SmBoP: Semi-autoregressive Bottom-up Semantic Parsing. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Online, 311–324. [20] Yewei Song, Saad Ezzini, Xunzhu Tang, Cedric Lothritz, Jacques Klein, Tegawendé Bissyandé, Andrey Boytsov, Ulrick Ble, and Anne Goujon. 2024. Enhancing text-to-sql translation for financial system design. In Proceedings of the 46th
International Conference on Software Engineering: Software Engineering in Practice. 252–262. [21] Ruoxi Sun, Sercan Arik, Rajarishi Sinha, Hootan Nakhost, Hanjun Dai, Pengcheng Yin, and Tomas Pfister. 2023. Sqlprompt: In-context text-to-sql with minimal labeled data. In Findings of the Association for Computational Linguistics: EMNLP 2023. 542–550. [22] Chang-Yu Tai, Ziru Chen, Tianshu Zhang, Xiang Deng, and Huan Sun. 2023. Exploring chain of thought style prompting for text-to-sql. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 5376–5393. [23] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017). [24] Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, and Matthew Richardson. 2020. RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online, 7567–7578. [25] Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. 2019. Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. arXiv:1809.08887 (2019). [26] Victor Zhong, Caiming Xiong, and Richard Socher. 2017. Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning. arXiv:1709.00103 (2017). [27] Xiaohu Zhu, Qian Li, Lizhen Cui, and Yuntao Du. 2025. Learning SQL Like a Human: Structure-Aware Curriculum Learning for Text-to-SQL Generation. In Findings of the Association for Computational Linguistics: EMNLP 2025. 3545–3559.