JOURNAL OF LATEX CLASS FILES
1
MiniOpt: Reasoning to Model and Solve General Optimization Problems with Limited Resources
arXiv:2606.25832v1 [cs.LG] 24 Jun 2026
Ke Zhao∗ , Zixiang Di∗ , Hong Qian, Xiang Shu, Yaolin Wen, Qitao Shi, Bingdong Li, Xingyu Lu, Xiangfeng Wang, Jun Zhou, Ke Tang, and Yang Yu
Abstract—Achieving strong optimization generalization across diverse optimization problems while requiring limited training resources remains a challenging problem for optimization-oriented large language models (LLMs). Existing approaches typically rely on large-scale supervised datasets, costly reasoning annotations, and expensive intermediate step verification, resulting in substantial training overhead. To address these challenges, we propose MiniOpt, a reinforcement learning framework that learns to solve optimization problems through an reasoning-to-model-andsolve paradigm. MiniOpt decomposes optimization reasoning into structured optimization modeling and executable solver generation. Building upon this paradigm, we introduce OptReward, a reward function with hierarchical score structure that jointly evaluates formulation and solution, enabling effective policy learning without expert demonstrations. We further develop an optimization-oriented policy optimization strategy that improves exploration efficiency and stabilizes reinforcement learning for compact models. Extensive experiments show that MiniOpt3B exhibits strong optimization generalization across various optimization types, problem scenarios, and task domains. For models with fewer than 10B parameters, MiniOpt series achieves the highest average solving accuracy (SA). For models with more than 10B parameters, MiniOpt still shows competitive performance. These results suggest that optimization-oriented reward design and reinforcement learning provide an effective pathway for developing compact optimization-specialized language models with strong optimization generalization capabilities. The code is available at https://github.com/Hsiang-1/MiniOpt. Index Terms—Optimization Generalization, Reasoning to Model and Solve, Limited Resources, Large Language Models for Optimization.
I. I NTRODUCTION PTIMIZATION problems are ubiquitous in real-world scenarios, profoundly affecting diverse domains, such as industrial production and transportation planning [1], [2].
O
These authors contributed equally to this work. Corresponding author: Hong Qian. Ke Zhao, Zixiang Di, Yaolin Wen, Bingdong Li, Xiangfeng Wang, and Hong Qian are with East China Normal University, Shanghai 200062, China (E-mail: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected]). Xiang Shu, Qitao Shi, Xingyu Lu, and Jun Zhou are with AntGroup, Hangzhou 310000, China. (E-mail: [email protected], [email protected], [email protected], [email protected]). Ke Tang is with Southern University of Science and Technology, Shenzhen 518055, China. (E-mail: [email protected]). Yang Yu is with Nanjing University, Nanjing 210023, China. (E-mail: [email protected]). This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. © 2026 IEEE.
While traditional optimization solvers are efficient, their application heavily relies on expert knowledge, requiring the manual conversion of problems described in natural language into precise mathematical formulations and executable code. This process is both time-consuming and non-generalizable. The rise of LLMs has opened new pathways for the automated modeling and solving of optimization problems using natural language descriptions [3]–[5], making them more applicable to application scenarios. Representative work like LLMOPT [6], BPP-Search [7], and Text2Zinc [8] significantly advances this field by parsing natural language descriptions into structured formulation that provide a unified representation of optimization problems, and subsequently generating solver code efficiently. However, deploying such LLM-based approaches faces three critical bottlenecks. First, ensuring accurate text generation with a small-scale model requires Supervised Fine-Tuning (SFT) with a large amount of high-quality training data [9], [10]. However, obtaining such data is difficult, often requires considerable time and effort, and is prone to errors [11]. Second, it is challenging to verify whether the result generated meets the requirements [12], [13]. The non-verifiability of this task also exposes the limitations of the learning-based solving paradigm. Previous methods often incorporate reflection or debugging mechanisms [14], [15] during the generation process, which significantly multiply the computational overhead. Finally, considering data privacy issues, local deployment of small-scale LLMs with strong performance and reducing training costs is important, including the volume and quality of training data. These limitations pose significant barriers to deploying LLMs with strong optimization-solving capabilities in small and medium-sized enterprises and even on mobile devices. To address these challenges, we propose MiniOpt, an RLVR-based methodology designed to enhance the optimization generalization of small-scale LLMs under limited data and training resources. Following LLMOPT [6], we use optimization generalization to denote the ability of a model to formulate and solve previously unseen optimization problems across different optimization types and scenarios. We systematically design the training process, training data, and algorithms of the model, introducing the reasoning-to-model-and-solve paradigm. Specifically, we adapt the classic “model and solve” paradigm of LLMs for optimization problems into a verifiable Chain-of-Thought (CoT) for Reinforcement Learning (RL), thereby facilitating effective RLVR training. To efficiently utilize limited data, we propose a two-stage RL framework
JOURNAL OF LATEX CLASS FILES
that divides the training into progressive stages aligned with the evolution of model capabilities. Initially, we perform MidTraining on a small subset of annotated data to prevent training collapse induced by reward sparsity. Subsequently, the first RL stage employs a training set of lower difficulty to enable the policy model to master the reasoning-to-model-and-solve paradigm. The second stage then further enhances performance and optimization generalization on more challenging datasets. Addressing the scarcity of annotated CoT data and the associated verification challenges, MiniOpt employs fiveelement tuple modeling as a verifiable structural abstraction of the CoT within the RL framework. This approach mitigates verification complexity while simultaneously enhancing result accuracy and model generalization. Furthermore, we design a hierarchical structured reward function, OptReward, which facilitates robust RLVR. Specifically, it incorporates a modeling completeness verification mechanism to circumvent the need for content verification of the optimization modeling, while simultaneously constraining the RL reasoning trajectory in the vicinity of the correct modeling formulation to mitigate reward hacking. Building upon this systematic architectural design, we also introduce tailored modifications to the GPRO algorithm, thereby fully leveraging limited samples to conduct highly efficient training and achieve robust optimization generalization. Ultimately, the aforementioned synergistic design of the RLVR framework significantly reduces the reliance on annotated data, as well as the overall training and inference costs. Building upon the aforementioned methodology, this paper conducts extensive experiments with MiniOpt-3B on 8 benchmarks across different optimization types and problem scenarios. The results demonstrate its strong optimization generalization. Compared with baselines with fewer than 10B parameters, MiniOpt-3B achieves the best performance. When evaluated against baselines exceeding 10B parameters, MiniOpt-3B remains competitive compared to LLMOPT14B, and surpasses the general thinking model GPT-5 by 2.11 percentage points. MiniOpt-3B achieves a competitive average SA compared with DeepSeek-R1 while consuming only about 24.40 percentage points of the output tokens used by DeepSeek-R1 across benchmarks of varying difficulty. Notably, MiniOpt lies on the empirical Pareto frontier of both parameter scale and the SA metric. Furthermore, results from ablation studies and discussion indicate that applying RLVR within the reasoning-to-model-and-solve paradigm yields substantial performance gains. This approach enhances the model’s proficiency in solving optimization problems while preserving generalization. The subsequent sections review the related work, introduce MiniOpt, present experimental results and analysis, provide an in-depth discussion, and finally conclude the paper. II. R ELATED W ORK LLMs for Modeling and Solving Optimization Problems. For modeling and solving optimization problems with LLMs, there are already a variety of benchmarks [14], [16], [17]. Challenging benchmarks like Mamo [16] and OptiBench [18]
2
have led to numerous studies utilizing LLMs to solve optimization problems. Prompt-based approaches such as OptiMUS [14], CoE [19], OptiTree [20] and LEAN-LLM-OPT [21] utilize the powerful generation capability of LLMs to generate the solver code of the optimization problem through multistage pipeline, without performing any post-training. Learningbased methods enhance LLMs’ capabilities in modeling and solving mathematical problems. For example, LLaMoCo [22] proposes an SFT-based framework comprising a meticulously designed instruction set and a two-stage training methodology that incorporates contrastive learning warm-up followed by SFT. LLMOPT [6] and NER4OPT [8], [23] adopt a twostage training process of modeling the optimization problems first and then solving them by generating solution code. StepORLM [24] synthesizes a large volume of process data to simultaneously learn a policy model and a Process-RewardModel (PRM), employing alternating training to produce a modeling-and-solving model with reliable process consistency. Similarly, BPP-Search [7] generates process data and trains a PRM alongside a preference selection model under a Treeof-Thought search paradigm, thereby simulating reasoning processes to derive robust results. Reinforcement Learning with Verifiable Reward. While Reinforcement Learning from Human Feedback (RLHF) [25] plays a crucial role in post-training alignment, it suffers from high annotation costs and inherent human bias [11]. Reinforcement Learning with Verifiable Reward (RLVR) [26] leverages externally grounded, easily verifiable rewards (e.g., rule-based reward) to provide dense and structurally simple supervision [27]–[29]. Its practicality is especially valuable in real-world black-box systems [30]–[32], where verification is typically feasible only at the output stage, making RLVR a broadly applicable paradigm for aligning LLMs. For example, SIRL [33] and OR-R1 [34] significantly improve the model’s performance through the RLVR training paradigm. III. M ETHODOLOGY: T HE P ROPOSED M INI O PT A. Overview This paper studies how to endow small-scale LLMs with strong optimization generalization under tight data and compute budgets. We introduce MiniOpt, a reasoning-driven paradigm for modeling and solving optimization problems, whose training framework is shown in Figure 1. MiniOpt integrates a five-element tuple structure into the Chain-ofThought (CoT) and leverages a hierarchical structured OptReward to achieve RLVR training with strong optimization generalization. Through this framework, MiniOpt formulates the pipeline from natural language problems to executable solver code as a single, end-to-end verifiable task. B. Reasoning-to-Model-and-Solve Paradigm As shown in subfigure (a) in Figure 1, we introduce a reasoning-to-model-and-solve paradigm that turns a natural language optimization problem into a single verifiable objective. The paradigm is enforced by two compulsory and parsable segments <think> . . . </think> and <answer> . . . </answer>. The first segment, enclosed by <think>
JOURNAL OF LATEX CLASS FILES
3
(a) Reasoning to Model and Solve Paradigm
(c) Efficient Training for Optimization Generalization
Optimization Problem:
OptReward Make Intermediate Process Easy to Verify
A school is planning its budget for the upcoming academic year. The main expense categories are...
MiniOpt
Format Score
The integrity of the <think> tags in the response
<think> We need to formulate it as an integer linear programming problem ... **Five-Element Model**: - ## Variables: ... - ## Objective: ... - ## Constraints: ... - ## Sets: ... - ## Parameters: ... ... The best choice is ``cbc`` solver ... </think> <answer> ```python from pyomo.environ import … ``` </answer>
OptReward
Five-element Score The integrity of the five-element math model
Reasoning Rationality Modeling Correctness Solving Accuracy
Accuracy Score The executability and solving accuracy of code
OptGRPO Adapting GRPO to RL Training for Optimization Problems 𝒥𝒥𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂 𝜃𝜃 = 𝔼𝔼𝑞𝑞~𝑃𝑃 𝑄𝑄 ,{𝑜𝑜 }𝐺𝐺 ~𝜋𝜋 𝑖𝑖 𝑖𝑖=1
𝑜𝑜𝑜𝑜𝑜𝑜 (�|𝑞𝑞)
𝐺𝐺 |𝑜𝑜𝑖𝑖 |
1 �� ∑𝐺𝐺𝑖𝑖=1 |𝑜𝑜𝑖𝑖 | 𝑖𝑖=1 𝑡𝑡=1
𝜋𝜋𝜃𝜃 (𝑜𝑜𝑖𝑖,𝑡𝑡 |𝑞𝑞, 𝑜𝑜𝑖𝑖,<𝑡𝑡 ) 𝜋𝜋𝜃𝜃 (𝑜𝑜𝑖𝑖,𝑡𝑡 |𝑞𝑞, 𝑜𝑜𝑖𝑖,<𝑡𝑡 ) 𝐴𝐴̂ , 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 , 1 − 𝜀𝜀𝑙𝑙𝑙𝑙𝑙𝑙 , 1 + 𝜀𝜀ℎ𝑖𝑖𝑖𝑖𝑖 𝐴𝐴̂ 𝑖𝑖,𝑡𝑡 ] min[ 𝜋𝜋𝜃𝜃𝑜𝑜𝑜𝑜𝑜𝑜 (𝑜𝑜𝑖𝑖,𝑡𝑡 |𝑞𝑞, 𝑜𝑜𝑖𝑖,<𝑡𝑡 ) 𝑖𝑖,𝑡𝑡 𝜋𝜋𝜃𝜃𝑜𝑜𝑜𝑜𝑜𝑜 (𝑜𝑜𝑖𝑖,𝑡𝑡 |𝑞𝑞, 𝑜𝑜𝑖𝑖,<𝑡𝑡 )
MiniOpt trains the small-scale LLM with high optimization generalization performance. The five-element math model is automatically generated and a solver is adaptively selected. The Pyomo code generated by the model can accurately model and solve the original problem.
Remove KL penalty to discover high-reward behaviors quickly
The asymmetric clipping Accumulate loss over tokens allows the rare but crucial to improve sample efficiency reasoning/code token to grow under limited budgets
(b) MiniOpt Training Pipeline with SFT Warm-up and Two-stage RL Mid-training SFT Warm-up
LLM-3B Endow the model with
Small-scale knowledge in the Model operational optimization
RL Stage-1
RL Stage-2
Fast-adapting
Peak-chasing
OptReward, OptGRPO
OptReward, OptGRPO
Rapidly learn the model-and-solve paradigm
Optimization generalization improvement
MiniOpt-3B
Strong Optimization Generalization & Low Inference Cost
Fig. 1: An overview of the proposed MiniOpt training paradigm. Sub-figure (a) demonstrates the reasoning-to-model-and-solve paradigm of MiniOpt, encompassing problem modeling and solver adaptation during the thinking (i.e., reasoning) process of RL, and solution code generation in the response. Sub-figure (b) illustrates the training pipeline of MiniOpt, which involves sequential execution of SFT (mid-training) as warm-up followed by two-stage RL (post-training). Sub-figure (c) presents the reward function OptReward and training algorithm OptGRPO used in MiniOpt’s RL training.
. . . </think>, contains all modeling-related contents. It analyzes the problem statement, specifies a verifiable five-element optimization formulation, and determines an appropriate opensource solver. Specifically, the optimization problem can be described as the following formulation: min
x∈X ⊆RD
f (x) , s.t. G(x) ≤ c ,
(1)
where x ∈ X ⊆ RD denotes the D-dimensional decision variable, and I = {1, 2, . . . , D} is the index set. X is the feasible region and c ∈ Rm provides the upper bounds. The constants in objective function f (x) and constraints G(x) form the parameter set, which also includes c. The five-element formulation, M =(Variables, Objective, Constraints,Sets, Parameters) maps one-to-one to the components of an optimization problem. Sets determine the dimensions and naming of decision and constraint families. Parameters supply exogenous constants such as costs, coefficients, budgets, and demands. Variables specify domains and bounds (e.g., continuous, nonnegative, integer, or binary), which jointly define the feasible region X . Domain-type restrictions such as “positive integers” may equivalently be encoded as explicit constraints, and our parser maps both styles to X . Objective gives the minimization or maximization expression, and Constraints provide
named families of linear or nonlinear equalities/inequalities composing G and the bound vector c. During the reasoning process, to enhance the capability of LLM for solving optimization problems across diverse optimization types, it analyzes the problem during the reasoning phase and selects different open-source solvers for the given problem depending on the optimization types and the characteristics of the solvers, thereby improving the match between the problem and the selected solver. Solver selection is guided by a prompt, which is introduced in the supplementary material. The second segment, enclosed by <answer> . . . </answer>, converts the five-element formulation into an executable Pyomo program that models the problem, invokes the solver, solves the instance, and prints the numerical answer. By constraining responses in this manner, we collapse the action space from free-form language to a programmable artifact whose intermediate structure and final result can be deterministically parsed and verified. The <answer> segment must implement the blueprint as a single Python code fence that contains a complete Pyomo script. Because these outputs are easy to parse and verify, OptReward in Section III-D1 can automatically score the response format, the five-element formulation and the numerical accuracy based on rules through
JOURNAL OF LATEX CLASS FILES
an automated procedure, providing low-cost supervision. The prompt used for this paradigm is provided in the supplementary material. C. Training Pipeline of MiniOpt Based on the paradigm described in Section III-B and the key techniques introduced in Section III-D, we propose a training pipeline that enables MiniOpt to learn from limited resource constraints while achieving powerful solving performance and strong optimization generalization. This pipeline begins with a lightweight SFT as mid-training followed by a two-stage RL under OptReward. The illustration of the training pipeline is shown in subfigure (b) of Figure 1. 1) Mid-Training based on SFT: LLMs acquire broad knowledge through large-scale general pretraining. However, for specialized domains, small-scale models often struggle to cultivate the necessary capabilities effectively or comprehensively. Mid-training serves as a bridge between pretraining and subsequent training stages, strengthening domain-specific skills through targeted training data [35]. In the proposed training pipeline, the mid-training phase employs a small-scale and diverse dataset from the operational optimization field to conduct SFT on the base model, thereby providing effective modeling and solving trajectories for the subsequent two-stage RL training. This SFT warm-up does not adopt the reasoning-to-modeland-solve paradigm introduced in Section III-B. Instead, it establishes a starting point upon which our two-stage RL can subsequently focus on paradigm acquisition and optimization generalization. For each instance, we prompt Qwen2.5-Coder32B-Instruct [36] using the prompt in supplementary material to rewrite the original GurobiPy program into an equivalent Pyomo implementation and to select an appropriate opensource solver according to the detected structure. Every rewritten instance is executed and only those that compile, solve, and print the correct structure are retained, the obtained dataset is referred to as OptMATH-Train-Pyomo. The remaining data serves as a candidate pool for the next stage of the twostage RL training. Based on the annotations of optimization types and problem-scenario tags assigned to each sample in this paper, the training data for mid-training are constructed from a small-scale subset of OptMATH-Train-Pyomo. Within this subset, we ensure an equal number of linear and nonlinear problems. Since nonlinear problems are scarce in OPTMATHTrain, we supplement the shortfall with MILP problems in the dataset to reach the desired equal distribution. Additionally, all samples labelled as ”unknown” are included to preserve both the difficulty and diversity of the data source during the warm-up training phase. The prompt used for data annotation is provided in supplementary material. 2) The Two-Stage RL: After the SFT phase, the models can undergo RL training more effectively. We employ a two-stage RL training under the same reward function (OptReward) for paradigm acquisition and optimization generalization, respectively. Both stages share the pipeline of parsing, executing, and scoring in Section III-B and the OptReward of Sections III-D1, they differ in training data and hyperparameter setting. Algo-
4
rithmically, both stages use the same OptGRPO (cf. paragraphs in Section III-D2). Stage-1 aims to enable the model to acquire the reasoningto-model-and-solve paradigm: the model must generate a valid <think> / <answer> pair, produce the executable Pyomo code, and make a coherent solver choice. To this end we train on 1,585 relatively easy problems, so that most signal arises from the formatting and structural components of OptReward, rapidly improving executability and the ability to solve the optimization problem of natural language description. Stage-2 focuses on optimization generalization once the paradigm is established. The training distribution shifts to the problems with diverse optimization types and problem scenarios, and the optimization emphasis moves to the accuracy score, encouraging refined modeling and solving behaviors (e.g., variable/constraint formulation and solver selection). We sample a data subset from the candidate pool mentioned in Section III-C1 subject to two constraints: (i) type-uniform coverage, with exactly 600 instances per type, and (ii) within each type, the scenario frequencies match the distribution in the full dataset. The resulting data serves as the Stage-2 RL training set. The dataset for the first stage is the union of the NL4Opt [37] and ICML Competition [38] training splits. Detailed information on the construction of the training set is provided in the supplementary material. Such a training strategy makes efficient use of limited data and reduces training costs, and ultimately allows for strong optimization generalization of LLMs with even small parameters and limited computational resources. D. Efficient Training for Optimization Generalization Building upon the reasoning-to-model-and-solve paradigm mentioned in Section III-B, we propose two key components for the RL training of MiniOpt as shown in subfigure (c) in Figure 1: An informative and easily verifiable reward function OptReward, and an improved algorithm OptGRPO builds upon GRPO [39]. 1) OptReward: Verifiable Rewards Designed for MiniOpt: Building upon the reasoning-to-model-and-solve pipeline, we propose OptReward as the core mechanism for RLVR training, serving not only as a verification tool for optimization modeling but also as a gating mechanism to control the stages of RL training. It comprises three automatically computed components: format correctness, structural completeness, and numerical accuracy. Each component is derived from the deterministic parsing or execution of the model’s output, which not only ensures the completeness of the problem modeling but also enables verification to scale with computational time, substantially reducing the overall verification cost. Format Score: The format score Sfmt validates the response format. A response must contain exactly one <think>. . . </think> and one <answer>. . . </answer> in the correct order. If all conditions hold, we assign Sfmt = +1, otherwise Sfmt = −1. If the specified format is not present in the response, we deterministically set the remaining components to their error defaults, Sfive = −1 and Sacc = −2, so that the total reward
JOURNAL OF LATEX CLASS FILES
5
immediately reaches the global minimum. This forces the model to adopt the correct response format early in training and prevents expensive evaluation of malformed samples. Five-element Score: To avoid ground-truth labeling of five-element content and the bias it may introduce, we use a presence-based rule aligned with the paradigm in Section III-B. Conditional on valid formatting, we compute the five-element score Sfive . In the <think> segment, the model response is expected to include five labelled summaries starting with “## Sets:”, “## Parameters:”, “## Variables:”, “## Objective:”, and “## Constraints:”. Each present summary contributes 0.2 points to the score; if none is present, we assign Sfive = −1. This structure shaping keeps the modeling blueprint parsable. 5 5 X X 0.2 I , if Ik ≥ 1 , k k=1 k=1 Sfive = (2) 5 X if Ik = 0 , −1, k=1
where Ik = 1 if the k-th required element ek is present in the <think> segment, and Ik = 0 otherwise. (e1 , . . . , e5 ) =(Sets, Parameters, Variables, Objective, Constraints). Accuracy Score: The accuracy score is obtained by executing the Pyomo program contained in <answer>. If the program cannot be extracted or execution fails, we assign Sacc = −2. When execution succeeds, we retrieve the optimal objective value fˆ from the model output and compare it with the ground-truth value f ⋆ ; if they are equal, we assign Sacc = 2, otherwise Sacc = −1.5. +2, Sacc = −1.5, −2,
if execution succeeds and fˆ = f ⋆ ,
the model’s ability to discriminate between correct and incorrect solutions. Similarly, the standard KL divergence penalty inherently restricts the exploration capacity of the policy model. To address these limitations, we set the coefficient on the KL penalty β = 0 to remove the KL penalty to encourage greater exploration by the model, and replace symmetric clipping with an asymmetric interval [1 − εlow , 1 + εhigh ] with a higher upper clipping threshold εhigh than the lower threshold εlow , to encourage the exploration of low-probability (yet potentially optimal) patterns. This relaxes the trust region on probability increases while keeping a firm lower bound on decreases, which empirically improves executability in stage-1 RL training and supports generalization in stage-2 RL training. Specifically, we raise εhigh to 0.28 during the training. Although a higher clipping threshold may potentially introduce high variance, training stability is empirically guaranteed because the modeling format of the rollouts is already strictly constrained by the five-element tuple structure within the CoT. Furthermore, we adopt a token-level policy gradient loss computation. This design amplifies the reward signals for valid solution patterns, ensuring that high-quality reasoning trajectories are not disproportionately penalized or underestimated, thereby providing a more precise and granular reward signal for the target solving behaviors. Ultimately, through OptGRPO, the reasoning-to-model-and-solve training paradigm is executed with significantly enhanced efficiency and efficacy. The final loss of OptGRPO is as follows:
JOptGRPO (θ) = Eq∼P (Q),{oi }G i=1 ∼πθ
By OptReward, the format score enforces the strict <think> / <answer> format, the five-element score shapes a complete modeling blueprint in the think phase, and the accuracy score certifies correctness through equality of optimal objective values, enabling low-cost verifiable RL for problems in the field of optimization. 2) OptGRPO: Training Small-Scale LLMs with Limited Resources: Considering the specific requirements of the task, we build upon the reasoning-to-model-and-solve paradigm and OptReward to refine the GRPO algorithm, introducing the OptGRPO. In the second stage of RL, MiniOpt is dedicated to exploring complex problem-solving patterns. In this context, a symmetrically designed upper bound for importance weight clipping can induce premature convergence, thereby hindering
old
|oi | G X X
(·|q) PG i=1 |oi | i=1 t=1
(5) min[ρθ Âi,t , clip(ρθ , 1 − εlow , 1 + εhigh )Âi,t ] ,
if execution succeeds but fˆ ̸= f ⋆ ,
if no executable code or execution fails . (3) Combining the components with the formatting gate yields the total OptReward as follows: ( −4, if Sfmt = −1, (4) R= Sfmt + Sfive + Sacc , if Sfmt = 1.
1
ρθ =
πθ (oi,t |q, oi,<t ) . πθold (oi,t |q, oi,<t )
(6)
IV. E XPERIMENTS We evaluate MiniOpt models on diverse optimization benchmarks spanning multiple types and scenarios to assess whether small-scale parameter LLMs (3B/7B) can achieve strong optimization generalization ability. To evaluate solution correctness, we report Solving Accuracy (SA) as the primary metric, SA measures the proportion of generated code samples whose execution results match the ground-truth optimal solutions provided by the benchmarks. A solution is deemed correct if the returned optimal objective value matches the reference solution within a tolerance of 10−1 ; otherwise, it is considered unsuccessful. Comparisons cover general LLMs, general reasoning LLMs, promptingbased baselines and learning-based baselines. The experiments are designed to answer the following four research questions. (Q1) Optimization Generalization Ability at Small-Scale LLMs. To what extent can MiniOpt at 3B/7B achieve high SA and ER across types and scenarios, and how does it compare with larger reasoning LLMs and prior learning-based approaches?
JOURNAL OF LATEX CLASS FILES
(Q2) Pareto Front of Performance vs. Cost. What is the limit of the scale of model parameters for achieving strong optimization generalization? (Q3) Importance of Each Stage in the Training Pipeline of MiniOpt. How critical are the lightweight SFT warm-up and the two-stage RL to the performance of MiniOpt? (Q4) Importance of the OptReward and OptGRPO. How does the proposed OptReward and OptGRPO in MiniOpt contribute to boost SA and ER in modeling and solving optimization problems? The four questions are answered sequentially in the following sections. We first provide a detailed description of the experimental setup and then present extensive analyses of the results. The code is available at https://github.com/Hsiang1/MiniOpt. A. Experimental Setup Since widely used packages such as Gurobi and COPT are closed-source, generating code for them may entail licensing costs, and a model’s generalization ability across different modeling languages and solvers depends on the proportion of relevant data in its pretraining and post-training corpora. Therefore, we adopted the open-source, solver-agnostic Pyomo modeling language as the language for our training and inference solver code. As for the solvers employed, the key consideration lies in their ability to select the appropriate solver for a specific optimization type. Therefore, this paper selects four types of solvers to cover the solving requirements of as many optimization problem types as possible, so as to automatically adapt to the problem types when generating the solving code. The training configuration of MiniOpt is as follows: SFT is conducted for 4 epochs; the first stage of RL involves 45 training steps, while the second stage consists of 25 steps. During inference, a temperature of 0.7 is applied. For additional experimental settings, please refer to the repository mentioned in the abstract of this paper. The evaluation encompasses eight benchmarks about operational optimization: NL4Opt [37], Mamo (Easy and Complex subsets, abbreviated as Mamo.E and Mamo.C, respectively) [16], IndustryOR [40], NLP4LP [14], ComplexOR [19], OptiBench [18], and ICML Competition (ICML.C) [38]. We follow the same setting in LLMOPT [6] to ensure consistency and comparability. For the newly included dataset, OptiBench and ICML Competition, we adhere to their original data splits provided by the authors. B. Analysis of Optimization Generalization In this section, we compare MiniOpt with general LLMs (Qwen2.5, DeepSeek-V3), general thinking LLMs (Qwen3, DeepSeek-R1, Gemini-2.5-Pro, GPT5), prompt-based methods (Chain of Experts, OptiMUS, Reflexion), learning-based methods (OptMATH-7B, LLMOPT-14B, Step-OPT-7B/3B), demonstrating the optimization generalization capability of MiniOpt. Table I summarizes SA on eight benchmarks that span 7 optimization types and 22 scenarios. The information on these methods and the statistics on problem categories and scenarios of benchmarks are provided in supplementary
6
material, as well as the Executable Rate metrics for all the methods on the 8 benchmarks and its analysis. Overall Performance (Answer to Q1). Across all eight benchmarks, MiniOpt-7B achieves the strongest average performance among all baselines. Notably, MiniOpt-3B surpasses all prompt-based and general methods. For example, compared to Gemini-2.5-Pro, the state-of-the-art general thinking model, MiniOpt-3B achieved an average SA of 2.26% higher. Compared to DeepSeek-V3 and DeepSeek-R1, MiniOpt-3B scored 0.49% and 1.20% lower, respectively. Furthermore, MiniOpt-7B achieves the highest average SA of 64.76%, significantly raising the performance ceiling of MiniOpt in practical applications. Competitiveness of Small-Scale Models (Answer to Q1). MiniOpt remains competitive even at smaller scales. MiniOpt3B reaches an average SA of 59.65%, which already matches several much larger reasoning models (e.g., the average SA higher than GPT-5 at 57.54%) and clearly outperforming general-purpose 14B pre-trained models (e.g., +12.19% over Qwen2.5-14B-Instruct on average). Performance grows smoothly with scale under the same training pipeline, the average SA increase by +5.11% when the model size grows from 3B to 7B. Challenging Benchmarks (Answer to Q1). On the most demanding sets that require faithful modeling and solver usage, MiniOpt shows clear advantages. On the latest challenging benchmark Mamo.Complex, MiniOpt-3B and -7B achieve 47.39% and 51.18% SA, respectively, outperforming all the general models and general thinking models. Even on extremely high-dimensional test sets such as Indus.OR, where the smaller MiniOpt models do not achieve the highest SA because of their limited instruction-following capability, it still delivers competitive performance levels in both metrics. Breadth across Types and Scenarios (Answer to Q1). To rigorously evaluate optimization generalization, we benchmark MiniOpt-3B and MiniOpt-7B across three difficulty tiers: simple (ICML.C), medium (OptiBench), and hard (Mamo.C). The evaluation encompasses diverse application scenarios (Supply Chain, Transportation, Resource) and problem formulations (LP, IP, MILP). The results highlight a clear scaling law within our framework. While MiniOpt-3B demonstrates competent baseline performance on simpler tasks (e.g., 94% in Transportation and 93% in Resource on ICML.C), MiniOpt-7B consistently bridges the gap on complex reasoning tasks. On the ICML.C, the 7B variant elevates MILP accuracy from 31% to 50%. This scaling advantage persists under increased difficulty; on the hard-difficulty Mamo.C benchmark, MiniOpt-7B maintains a 10% to 13% absolute improvement over the 3B model in LP, IP, and MILP formulations. Furthermore, both models exhibit remarkable robustness in Resource scenarios, with the 7B model sustaining a 75% success rate even on the most complex tasks. These findings validate that MiniOpt provides a scalable and reliable foundation for LLM-based optimization modeling. C. Pareto Front of Performance vs. Cost Analysis of the Pareto Front (Answer to Q2). Figure 2 indicates that the MiniOpt family (represented by the solid red
JOURNAL OF LATEX CLASS FILES
7
TABLE I: Comparison of the SA metric across 8 benchmarks with rankings (NL4Opt, ICML Competition, Mamo Easy, Mamo Complex, NLP4LP, ComplexOR, IndustryOR, OptiBench). Bold indicates 1st, wavy underline indicates 2nd, underline indicates :::::::::::: 3rd. “Rank∗ ” represents the result of sorting methods among parameter sizes below 10B. Category
Models / Methods
Avg.
Rank Rank∗ NL4Opt ICML.C Mamo.E Mamo.C NLP4LP Com.OR Indus.OR OptiBench
General Models
Qwen2.5-3B-Instruct Qwen2.5-7B-Instruct Qwen2.5-14B-Instruct DeepSeek-V3 (671B)
11.23 33.20 47.46 60.14
18 14 10 3
9 6 -
19.13 53.48 67.39 78.26
18.78 51.71 63.17 77.56
17.18 35.58 80.21 84.82
2.37 4.27 15.64 26.54
18.60 55.37 67.36 79.34
0.00 16.67 22.22 44.44
2.00 13.00 22.00 26.00
11.74 35.54 41.65 64.13
Qwen3-4B Qwen3-8B General Models Qwen3-14B (Thinking) DeepSeek-R1 (671B) Gemini-2.5-Pro GPT-5
11.16 21.79 23.78 60.85 ::: 57.39 57.54
19 16 15 2 7 6
8 7 -
16.52 30.87 24.35 83.91 78.26 80.43
17.56 29.51 22.68 75.37 71.22 73.66
13.80 23.93 36.20 74.54 65.95 58.12
6.64 9.95 11.37 39.81 30.81 23.22
15.29 34.71 19.42 69.83 73.55 73.14
5.56 11.11 38.89 44.44 50.00 ::: 61.11
2.00 6.00 15.00 32.00 28.00 26.00
11.90 28.26 22.31 66.94 61.32 64.63 :::
Prompt-based Methods
Chain-of-Experts OptiMUS Reflexion
45.78 20.65 45.54
11 17 12
-
66.52 13.48 56.52
56.59 33.17 52.20
63.65 37.27 84.82
22.75 11.85 18.01
59.09 18.18 53.72
33.33 16.67 38.89
19.00 8.00 19.00
45.29 26.61 41.16
Learning-based Models
Step-OPT-Qwen2.5-3B Step-OPT-Qwen2.5-7B OptMATH-7B LLMOPT-14B
39.76 52.22 54.62 60.10
13 9 8 4
5 4 3 -
41.30 77.83 78.70 80.28
38.54 57.32 66.83 75.35
75.31 69.33 84.20 89.53
20.85 50.24 ::: 34.12 44.08
53.31 48.35 68.60 73.42
27.78 38.89 33.33 35.29
21.00 27.00 19.00 29.00
40.00 48.76 52.23 53.83
MiniOpt-3B MiniOpt-7B
59.65 64.76
5 1
2 1
93.04 ::: 95.22
78.05 ::: 83.41
84.36 89.57
47.39 51.18
74.38 74.79
27.78 38.89
17.00 25.00
55.21 60.00
Ours
:::
:::
:::
TABLE II: Ablation study (MiniOpt-3B) on the SA metric across 8 benchmarks. Herein, “w” denotes “with” and “w/o” denotes “without”. Category
Model / Method
MiniOpt-3B MiniOpt-3B w/o SFT Warm-up MiniOpt-3B w/o RL MiniOpt-3B w/o Two-stage RL Ablations MiniOpt-3B w/ Random Selection MiniOpt-3B w/o OptReward MiniOpt-3B w/ GRPO
Avg.
NL4Opt ICML.C Mamo.E Mamo.C NLP4LP Com.OR Indus.OR OptiBench
59.65 56.60 27.24 55.07 58.33 57.96 56.54
93.04 89.13 40.00 89.13 93.04 89.13 90.43
78.05 76.83 35.37 73.41 79.02 76.34 77.07
line) lies on the empirical Pareto frontier among the compared open-weight methods in the performance-versus-cost trade-off. Since the parameter size of GPT-5 and Gemini-2.5-Pro have not been disclosed, we do not label these two models in the figures.
As the scale of the model increases, the average SA performance of MiniOpt also grows steadily. It achieves a comprehensive performance lead while having substantially fewer parameters than top-tier general reasoning models such as DeepSeek-R1 and learning-based model LLMOPT-14B. Compared to the similar modeling and solving model Step-OPT, Step-OPT-Qwen2.5-7B achieves an average SA of 52.22% across 8 benchmarks, while the proposed MiniOpt attains an average SA of 64.76%. When the parameter scale of both models is reduced to 3B, Step-OPT-Qwen2.5-3B exhibits a performance drop of 12.46%, while MiniOpt only decreases by 5.11%. This indicates that the key advantage of MiniOpt lies in its ability to maintain superior performance even with a reduced parameter scale. From the perspective of capability density [41], MiniOpt effectively achieves lower parameter requirements and inference costs while preserving comparable performance.
84.36 84.36 50.15 79.75 82.06 83.90 80.06
47.39 28.44 3.79 33.18 48.34 38.86 45.02
74.38 72.73 41.74 74.79 73.97 72.73 73.55
27.78 27.78 11.11 22.22 16.67 33.33 16.67
17.00 19.00 7.00 16.00 17.00 16.00 14.00
55.21 54.55 28.76 52.07 56.53 53.39 55.54
D. Ablation Study We ablate core components of MiniOpt-3B and report results of the SA metric in Table II and the ER metric in Table S4 in supplementary material, where “w” denotes “with” and “w/o” denotes “without”. As evidenced in Table II, each module of the proposed reasoning-to-model-and-solve paradigm demonstrates substantial contributions to model and solve optimization problems with smaller-scale models under limited training resources. Among these, Importance of the Training Pipeline of MiniOpt (Answer to Q3). In the training pipeline of MiniOpt, each module plays a distinct role in improving SA and ER. First, the SFT warmup of mid-training phase provides a better starting point for RL training. Without it, averages for SA fall to 56.60%. The decreases are ∆SA=–3.05. Second, collapsing the two-stage RL removes the progressive training that first consolidates the paradigm (stage-1) and then targets generalization (stage-2), the average SA drop to 55.07% and the average ER of 82.73% (∆ SA=–4.58%, ∆ ER=–5.19%). Finally, reverting our OptGRPO to the original GRPO further decreases both metrics, the average SA of 56.54% and ER of 82.35%. Together these changes of OptGRPO improve sample efficiency and training stability, which is critical for eliciting strong optimization
JOURNAL OF LATEX CLASS FILES
8
TABLE III: Comparison of the SA metric between MiniOpt-1.5B and larger scale counterparts across 8 benchmarks. Avg.
Solving Accuracy (SA)
NL4Opt ICML.C Mamo.E Mamo.C NLP4LP Com.OR Indus.OR OptiBench
Number of Samples in Datasets MiniOpt-1.5B Ours MiniOpt-3B MiniOpt-7B
50.33 59.65 64.76
230
410
652
211
242
18
100
605
88.7 93.04 95.22
72.2 78.05 83.41
78.99 84.36 89.57
8.06 47.39 51.18
71.07 74.38 74.79
16.67 27.78 38.89
16 17.00 25.00
50.91 55.21 60.00