Case-Based Calibration of Adaptive Reasoning and Execution for LLM Tool Use
arXiv:2605.15041v1 [cs.AI] 14 May 2026
Renning Pang , Tian Lan , Leyuan Liu(B) , Piao Tong , Sheng Cao , and Xiaosong Zhang University of Electronic Science and Technology of China, Chengdu 611731, China {prn,piaot}@std.uestc.edu.cn, {lantian1029,leyuanliu,caosheng,johnsonzxs}@uestc.edu.cn
Abstract. Tool use extends large language models beyond parametric knowledge, but reliable execution requires balancing appropriate reasoning depth with strict structural validity. We approach this problem from a case-based perspective to present CAST, a case-driven framework that treats historical execution trajectories as structured cases. Instead of reusing raw exemplar outputs, CAST extracts case-derived signals to identify complexity profiles for estimating optimal reasoning strategies, alongside failure profiles to map likely structural breakdowns. The framework translates this knowledge into a fine-grained reward design and adaptive reasoning, enabling the model to autonomously internalize casebased strategies during reinforcement learning. Experiments on BFCLv2 and ToolBench demonstrate that CAST improves both schema-faithful execution and task-level tool-use success while reducing unnecessary deliberation. The approach achieves up to 5.85 percentage points gain in overall execution accuracy and reduces average reasoning length by 26%, significantly mitigating high-impact structural errors. Ultimately, this demonstrates how historical execution cases can provide reusable adaptation knowledge for calibrated tool use. Keywords: Case-Based Reasoning · Large Language Models · Tool Use · Reinforcement Learning · Reasoning Budget Calibration
1
Introduction
Tool use has become a central mechanism for extending large language models (LLMs) beyond parametric knowledge, and recent surveys identify tool invocation, planning, and action coordination as core components of LLM-agent systems [25,4,6,21]. At the same time, benchmark-oriented studies suggest that reliable function calling remains far from solved, especially in settings that require memory, dynamic decision-making, and long-horizon reasoning rather than single-turn text generation alone [22]. Representative frameworks such as Toolformer [26], ReAct [29], and ToolLLM [24] further show that the key challenge is no longer simply whether LLMs can call tools, but whether they can allocate reasoning effort and execute structured tool actions in a stable and task-sensitive manner. This challenge arises because tool use places two simultaneous demands on the model. Different queries require different amounts of intermediate reasoning:
2
Pang et al.
Fig. 1. Comparison of reasoning workflows with and without a case-based mechanism.
some can be solved with little deliberation, whereas others require additional reasoning to verify constraints, normalize arguments, or compose multi-step tool calls. Recent work on adaptive chain-of-thought (CoT) and efficient tool calling suggests that treating all inputs with the same reasoning policy is often inefficient and can even be counterproductive [20,19,27]. At the same time, tool execution is governed by strict structural constraints, while standard reinforcement learning (RL) often provides only coarse end-task feedback, making it difficult to identify whether failure originates from tool selection, parameter coverage, type mismatch, schema violation, or value construction. The central difficulty in tool use is therefore to calibrate reasoning depth and execution structure jointly under heterogeneous task demands. We approach this problem from a modern Case-Based Reasoning (CBR) perspective on tool-use adaptation in LLMs. In our setting, historical execution trajectories are treated as cases that record how a task was solved, how much reasoning preceded action, whether the tool invocation succeeded, and what kinds of failures arose when it did not. This view is consistent with classical CBR [1] and with subsequent work emphasizing adaptation bottlenecks and case-base maintenance [11,7]. Recent studies further suggest that LLMs can support several stages of the CBR process, including case adaptation, similarity assessment, and experience-grounded agent reasoning [13,14,18,3]. Together, these developments motivate the use of historical execution cases as structured sources of calibration knowledge for tool use. In this paper, we formulate tool use as a case-based adaptation problem and introduce Case-driven Adaptation for Schema-faithful Tool use (CAST), a case-driven framework that extracts two forms of casederived signals from past trajectories. Specifically, it derives a complexity profile to estimate the necessary reasoning depth and a failure profile to map likely structural breakdowns. As shown in Figure1, rather than imposing a static reasoning limit or a uniform chain-of-thought policy, CAST translates this case-derived knowledge into a fine-grained reward design that supports adaptive reasoning. In particular, the model learns to shorten deliberation for easy cases while pre-
Case-Based Adaptive LLM Tool Use
3
serving sufficient reasoning steps for cases that require constraint verification, argument normalization, or multi-step tool composition. This approach enables the model to internalize historical case experiences during reinforcement learning, empowering it to autonomously orchestrate the model’s reasoning budget and execute schema-faithful tool actions. Experiments on BFCLv2 and ToolBench show that CAST improves both schema-faithful execution and task-level tool-use success while reducing unnecessary reasoning, with especially strong gains on more complex cases and clear reductions in high-impact structural failures. The remainder of this paper is organized as follows. Section 2 reviews related work on CBR for LLMs and agents, adaptive reasoning, and tool-use alignment. Section 3 formulates tool use as a case-based adaptation problem. Section 4 presents the CAST framework. Section 5 reports the experimental results and analyses. Section 6 concludes the paper and discusses limitations and future work.
2
Related Work
Case-based reasoning (CBR) addresses new problems by reusing and adapting solutions from similar prior cases through retrieval, reuse, revision, and retention [1]. In recent years, this perspective has increasingly intersected with large language models and agentic systems. Prior work has explored the use of LLMs for case adaptation in argumentative reasoning [13], while more recent studies investigate how LLMs can support similarity assessment in case retrieval [14]. Other work extends case-based ideas to reinforcement learning and agent settings, for example through case-based knowledge distillation for reinforcement learning [18], and unified experience-grounded agentic reasoning architectures [3]. Recent work has also examined how LLMs can support case-base population from unstructured sources [10] and how CBR can be integrated with LLMs in practical decision-support settings such as fraud detection [9]. Taken together, these studies suggest that LLMs can support multiple stages of the CBR process, including similarity assessment, adaptation, case acquisition, and the use of structured prior experience in agent reasoning. A related line of research studies how to regulate the amount of reasoning generated by large language models. Existing methods seek to shorten, compress, or selectively trigger chain-of-thought in order to improve efficiency while maintaining competitive task performance [20,19]. This line of work is directly relevant to our setting because it recognizes that different inputs may require different amounts of intermediate reasoning. In parallel, another line of work focuses on tool use and reasoning–acting behavior in LLMs. Toolformer shows that language models can learn when and how to invoke external tools [26], while ReAct demonstrates the value of interleaving reasoning traces with external actions [29]. Tool-use research has also been accompanied by increasingly realistic evaluation settings, including large-scale API benchmarks and function-calling leaderboards [24,22]. More recent work further improves invocation discipline and efficiency, for example through meta-cognitive triggering [15], alignment for efficient tool calling [27], or token-level policy gradient reshaping for tool-use
4
Pang et al.
LLMs [16]. These studies substantially advance reasoning control and tool-use alignment, but they generally treat reasoning depth and execution structure as separate concerns rather than as jointly case-conditioned aspects of the same problem. Our work is motivated by the gap between these directions. Existing CBRoriented research suggests that historical cases can provide reusable structure for future reasoning and action, while adaptive reasoning and tool-alignment research shows that both reasoning cost and execution discipline must be carefully controlled. What remains underexplored is how historical execution cases can be used to derive calibration signals for these two aspects jointly. CAST is a case-driven framework in which historical execution cases provide case-derived complexity and failure signals, designed to explicitly internalize the case-based experience that calibrates reinforcement learning through reasoning-budget control and schema-faithful tool optimization.
3
Problem Formulation
Given a user query q and a tool set T , tool use requires a model to generate a trajectory τ = (r, c), where r denotes the reasoning trace and c = call(f, z) denotes a structured tool invocation with function f and arguments z. We formulate this problem from a case-based adaptation perspective: historical execution trajectories are treated as execution cases, ξi = (qi , ri , ci , oi , ϕi ), where qi is the input query, ri is the reasoning trace, ci is the executed tool call, oi is the execution outcome, and ϕi = (hi , ei ) is the case profile consisting of a complexity profile hi and a failure profile ei . Under this formulation, historical executions are not treated merely as supervision traces, but as structured cases from which case-derived calibration signals can be obtained for reasoning-budget control and schema-faithful tool use. The objective is therefore to learn a policy πθ calibrated by these historical cases, such that the generated trajectory τ ∼ πθ (· | q, T ) produces tool actions that are both semantically appropriate and structurally executable.
4
Method
Figure 2 illustrates the overall architecture of CAST. From a case-based perspective, CAST organizes historical tool-use trajectories as structured execution cases, derives case-derived complexity and failure signals from these cases, and uses them to guide two coordinated adaptation processes: reasoning-budget calibration and schema-faithful tool optimization. In implementation, CAST is trained with a supervised warm-up stage followed by reinforcement learning calibrated by these case-derived signals. In this sense, reasoning-budget calibration model is encouraged to vary its deliberation depth according to case complexity before producing a schema-faithful tool action. Under this formulation, reinforcement learning serves as the optimization mechanism, while the methodological core lies in case-driven capability assessment and adaptation.
Case-Based Adaptive LLM Tool Use
5
Fig. 2. Overview of CAST. Historical tool-use trajectories are organized as execution cases. These case signals guide two coordinated adaptation processes: reasoning-budget calibration and schema-faithful tool optimization. Reinforcement learning serves as the optimization mechanism for this case-based adaptation framework.
4.1 Execution Case Construction CAST first converts historical tool-use trajectories into execution cases. Unlike ordinary supervision pairs, an execution case preserves not only the input query and target tool call, but also the intermediate reasoning process and the observed execution outcome. This allows historical trajectories to function as structured experience rather than isolated input–output examples. Formally, each case is represented as ξi = (qi , ri , ci , oi , ϕi ), where qi is the user query, ri is the reasoning trace, ci is the structured tool invocation, oi is the execution outcome, and ϕi is the case profile defined in Section 4.2. In practice, we bootstrap the case base from tool-augmented reasoning trajectories distilled from DeepSeek-R1, normalize all tool interactions into a unified function-schema format, and filter malformed or unverifiable traces during preprocessing. 4.2 Case Representation: Complexity and Failure Profiles After constructing the execution case base, CAST represents each case through a profile ϕi = (hi , ei ), where hi is a complexity profile and ei is a failure profile. The complexity profile characterizes how much intermediate reasoning is typically required before action, whereas the failure profile characterizes where execution is most likely to break, such as function selection, argument construction, type mismatch, or schema violation. Together, these profiles provide the case-derived calibration signals used in the subsequent adaptation stages. We estimate case complexity from the base model’s observed execution behavior rather than from a manually specified heuristic. For each historical query-target
6
Pang et al.
pair (q, a), we sample a trajectory y ∼ πbase (· | q) and evaluate it with the verifier V . If the generated tool call is fully correct and schema-faithful, the case is assigned zero hardness. Otherwise, the failed trajectory is assessed by an external rubric-guided judge using reference examples of simple and difficult cases. In our implementation, this judge is instantiated with Gemini-Pro. We define the resulting hardness score as ( 0, V (y, a) = 1, (1) H(q) = 1 − Sjudge (q, y), V (y, a) = 0, where Sjudge (q, y) ∈ [0, 1] denotes the judged adequacy of the failed trajectory with respect to logical coherence, parameter construction, and schema adherence. We use this score as an operational proxy for reasoning demand, and partition the case base into Deasy and Dhard for curriculum scheduling while retaining the continuous score for finer-grained analysis. The failure profile is derived from the mismatch between the generated tool call and the reference execution. For each failed case, we record the principal failure dimensions involved in the mismatch, including function-name errors, parameterkey omissions, type mismatches, constraint violations, and value mismatches, and summarize them as type constraint value ei = (ename , ekey , ei , ei ). i i , ei
(2)
Each component indicates whether the corresponding failure pattern is present in the trajectory. 4.3 Case-Based Adaptation for Reasoning-Budget Calibration The adaptation problem in tool use is to determine how much reasoning should precede action for a given case. A uniform reasoning policy is inadequate because different cases require different amounts of intermediate deliberation: simple cases may suffer from unnecessary verbosity. CAST addresses this problem by using the case complexity profile to internalize historical execution strategies into the model’s policy. Through a fine-grained reward design, the model learns to autonomously produce appropriately short or long CoT traces, mirroring the successful adaptation patterns found in similar prior cases. Concretely, the complexity profile introduced in Section 4.2 is operationalized by the hardness score H(q) ∈ [0, 1], where smaller values indicate easier cases and larger values indicate more difficult ones. We discretize H(q) into a small number of difficulty bands d(q) only for estimating stable empirical length baselines, while retaining the continuous value H(q) itself as the control signal for reward shaping. Let ra ∈ [−2, 2] denote the final answer score and L the response length. We define a difficulty-conditioned length baseline as: t d(q) d(q) d(q) Ld(q) (t) = L − L − L · min 1, , (3) target emp max max Twarmup d(q)
d(q)
where Lmax is the initial relaxed length limit for difficulty band d(q), Ltarget is the target concise length, and Twarmup is the curriculum duration.
Case-Based Adaptive LLM Tool Use
7
We then define the excess-length ratio ρ(q, L) = max 0,
L d(q)
! −1 ,
(4)
Lemp which measures how much the current response exceeds the expected reasoning budget, and the complexity-sensitive gating weight λ(q) = 1 − H(q),
(5)
so that easy cases are more strongly penalized for unnecessary overthinking, whereas difficult cases are less sensitive to length and remain primarily correctnessdriven. The shaping coefficient is defined as max(0, 1 − λ(q)ρ(q, L)) , ra > 0, α(q, ra , L) = 1 + λ(q)ρ(q, L), (6) ra < 0, 1, ra = 0, and the reasoning-side reward is RThink = α(q, ra , L) · ra .
(7)
Under this formulation, the same complexity signal plays two roles. Through d(q), it provides a stable empirical baseline for expected reasoning length; through H(q), it determines how strongly overlong reasoning should be penalized. As a result, easy cases are encouraged to remain concise, while difficult cases preserve a larger reasoning workspace and are optimized primarily with respect to answer correctness. 4.4
Failure-Profile-Grounded Optimization for Schema-Faithful Tool Use While reasoning-budget calibration governs how much deliberation precedes action, a complementary challenge remains: ensuring that the final tool action is structurally executable. Even when a reasoning trace appears semantically plausible, execution can still fail because of errors in function names, argument keys, types, constraints, or values. CAST therefore treats tool optimization as a separate but coordinated objective grounded in the failure profile ei = key type constraint value ename , e , e , e , e . Let G and P denote the ground-truth and i i i i i predicted collections of tool calls. To handle multi-call settings, we align them by maximum-weight bipartite matching: X J = arg max smatch (G, P ), (8) M ∈M(G,P)
(G,P )∈M G
P
∩K | where smatch (G, P ) = δ(name(G), name(P )) + |K , and K G , K P are the |K G ∪K P | parameter-key sets of G, P . Repeated tool calls remain distinct nodes in the matching graph.
8
Pang et al.
We then define a six-dimensional structural reward vector: rtool = (rname , rkey , rtype , rconstraint , rvalue , rexact )⊤ .
(9)
Let NG , NP be the overall function-name sets of G, P. The name score is the G ∩NP | Jaccard overlap rname = |N |NG ∪NP | . The key score averages parameter-key overlap P |K G ∩K P | over aligned calls: rkey = |J1 | j∈J |KjG ∪KjP | when |J | > 0, and 0 otherwise; j
j
when KjG = KjP = ∅, the corresponding overlap term is set to 1. We define rtype , rconstraint , rvalue as the average indicator matches over overlapping keys, with default value 0 when the number of overlapping keys is zero. The exactmatch term is rexact = VAST (P, G) ∈ {0, 1}, where AST denotes Abstract Syntax Tree matching. The raw structural score is: Rraw = rname + rkey + rtype + rconstraint + rvalue + rexact .
(10)
Since each component is bounded in [0, 1], the global maximum is Smax = 6. We therefore define the final tool-side reward as RTool = 2 ·
Rraw − 1 ∈ [−1, 1]. Smax
(11)
As a result, CAST provides dense and interpretable credit assignment for schema-faithful tool use without hand-tuned gating rules or query-dependent scaling constants. 4.5 Training Objective and Optimization Given a query q and a tool set T , the model generates a trajectory τ = (r, c). To guide the optimization securely and prevent degenerate behaviors, we formulate a composite reward that provides feedback at three complementary levels of abstraction: RC = RThink + RFormat + RTool . (12) The first term, RThink , is derived from the complexity profile and aligns the model toward adaptive reasoning by calibrating the intermediate deliberation budget for each case. (Section 4.3). The second, RFormat , is a rule-based guardrail that enforces valid tag encapsulation. The third, RTool , relies on the failure profile to evaluate fine-grained schema-faithful accuracy (Section 4.4). In practice, we first initialize the model using SFT on curated tool-augmented trajectories with the standard next-token prediction loss: LSFT = −
1 |DSFT |
X
|y| X
log πθ (yt | x, y<t ).
(13)
(x,y)∈DSFT t=1
After initialization, we optimize the policy πθ using a Group Relative Policy Optimization (GRPO)-based procedure augmented with the case-derived composite reward RC , which replaces the standard scalar reward with our decomposed complexity- and failure-conditioned signals. To stabilize the learning dynamics,
Case-Based Adaptive LLM Tool Use
9
we organize the RL training following an Easy-to-Hard curriculum based on the difficulty subsets partitioned by the complexity profile. For each query q, we sample a group of G trajectories {τ1 , . . . , τG }. The group-relative advantage for the i-th trajectory is computed by normalizing the composite rewards within the group: RC (τi ) − µR Ai = , (14) σR + ϵstab where µR and σR are the mean and standard deviation of the group rewards, and ϵstab is a small constant for numerical stability. The policy is then updated by maximizing the clipped surrogate objective augmented with a KL divergence penalty:
JCAST (θ) = Eq,{τi }G min ρ A , clip ρ , 1 − ϵ, 1 + ϵ Ai i i i i=1 − β · DKL πθ (· | q) ∥ πref (· | q) , (15)
where ρi = πθπθ (τ(τi |q) is the probability ratio, ϵ is the clipping margin, β controls i |q) old the KL penalty strength, and πref is the reference policy. Under this formulation, SFT provides warm-up initialization and the GRPO-based procedure serves as the optimization engine, while the methodological core of CAST lies in representing historical executions as structured cases, deriving complexity and failure profiles, and translating them into the decomposed case-derived signals that drive the GRPO updates.
5
Experiments
5.1
Experimental Setup
We evaluate CAST on BFCLv2, including 5,551 instances covering single-turn, parallel, multi-step, and irrelevance-detection scenarios, and on ToolBench with diverse queries mapping to REST APIs across 49 categories. Model-level baselines include the original backbones alongside their SFT and GRPO variants. We primarily use Qwen2.5-7B-Instruct, adding Qwen2.5-Coder-7B-Instruct and Llama3.2-8B-Instruct to examine cross-backbone robustness, and report closed models (GPT-4o [12], Qwen-Max [28], DeepSeek-V3 [17]) for context. Method-level comparisons feature Toolformer [26], ReAct [29], ToolAlign [5], CoT-Valve [20], OTC [8], Granite [2], and Gorilla [23] evaluated under identical protocols. We utilize CAST via Megatron with an 8K maximum response length and evaluate inference using SGLang. To balance exploration and warm-up, we train for 2 epochs during the SFT phase. For RL training, we sample a group of G = 8 rollouts per query at a temperature of 0.9. The policy is optimized using AdamW with a peak learning rate of 1 × 10−6 , a cosine learning rate scheduler, and a weight decay of 0.01. To stabilize the reinforcement learning dynamics, we apply a KL divergence penalty coefficient of β = 0.01.
10
Pang et al.
Table 1. Main results on BFCLv2 and ToolBench. We report execution accuracy on BFCLv2 (Non-Live AST, Live, and Overall) and task success on ToolBench (Pass and Win). Category Model / Method
Non-Live AST Live Overall Pass BFCLv2 (%)
Closed Source
Open Source
Other
5.2
Win
ToolBench (%)
GPT-4o Qwen-Max DeepSeek-V3
86.83 84.97 86.37
78.92 80.85 75.28
82.88 82.91 80.83
64.43 70.93 70.73
67.63 71.83 72.93
Llama-3.2-8B-Instruct-SFT Llama-3.2-8B-Instruct-GRPO Llama-3.2-8B-Instruct-CAST
83.14 82.18 83.95
73.82 75.21 76.84
78.48 78.73 80.43
70.63 71.23 75.93
45.83 45.23 47.23
Qwen2.5-Coder-7B-Instruct-SFT Qwen2.5-Coder-7B-Instruct-GRPO Qwen2.5-Coder-7B-Instruct-CAST
86.07 86.51 87.12
74.92 75.07 82.43
80.86 80.79 84.70
67.84 71.13 80.61
65.37 67.82 79.14
Qwen2.5-7B-Instruct-SFT Qwen2.5-7B-Instruct-GRPO Qwen2.5-7B-Instruct-CAST
86.23 87.05 88.24
78.90 80.29 87.40
82.58 83.67 88.43
68.67 72.71 80.67
65.23 68.23 79.43
Granite Gorilla Toolformer ReAct ToolAlign OTC
86.17 86.02 76.11 73.58 77.26 82.64
79.19 80.44 59.47 58.43 61.47 71.28
84.71 82.21 67.07 66.08 71.16 78.33
68.47 62.27 48.92 43.37 46.78 65.49
50.17 46.29 22.11 18.22 22.36 36.12
Overall Performance and Efficiency
Table 1 reports the main results on BFCLv2 and ToolBench. On BFCLv2, CAST consistently improves over both SFT and GRPO across all three open-source backbones, indicating that case-derived calibration transfers across different model families and pretraining styles. The strongest result is obtained with Qwen2.5-7BInstruct, where CAST reaches 88.43% overall, gaining 5.85 points over SFT and 4.76 points over GRPO. The narrow 0.84-point gap between Non-Live AST and Live scores suggests that the gains transfer from offline schema matching to actual tool execution. The same trend is visible on the other open-source backbones. On Llama-3.2-8B-Instruct, CAST improves BFCLv2 Overall from 78.48% under SFT and 78.73% under GRPO to 80.43%. On Qwen2.5-Coder-7B-Instruct, CAST reaches 84.70% overall, with especially large gains on Live execution, rising from 74.92% and 75.07% to 82.43%. Together, these results suggest that CAST is particularly effective when evaluation places stricter weight on execution validity. On ToolBench, the transfer pattern is also consistently positive. CAST achieves the strongest ToolBench results among CAST-trained models with Qwen2.57B-Instruct, reaching 80.67% Pass and 79.43% Win, both substantially above its SFT and GRPO counterparts. Llama-3.2-8B-Instruct shows a similar trend. Notably, the updated results on Qwen2.5-Coder-7B-Instruct now reveal a strong monotonic improvement as well: ToolBench Pass increases from 67.84% under SFT and 71.13% under GRPO to 80.61% under CAST, while Win rises from 65.37% and 67.82% to 79.14%. Rather than indicating a trade-off between schemafaithful calibration and end-to-end task success, the coder backbone now provides additional evidence that case-derived calibration transfers beyond local structural correctness and yields substantial gains on task-level tool-use performance.
Case-Based Adaptive LLM Tool Use
11
Table 2. Ablation of reasoning-side and tool-side adaptation on BFCLv2. Variant base-GRPO dynamic-only schema-only CAST
Non-Live AST ↑ Avg. Reasoning Length (tok) ↓ 87.05 85.50 86.59 88.24
236.9 164.7 214.5 175.4
5.3 Evidence for Case-Conditioned Adaptation We evaluate CAST through ablations, budget sensitivity, and training stability. Figure 3 shows that removing any major component consistently lowers average reward across backbones, indicating that CAST depends on the interaction between reasoning-side and tool-side adaptation. Table 2 confirms this on BFCLv2. The dynamic-only variant, which keeps the adaptive reasoning-budget component without the full schema-level reward, reduces average length from 236.9 to 164.7 tokens while reaching 85.50 Non-Live AST. This confirms that the adaptive reasoning component alone can suppress unnecessary deliberation, although schema-faithful optimization is still needed for the best execution accuracy. Combining both gives the best result 88.24% Non-Live AST at 175.4 tokens. Table 3 shows the importance of budget control. Removing the cap increases average length to 486.2 tokens and lowers BFCLv2 Overall to 86.3%. A strict 50th-percentile cap shortens outputs to 140.8 tokens but drops ToolBench Pass to 76.5%. The 80th-percentile setting achieves the best overall trade-off across BFCLv2 and ToolBench, supporting case-conditioned reasoning budgets for heterogeneous tool-use tasks.
Fig. 3. Ablation study of the major adaptation components.
Fig. 4. Training accuracy and normalized advantage variance.
Figure 4 shows that CAST also stabilizes RL training. Its normalized advantage variance decreases from 0.48 to 0.10, while GRPO remains around 0.21, indicating cleaner credit assignment and fewer oscillations between overthinking and underthinking. 5.4 Case Complexity and Failure Profiles To verify that the case-derived profiles operate as intended, we analyze their impact across three dimensions: instance-level budget allocation, structural error suppression, and global curriculum organization.
12
Pang et al.
Table 3. Impact of varying reasoning length budgets on task accuracy and schema alignment. Threshold
BFCLv2 (Overall)
ToolBench (Pass)
ToolBench (Win)
Length
0 (no cap) 100th (loose) 80th (default) 50th (strict)
86.3 87.5 88.43 86.8
69.9 80.3 80.67 76.5
65.7 79.6 79.43 73.1
486.2 240.9 175.4 140.8
Fig. 5. Performance across fine-grained difficulty levels.
Fig. 6. Error-rate distribution by category.
Adaptive Budget Allocation via Complexity Profiles. Figure 5 reports execution accuracy across fine-grained difficulty levels. If the complexity profile merely acted as a length penalty, its effect would be roughly uniform across buckets or even harmful on harder cases. Instead, CAST yields only small gains on easy instances, where budget control mainly reduces verbosity, but substantially larger gains as difficulty increases. This pattern suggests that the complexity profile enables adaptive reasoning allocation: CAST shortens reasoning when additional deliberation is unnecessary, but preserves or expands the reasoning budget when compositional reasoning is required. Structural Error Suppression via Failure Profiles. We next study whether schema-faithful tool optimization targets specific failure modes. Figure 6 separates execution errors into structural violations and localized content errors. As shown in the Reward Calculation module of Figure 2, RTool explicitly penalizes structural deviations. Accordingly, CAST reduces structural failures much more than the GRPO baseline, leaving most residual errors in localized value prediction. This result indicates that CAST improves the conversion of free-form reasoning into schema-compliant tool execution. Table 4. Effect of curriculum strategy on ToolBench. Method No Selection Two Stage Hard to Easy Easy to Hard
Pass (%)
Win (%)
Length
73.2 76.8 68.5 80.7
69.5 74.2 64.3 79.4
2 417.3 297.3 426.3 175.4
Case-Based Adaptive LLM Tool Use
13
At the training level, Table 4 tests whether the case-derived complexity score H(q) provides a useful curriculum signal. The easy-to-hard schedule achieves the best accuracy and the shortest outputs, with an average generation length of 175.4 tokens. In contrast, the hard-to-easy schedule performs worst and produces much longer outputs (426.3 tokens on average), suggesting that early exposure to difficult cases encourages unstable trial-and-error behavior and persistent overthinking. Overall, case-derived complexity serves as an effective signal for both local budget control and global curriculum design. Easy Instance Case Query: Could you check the current weather conditions in Beijing and New York City for me?
SFT
The user asks for weather in Beijing and New York City. I should use get_current_weather for each location. Two calls, one for Beijing, one for NYC.
get_current_weather (location=’Beijing’) get_current_weather (location=’NYC’)
GRPO
The query involves retrieving weather for two cities. I have access to get_current_weather. Since both Beijing and NYC are mentioned, I should invoke this for each city. No single API returns multiple cities. Wait, let me pause. Is parallel execution safe? While it reduces latency, there is a non-zero probability of hitting a 429 error if I trigger both at the same millisecond. Should I introduce a micro-delay? Or execute sequentially? Sequential adds latency but guarantees safety. However, the user likely values speed. No complex transformation needed, just passing city names. To maximize efficiency, I will call the function twice in parallel.
get_current_weather (location=’Beijing’) get_current_weather (location=’NYC’)
CAST
The user wants weather for Beijing and NYC. The simplest approach is to call the weather function for each city. I will use get_current_weather twice, one for each city, possibly in parallel. No further reasoning needed, the task is straightforward.
get_current_weather (location=’Beijing’) get_current_weather (location=’NYC’)
Fig. 7. Easy case. Hard Instance Case Query: Calculate density (mass=50kg, volume=10m3), future value ($5000 at 5% for 10 years), Apple stock price, and Amazon rating (ASIN: B08PPDJWC8)
SFT
The request involves multiple tasks: density, future value, stock price, and product rating. I will tackle them in order. For density, I have calculate_density with mass=50, volume=10. For future value, calculate_future_value with present_value=5000, interest_rate=5, periods=10. Then Apple stock with get_stock_price, and Amazon rating with get_rating_by_ASIN.
calculate_density (mass=50, volume=10) calculate_future_value (5000, 5, 10) get_stock_price (‘Apple’) get_rating_by_ASIN (‘B08PPDJWC8’)
GRPO
The user requests four independent results. The safest plan is to map each task to one tool call and validate arguments. For density: mass=50, volume=10, straightforward. For future value: present_value=5000, interest_rate=5%, periods=10. I need to decide how to encode 5%. Many functions expect decimal 0.05, but some accept 5. I’ll use 5, keeping closer to surface form. For Apple stock: the tool likely expects ticker symbol AAPL, not company name. For ASIN: pass B08PPDJWC8 exactly.
calculate_density (mass=50, volume=10) calculate_future_value (5000, 5, 10) get_stock_price (‘AAPL’) get_rating_by_ASIN (‘B08PPDJWC8’)
CAST
The request comprises four diverse tasks requiring careful handling. I will approach each methodically. Density: calculate_density(mass=50, volume=10), straightforward. Future value: Must be precise with interest_rate. 5% = 0.05 decimal. Crucial to pass 0.05, not integer 5. Stock price: Safest to use ticker symbol AAPL, not company name. Amazon rating: Pass ASIN exactly as given.
calculate_density (mass=50, volume=10) calculate_future_value (5000, 0.05, 10) get_stock_price (‘AAPL’) get_rating_by_ASIN (‘B08PPDJWC8’)
Fig. 8. Hard case.
14
Pang et al.
5.5 Case Study Figures 7 and 8 illustrate CAST’s adaptive reasoning-execution boundary. In the easy weather-query case, GRPO produces redundant deliberation about rate limits and execution order, whereas CAST directly emits the necessary tool calls. In the harder compositional case, SFT and GRPO copy the surface form of 5% into the tool arguments, while CAST preserves enough reasoning to normalize it to 0.05 and keeps the remaining calls structurally valid. This shows that CAST shortens reasoning when execution is straightforward and preserves it when semantic normalization is required.
6
Conclusion
This paper revisits tool use from a case-based reasoning perspective and proposes CAST, a framework that uses signals distilled from past execution cases to guide reinforcement learning. Specifically, CAST summarizes historical trajectories through complexity and failure profiles, and uses them to regulate reasoning length and supervise schema-level tool execution. Experiments on BFCLv2 and ToolBench show that this case-conditioned calibration improves execution accuracy, transfers to end-to-end tool-use success, and reduces unnecessary reasoning, with the largest gains on more complex cases. Long-horizon planning remains challenging, but the results suggest that case-derived supervision provides a practical basis for improving both reliable tool calling and downstream task completion. A natural next step is to extend this framework to richer case memories, stronger retrieval and reuse, and more interactive agent settings.
Case-Based Adaptive LLM Tool Use
15
Acknowledgments. This work was partially supported by the National Natural Science Foundation of China (U2336204), Chengdu Industrial Chain Collaborative Innovation Project (Grant No. 2025-XT00-00017-GX) and the Open bidding for selecting the best candidates of Sichuan Provincial Department of Science and Technology (2024YFCY0003). Generative AI Disclosure LLMs were used exclusively for stylistic refinement of the manuscript. All textual content was initially drafted in full by the authors and subsequently polished with the assistance of LLM-based tools, including ChatGPT and Gemini. All scientific contributions, technical methods, ideas, and core results presented in this work are entirely the original work of the authors.
References 1. Aamodt, A., Plaza, E.: Case-based reasoning: Foundational issues, methodological variations, and system approaches. AI communications 7(1), 39–59 (1994) 2. Abdelaziz, I., Basu, K., Agarwal, M., Kumaravel, S., Stallone, M., Panda, R., et al.: Granite-function calling model: Introducing function calling abilities via multi-task learning of granular tasks. In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track. pp. 1131–1139 (2024) 3. Bergmann, R., Brand, F., Lenz, M., Malburg, L.: Exar: A unified experiencegrounded agentic reasoning architecture. In: International Conference on Case-Based Reasoning. pp. 3–17. Springer (2025) 4. Chen, J., Wu, H., Pang, J., Wang, Y., Zhang, D., Sun, C.: Tool learning with language models: a comprehensive survey of methods, pipelines, and benchmarks. Vicinagearth 2(1), 16 (2025) 5. Chen, Z.Y., Shen, S., Shen, G., Zhi, G., Chen, X., Lin, Y.: Towards tool use alignment of large language models. In: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. pp. 1382–1400 (2024) 6. Chowa, S.S., Alvi, R., Rahman, S.S., Rahman, M.A., Raiaan, M.A.K., et al.: From language to action: a review of large language models as autonomous agents and tool users. Artificial Intelligence Review (2026) 7. Craw, S., Wiratunga, N., Rowe, R.C.: Learning adaptation knowledge to improve case-based reasoning. Artificial intelligence 170(16-17), 1175–1192 (2006) 8. El-Tantawy, S., Abdulhai, B.: Towards multi-agent reinforcement learning for integrated network of optimal traffic controllers (marlin-otc). Transportation Letters 2(2), 89–110 (2010) 9. Ge, X., Xu, J.: Integrating case-based reasoning with llm for expense fraud detection. In: International Conference on Case-Based Reasoning. pp. 52–66. Springer (2025) 10. Ghazouani, F., Giustozzi, F., Le Ber, F.: Llm-driven case-base populating for structuring and integrating restoration experiences. In: International Conference on Case-Based Reasoning. pp. 67–80. Springer (2025) 11. Hanney, K., Keane, M.T.: The adaptation knowledge bottleneck: How to ease it by learning from cases. In: International Conference on Case-Based Reasoning. pp. 359–370. Springer (1997) 12. Hurst, A., Lerer, A., Goucher, A.P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., et al.: Gpt-4o system card. arXiv preprint arXiv:2410.21276 (2024) 13. Lenz, M., Bergmann, R.: Case-based adaptation of argument graphs with wordnet and large language models. In: International Conference on Case-Based Reasoning. pp. 263–278. Springer (2023)
16
Pang et al.
14. Lenz, M., Hoffmann, M., Bergmann, R.: Llsim: large language models for similarity assessment in case-based reasoning. In: International Conference on Case-Based Reasoning. pp. 126–141. Springer (2025) 15. Li, W., Li, D., Dong, K., Zhang, C., Zhang, H., Liu, W., Wang, Y., Tang, R., Liu, Y.: Adaptive tool use in large language models with meta-cognition trigger. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 13346–13370 (2025) 16. Lin, Z., Wang, X., Cao, J., Chai, J.: Rest: Reshaping token-level policy gradients for tool-use large language models. In: ICLR 2026 Conference Proceedings (2026) 17. Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al.: Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 (2024) 18. Liu, H., Liu, Q., Wu, L., Shi, M., Cui, Z.: Offline-to-online: Case-based knowledge distillation with large language models for reinforcement learning. In: ICCBR 2025. LNCS, vol. 15662, pp. 142–156. Springer (2025) 19. Lou, C., Sun, Z., Liang, X., Qu, M., Shen, W., Wang, W., Li, Y., Yang, Q., Wu, S.: Adacot: Pareto-optimal adaptive chain-of-thought triggering via reinforcement learning. arXiv preprint arXiv:2505.11896 (2025) 20. Ma, X., Wan, G., Yu, R., Fang, G., Wang, X.: Cot-valve: Length-compressible chainof-thought tuning. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 6025–6035 (2025) 21. Mohammadi, M., Li, Y., Lo, J., Yip, W.: Evaluation and benchmarking of llm agents: A survey. In: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. pp. 6129–6139 (2025) 22. Patil, S.G., Mao, H., Yan, F., Ji, C.C.J., Suresh, V., Stoica, I., Gonzalez, J.E.: The berkeley function calling leaderboard (BFCL) (2025) 23. Patil, S.G., Zhang, T., Wang, X., Gonzalez, J.E.: Gorilla: Large language model connected with massive apis. In: Advances in Neural Information Processing Systems 37 (NeurIPS 2024) (2024) 24. Qin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., et al.: Toolllm: Facilitating large language models to master 16000+ real-world apis. In: The Twelfth International Conference on Learning Representations (ICLR) (2024) 25. Ruan, J., Chen, Y., Zhang, B., Xu, Z., Bao, T., Mao, H., Li, Z., Zeng, X., Zhao, R., et al.: Tptu: Task planning and tool usage of large language model-based ai agents. In: NeurIPS 2023 foundation models for decision making workshop (2023) 26. Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., Scialom, T.: Toolformer: Language models can teach themselves to use tools. Advances in neural information processing systems 36, 68539–68551 (2023) 27. Xu, H., Wang, Z., Zhu, Z., Pan, L., Chen, X., Fan, S., Chen, L., Yu, K.: Alignment for efficient tool calling of large language models. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. pp. 17787–17803 (2025) 28. Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., et al.: Qwen2.5 technical report (2025), https://arxiv.org/abs/2412.15115 29. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K.R., Cao, Y.: React: Synergizing reasoning and acting in language models. In: The eleventh international conference on learning representations (2022)