UniToolCall: Unifying Tool-Use Representation, Data, and Evaluation for LLM Agents Yijuan Liang1,2 , Xinghao Chen2,3 , Yifan Ge2 , Ziyi Wu2 , Hao Wu2 , Changyu Zeng2 Wei Xing2 , Xiaoyu Shen2 * 1 University of Science and Technology of China 2 Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo 3 Department of Computing, The Hong Kong Polytechnic University [email protected] Abstract
arXiv:2604.11557v1 [cs.AI] 13 Apr 2026
Tool-use capability is a fundamental component of LLM agents, enabling them to interact with external systems through structured function calls. However, existing research exhibits inconsistent interaction representations, largely overlooks the structural distribution of tool-use trajectories, and relies on incompatible evaluation benchmarks. We present UniToolCall, a unified framework for tool learning that standardizes the entire pipeline from toolset construction and dataset generation to evaluation. The framework curates a large tool pool of 22k+ tools and constructs a hybrid training corpus of 390k+ instances by combining 10 standardized public datasets with structurally controlled synthetic trajectories. It explicitly models diverse interaction patterns, including single-hop vs. multi-hop and single-turn vs. multi-turn, while capturing both serial and parallel execution structures. To support coherent multiturn reasoning, we further introduce an Anchor Linkage mechanism that enforces cross-turn dependencies. Furthermore, we convert 7 public benchmarks into a unified Query–Action– Observation–Answer (QAOA) representation with fine-grained evaluation at the functioncall, turn, and conversation levels. Experiments show that fine-tuning Qwen3-8B on our dataset substantially improves tool-use performance. Under the distractor-heavy Hybrid-20 setting, UniToolCall achieves 93.0% single-turn Strict Precision, outperforming commercial models including GPT, Gemini, and Claude. 1
1
Introduction
The emergence of LLM agents marks a shift from passive text generation to goal-directed interaction with external environments (Durante et al., 2024; Luo et al., 2025a; Sapkota et al., 2026). A key capability underlying this shift is tool use, which * Corresponding Author 1
https://github.com/EIT-NLP/UniToolCall.
Figure 1: Existing datasets are severely limited by the fragmentation problem. To address these challenges, UniToolCall introduces a standardized framework that provides robust structural constraints, yielding SOTA over strong baselines.
enables agents to take actions by translating natural language instructions into executable function calls. Through tool use, LLM agents can access external knowledge, invoke APIs, and perform multi-step operations, extending their capabilities beyond parametric knowledge (Schick et al., 2023; Yu et al., 2025). Consequently, an agent’s effectiveness largely depends on its ability to select, compose, and execute tools reliably, making tool learning a central problem in agent research (Paprunia et al., 2025; Lu et al., 2026). In the current data-driven paradigm, progress in tool learning is largely determined by the availability and quality of training data, particularly tool-use trajectories that capture how agents interact with external environments. Early efforts such as ToolLLM (Qin et al., 2024), ToolBench (Patil et al., 2024), and API-Bank (Li et al., 2023) construct such data by executing real-world APIs. While providing realistic supervision signals, they suffer from limited scalability and instability due to their
reliance on external systems. To address these limitations, more recent works have shifted toward synthetic data generation, building simulated tool environments and automatically generating interaction trajectories (e.g., ToolForge (Chen et al., 2025b), LoopTool (Zhang et al., 2025), ASTRA (Tian et al., 2026)). In parallel, a number of benchmarks have been proposed to evaluate tool-use capability, including ComplexFuncBench (Zhong et al., 2025), HammerBench (Wang et al., 2024), and ACEBench (Chen et al., 2025a). Despite this progress, existing efforts are largely developed in isolation, leading to a fundamental fragmentation problem in tool learning. This fragmentation manifests along three key dimensions. First, representation inconsistency: different datasets adopt incompatible schemas to encode tool calls, arguments, and observations, making joint training across sources difficult. Second, structural under-modeling: current pipelines largely overlook the diversity of execution structures, particularly the distinction between serial and parallel tool invocation patterns. Third, evaluation mismatch: existing benchmarks rely on disparate protocols, tool definitions, and evaluation scripts, preventing fair and reproducible cross-dataset comparisons. Together, these issues hinder both scalable training and systematic evaluation of tool-use capabilities. To address these limitations, we propose UniToolCall, a unified framework for tool learning that standardizes the entire pipeline from toolset construction and data generation to evaluation under a shared representation. We first curate a largescale tool pool by aggregating tools from multiple sources, resulting in a filtered set of over 22K tools. Building on this, we construct a hybrid training corpus that combines standardized public datasets with structurally controlled synthetic trajectories, yielding 390K instances spanning single-hop, multi-hop, single-turn, and multiturn interactions. Crucially, our synthetic pipeline explicitly models both serial and parallel execution structures, enabling fine-grained analysis of execution patterns. Finally, we unify all data into a Query–Action–Observation–Answer (QAOA) representation 2 and introduce a standardized evaluation protocol with comprehensive metrics, enabling consistent and fair comparison across diverse settings. Our contributions are as follows: • Structurally-Aware data generation We pro2
A single-hop sample is shown in Appendix C.2.
pose a synthetic data generation pipeline that provides controlled supervision for single/multi-hop and single/multi-turn interactions. The pipeline explicitly models both serial and parallel execution patterns and introduces an Anchor Linkage mechanism to enforce cross-turn dependencies. • A standardized unified benchmark We convert heterogeneous public benchmarks into a unified QAOA format with shared matching rules and metrics, enabling fine-grained evaluation across function-call, turn, and conversation levels and fair comparison across diverse task structures. • Strong empirical Performance Fine-tuning a lightweight Q WEN 3-8B model on our framework achieves state-of-the-art results. Under the distractor-heavy Hybrid-20 setting, UniToolCall attains 93.0% single-turn Strict Precision, outperforming leading commercial models including GPT, Gemini, and Claude.
2
Related work
Synthetic data generation Early work (Tang et al., 2023) generates instruction-style tool-use examples to teach basic API usage. Subsequent pipelines further automate dataset construction (Liu et al., 2024; Chen et al., 2025b; Zhang et al., 2025), synthesizing tool-use trajectories at larger scale. Despite this, most generated trajectories tend to follow relatively simple interaction patterns. Moreover, the balance between serial and parallel tool execution is rarely considered. In contrast, our synthetic pipeline explicitly models structural diversity by considering both serial and parallel execution patterns across four interaction structures. Tool-use benchmarks Some studies rely on real environments (Qin et al., 2024; Wang et al., 2025; Gao et al., 2025), where models interact with external tools through actual execution. To improve reproducibility, several benchmarks evaluate tool usage through simulated invocation while retaining real tool definitions (Chen et al., 2025a; Moon et al., 2024). However, these benchmarks adopt heterogeneous schemas, evaluation rules, and task structures, which hinder fair comparison. To address these limitations, we construct a unified benchmark that evaluates heterogeneous datasets under a shared QAOA representation, enabling multigranularity evaluation and providing a more comprehensive assessment of tool learning.
Toolset Construction
Unified Data Synthesis Engine
Structural Integration Validated Syntheic Data
Academic Benchmarks
MCP Servers
Constructed Datasets
Tool Pool
Chonsen Tools QAOA Formatting
Hop-Level Trajectory (Parallel & Serial)
Multi-Stage Filtering
Sampling
Cleaned Public Data
LLM Dual-path Planning
Query Find a flight to Beijing. Turn-Level Trajectory (Cross-Turn Dialogues)
Action SearchFlights (dest='NRT') Observation Flight found: [JL123] Answer I found flight JL123 to Beijing
Exact Deduplication
Schema Validation
Semantic Deduplication
Query Trajectory
Self-Evaluation Rubric
Validated Syntheic Data
Gold-anchored Hybrid Sampling Target Tools n
Evaluation Protocol
20 Candidate Tools in Total
1. Large-Scale: 22k+ tools System Prompt
2. Domain Diversity: 13 domins Tool Pool 3. Category Diversity: 6 categories
Matching Criteria
Granularity
Unified Metrics
Unidied Dataset
Figure 2: The overall architecture of UniToolCall, comprising several interconnected modules: (1) Toolset construction; (2) Unified data synthesis engine; (3) Structural integration; (4) Evaluation protocol.
3
UniToolCall
In this section, we present UniToolCall, a unified framework for tool learning. At the core of our framework is a standardized QAOA representation, which provides a consistent format for modeling tool interactions across datasets. As illustrated in Figure 2, the framework consists of three components: a curated toolset, a data generation pipeline, and a structural assembly stage. In addition, we introduce a unified benchmark to enable consistent evaluation across tool-use scenarios. 3.1
Toolset construction
To serve as the candidate pool for dataset construction, we construct a comprehensive toolset after applying the multi-stage filtering mechanism (Appendix A.3), denoted as T . As illustrated in Figure 7, the toolset is formed from three primary sources: (1) Academic benchmarks, (2) MCP servers, and (3) Constructed datasets. All tools are standardized into a unified JSON Schema format. To facilitate semantic organization and balanced sampling during data generation, we categorize tools along two dimensions: functional category and application domain. Based on common API usage patterns in agent systems, we define 6 functional categories (e.g., visualization, analysis) to capture the operational roles of tools and 13 application domains (e.g., finance, technology) to represent typical real-world usage scenarios.3 3
The details, complete taxonomy and category definitions are provided in Appendix A.1 and A.2.
3.2
Training dataset construction
Source To equip the agent with robust tool-use and planning capabilities, we construct a largescale hybrid training dataset, denoted as Dtrain . The dataset is composed of two parts: (1) Public data integration (Dpub ) We collected and integrated 10 distinct tool-use datasets4 . To ensure validity and reliability, we implemented a two-stage filtering strategy applied before and after format conversion (Appendix A.4). (2) Synthetic augmentation (Dsyn ) To overcome the structural shallowness inherent in public corpora, we construct a synthetic dataset Dsyn based on the toolset T . In particular, the pipeline controls both execution patterns (serial vs. parallel tool invocation) and interaction complexities (single-hop, multi-hop, singleturn, and multi-turn scenarios). Dsyn is filtered using an LLM-based self-evaluation framework using six core metrics (e.g., Tool-fit, Success), plus an additional anchor-linkage metric for multi-turn episodes (Appendix A.5). Unified synthetic data pipeline We design a unified generative framework equipped with stringent quality control. Formally, the construction of any synthetic subset Dx ∈ {Dsh , Dmh , Dmt } is generalized as follows: Dx = Ψ(τ, Psys , Lcand ) | S ⊆ T , τ ∼ M(S), Φeval (τ ) = 1
where S is a sampled subset from the filtered tool pool T , τ represents the raw interaction trajectory 4
Table 5 provides detailed statistics for each dataset.
Figure 3: Detailed illustration of our synthetic trajectory generation pipelines. The single-turn pipeline encompasses both fundamental single-hop invocations (K = 1) and complex multi-hop scenarios (K ≥ 2), which are further categorized into parallel and serial execution strategies. The multi-turn pipeline extends the interaction to long-horizon conversational settings, explicitly enforcing strict cross-turn state dependencies via Anchor Linkage mechanism.
generated by the LLM M, and Φeval acts as the heuristic self-evaluation gate. Across all scenarios, the structural assembly function Ψ standardizes the validated trajectories into our QAOA format. Crucially, Ψ constructs the candidate list Lcand using a uniform Hybrid-20 setting: retaining the ground-truth tools from S as anchors, retrieving top-ranking hard negatives via embedding similarity, and appending 5 random easy negatives to yield exactly 20 candidates (Esfandiarpoor et al., 2026). Finally, a system prompt Psys (Appendix C.2) detailing tool-use constraints is injected. While sharing this core formulation, the specific definitions of the tool subset S and the trajectory τ diverge to target distinct agentic capabilities: Single-Hop (Dsh ) Focuses on fundamental invocation mapping. We sample a single tool (|S| = 1), and the model M deterministically generates a onestep trajectory τ = ⟨q, a, o, r⟩ based strictly on the tool’s schema, where q, a, o, and r denote Query, Action, Observation, and Answer, respectively. Multi-Hop (Dmh ) Trains the agent to coordinate sequences of tool calls. We sample a domainconstrained subset S (|S| ∈ {2, . . . , 5}). The trajectory extends to K steps (K ≥ 2). Crucial distinction: We explicitly control the execution routing. For serial instances, M generates steps iteratively, constraining subsequent turns to reference concrete values from earlier observations to form genuine inter-step dependencies. For parallel instances, the query q and all tool calls are synchronized in a one-shot generation to prevent intention-tool mismatches.
Multi-Turn (Dmt ) Models long-horizon, stateful interactions across T ∈ {2, 3, 4} dialogue turns. We sample a usage-balanced subset S (|S| = 10). Generation requires a Two-Stage planning mechanism (episode-level storyline followed by turnlevel intent). Furthermore, to address the disjointed context shifts common in existing datasets (Ma et al., 2024), we introduce explicit Anchor Linkage: a strict adjacent-turn constraint ensuring that the user query at turn t deterministically inherits state variables (e.g., transaction IDs) generated by the tool observations at t − 1. 3.3
Evaluation protocol
To evaluate model performance across complex tool-use scenarios, we construct a unified benchmark Dtest . This unified benchmark focuses on two fundamental capabilities of tool-use agents: accurate tool selection and correct parameter generation. We convert all raw data into the standardized QAOA framework. This standardization enables fairer comparison by applying a unified set of evaluation metrics across all datasets. Agent interactions exhibit a hierarchical structure: a full conversation consists of multiple turns, and each turn contains one or more individual function calls. To accurately capture performance across these nested levels, we decouple our evaluation logic into the following granularities: Function call-level verification At the most fundamental level, the validity of each individual tool invocation is assessed by matching the predicted tool name and generated arguments against the ground truth. This call-level correctness serves as the computational basis for calculating proportional scores in flexible metrics. Turn & Conversation-level aggregation To evaluate task-level capabilities, the aforementioned call-level results are aggregated at higher dimensions, denoted by N . We explicitly map the aggregation granularity to the specific type of task complexity being assessed: (1) Turn-level: For single/multi-hop scenarios, we compute metrics across individual dialogue turns. (2) Conversationlevel: For single/multi-turn scenarios, we compute metrics across the entire dialogue trajectory. During aggregation, Strict metrics employ an all-ornothing penalty (the instance scores 0 if any function call is flawed), whereas Flexible metrics award credit based on the ratio of correct function calls
within the instance. These instance scores are subsequently macro-averaged across the dataset. Matching criteria At the Function Call-level, we employ a cascaded strategy to determine the validity of a predicted function call: (1) Rulebased matching: Serving as the primary strategy, this method achieves exact matching through rigorous standardization5 . A match is confirmed if the standardized prediction aligns perfectly with the ground truth. (2) Semantic matching: We calculate the ROUGE-L similarity score between the prediction and the reference. A prediction is deemed a semantic match if the score is ≥ 0.7.
4
Experiments
4.1
Experimental setup
Models and training data We instantiate UniToolCall on the open-source backbone Qwen38B (Yang et al., 2025). The model is fine-tuned on our comprehensive dataset Dtrain , which contains 390,060 instances grounded in our tool pool T of 22,606 tools. Specifically, this consists of 387,123 high-quality conversations retained from the standardized public corpus (Dpub ), and 2,937 synthetic trajectories (Dsyn ). The synthetic subset is generated and rigorously self-evaluated by Qwen3-32B (Yang et al., 2025), comprising 979 instances each for single-hop, multi-hop, and multiturn scenarios respectively. Implementation details We employ the LLaMAFactory framework integrated with DeepSpeed optimization to fine-tune the base model. Training is conducted with LoRA (Hu et al., 2022). We target all linear modules with rank r = 8 and scaling factor α = 16. The model is trained for 1 epoch using AdamW with a learning rate of 1 × 10−5 and a warmup ratio of 0.03. The maximum sequence length is set to 8192 tokens. The effective batch size is 8. We use bfloat16 precision throughout training. All experiments are conducted on a single node with 4×NVIDIA A800-SXM4 GPUs (40GB each) and an Intel Xeon Platinum 8378A CPU. Baselines We compare UniToolCall against six strong LLM baselines. To improve inference efficiency, we standardize the inference setup by disabling explicit reasoning traces when supported (e.g., <think> style outputs). Our finetuned Qwen3-8B is trained and evaluated in 5
Refer to Appendix B.2 for the detailed rules.
an enable-thinking=false setting, and DeepSeekV3.2 (DeepSeek-AI et al., 2025), Qwen332B (Yang et al., 2025), and Claude 4.6 Sonnet (Anthropic, 2026) are evaluated with reasoning disabled also through the available API options. For other proprietary models, we use their nonreasoning or efficiency-oriented variants, including GPT-5.2 Instant (OpenAI, 2025) and Gemini 3 Flash Preview (Google, 2025). Evaluation settings Our main evaluation is conducted under the Hybrid-20 setting mentioned in Section 3.2. We additionally report results under the Ground Truth (GT) setting, in which the candidate list contains only the required target tools. The main result tables report a single representative run for each model. To characterize training stability, we further report multi-run summary statistics for UniToolCall and the vanilla Qwen3-8B in Appendix C.1 for reference. The unified evaluation Dtest mentioned in Section 3.3 comprises 7 public benchmarks, yielding 6,163 high-quality conversations after filtering (Table 5). Based on this evaluation protocol, we define four macro-averaged quantitative metrics. We first introduce three indicator functions for a predicted function call p within an instance’s prediction set Pi (Missed cases will be filled with null), evaluated against the ground truth set Gi : • mn (p): Returns 1 if p has a correctly matching tool name in Gi ; otherwise 0. • ms (p): Returns 1 if p strictly matches a call in Gi in both name and all argument values. • mf (p): Returns 1 if p matches the tool name and satisfies the semantic similarity threshold for arguments. Note: Under-predicted calls in Pi are padded with null to penalize omissions, directly yielding a score of 0 when no tools are invoked (|Pi | = 0). Strict Precision (SP) This metric establishes the rigorous lower bound for tool selection. An instance scores 1 if and only if every predicted tool name perfectly matches the ground truth: SP = N1
PN
i=1 1 |Pi | = |Gi | ∧ ∀p ∈ Pi , mn (p) = 1
Flexible Precision (FP) As a tolerant tool selection metric, this macro-averaged precision calculates the proportion of correctly named tools: FP = N1
PN
1 P i=1 |Pi | p∈Pi mn (p)
Strict Parameter Accuracy (SPA) This metric assesses the exactness of argument generation. The denominator is the total number of predicted calls (|Pi |). A prediction only contributes to the score if both its name and arguments are perfectly correct: SPA = N1
PN
1 P p∈Pi ms (p) i=1 |Pi |
Flexible Parameter Accuracy (FPA) This metric measures the proportion of predicted tools that pass argument matching: either exact rule-based matching or ROUGE-L similarity: FPA = N1
4.2
PN
1 P p∈Pi mf (p) i=1 |Pi |
Main results
Tables 1 summarizes the overall performance of UniToolCall and the baselines under the Hybrid-20 setting. We highlight three main observations. Strong gains in tool selection As shown in Figure 4, UniToolCall achieves the best strict tool-selection performance in both single-hop and single-turn settings, reaching SP scores of 92.9% and 93.0%, respectively. These results substantially improve over the vanilla Qwen3-8B and also exceed stronger open-source and proprietary baselines such as Qwen3-32B and Gemini 3 Flash Preview. In multi-hop settings, UniToolCall obtains the second-best SP (80.7%) and FP (89.6%), while remaining close to Claude 4.6 Sonnet on strict selection. This suggests that our framework is particularly effective at improving precise tool localization under distractor-heavy retrieval conditions. Improved parameter grounding Beyond tool selection, UniToolCall also improves parameter generation quality. In the single-turn setting, it achieves the best SPA and FPA across all compared models. In multi-turn scenarios, UniToolCall improves FP and parameter-level metrics compared to the vanilla backbone, although strict conversationlevel matching remains challenging for all models due to the long-horizon nature of the task. Comparison to the GT setting The GT setting removes distractor tools and therefore serves as a useful reference point for analyzing retrieval difficulty. In single-hop evaluation, UniToolCall under Hybrid-20 approaches the vanilla model’s GT performance. In single-turn and multi-hop settings, the fine-tuned model even surpasses the vanilla model evaluated in the GT condition, suggesting
that the gains are not limited to distractor resistance alone, but also reflect improved intrinsic capability in structured tool-use prediction. 4.3
Ablations
Our synthetic pipeline is designed to provide structurally controlled supervision rather than to replace the scale and domain breadth of large public corpora. We conduct ablations from two distinct perspectives: downstream model training and intrinsic data quality. To evaluate downstream training efficacy, we investigate two design questions under a matched data budget (N = 979): (1) how the structural profile of pipeline-generated data differs from that of collected public corpora, and (2) whether mixing different structural complexities is beneficial within synthetic training. To ensure a comprehensive assessment, Table 2 reports the macro-averaged results across our unified benchmark. Furthermore, to continuously observe performance dynamics under varying execution structures, we conduct a targeted evaluation on the BFCL v3 (Patil et al., 2025) subset (Figure 5). BFCL v3 was selected due to its sufficient volume of multi-hop instances, allowing us to dynamically control parallel-to-serial ratios via stratified random sampling. Separately, to validate the generation mechanism itself, we address a third question: (3) the efficacy of explicit state-tracking constraints. We explore this by directly assessing the intrinsic quality of synthesized multi-turn trajectories with and without our Anchor Linkage mechanism. Ablation I: Structural profile of public vs. synthetic data Evaluated globally across the entire benchmark (Table 2), the public subset naturally achieves strong overall scores due to its broad linguistic and domain coverage. However, its inherent serial-to-parallel ratio is severely skewed (1:5.69), indicating a dominance of flatter, independent invocation patterns. In contrast, our synthetic pipeline explicitly injects denser sequential dependencies (1:1.91). We isolate the impact of this structural bias using the BFCL v3 fine-grained analysis in Figure 5. As the proportion of parallel tasks increases (x-axis), all evaluation scores artificially inflate, confirming that serial dependencies are inherently more challenging. Crucially, the comparative advantage between synthetic and public data dynamically shifts across this spectrum. In heavily sequential scenarios, the Synthetic Mixed data demonstrates clear superiority over the Public
SP (%) ↑
Models SH
MH
ST
FP (%) ↑ MT
SH
MH
SPA (%) ↑
ST
MT
SH
MH
ST
FPA (%) ↑ MT
SH
MH
ST
MT
GT Setting Qwen3-8B (Upper Bound)
96.1 47.5 92.6 0.0
96.1 76.4 95.5 39.5
28.7 57.2 32.1 18.8
52.3 70.0 54.9 24.9
Hybrid-20 Setting Proprietary Models GPT-5.2 Instant Gemini 3 Flash Preview Claude 4.6 Sonnet Open-Source Models Kimi-K2-Instruct DeepSeek-V3.2 Qwen3-32B Qwen3-8B (Vanilla) UniToolCall (Ours)
50.9 39.0 50.5 0.0 68.3 77.2 70.3 0.0 58.8 83.3 62.1 0.0
50.9 58.2 52.5 16.1 68.3 83.6 70.9 22.5 58.8 89.6 62.7 34.8
23.2 49.1 26.3 9.1 25.2 69.3 30.3 14.2 24.9 74.1 30.3 19.4
39.2 55.3 41.5 9.8 46.3 78.5 50.5 15.6 43.1 84.1 47.9 25.2
55.9 46.9 72.6 66.9 92.9
55.9 46.9 72.6 66.9 92.9
21.3 18.8 23.7 19.8 27.1
37.1 32.8 43.8 19.8 48.6
75.9 39.7 64.0 22.7 80.7
58.7 47.0 72.7 63.3 93.0
0.0 0.0 8.3 0.0 0.0
↑ 26.0 ↑ 58.0 ↑ 29.7 ↓ 0.0
86.1 65.3 79.9 53.9 89.6
59.7 49.6 74.3 66.5 93.8
31.1 15.0 38.2 25.8 39.4
↑ 26.0 ↑ 35.7 ↑ 27.3 ↑ 13.6
69.6 54.3 61.3 38.1 66.6
26.5 18.4 22.9 10.3 27.9 21.6 22.1 12.1 31.6 21.2
↑ 7.3 ↑ 28.5 ↑ 9.5
↑ 9.1
80.1 60.9 72.9 38.1 78.8
42.0 36.3 47.4 22.1 52.4
24.9 13.0 28.8 12.1 26.1
↑ 28.8 ↑ 40.7 ↑ 30.3 ↑ 14.0
Table 1: Comprehensive evaluation results across varying tool-use complexities. SH, MH, ST, and MT denote Single-Hop, Multi-Hop, Single-Turn, and Multi-Turn scenarios, respectively. All reported metrics are scaled to percentages (%). The best results are bolded and the second best results are underlined in all following tables.
Figure 4: Performance breakdown of UniToolCall across the 7 sub-datasets in our unified evaluation benchmark Dtest . For each benchmark, we report both hop-level (left) and turn-level (right) results using the unified metrics introduced in Section 3.3.
Mixed baseline across metrics. This indicates that our pipeline’s explicit constraint modeling effectively tackles deep inter-step dependencies. Conversely, as the parallel proportion rises, the Public Mixed baseline gradually catches up, benefiting from its inherent abundance of flat, independent invocations. This dynamic complementarity proves that while public data provides a robust baseline for parallel tasks through massive domain exposure, synthetic data is an indispensable supplement for injecting precise, controllable sequential reasoning. Ablation II: Mixing structural complexities within synthetic data We next zoom into the synthetic pipeline to compare pure homogeneous datasets against the mixed configuration. On the global benchmark (Table 2), while all synthetic variants improve over the vanilla model, taskspecific concentration only benefits in-domain metrics (e.g., pure multi-turn yields the strongest multi-
turn FP but suboptimal multi-hop SP). The necessity of a mixed curriculum is visually corroborated in our targeted BFCL v3 analysis (Figure 5). In the highly challenging sequential and balanced regions (low to medium x-axis values), the Synthetic Mixed setting maintains a strong upper bound among all synthetic variants. While homogeneous datasets like Pure Multi-Hop can perform competitively in highly parallel scenarios, they exhibit noticeable degradation when strict sequential logic is required. Meanwhile, Pure Single-Hop consistently lags behind across the entire spectrum. This demonstrates that specializing in a single interaction pattern limits generalization. Combining simpler extraction tasks with complex sequential routing creates a positive knowledge transfer, providing the most robust and balanced performance across varying reasoning complexities.
Figure 5: Performance trends across different data compositions under varying parallel-to-serial ratios.
Method
MH Ratio
MH
MT
(Ser:Par) SP (%)↑ SPA (%)↑ Vanilla Qwen3-8B
-
22.7
38.1
Total Δ-0.20
FP (%)↑
Tool-fit Δ-0.17
25.8
1 : 1.9 1 : 5.7
57.1 59.7
56.8 58.2
Δ-0.25
9.0
Ablation I: Public vs. Synthetic under matched budget Synthetic Mixed Public Mixed
10.0 Clarity 9.5
8.5
Anchor Δ-0.85
39.6 47.9
Naturalness Δ-0.19
8.0
Ablation II: Synthetic-only comparison (Homogeneous vs. Mixed) Pure Single-hop Pure Multi-hop Pure Multi-turn Synthetic Mixed
1 : 1.3 1 : 0.9 1 : 1.9
51.9 53.4 54.3 57.1
54.9 55.6 56.4 56.8
38.5 42.1 44.4 39.6
Table 2: Controlled analysis under a matched data budget (N = 979). MH Ratio denotes the proportion of serial to parallel multi-hop trajectories. Both Mixed datasets are constructed by proportionally sampling from their respective structural subsets.
Efficiency Δ+0.21
With Anchor
Success Δ+0.03
Without Anchor
Figure 6: Intrinsic data quality evaluation for the Anchor Linkage mechanism. Deltas (∆) indicate the score reduction or increase when the mechanism is removed.
5 Ablation III: Efficacy of the Anchor Linkage Mechanism We randomly sampled 10 multiturn samples generated with the Anchor Linkage constraint and 10 generated without it, which were evaluated using our LLM-based rubric (Appendix A.5). As illustrated in the radar chart (Figure 6), removing the anchor mechanism leads to a substantial performance drop on the specific Anchor dimension. This confirms that without explicit constraints, generative models struggle to produce later turns that consistently and functionally reference preceding states. Consequently, this lack of cross-turn continuity negatively impacts the Query Evaluation dimensions. Conversely, the unconstrained baseline yields slightly higher scores in Trajectory Evaluation metrics. This dynamic reflects a natural structural trade-off: when multiturn episodes lack strict inter-turn dependencies, they tend to degenerate into a series of decoupled, simpler single-turn interactions. Ultimately, these results demonstrate that the Anchor Linkage mechanism is indispensable for synthesizing genuinely coherent, complex multi-turn datasets.
Grounding Δ+0.16
Conclusion
In this paper, we presented UniToolCall, a unified framework for tool learning in LLM agents. Our framework standardizes the entire pipeline from toolset construction and hybrid data synthesis to evaluation under a shared QAOA representation. By integrating large-scale public corpora with structurally controlled synthetic trajectories, the resulting training dataset contains 390k+ instances covering diverse interaction patterns, including singlehop, multi-hop, single-turn and multi-turn scenarios with both serial and parallel execution structures. In addition, we construct a unified benchmark that enables fine-grained evaluation across function-call, turn, and conversation levels. Experiments show that models trained with our framework achieve strong improvements in tool selection and parameter generation, highlighting the importance of explicitly modeling structural diversity in tool-use data. In future work, we plan to extend the framework to longer-horizon agent interactions and further evaluate it in real-world environments with live tool execution.
Limitations Due to computational constraints, our experiments were conducted with a maximum context length of 8192 tokens, which restricts our exploration of extremely long-horizon interactions or scenarios involving large tool outputs (e.g., lengthy documents or database results). Second, our experiments primarily focus on a lightweight backbone (Qwen38B). While the framework significantly improves its performance and even surpasses several larger models, we did not systematically investigate scaling behavior on larger backbones (e.g., 30B+ or 70B+ models). Finally, there is a potential risk of evaluation distortion introduced by our rigorous data filtering and format conversion processes. Because our methodology relies on standardizing highly heterogeneous datasets into a unified benchmark, readers should be aware that the final evaluation results may not fully preserve all original task attributes or idiosyncratic features of the source benchmarks.
Ethics Statement For the integration of public data, we exclusively utilized open-source datasets that have been previously released under permissive licenses. During the synthetic data generation process, our prompting mechanisms and LLM-based planners were explicitly instructed to simulate fictitious user intents and generic business scenarios. We confirm that no personally identifiable information or sensitive user data was scraped, generated, or included in our final dataset. Furthermore, while our data synthesis relies on LLMs, which may inherently reflect societal biases, our multi-stage quality filtering and strict argument-grounding rubrics significantly mitigate the risk of generating unsafe or hallucinated content. All scientific artifacts, including base models and MCP server definitions, were used strictly in accordance with their intended purposes and licenses. Therefore, we believe that our research complies with the ACL Code of Ethics. We used ChatGPT and Gemini for minor language polishing and grammar correction. All technical content, experiments, and conclusions were generated and verified by the authors.
References Mayank Agarwal, Ibrahim Abdelaziz, Kinjal Basu, Merve Unuvar, Luis A. Lastras, Yara Rizk, and Pa-
van Kapanipathi. 2025. Toolrm: Outcome reward models for tool-calling large language models. arXiv preprint arXiv:2509.11963. Anthropic. 2026. Claude 4.6 sonnet. https://www. anthropic.com/news/claude-sonnet-4-6. Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang, Xingshan Zeng, Shuai Yu, Dexun Li, Yuefeng Huang, Xiangcheng Liu, Xinzhi Wang, and Wu Liu. 2025a. ACEBench: A comprehensive evaluation of LLM tool usage. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 12970– 12998, Suzhou, China. Association for Computational Linguistics. Hao Chen, Zhexin Hu, Jiajun Chai, Haocheng Yang, Hang He, Xiaohan Wang, Wei Lin, Luhang Wang, Guojun Yin, and Zhuofeng Zhao. 2025b. Toolforge: A data synthesis pipeline for multi-hop search without real-world apis. arXiv preprint arXiv:2512.16149. DeepSeek-AI, Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenhao Xu, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Erhang Li, Fangqi Zhou, Fangyun Lin, Fucong Dai, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Hao Li, Haofen Liang, Haoran Wei, Haowei Zhang, Haowen Luo, Haozhe Ji, Honghui Ding, Hongxuan Tang, Huangqi Cao, Huazuo Gao, Hui Qu, Hui Zeng, Jialiang Huang, Jiashi Li, Jiayin Xu, Jiewen Hu, Jingchang Chen, Jingting Xiang, Jingyang Yuan, Jingyuan Cheng, Jinhua Zhu, Jun Ran, Junguang Jiang, Junjie Qiu, Junlong Li, Junxiao Song, Kai Dong, Kaige Gao, Kang Guan, Kexin Huang, Kexing Zhou, Kezhao Huang, Kuai Yu, Lean Wang, Lecong Zhang, Lei Wang, Liang Zhao, Liangsheng Yin, Lihua Guo, Lingxiao Luo, Linwang Ma, Litong Wang, Liyue Zhang, M.S. Di, M.Y. Xu, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingxu Zhou, Panpan Huang, Peixin Cong, Peiyi Wang, Qiancheng Wang, Qihao Zhu, Qingyang Li, Qinyu Chen, Qiushi Du, Ruiling Xu, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, Runqiu Yin, Runxin Xu, Ruomeng Shen, Ruoyu Zhang, S.H. Liu, Shanghao Lu, Shangyan Zhou, Shanshuang Chen, Shaofei Cai, et al. 2025. Deepseek-v3.2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556. Zane Durante, Qiuyuan Huang, Naoki Wake, Ran Gong, Jae Sung Park, Bidipta Sarkar, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Yejin Choi, Katsushi Ikeuchi, Hoi Vo, Li Fei-Fei, and Jianfeng Gao. 2024. Agent ai: Surveying the horizons of multimodal interaction. CoRR, abs/2401.03568. Reza Esfandiarpoor, Vishwas Suryanarayanan, Stephen Bach, Vishal Chowdhary, and Anthony Aue. 2026. TheMCPCompany: Creating general-purpose agents with task-specific tools.
Xuanqi Gao, Siyi Xie, Juan Zhai, Shqing Ma, and Chao Shen. 2025. Mcp-radar: A multi-dimensional benchmark for evaluating tool use capabilities in large language models. CoRR, abs/2505.16700. Google. 2025. Gemini 3 flash — frontier intelligence built for speed. https: //blog.google/products-and-platforms/ products/gemini/gemini-3-flash/. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR).
Chang Ma, Junlei Zhang, Zhihao Zhu, Cheng Yang, Yujiu Yang, Yaohui Jin, Zhenzhong Lan, Lingpeng Kong, and Junxian He. 2024. Agentboard: An analytical evaluation board of multi-turn LLM agents. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track. MathAndMagic. 2024. Function calling dataset. https://huggingface.co/datasets/ MathAndMagic/function-calling. Hugging Face Dataset. MCP.so Team. 2025. Mcp.so: A registry for model context protocol servers. https://mcp.so/. Accessed: 2025-11-20.
Interstellar Ninja. 2025. Hermes reasoning tool use dataset. https://huggingface.co/datasets/ interstellarninja/hermes_reasoning_tool_ use. Hugging Face Dataset.
Suhong Moon, Siddharth Jha, Lutfi Eren Erdogan, Sehoon Kim, Woosang Lim, Kurt Keutzer, and Amir Gholami. 2024. Efficient and scalable estimation of tool representations in vector space. CoRR, abs/2409.02141.
Junaidjk. 2024. Function calling dataset. https://huggingface.co/datasets/Junaidjk/ function_calling. Hugging Face Dataset.
OpenAI. 2025. Introducing gpt-5.2. https://openai. com/index/introducing-gpt-5-2/.
Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, and Yongbin Li. 2023. API-bank: A comprehensive benchmark for tool-augmented LLMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3102–3116, Singapore. Association for Computational Linguistics. Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, Rithesh R. N., Liangwei Yang, Silvio Savarese, Juan Carlos Niebles, Huan Wang, Shelby Heinecke, and Caiming Xiong. 2024. Apigen: Automated pipeline for generating verifiable and diverse function-calling datasets. In NeurIPS. Xuan Lu, Haohang Huang, Rui Meng, Yaohui Jin, Wenjun Zeng, and Xiaoyu Shen. 2026. Tools are underdocumented: Simple document expansion boosts tool retrieval. In The Fourteenth International Conference on Learning Representations. Junyu Luo, Weizhi Zhang, Ye Yuan, Yusheng Zhao, Junwei Yang, Yiyang Gu, Bohan Wu, Binqi Chen, Ziyue Qiao, Qingqing Long, Rongcheng Tu, Xiao Luo, Wei Ju, Zhiping Xiao, Yifan Wang, Meng Xiao, Chenwu Liu, Jingyang Yuan, Shichang Zhang, Yiqiao Jin, Fan Zhang, Xian Wu, Hanqing Zhao, Dacheng Tao, Philip S. Yu, and Ming Zhang. 2025a. Large language model agent: A survey on methodology, applications and challenges. CoRR, abs/2503.21460. Ziyang Luo, Zhiqi Shen, Wenzhuo Yang, Zirui Zhao, Prathyusha Jwalapuram, Amrita Saha, Doyen Sahoo, Silvio Savarese, Caiming Xiong, and Junnan Li. 2025b. MCP-universe: Benchmarking large language models with real-world model context protocol servers. In Workshop on Scaling Environments for Agents.
Dhruvi Paprunia, Vansh Kharidia, and Pankti Doshi. 2025. Advancing slm tool-use capability using reinforcement learning. Preprint, arXiv:2509.04518. Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025. The berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning. Shishir G Patil, Tianjun Zhang, Xin Wang, et al. 2024. Gorilla: Large language model connected with massive apis. In Advances in Neural Information Processing Systems (NeurIPS), volume 37, pages 126544–126565. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, dahai li, Zhiyuan Liu, and Maosong Sun. 2024. ToolLLM: Facilitating large language models to master 16000+ real-world APIs. In The Twelfth International Conference on Learning Representations. Ranjan Sapkota, Konstantinos I. Roumeliotis, and Manoj Karkee. 2026. Ai agents vs. agentic ai: A conceptual taxonomy, applications and challenges. Information Fusion, 126:103599. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. CoRR, abs/2302.04761. Zhengliang Shi, Yuhan Wang, Lingyong Yan, Pengjie Ren, Shuaiqiang Wang, Dawei Yin, and Zhaochun Ren. 2025. Retrieval models aren’t tool-savvy: Benchmarking tool retrieval for large language models. In ACL (Findings), pages 24497–24524.
Qiaoyu Tang, Ziliang Deng, Hongyu Lin, Xianpei Han, Qiao Liang, and Le Sun. 2023. Toolalpaca: Generalized tool learning for language models with 3000 simulated cases. CoRR, abs/2306.05301.
Huang, and Jiecao Chen. 2025. Toolhop: A querydriven benchmark for evaluating large language models in multi-hop tool use. In ACL (1), pages 2995– 3021.
Xiaoyu Tian, Haotian Wang, Shuaiting Chen, Hao Zhou, Kaichi Yu, Yudian Zhang, Jade Ouyang, Junxi Yin, Jiong Chen, Baoyan Guo, Lei Zhang, Junjie Tao, Yuansheng Song, Ming Cui, and Chengwei Liu. 2026. Astra: Automated synthesis of agentic trajectories and reinforcement arenas. Preprint, arXiv:2601.21558.
Chaojia Yu, Zihan Cheng, Hanwen Cui, Yishuo Gao, Zexu Luo, Yijin Wang, Hangbin Zheng, and Yong Zhao. 2025. A survey on agent workflow – status and future. In Proceedings of the 2025 8th International Conference on Artificial Intelligence and Big Data (ICAIBD), pages 770–781. IEEE.
Vikhrmodels. 2024. Tool plannings v0.2. https: //huggingface.co/datasets/Vikhrmodels/ tool-plannings-v0.2. Hugging Face Dataset. Jun Wang, Jiamu Zhou, Muning Wen, Xiaoyun Mo, Haoyu Zhang, Qiqiang Lin, Cheng Jin, Xihuai Wang, Weinan Zhang, Qiuying Peng, and Jun Wang. 2024. Hammerbench: Fine-grained function-calling evaluation in real mobile device scenarios. CoRR, abs/2412.16516. Zhenting Wang, Qi Chang, Hemani Patel, Shashank Biju, Cheng-En Wu, Quan Liu, Aolin Ding, Alireza Rezazadeh, Ankit Shah, Yujia Bao, and Eugene Siow. 2025. MCP-bench: Benchmarking tool-using LLM agents with complex real-world tasks via MCP servers. In Workshop on Scaling Environments for Agents. Mengsong Wu, Tong Zhu, Han Han, Chuanyuan Tan, Xiang Zhang, and Wenliang Chen. 2024. Seal-tools: Self-instruct tool learning dataset for agent tuning and detailed benchmark. In NLPCC (2), pages 372– 384. Zhangchen Xu, Adriana Meza Soria, Shawn Tan, Anurag Roy, Ashish Sunil Agrawal, Radha Poovendran, and Rameswar Panda. 2026. TOUCAN: Synthesizing 1.5m tool-agentic data from real-world MCP environments. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jian Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. 2025. Qwen3 technical report. CoRR, abs/2505.09388. Junjie Ye, Zhengyin Du, Xuesong Yao, Weijian Lin, Yufei Xu, Zehui Chen, Zaiyuan Wang, Sining Zhu, Zhiheng Xi, Siyu Yuan, Tao Gui, Qi Zhang, Xuanjing
Kangning Zhang, Wenxiang Jiao, Kounianhua Du, Yuan Lu, Weiwen Liu, Weinan Zhang, Lei Zhang, and Yong Yu. 2025. Looptool: Closing the data-training loop for robust llm tool calls. CoRR, abs/2511.09148. Lucen Zhong, Zhengxiao Du, Xiaohan Zhang, Haiyi Hu, and Jie Tang. 2025. Complexfuncbench: Exploring multi-step and constrained function calling under long-context scenario. arXiv preprint arXiv:2501.10132.
Appendix A
Dataset details
A.1
Toolset construction
As illustrated in Figure 7, the toolset is formally defined as the union of six distinct subsets drawn from three primary sources: (1) Academic benchmarks: We integrated tools from established benchmarks to ensure comparability. This includes FCR EWARD B ENCH (Tfc ) (Agarwal et al., 2025) and T OOL R ET- TRAIN (Tret ) (Shi et al., 2025). (2) MCP servers: To capture real-world tool usage patterns, we collected Model Context Protocol (MCP) servers. This subset comprises the top 40 servers listed on mcp.so at the time of collection (Tso ) (MCP.so Team, 2025) and 11 servers utilized in MCP-U NIVERSE (Tuni ) (Luo et al., 2025b). The specific list of MCP servers is provided in Table 3. (3) Constructed datasets: This subset includes the specific tool definitions extracted from the training (Ttrain ) and test (Ttest ) datasets constructed in this study. A.2
Tool classification taxonomy
Functional Categories Based on common API usage patterns observed in agent systems, we define six functional categories: • Analysis: Data analysis and insights (statistical analysis, trend analysis, data mining, predictive analysis, business intelligence, etc.) • Operations: Business process operations (create, update, delete, workflow management, business logic execution, etc.) • System: System administration and maintenance (system configuration, user management, system monitoring, technical maintenance, etc.) • Visualization: Data visualization and presentation (chart generation, report creation, data display, dashboard creation, etc.) • Search: Information retrieval and search (fulltext search, fuzzy search, index query, structured query, data lookup, etc.) • Generate: Content and data generation (content generation, code generation, intelligent recommendation, AI generation, automated creation, etc.) Application domains Tools are further associated with one of thirteen application domains to reflect real-world usage scenarios:
• Finance: Finance related (payment, investment, wealth management, insurance, trading, etc.) • Technology: Technology and software development (programming, system management, software tools, IT infrastructure, etc.) • Education: Education and learning (academic courses, training programs, educational content, learning management, etc.) • Healthcare: Medical and health services (medical treatment, health monitoring, medical devices, healthcare management, etc.) • Entertainment: Entertainment and media (music, games, film/TV, social entertainment, news, content creation, etc.) • Travel: Travel and transportation (tourism, transportation, accommodation, attractions, travel planning, etc.) • Business: Business management (enterprise operations, marketing, customer relations, business processes, etc.) • Lifestyle: Daily life services (shopping, food, housekeeping, personal tools, consumer services, etc.) • Science: Scientific research and analysis (research projects, scientific experiments, academic studies, data analysis, etc.) • Social: Social communication and community (social networking, communication tools, community management, collaboration, etc.) • Sports: Sports and fitness (sports activities, fitness training, sports events, athletic performance, etc.) • Environment: Environment and sustainability (environmental protection, climate monitoring, ecology, sustainable development, etc.) • Culture: Culture and arts (art, literature, history, cultural events, language learning, creative content, etc.) A.3
Toolset filtering details
Because the collected tools originate from heterogeneous sources, the raw pool contains redundancy and incomplete definitions. As illustrated in Figure 7, we therefore apply a multi-stage filtering pipeline to improve tool quality and ensure fair evaluation. First, we remove exact duplicates within and across subsets based on tool names and descriptions. Second, we exclude tools whose schemas rely on temporal attributes, since different benchmarks adopt inconsistent conventions for resolving relative time expressions (Appendix B.1). Third,
Servers from mcp.so (Top 40) 302_browser_use_mcp aws-kb-retrieval-server context7 fetch gitlab mailtrap-email-sending-mcp neon-mcp-server postgresql search1api slack
302_sandbox_mcp baidu-map devcontext firecrawl-mcp-server google-maps mcp-advisor notion-mcp-server puppeteer sentry time
agentql-mcp-server blender edgeone-pages-mcp framelink-figma-mcp-server howtocook-mcp mcp-server-flomo-mcp-server perplexity-ask-mcp-server qiniu-mcp-server sequential-thinking todoist-mcp
amap-maps brave-search everart github jina-ai-mcp-tools minimax-mcp playwright-mcp redis serper-mcp-server zhipu-web-search
Servers from MCP-Universe (11 Servers) blender github playwright
calculator google-maps weather
date google-search yfinance
fetch notion
Table 3: The complete list of collected MCP servers used in our toolset construction.
we discard tools with missing or invalid parameter schemas to ensure that each tool provides sufficient information for argument generation. Finally, we perform semantic deduplication using embedding similarity to remove functionally redundant tools with different names. Exact deduplication Tools extracted from public datasets often contain duplicates because identical tool definitions appear in multiple query–tool pairs. We first perform intra-subset deduplication by removing entries with identical tool names and descriptions. This is followed by inter-subset deduplication across different tool sources. To preserve dataset consistency, tools belonging to Ttrain and Ttest are retained even when duplicates are detected across external subsets. Schema validation We remove tools with incomplete definitions, such as those lacking a valid parameter schema. Tools that contain only a name or description without argument specifications cannot provide sufficient supervision for learning the mapping between user queries and structured tool arguments. Semantic deduplication To identify semantically redundant tools with different names, we encode the concatenation of each tool’s name and description using Q WEN 3-E MBEDDING -8B. Cosine similarity is computed using FAISS. Tools with similarity greater than 0.9 are considered duplicates. When duplicates are detected, instances from external subsets are removed while those belonging to Ttrain and Ttest are retained to preserve
dataset consistency. A.4
Public data quality evaluation and filtering
To construct a structurally consistent training and evaluation corpus, we apply uniform filtering principles across all public datasets. Table 4 summarizes the dataset-level normalization and filtering applied to all public corpora prior to integration. Across all datasets, we enforce the following dataset-agnostic criteria: • Schema completeness: each sample must contain a well-formed user query, a valid tool call (or candidate API schema), and—when applicable—observations and final answers. • Executable supervision: we discard items with missing function calls, incomplete parameter specifications, invalid JSON structure, or empty/malformed ground-truth traces. • Language normalization: only Englishlanguage user queries and assistant messages are retained. • Toolset compatibility: samples referencing tools removed during tool filtering (e.g., temporal-sensitive or redundant tools) are excluded. • Invalid-category filtering: subsets explicitly marked as irrelevant or lacking actionable ground truth are removed. • Deterministic evaluation: for test sets, we retain only samples for which function calls and argument mappings can be deterministically reconstructed.
Multi-Stage Toolset Quality Filtering Flow Ttrain(15,078) Preserved
Ttrain(15,078) Ttrain(30,199)
Ttrain(30,199) Preserved
Ttest(2,257) Preserved
Ttrain(23,372) Removed (Empty Schema) Ttest(2,257)
Ttest(5,113)
Ttest(5,113) Preserved
Ttest(3,610) Removed (Temporal)
Removed (Exact Match) Tret(31,123)
Tfc(1,983) Tuni(119) Tso(275)
Tret(25,014)
Tfc(1,980) Tuni(90) Tso(247)
Tret(19,516)
Tfc(974) Tuni(68) Tso(205)
Tret(16,603) Removed (Semantic)
Tfc(949)
Tret(4,944)
Tuni(64) Tso(197)
Tfc(127) Tso(150) Tuni(50)
Figure 7: The multi-stage data reduction flow of our toolset quality filtering process. Gray indicates the tool being deduplicated, and purple indicates the tool being retained.
After standardization, we further filtered samples based on the finalized training and evaluation tool inventories, removing conversations that referenced tools excluded during toolset filtering. The statistics are illustrated in Table5. A.5
Synthetic data quality evaluation and filtering
To ensure the quality and consistency of the synthetically generated dataset without relying on external proprietary models, we design a unified LLMbased self-evaluation framework. The generator model (Q WEN 3-32B) evaluates its own generated QAOA trajectories across a set of fine-grained metrics. The framework is shared across single/multihop and single/multi-turn datasets, with minor extensions for multi-turn episodes. Evaluation dimensions The evaluation rubric consists of six core metrics grouped into two dimensions: Query Evaluation and Trajectory Evaluation. Each metric is scored on a scale from 1 to 10 by the generator model. Query evaluation This dimension evaluates the initial user query q with respect to the available tools:
• Tool-fit: Whether the query is appropriately designed around the available tool capabilities and implicitly or explicitly provides the necessary parameters. • Clarity: Whether the task specification is unambiguous, well-defined, and provides sufficient constraints for planning a valid solution. • Naturalness: Whether the query resembles a realistic user request in a practical scenario rather than a templated or system-style prompt. Trajectory evaluation This dimension evaluates the correctness and coherence of the generated trajectory consisting of Action (a), Observation (o), and Answer (r): • Success: Whether the generated tool calls and final answer successfully complete the user’s task. • Grounding: Whether the final response is strictly supported by the simulated observations, without hallucinated facts or inconsistent parameters. • Efficiency: Whether the trajectory completes the task using a concise and non-redundant sequence of tool calls.
Dataset
Processing Summary
BFCL (Patil et al., 2025)
Removed irrelevance, live_irrelevance, live_relevance, and multi_turn_miss_func subsets; dropped non-English queries; converted 3,065 valid samples to the unified QAOA format. Retained subsets with deterministic function-call mapping; removed atom-type subsets; normalized query–function structures; added consistent gold function-call annotations. Preserved train/dev/test-in/out-domain partitions; standardized tool schemas; converted all entries into QAOA with explicit tool definitions. Excluded multi-turn parameter-filling subsets; processed single-turn samples with new identifiers; mapped tool definitions and integrated unified system prompts. Standardized multi-step API sequences into multi-hop trajectories; ensured consistent JSON formatting; retained 1,000 normalized samples. Kept only Level-3 (Plan+Retrieve+Call) samples; removed Level-1/2 subsets requiring missing user inputs; transformed remaining items into QAOA structure. Transformed API descriptions and queries into single-hop QAOA format; removed structurally inconsistent items. Retained items convertible to multi-hop trajectories; removed unresolved-hop subsets; normalized argument formats and tool identifiers. Filtered structurally invalid entries from 60,000 raw items; standardized tool schemas; integrated 17,178 valid samples into QAOA. Unified formatting into QAOA; removed schema-mismatched or incomplete entries; retained samples with valid function-call traces. Standardized tool schemas; resolved formatting inconsistencies; retained entries convertible to well-formed function calls. Normalized function-calling traces; removed incomplete or invalid entries; retained consistent QAOA-formatted samples. Converted 1.37M raw samples; removed incomplete or structurally inconsistent trajectories; retained 319,669 QAOA-normalized conversations.
ACEBench (Chen et al., 2025a) Seal-Tools (Wu et al., 2024) HammerBench (Wang et al., 2024) ComplexFuncBench (Zhong et al., 2025) API-Bank (Li et al., 2023) ToolAlpaca (Tang et al., 2023) ToolHop (Ye et al., 2025) APIGen (Liu et al., 2024) Junaidjk (Junaidjk, 2024) Vikhrmodels (Vikhrmodels, 2024) MathAndMagic (MathAndMagic, 2024) Toucan (Xu et al., 2026)
Table 4: Public data quality evaluation and filtering.
Multi-Turn anchor evaluation Dataset
Conv.
Filt.
API-Bank (Li et al., 2023) ToolAlpaca (Tang et al., 2023) ToolHop (Ye et al., 2025) APIGen (Liu et al., 2024) Seal-Tools (Wu et al., 2024) Toucan (Xu et al., 2026) Tool-calling (Interstellar Ninja, 2025) Junaidjk (Junaidjk, 2024) Vikhrmodels (Vikhrmodels, 2024) MathAndMagic (MathAndMagic, 2024)
338 4,096 995 60,000 12,022 1,367,983 35,786 13,850 3,396 22,218
122 2,429 7 28,666 5,214 319,669 8,692 3,470 2,493 16,361
Total (Train)
1,520,684
387,123
Training Data (Dpub )
Evaluation Benchmark (Dtest ) BFCL V3 (Patil et al., 2025) ACEBench (Chen et al., 2025a) Seal-Tools (Wu et al., 2024) HammerBench (Wang et al., 2024) ComplexFuncBench (Zhong et al., 2025) API-Bank (Li et al., 2023) ToolAlpaca (Tang et al., 2023)
3,065 250 1,354 6,531 1,000 50 209
984 59 579 4,340 51 35 145
Total (Test)
12,459
6,163
Table 5: Statistics of the public datasets integrated into our framework. Conv. (Converted Count) represents the initial number of conversations obtained after standardizing the raw heterogeneous data into our unified QAOA format. Filt. (Filtered Count) indicates the final retained size after our rigorous quality filtering mechanism.
For multi-turn episodes, we extend the sixdimensional rubric with an additional episode-level metric: • Anchor Linkage (sanchor ): Measures whether later turns explicitly and consistently reference anchors introduced in previous turns, and whether such references are functionally meaningful for subsequent tool usage. Acceptance thresholds For all synthetically generated candidates that pass basic schema and formatting checks, we apply strict acceptance criteria based on the evaluation scores. Single-Hop and Multi-Hop instances A trajectory is accepted only if the following conditions are simultaneously satisfied: • Minimum score constraint: The lowest score among all six metrics must be at least 4.0 (min(S) ≥ 4.0). • Average score constraint: The average score across the six metrics must be at least 8.0 (avg(S) ≥ 8.0).
These constraints ensure that no individual dimension is critically flawed while maintaining high overall quality.
• Specific scenarios: start_time, end_time, start_date, end_date, pickup_time, dropoff_time, etc.
Multi-Turn episodes For multi-turn data, acceptance is determined by a weighted comprehensive score:
Matching patterns We support multiple naming conventions to ensure comprehensive coverage:
S = 0.4 Queryavg + 0.4 Trajectoryavg + 0.2 sanchor
(1)
A multi-turn episode is accepted only if S ≥ 8.0 and the minimum score across all dimensions is at least 4.0. Self-Refinement loop If a generated trajectory fails to satisfy the above criteria, the pipeline triggers an automatic selfcorrection loop. The generator model is instructed to regenerate the trajectory for the same target tool, with a maximum of three retries. If no valid trajectory is produced after all attempts, the corresponding tool is excluded from the synthetic dataset Dsyn . A.6
Statistics of DPub
Figure 8 shows the comprehensive statistics of our unified training dataset DPub . Collectively, these distributions highlight the dataset’s high diversity and rigorous complexity, highlighting the diversity and structural complexity of the dataset.
B
Benchmark details
B.1
Temporal filtering criteria
Different benchmarks adopt inconsistent conventions for handling temporal parameters. For example, ACEBench resolves relative expressions (e.g., tomorrow) into absolute timestamps, whereas HammerBench preserves the original relative expressions. To ensure the exclusion of time-sensitive tools that may introduce evaluation bias, we implemented a keyword-based filtering mechanism based on the following criteria: Keyword list • Core keywords: date, dates, time, times, datetime, timestamp • Units: day, days, hour, hours, minute, minutes, second, seconds • Periods: year, years, month, months, week, weeks • Actions/Properties: when, schedule, scheduled, duration, period, periods
• Snake case: e.g., travel_date, start_time • Kebab case: e.g., travel-date, start-time • Camel case: e.g., travelDate, startTime • Word boundary: Isolated occurrences of keywords (e.g., date, time) B.2
Rule-based matching details
Rule-based matching is implemented as strict exact matching after deterministic normalization. This stage is designed to treat formatting variance as equivalent while preserving hard correctness constraints. Tool name normalization Tool names are normalized by removing punctuation, digits, and separators, and then converting to lowercase. For example, uber.ride and uber_ride become identical after normalization. Parameter value normalization. To robustly compare parameter values across heterogeneous outputs, we apply the following canonicalization rules: • Date canonicalization: date strings in multiple formats (e.g., April 1, 2023, 2023-04-01, and 2023/04/01) are normalized to YYYYMM-DD. • Array parsing: stringified arrays (e.g., [1, 2, 3]) are parsed into actual arrays before comparison. • String normalization: strings are lowercased, punctuation and articles (a/an/the) are removed, and whitespace is ignored; e.g., A black cat and blackcat are treated as identical. • Type casting: mixed representations of the same value are unified, including numeric string–number equivalence (e.g., "40.7128" and 40.7128). Matching criteria After normalization, rulebased matching requires full equality on both normalized tool names and all normalized argument key-value pairs. Such normalization improves evaluation fairness and reproducibility by removing superficial formatting variance while preserving exact semantic correctness constraints.
Figure 8: Comprehensive statistics of our unified training dataset DPub . The top row illustrates the structural complexity and scale, including the distribution of tool calls per sample, token length density, and the proportions of multi-turn and multi-hop trajectories. The bottom row demonstrates the broad semantic diversity across tool domains and functional categories, alongside conversation density metrics. Note that Messages per Sample reflects the total count of user queries, tool calls, environment observations, and assistant answers within a single dialogue episode. Model
SH SP (%)
MH FP (%)
SH SPA (%)
MH FPA (%)
Qwen3-8B (Vanilla) UniToolCall
67.7 ± 1.3 93.9 ± 1.0
22.9 ± 1.7 80.5 ± 0.5
39.1 ± 2.4 67.0 ± 0.7
39.1 ± 2.4 78.6 ± 0.3
Table 6: Repeated-run statistics for hop-level metrics. Model
ST SP (%)
MT FP (%)
ST FPA (%)
MT SPA (%)
Qwen3-8B (Vanilla) UniToolCall
64.1 ± 1.3 93.8 ± 0.8
27.5 ± 3.4 37.8 ± 1.6
22.5 ± 0.7 52.8 ± 0.4
13.2 ± 1.3 16.6 ± 4.0
Table 7: Repeated-run statistics for turn-level metrics.
C
Experimental details
C.1
Run-level stability statistics
Tables 6 and 7 report repeated-run statistics for the trainable backbone models. Both UniToolCall and the vanilla Qwen3-8B are trained three times with independent runs. The reported values are mean ± sample standard deviation across runs. The repeated-run statistics show that the improvements of UniToolCall over the vanilla backbone remain consistent across runs, particularly for single/multi-hop, and single-turn tool selection. Variance remains relatively small for most metrics. Multi-turn metrics exhibit larger fluctuations due to the small evaluation size (36 conversations) and the inherent difficulty of strict conversation-level matching. C.2
Example of the action-only training format
Observation and Answer fields are retained in the dataset for evaluation purposes but are not part of the prediction target during fine-tuning.
This design isolates the model’s tool-selection and parameter-generation capabilities from downstream response realization. To illustrate this structure, a concrete data sample of a single-hop scenario is presented below.
Data Sample (Psys ) {
}
"conversations":[ { "from": "human", "value": "Provide secure access to medical records for a patient named John Smith." }, { "from": "function_call", "value": "{\"name\":\"MedicalRecordAccess\",\"arguments\":{\"patient_name\":\"John Smith\"}}" }, { "from": "observation", "value": "" }, { "from": "gpt", "value": "<answer></answer>" } ], "system": "# Role\n\nYou are an AI assistant capable of calling various functions to help users solve their problems.\n \n# Tool Selection\n\n **Important**: The available function signatures are provided in the <tools></tools> section. You must carefully select one or more appropriate tools from this section that can solve the user's request.\n\n # Output Rules\n\nYou must strictly follow the rules below when responding:\n\n ## 1. Function Call Format\nWhen you need to call a function, you must output only one function call per round in the following format:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call>\n\n **Parameter Parsing**: The arguments must be parsed based on the user's query. **Do not fabricate parameters that are not mentioned or cannot be reasonably inferred from the query.** Only use parameters that can be reasonably extracted or inferred from the user's request.\n\n **Basis for Generating Function Call Content**:\n- **First function call**: The user's query and available tools information.\n- **n-th function call (n > 1)**: The user's query, available tools information, and the complete conversation history in <chat_history></chat_history> from the previous n-1 rounds (including all prior function calls, observations, and answers). In some scenarios, observations may be empty; this is acceptable for generating function calls.\n\n **Example**:\n<tool_call>\n{\\\"name\\\": \\\"cancel_booking\\\", \\\"arguments\\\": {\\\"access_token\\\": \\\" abc123xyz\\\", \\\"booking_id\\\": \\\"flight_001\\\"}}\n</tool_call>\n\n ## 2. Answer Format\nWhen you judge from the chat history that all necessary tools have been called, you must immediately stop calling tools and provide the final answer in the following format:\n<answer>\nYour final answer here\n</answer>\n\n **Answer Generation Requirements**:\n- **Critical**: If all observations in chat-history are empty (meaning tools were called but returned no data), you MUST reply exactly: \"Sorry, I did not obtain sufficient information to complete your request.\" Do NOT fabricate, invent, or generate any content based on assumptions. Do NOT create imaginary results or responses. Only output this exact message.\n- **Important**: The provided tools may include tools that are irrelevant or unsuitable for the current query. If you determine there are no suitable tools to answer the user 's request, reply: \"Sorry, there are no suitable tools to answer your request.\"\n- **Important**: If you have called some tools and obtained observations, but the available tools are insufficient to fully satisfy the user's request (e.g., some required tools are missing from the available tool list), you MUST reply exactly: \"Sorry, there are not enough tools to fully satisfy your request.\" Do NOT fabricate or generate partial answers based on incomplete information.\n- Carefully analyze the conversation history to determine the current turn. The answer must be based on the user's query and all available observation results in the conversation.\n\n ## 3. Intelligent Process Stage Judgment\n- single-hop: Typically requires only one tool call to complete the task.\nmulti-hop: Requires multiple tool calls to complete the task.\n- single-turn: Involves only one user query.\nmulti-turn: Involves multiple user queries; later queries may refer to or build upon earlier exchanges.\n- When you see that the assistant has issued a tool call and received an observation, that tool call is considered complete.\ n\n **Special Note**: By examining the conversation history, you can clearly see:\n- Previous interactions between the user and the assistant\n- Tool calls that have already been executed\n- Results returned by tools\n- The stage the current conversation has reached\n\n ## 4. Strictly Prohibited Behaviors\n- Do not output a function call and an answer in the same round.\n- Do not repeatedly call the same tool with identical parameters.\n- Do not ignore existing tool calls and their returned information in the conversation history.\n- Do not fabricate parameters that are not present in or reasonably implied by the user's query.\n\n ## 5. Error Handling and Data Quality Assessment\n- If the tool returns an empty observation, it may indicate there is no data under the current query conditions or that observation data is unavailable in the current context.\n- If the tool returns error messages (e.g., \"resource not found\", \"invalid parameters\"), do not repeat the same tool call.\n- In such cases, provide an explanatory answer describing the specific error cause or data condition.\nAbsolutely do not repeatedly call the same tool because it returned an error or empty data.\n\n\n", "tools": "[\n {\"name\": \"MedicalRecordAccess\", \n \"description\": \"API for providing secure access to medical records.\", \n \"category\": \"operations\", \n \"domain\": \"healthcare\", \n \"inputSchema\": {\"type\": \"object\", \"properties\": {\"patient_name\": {\"type\": \"str\", \"description\": \"The name of the patient.\"}}, \"required\": [\"patient_name\"]}\n }, \n ... [18 distractor tools omitted for brevity] ..., \n \n {\"name\": \"update_with_defaults\", \n \"description\": \"Updates the defaults dictionary with the values from the updates dictionary.\", \n \"inputSchema\": {\"type\": \"object\", \"properties\": {\"defaults\": {\"type\": \"object\", \"additionalProperties\": {\"type\": \"integer\"}, \"description\": \"The default dictionary to be updated.\"}, \"updates\": {\"type\": \"object\", \"additionalProperties\": {\"type\": \"integer\"}, \"description\": \"The dictionary containing updates to apply to the defaults.\"}}, \"required\": [\"defaults\", \"updates\"]}, \n \"category\": \"operations\", \n \"domain\": \"technology\"\n }\n]"