arXiv:2606.06566v1 [cs.SE] 4 Jun 2026
NTILC: Neural Tool Invocation via Learned Compression
Andrew Krikorian Department of Robotics University of Michigan Ann Arbor, MI 48104 [email protected]
Yayuan Li Department of ECE University of Michigan Ann Arbor, MI 48104 [email protected]
Jason Corso Department of Robotics University of Michigan Ann Arbor, MI 48104 [email protected]
Abstract Agentic tool-calling language models depend on large registries of callable APIs, functions, and local actions. Placing full tool specifications directly in the prompt incurs a cost that scales linearly with the size of the tool registry, rapidly consuming the context budget. As the registry grows, this leads to higher latency and degrades selection accuracy, particularly due to interference from irrelevant tools. We overcome these limitations by introducing NTILC, a neural tool selection and invocation framework that replaces in-context registry look-up with learned latent retrieval. NTILC maps both user intent and tool specifications into a shared embedding space, enabling tool selection via external retrieval rather than incontext lookup. The language model is conditioned only on the selected tool schema, allowing for precise, constrained argument generation. Central to our approach is a signature-aware composite objective, which augments semantic similarity with constraints derived from tool signatures (e.g., argument schema, type compatibility, and return types). By combining Circle Loss with a Functional Margin Loss, the model enforces separation between tools that are semantically similar but incompatible under their execution signatures. We evaluate NTILC on public tool-selection and function-calling datasets and report context token usage, retrieval accuracy, and selection latency metrics. Across these settings, NTILC reduces context window consumption by over 95% and inference latency by up to 74% compared to long-context ICT baselines.
1
Introduction
Language models have increasingly become the reasoning core of agents Parisi et al. [2022], Li et al. [2025], Plaat et al. [2025]. In this architectural framework, agents are defined as LLMs augmented with tool registries, operating through a modular pipeline comprising task planning, tool discovery, parameter generation, and response synthesis. Modern agents extend these capabilities by increasing their tool registries: web APIs, code interpreters, database queries, and shell commands, instead of encoding all information in model weights. While this integration has unlocked a new tier of agentic autonomy, it has also induced a practical bottleneck due to vastly increased token usage, causing latency issues and context rot. The standard approach to tool-augmented inference Mialon et al. [2023], Shtok et al. [2024], Ye et al. [2026], which we refer to as In-Context Tooling (ICT), works as follows: before the model sees the user’s request, every available tool definition is inserted into the context window. The model then receives a query from the user, reads the full registry, and generates a structured invocation. This approach is simple and effective for small tool registries, but it imposes a token cost that scales linearly with the number of tools. Across the datasets used in our evaluation, tool schemas contain approximately 120 tokens on average. Therefore, a registry with 5, 000 tools would require roughly Preprint.
Context Window
(a) Baseline ICT Method (Full Context)
ICT Tool Selection
(Full Registry Parsing)
Tool Registry
System
Prompt
Tool: Edit - Edits a text
Tool: Calculator - Adds and subtracts numbers
Tool: Search - Queries the web using Google
Tool: Edit - Edits a text
Tool: Search - Queries the web using Google
...
User
Query
asdasdaasdasdasdasda sdasdasdasasdasdaasd asdasdasdasdasdasdas asdasdaasdasddasdasd asdasdasasdasdaasdas dasdasdasdasdasdasas dasdaasdasdasdasdasd asdasdasasdasdaasdas dasdasdasdasdasdas
Tool
Loop
Result: Higher context and irrelevant tools in context
User
Query
95%+
Context
Reduction
NTILC Tool Selection
(Efficient Metric-based)
Saved Context
Saved Tool Registry Context
Tool Loop
Context Window
(b) NTILC Method (Reduced Context)
System
Prompt
~100K+
Mechanism: Implicit text-based reasoning across all tools at every step
Full Context Used (~100K tokens)
(Pre-computed)
Input: Full tool registry Tool: Calculator - Adds and subtracts numbers
Tool: Search - Queries the web using Google
Relative Context
Usage (Tokens)
Intent Latent
Space
Functional
Margin Loss
< 5K Mechanism: Uses pre-computed tool embeddings to retrieve a small, relevant subset.
Result: Lower context, fewer irrelevant tools
Reduced Context Used (<5K tokens)
Baseline
(ICT)
Proposed
Proposed
(NTILC) (NTILC)
Figure 1: Relative context reduction achieved by NTILC compared to baseline ICT methods.
600, 000 input tokens before the model sees any task-specific information. Although a long-context model can technically accommodate such registries, the overhead causes increased inference costs and degrades accuracy Liu et al. [2023], Hosseini et al. [2024], Wang et al. [2026], Paulsen [2026]. To address these limitations, we propose NTILC (Neural Tool Invocation via Learned Compression). Rather than forcing the agent to read raw text schemas at every call, NTILC compresses the entire tool registry into an embedding space. At inference time, the agent produces a natural language intent: a concise description of the action required to obtain a specific piece of information. This intent serves as a query, which is embedded and used to retrieve the most relevant tool via nearest-neighbor search in the shared embedding space. This eliminates the tool registry’s footprint from the context window entirely, reducing tool registry prompt tokens from O(N ) to O(1) regardless of how many tools are registered. A key challenge in tool selection is that tools frequently share nearly identical natural language descriptions yet have incompatible arguments. For example, getWeather(zipcode: int) and getWeather(lat: float, lon: float) describe the same concept but cannot be substituted for one another. This causes models to select semantically relevant but functionally incompatible tools. We term this phenomenon semantic blur. Our contributions: • We introduce NTILC, a framework that replaces linear-scaling in-context registry scanning with an external learned dispatch step, reducing registry prompt tokens from O(N ) to 0 with respect to the number of registered tools. • We propose a Functional Margin (FM) Loss that applies a repulsive force between tools whose descriptions are similar but whose executable signatures differ. Combined with Circle Loss for query-tool alignment, this composite objective produces an embedding space that is not only semantically coherent but also functionally compatible. • We evaluate NTILC on public tool-selection and function-calling datasets including ToolBench, BFCL, API-Bank, MetaTool, and ToolEyes, and report token cost, latency, and accuracy under increasing registry size.
2
Related Work
The evolution of tool-augmented large language models has primarily focused on expanding the knowledge boundaries of models through external interfaces Mialon et al. [2023]. We survey three lines of work that are most relevant to NTILC. 2
In-Context Tooling (ICT) and Prompt-Based Methods. Prompt-based tool-use systems provide tool descriptions directly in the system prompt Mialon et al. [2023]. ICT is effective for small toolsets but suffers from a linear registry cost: as new tools are added, the registry consumes a growing share of the context window, increases prefill work, raises input-token costs, and exposes the model to more distractor tools Patil et al. [2023], Wang et al. [2025]. NTILC addresses this by removing the full registry from the prompt. Retrieval-Augmented Tool Selection. As tool libraries scale to hundreds or thousands of APIs, research has shifted toward retrieval-based architectures Qin et al. [2023a]. Typical implementations use sparse or dense retrieval to fetch a small set of relevant tool schemas into the context window dynamically Patil et al. [2023], Qin et al. [2023a]. This reduces the prompt from the full registry to top-k candidate schemas, but it does not remove tool text from the LLM prompt and remains vulnerable to semantically plausible distractors Wang et al. [2025]. Semantic retrieval often fails when tools share a topic but have incompatible functional signatures. Unlike standard RAG-style tool selection, NTILC treats retrieval as the dispatch decision itself: the full registry is never injected into the prompt, and only the selected schema is exposed to the decoder as an output constraint for argument generation. Efficiency and Context Optimization. Recent efforts have focused on memory compression, prompt caching, and long-context models Kang et al. [2025]. These techniques reduce the prompt length but do not change the fact that in-context tool registries scale linearly with the number and verbosity of tools Wang et al. [2025]. NTILC is complementary to long-context and caching techniques: while those methods reduce the cost of what is already in the prompt, NTILC eliminates the registry from the prompt entirely, meaning the two can be composed to reduce both residual prompt overhead and registry footprint simultaneously.
3
Method
Problem Setting Given a user request χ and a registry of tools T = {t1 , . . . , tN }, the objective is to (i) identify the appropriate tool t∗ ∈ T that satisfies the request, and (ii) generate a valid set of arguments a∗ consistent with the selected tool’s interface. Each tool t ∈ T is defined by a schema specifying its functionality and input signature. The problem thus requires both semantic alignment between χ and t∗ , and functional alignment between a∗ and the schema of t∗ . 3.1
Inference
Algorithm 1 NTILC Tool Loop Require: Query χ, encoder E, tool index V, backbone LLM M, finite state machine S 1: P ← M(χ) ▷ LLM produces plan block of tool intents 2: {p1 , . . . , pk } ← Split(P) ▷ Decompose plan block into individual intents 3: vχ ← E(pi ) ▷ Encode intent to NTILC embedding space 4: t̂ ← NNSearch(V, vχ ) ▷ Retrieve nearest tool cluster from index 5: b ← M(χ, S(t̂.schema)) ▷ Constrained argument generation produces dispatch block 6: r ← Dispatch(t̂, b) ▷ Execute tool, return response block 7: y ← M(χ, r) ▷ Synthesize final response 8: return y As shown in Algorithm 1 and Figure 2-(a), at inference time, a backbone LLM M receives the user request χ and produces a plan block P consisting of one or more natural-language tool intents. A splitting function decomposes P into individual intents {p1 , . . . , pk }, each of which is encoded by E into the trained NTILC embedding space (introduced in Section 3.2) to produce a query vector vχ . Nearest-neighbor search Johnson et al. [2017], Douze et al. [2025] over the pre-built tool index V retrieves a cluster identifier t̂, which is passed to a mapping function that resolves it to the corresponding tool schema. This schema is handed to the Outlines library Willard and Louf [2023], which enforces a finite-state machine (FSM) constraint over M’s output vocabulary during argument generation, ensuring that the decoder can only produce syntactically valid arguments that conform to 3
(a) Inference
Tool Mapping
Dispatcher
NTILC Embedding
<response> result: 72 degrees
status: Success
Query
LLM
</response>
“Whats the weather in Ann
Response <dispatch>
Arbor?”
tool: getWeather
The weather is 72
arg: city
degrees in Ann
input: Ann Arbor
Arbor.
</dispatch>
...
...
...
...
...
...
FSM
<plan> intent: get current
weather ann arbor
</plan>
(b) Training Circle + Functional
...
tool: getWeather
Margin Loss
intents: [get weather in ann arbor, whats the weather in ann arbor?, ... ]
tool: editFile
...
intents: [edit a file, revise a text file, ... ]
Figure 2: NTILC inference and training pipelines. The backbone LLM M produces a plan block P from user request χ, which is decomposed into individual intents and encoded into the NTILC embedding space. Nearest-neighbor search over the pre-built tool index V retrieves a tool identifier t̂, whose schema conditions constrained argument generation via an FSM to produce dispatch block b. The dispatcher executes the tool and returns response block r to M, which synthesizes the final response y. the selected tool signature and preventing malformed or schema-incompatible tool calls. The model then generates a dispatch block b containing syntactically valid arguments for t̂. The dispatcher executes the tool call and returns a response block r to M, which synthesizes the final response y for the user. Because tool embeddings are query-independent, the index V is built offline and updated incrementally without retraining the encoder. This removes the full tool registry from the active prompt entirely (as shown in Figure 1), reducing registry prompt tokens from O(N ) to O(1) regardless of how many tools are registered. 3.2
Training
Encoder Architecture. The NTILC encoder E uses a all-MiniLM-L6-v2 backbone (22.7M parameters). Final hidden states are mean-pooled and passed through a lightweight two-linear-layer MLP projection head, mapping the pooled representation to a 128-dimensional embedding space. All embeddings are L2-normalized, bounding pairwise distances and stabilizing margin-based objectives across registries. Each tool t ∈ T is represented as t = (n, d, A, R, e),
(1)
where n is the name, d is a natural-language description, A is the argument schema, R is the return schema, and e contains metadata. The encoder E operates on a textual rendering of this schema, while functional signature terms in the loss are derived from A, R, and e. Training Data. To train E, we construct a dataset of (q, t) query–tool pairs, where q is a naturallanguage intent derived from χ. For each tool in the registry, an LLM generates natural-language intents that a user might express when wanting to invoke that tool, producing a supervised training signal without requiring manual annotation. When dataset-provided train/test splits are available, they are used directly; otherwise, validation splits are stratified from the training data. Semantic Topology via Circle Loss. We adopt Circle Loss Sun et al. [2020] as the query-tool alignment term. Compared to standard contrastive losses, Circle Loss weights each pairwise similarity score individually, concentrating more gradient signal on examples near the retrieval decision 4
boundary. Let K + denote the number of positive pairs and K − denote the number of negative pairs for a given query. The objective is: K− X K+ X Lcircle = log 1 + exp γ(αnj (sjn − ∆n ) − αpi (sip − ∆p )) (2) j=1 i=1
where γ scales the similarity scores and the α factors adaptively weight positive and negative pairs. We set γ = 32.0, ∆p = 0.75, ∆n = 0.25 in all experiments. Resolving Semantic Blur via Functional Margin Loss. Circle Loss alone is insufficient because semantic proximity does not guarantee executable compatibility, as described in Section 1. To handle these cases, we propose the Functional Margin (FM) Loss, which applies an explicit repulsive force to semantic blur hard negatives. Rather than filtering to a discrete hard-negative set, we assign each negative pair a continuous weight representing the degree of functional incompatibility: wij = 1 − compat(ti , tj ),
(3)
where compat is a normalized score in [0, 1] derived from shared argument names, argument types, required fields, and interface metadata. For each tool t, define Sig(t) as the set of required argument names, argument types, return type, and metadata fields. The compatibility score is: compat(ti , tj ) =
| Sig(ti ) ∩ Sig(tj )| . | Sig(ti ) ∪ Sig(tj )|
(4)
Pairs whose signatures are nearly identical receive a weight near zero and contribute negligible gradient; pairs with incompatible signatures receive a weight near one and are pushed apart strongly. Let zi , zj ∈ Rd be ℓ2 -normalized tool embeddings. Define the cosine distance dembed (ti , tj ) = 1 − zi⊤ zj , a fixed margin hyperparameter m0 > 0, and the margin residual ρij = max(0, m0 − dembed (ti , tj )). The FM loss over a batch of B examples is: X wij 1[ρij > 0] ρ2ij B 1 X j̸=gi X LFM = , (5) B i=1 wij 1[ρij > 0] + ε j̸=gi
where B is the batch size, gi is the index of the ground-truth tool for query i, and ε is a small constant for numerical stability. Only pairs with an active margin violation (ρij > 0) contribute to the numerator; the denominator normalizes by the total weight of those active pairs rather than the full negative set, concentrating gradient on the hardest functionally-incompatible negatives. Composite Training Objective. The full training objective combines query-tool alignment with signature-aware tool separation: Ldispatch = λ1 Lcircle + λ2 LFM .
(6)
The λ1 coefficient governs the broad semantic structure of the embedding space, while λ2 controls the strength of functional discrimination among confusable tools. We sweep λ1 ∈ {0.1, 0.5, 1.0} and λ2 ∈ {0.1, 0.5, 1.0, 2.0, 5.0, 10.0}, with additional grids for m0 reported in the appendix. The optimal configuration is λ1 = 1.0 and λ2 = 2.0.
4
Experiments
We evaluate NTILC on two claims: (1) it reduces context length during tool selection, and (2) it preserves tool-retrieval accuracy while reducing inference time. All results are reported on public tool-selection or function-calling datasets. All NTILC models were trained and evaluated on a single NVIDIA H100 80GB GPU. Inference latency for NTILC is measured on this local hardware. Training took on average ∼40 minutes, and inference is performed with batch size 32. For closed-source baselines (ChatGPT, Claude, Gemini), we use their public APIs and do not have access to underlying hardware configurations. 5
Figure 3: Context length as a function of the number of available tools. ICT incurs linear growth by inserting the full tool registry into the prompt, whereas NTILC externalizes the registry through latent retrieval and preserves an approximately constant context length. 4.1
Experimental Setup
We evaluate on five public datasets, each normalized into a common representation: x = (q, T , t∗ , a∗ , σ),
(7) ∗
∗
where q is the user request, T is the available tool registry, t is the target tool, a contains correct arguments when provided, and σ is a dataset evaluator signal. Table 1: Public dataset coverage. Each row corresponds to an adapter that maps the original dataset into the NTILC tool-selection format. Blur pairs denote tool pairs that are semantically similar but functionally incompatible. Dataset ToolBench Qin et al. [2023b] API-Bank Li et al. [2023] BFCL Patil et al. [2025] ToolEyes Ye et al. [2024] MetaTool Huang et al. [2024]
Primary Signal
Tool Registry Size
Blur Pairs
Pass Rate / Win Rate Tool Call Accuracy AST / Exec. Success Planning / Tool Call Selection / Awareness
16,464 3,169 2,138 588 199
808 190 133 49 32
The datasets above allow us to measure how performance and cost scale with registry size. Semanticblur evaluation subsets are constructed per dataset by computing wij = 1 − compat(ti , tj ) for all tool pairs and retaining pairs where wij exceeds a fixed threshold, producing dataset-specific hard-negative subsets used for both training and evaluation. 4.2
Context Token Cost
Traditional ICT places the full tool registry in the prompt on every tool-selection call. Let S be non-tool system-prompt tokens, Q be user-query tokens, D be the fixed NTILC dispatcher instruction, N be the number of available tools, and r̄ be the average schema length in tokens. NTILC keeps the registry outside the prompt as an embedding index, yielding constant prompt length regardless of N . Table 2: Token cost for ICT versus NTILC. Registry prompt tokens scale linearly with tools under ICT; NTILC keeps the registry external to the LLM prompt. Method
System Prompt
Registry Prompt Cost
Total Prompt Tokens
Scaling
ICT NTILC
S S+D
N r̄ 0
S + Q + N r̄ S+D+Q
O(N ) O(1)
6
Table 3: Inference-time performance as tool registry size increases (ICT baseline), evaluated using a Qwen3-27B model on the ToolBench dataset, which enables experiments with large tool registries. #Tools
Registry Tokens ↓
Top-1 ↑
Top-5 ↑
Latency (ms) ↓
10 50 100 150 200 250
667 2390 4787 6884 9216 11575
1.000 0.970 0.950 0.920 0.920 0.870
1.000 1.000 0.980 0.970 0.980 0.960
4934 5821 6119 7668 7045 12120
Table 4: Main inference-time comparison, including parameter counts. Registry token counts are constant within each dataset, as identical prompts are used across all tools. “Gen. Tokens” denotes the number of tokens generated by the model during reasoning and tool selection, and latency is reported in milliseconds (ms). Dataset
Method
Params
Registry ↓
Gen. Tokens ↓
Top-1 ↑
Top-5 ↑
Latency ↓
ToolEyes
Qwen3-27B (ICT) Ministral 3 Kimi Moonlight ChatGPT 5 (ICT) Gemini 2.5 Flash (ICT) Claude Sonnet 4.6 (ICT) NTILC (Ours)
27B 14B 16B 1T+ 1T+ 1T+ 27.2B
22757 22757 22757 22757 22757 22757 0
2303 2518 2864 3092 2580 4222 115
93% 91% 92% 93% 93% 96% 94%
98% 97% 98% 99% 100% 100% 100%
4609 4892 5127 – – – 1127
MetaTool
Qwen3-27B (ICT) Ministral 3 Kimi Moonlight ChatGPT 5 (ICT) Gemini 2.5 Flash (ICT) Claude Sonnet 4.6 (ICT) NTILC (Ours)
27B 14B 16B 1T+ 1T+ 1T+ 27.2B
6593 6593 6593 6593 6593 6593 0
1911 2048 2375 2649 2113 3651 162
97% 94% 95% 95% 92% 97% 97%
99% 98% 99% 100% 99% 100% 100%
4299 4516 4868 – – – 1512
API-Bank
Qwen3-27B (ICT) Ministral 3 Kimi Moonlight ChatGPT 5 (ICT) Gemini 2.5 Flash (ICT) Claude Sonnet 4.6 (ICT) NTILC (Ours)
27B 14B 16B 1T+ 1T+ 1T+ 27.2B
115189 115189 115189 115189 115189 115189 0
2073 2294 2667 2856 2236 3883 137
95% 93% 94% 96% 92% 97% 96%
99% 98% 99% 99% 100% 100% 100%
4539 4811 5194 – – – 1268
BFCL
Qwen3-27B (ICT) Ministral 3 Kimi Moonlight ChatGPT 5 (ICT) Gemini 2.5 Flash (ICT) Claude Sonnet 4.6 (ICT) NTILC (Ours)
27B 14B 16B 1T+ 1T+ 1T+ 27.2B
21288 21288 21288 21288 21288 21288 0
2195 2391 2716 2983 2365 4074 153
98% 95% 96% 94% 92% 97% 98%
99% 98% 99% 99% 99% 99% 100%
4725 5018 5332 – – – 1398
ToolBench
Qwen3-27B (ICT) Ministral 3 Kimi Moonlight ChatGPT 5 (ICT) Gemini 2.5 Flash (ICT) Claude Sonnet 4.6 (ICT) NTILC (Ours)
27B 14B 16B 1T+ 1T+ 1T+ 27.2B
386355 386355 386355 386355 386355 386355 0
2446 2685 3217 3688 2788 5094 147
98% 96% 97% 99% 95% 97% 98%
98% 98% 99% 100% 99% 99% 100%
5663 5924 6410 – – – 1452
4.3
Inference Time and Tool-Retrieval Accuracy
We compare NTILC against ICT baselines in table 4 using Qwen3-27B, Ministral 3, Kimi Moonlight, ChatGPT 5, Gemini 2.5 Flash, and Claude Sonnet 4.6. Each ICT baseline receives the same tool registry in the prompt. NTILC uses Qwen3-27B for constrained argument generation and the learned tool-embedding index for tool selection. 4.4
Retrieval and Loss Ablations
Table 5 isolates the tool-retrieval component to evaluate different encoders and training objectives. This table provides the primary evidence that our proposed Functional Margin Loss (LFM ) is the specific driver for performance improvements under semantic blur. While moving from sparse retrieval (BM25) to a strong off-the-shelf dense encoder (Qwen3-Embedding-8B) provides a baseline performance jump, the critical ablation lies in the bottom two rows. When the embeddings are finetuned using standard metric learning (Circle Loss only), the Semantic-Blur Accuracy reaches 70.8%. However, by augmenting the objective with our Functional Margin Loss (NTILC), the Semantic-Blur Accuracy increases significantly to 75.0%, alongside a drop in Functional Error to 8.7%. Because 7
Figure 4: Qualitative examples illustrating semantic blur. While baseline methods frequently misclassify user intents by selecting tools with similar natural language descriptions but executionally incompatible arguments, NTILC leverages Functional Margin (FM) Loss to separate these confusable tools in the embedding space, ensuring accurate and functionally viable tool selection. Table 5: Tool-retrieval ablation on pooled public dataset test splits. Functional error rate measures the fraction of selected tools whose specification is functionally incompatible with the correct tool. Retriever / Loss BM25 Qwen3-Embedding-8B Circle Loss only NTILC (LFM )
Top-1 Acc. ↑
Top-5 Acc. ↑
Semantic-Blur Acc. ↑
Functional Error ↓
26.6% 82.4% 89.9% 91.3%
44.7% 95.4% 96.4% 97.4%
12.5% 66.7% 70.8% 75.0%
72.7% 17.1% 10.1% 8.7%
the underlying encoder architecture remains identical between these two configurations, this 4.2% absolute improvement clearly demonstrates that LFM , and not just the encoder itself, is directly responsible for teaching the model to resolve ambiguous, semantically blurred queries.
5
Conclusion
NTILC achieves substantial reductions in prompt token cost and selection latency without sacrificing accuracy. On ToolBench, it reaches 98% Top-1 accuracy with zero registry prompt tokens, maintaining parity with state-of-the-art ICT models while reducing selection latency by approximately 74% (from 5663ms to 1452ms). The Functional Margin Loss is the key driver of gains on semantic-blur subsets, as confirmed by ablations. More broadly, NTILC demonstrates that the tool registry need not live in the context window: moving it to a compact embedding index decouples agent capability from prompt length, a property that becomes increasingly important as tool registries grow. Limitations and Future Work. NTILC requires the tool registry to be indexed ahead of time. Static tool schemas are easily handled, but tools with highly dynamic or state-dependent documentation may require frequent re-indexing or a hybrid approach that retrieves a short schema snippet at query time. NTILC also focuses on selecting the correct tool; argument generation and tool-execution safety remain separate evaluation surfaces. Future work will examine larger and more dynamic registries, online index updates when tools are added or modified, and richer functional-distance metrics that capture schema compatibility beyond argument names and primitive types. Broader Impact. NTILC changes how tools are selected, not what policy governs their use. A malicious or ambiguous query could still cause a model to invoke an unintended tool, particularly 8
when the registry contains tools with external side effects. We recommend pairing NTILC with allow-lists, per-tool risk labels, and pre-dispatch policy filters. The efficiency gains also lower the barrier to deploying large-registry agents, which warrants care around intent-level safety filtering applied before the dispatch step.
References M. Douze, A. Guzhva, C. Deng, J. Johnson, G. Szilvasy, P.-E. Mazaré, M. Lomeli, L. Hosseini, and H. Jégou. The faiss library, 2025. URL https://arxiv.org/abs/2401.08281. P. Hosseini, I. Castro, I. Ghinassi, and M. Purver. Efficient solutions for an intriguing failure of llms: Long context window does not mean llms can analyze long sequences flawlessly, 2024. URL https://arxiv.org/abs/2408.01866. Y. Huang, J. Shi, Y. Li, C. Fan, S. Wu, Q. Zhang, Y. Liu, P. Zhou, Y. Wan, N. Z. Gong, and L. Sun. Metatool benchmark for large language models: Deciding whether to use tools and which to use, 2024. URL https://arxiv.org/abs/2310.03128. J. Johnson, M. Douze, and H. Jégou. Billion-scale similarity search with gpus, 2017. URL https: //arxiv.org/abs/1702.08734. M. Kang, W.-N. Chen, D. Han, H. A. Inan, L. Wutschitz, Y. Chen, R. Sim, and S. Rajmohan. Acon: Optimizing context compression for long-horizon llm agents, 2025. URL https://arxiv.org/ abs/2510.00615. C. Li, Z. Tang, Z. Li, M. Xue, K. Bao, T. Ding, R. Sun, B. Wang, X. Wang, J. Lin, and D. Liu. Teaching language models to reason with tools, 2025. URL https://arxiv.org/abs/2510.20342. M. Li, Y. Zhao, B. Yu, F. Song, H. Li, H. Yu, Z. Li, F. Huang, and Y. Li. Api-bank: A comprehensive benchmark for tool-augmented llms, 2023. URL https://arxiv.org/abs/2304.08244. N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang. Lost in the middle: How language models use long contexts, 2023. URL https://arxiv.org/abs/2307.03172. G. Mialon, R. Dessì, M. Lomeli, C. Nalmpantis, R. Pasunuru, R. Raileanu, B. Rozière, T. Schick, J. Dwivedi-Yu, A. Celikyilmaz, E. Grave, Y. LeCun, and T. Scialom. Augmented language models: a survey, 2023. URL https://arxiv.org/abs/2302.07842. A. Parisi, Y. Zhao, and N. Fiedel. Talm: Tool augmented language models, 2022. URL https: //arxiv.org/abs/2205.12255. S. G. Patil, T. Zhang, X. Wang, and J. E. Gonzalez. Gorilla: Large language model connected with massive apis, 2023. URL https://arxiv.org/abs/2305.15334. S. G. Patil, H. Mao, F. Yan, C. C.-J. Ji, V. Suresh, I. Stoica, and J. E. Gonzalez. The berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning, 2025. URL https: //openreview.net/forum?id=2GmDdhBdDk. N. Paulsen. Context is what you need: The maximum effective context window for real world limits of llms. Advances in Artificial Intelligence and Machine Learning, 06(01):01–26, 2026. ISSN 2582-9793. doi: 10.54364/aaiml.2026.61268. URL http://dx.doi.org/10.54364/AAIML. 2026.61268. A. Plaat, M. Van Duijn, N. Van Stein, M. Preuss, P. Van der Putten, and K. J. Batenburg. Agentic large language models, a survey. Journal of Artificial Intelligence Research, 84, Dec. 2025. ISSN 1076-9757. doi: 10.1613/jair.1.18675. URL http://dx.doi.org/10.1613/jair.1.18675. Y. Qin, S. Liang, Y. Ye, K. Zhu, L. Yan, Y. Lu, Y. Lin, X. Cong, X. Tang, B. Qian, S. Zhao, L. Hong, R. Tian, R. Xie, J. Zhou, M. Gerstein, D. Li, Z. Liu, and M. Sun. Toolllm: Facilitating large language models to master 16000+ real-world apis, 2023a. URL https://arxiv.org/abs/ 2307.16789. 9
Y. Qin, S. Liang, Y. Ye, K. Zhu, L. Yan, Y. Lu, Y. Lin, X. Cong, X. Tang, B. Qian, S. Zhao, L. Hong, R. Tian, R. Xie, J. Zhou, M. Gerstein, D. Li, Z. Liu, and M. Sun. Toolllm: Facilitating large language models to master 16000+ real-world apis, 2023b. URL https://arxiv.org/abs/ 2307.16789. J. Shtok, A. Alfassy, F. A. Dahood, E. Schwartz, S. Doveh, and A. Arbelle. Augmenting in-contextlearning in llms via automatic data labeling and refinement, 2024. URL https://arxiv.org/ abs/2410.10348. Y. Sun, C. Cheng, Y. Zhang, C. Zhang, L. Zheng, Z. Wang, and Y. Wei. Circle loss: A unified perspective of pair similarity optimization, 2020. URL https://arxiv.org/abs/2002.10857. R. Wang, X. Han, L. Ji, S. Wang, T. Baldwin, and H. Li. Toolgen: Unified tool retrieval and calling via generation, 2025. URL https://arxiv.org/abs/2410.03439. W. Wang, J. Min, and W. Zou. Intelligence degradation in long-context llms: Critical threshold determination via natural length distribution analysis, 2026. URL https://arxiv.org/abs/ 2601.15300. B. T. Willard and R. Louf. Efficient guided generation for large language models, 2023. URL https://arxiv.org/abs/2307.09702. J. Ye, G. Li, S. Gao, C. Huang, Y. Wu, S. Li, X. Fan, S. Dou, T. Ji, Q. Zhang, T. Gui, and X. Huang. Tooleyes: Fine-grained evaluation for tool learning capabilities of large language models in real-world scenarios, 2024. URL https://arxiv.org/abs/2401.00741. Y. Ye, Y. Zhao, K. Duan, Z. Zheng, K. Kawaguchi, C. Xie, and M. Q. Shieh. In-context reinforcement learning for tool use in large language models, 2026. URL https://arxiv.org/abs/2603. 08068.
10