Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use
Song-Lin Lv * 1 2 Weiming Wu * 1 Rui Zhu 1 Zi-Jian Cheng 1 2 Lan-Zhe Guo 1 2
Abstract
them to navigate external environments to solve complex, multi-step tasks (Guo et al., 2024; Wang et al., 2024b; Qu et al., 2025). Optimized via Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), recent open-source models (Hsieh et al., 2023; Qu et al., 2024; Qwen et al., 2024; Bai et al., 2025) have achieved tool invocation proficiency that rivals proprietary frontiers like GPT-4 (Achiam et al., 2023) and Gemini (Team et al., 2023). As illustrated in Figure 1, under the prevailing static world assumption, where the distribution of tools, schemas, and interaction logic remains consistent between training and inference (Shen et al., 2023; Yin et al., 2025; He et al., 2025), both SFT and RL paradigms demonstrate stable and continuous performance gains, eventually converging on near-perfect success rates.
arXiv:2607.01084v1 [cs.AI] 1 Jul 2026
While Large Language Model (LLM) agents demonstrate proficiency in static benchmarks, their deployment in real-world scenarios is hindered by the dynamic nature of user queries, tool sets, and interaction dynamics. To address this generalization gap, we formalize OpenAgent (Tool-Use Agent in Open-World), a problem setting characterized by distributional shifts across query, action, observation, and domain dimensions. To systematically diagnose its impact, we construct a controlled sandbox environment where we define fine-grained environmental shifts across a four-tier hierarchy, Perception, Interaction, Reasoning, and Internalization, and conduct a comprehensive series of experiments. Our analysis yields a series of key insights, demonstrating that agents trained via both Supervised Fine-Tuning(SFT) and Reinforcement Learning suffer from varying degrees of performance degradation when confronting open environmental shifts. Building on these insights, we propose Perturbation-Augmented Fine-Tuning, a disturbance-based intervention strategy for SFT that lays the foundation for enhancing agent robustness and utility in realistic environments. Our code will be released at: https://github. com/LAMDA-NeSy/OpenAgent.
However, this stability is often an artifact of the closed-set nature of current benchmarks. Real-world deployment is fundamentally non-stationary: APIs are deprecated, tool schemas evolve, and user instructions are frequently ambiguous. When these environmental dynamics diverge from training priors, the apparent mastery observed in static benchmarks often proves fragile. This discrepancy raises a fundamental research question: Do current training paradigms enable agents to generalize to the open world? To rigorously address this, we formally define OpenAgent (Tool-Use Agent in Open-World), a problem setting characterizing shifts across four dimensions: User Queries (∆Q), Tool Sets (∆A), Interaction Dynamics (∆O) and Task Domain (∆D). To isolate the impact of these shifts from the inherent instability and transient noise of real-world APIs, we establish a controlled sandbox environment. This setup allows for controlled probing, enabling us to maintain a pure closed-set baseline while systematically injecting openworld perturbations across a four-tier diagnostic framework: Perception, Interaction, Reasoning, and Internalization.
1. Introduction The integration of Tool Learning and the Model Context Protocol (MCP) (Anthropic, 2024) has catalyzed a paradigm shift in Large Language Model (LLM) agents, enabling
Our comprehensive evaluations reveal varying degrees of generalization and adaptability in SFT and RL models under open-world settings, while further identifying their respective structural weaknesses. We find that both paradigms suffer from performance degradation in open environments, albeit through distinct failure modes. While SFT agents are prone to trajectory overfitting and brittle symbolic anchoring, RL agents, despite demonstrating better semantic
*
Equal contribution 1 School of Intelligence Science and Technology, Nanjing University, Nanjing, China 2 National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China. Correspondence to: Lan-Zhe Guo <[email protected]>. Proceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026. Copyright 2026 by the author(s).
1
Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use