arXiv CSopen access

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

softwarearchitecture softwareengineering testing

software engineering, software architecture, testing

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation SHIYU HE, ZHIMAN CHEN, YUQI ZHAO, NENG ZHANG, RAN MO, and YUTAO MA∗ , School of Computer Science & Hubei Provincial Key Laboratory of Artificial Intelligence and Smart

arXiv:2604.17234v1 [cs.SE] 19 Apr 2026

Learning, Central China Normal University, China The rapid expansion of the model context protocol (MCP) ecosystem enables large language model (LLM)-based agents to access a wide range of external tools via a standardized interface. However, identifying appropriate MCP servers for a specific development task remains challenging. This difficulty arises from several factors, including ecosystem fragmentation, mismatching between task requirements and tool constraints at both the semantic and structural levels, and the absence of transparent rationales for recommendation results. Existing studies primarily focus on measuring the MCP ecosystem or optimizing tool invocation mechanisms, while systematic recommendation frameworks and reproducible benchmarks for real-world development tasks remain largely unexplored. To address this limitation, we formulate task-oriented MCP server recommendation as a structured retrieval-and-ranking problem that jointly considers semantic relevance and engineering constraints. We first construct Task2MCP, a task-centered dataset that systematically associates taxonomygrounded development tasks with curated MCP servers. This dataset provides structured supervision and a reproducible evaluation environment for research on MCP tool recommendations. Building on this dataset, we propose T2MRec, a task-to-MCP server recommendation model. It models semantic relevance and structural compatibility to construct an initial candidate set. Then it improves coverage and ranking quality through centroid-based candidate expansion and constrained LLM-based re-ranking. In addition, we design and implement an interactive MCP server recommendation agent prototype that operates in conversational environments to support dynamic decision-making. The agent assists developers in efficiently evaluating and integrating tools by providing recommended MCP servers together with usage guidelines. Experimental results indicate that T2MRec consistently improves ranking performance over traditional, graph-based, and LLM-based recommendation models across multiple evaluation metrics. These findings provide practical insights into task-aware MCP server recommendation and contribute a reproducible benchmark for future research on tool selection in LLM-based agent systems. CCS Concepts: • Information systems → Recommender systems; • Software and its engineering → Software development techniques. Additional Key Words and Phrases: Model context protocol, task-oriented recommendation, tool selection, large language models, agent

Introduction

Large language models (LLMs) have demonstrated remarkable capabilities in code generation [1], multi-step reasoning [2], and instruction following [3], leading to their increasing adoption in software development workflows [4]. Despite these advances, a significant challenge remains: LLM-based agents often struggle to select appropriate tools for specific tasks [5]. Rather than lacking reasoning power, these agents typically rely on trial-and-error approaches to identify usable tools. This difficulty is further compounded as the software tool ecosystem continues to expand, creating a complex environment in which LLMs must operate effectively. The model context protocol (MCP) offers a promising solution by providing a unifying standard for accessing and invoking external tools consistently. By decoupling tool implementation from ∗ Corresponding author.

Authors’ Contact Information: Shiyu He, [email protected]; Zhiman Chen, [email protected]; Yuqi Zhao, [email protected]; Neng Zhang, [email protected]; Ran Mo, [email protected]; Yutao Ma, ytma@ ccnu.edu.cn, School of Computer Science & Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning, Central China Normal University, Wuhan, Hubei, China. , Vol. 1, No. 1, Article . Publication date: April 2026.

He et al.

agent logic, MCP makes it easier to integrate diverse tools into LLM-based agent systems. As a result, the number and diversity of available MCP servers have grown rapidly [6]. Despite this progress, MCP primarily addresses how tools can be accessed rather than which tools should be selected for a given task. This gap in effective tool recommendation presents several challenges: (1) MCP ecosystem fragmentation. Although the MCP ecosystem has grown rapidly, existing MCP servers are distributed across heterogeneous platforms with inconsistent descriptions, incomplete metadata, and varying levels of documentation. This fragmentation makes it difficult for developers and agents to obtain a unified view of available MCP servers or to discover reusable tools for specific development tasks systematically. Therefore, MCP discovery often relies on manual search or prior knowledge, which introduces a strong exposure bias toward well-known MCP servers or frequently referenced MCP tools. Many potentially suitable but less visible MCP servers remain underutilized, leading to redundant tool implementations and inefficient reuse of existing capabilities. (2) Semantic–structural mismatch. Most existing tool recommendation approaches emphasize semantic similarity between task descriptions and tool documentation, assuming that semantic relevance is sufficient for effective tool selection. However, in practical development scenarios, an MCP server that appears semantically relevant may still be unusable due to mismatches in programming language, system environment, or functional interfaces. This mismatch arises from treating MCP server recommendations as a purely semantic matching problem while overlooking structural constraints that determine whether a tool can be directly integrated into a development workflow. Consequently, LLM-based agents may select MCP tools that conceptually match the task intent but fail during execution, leading to fragile pipelines and higher trial-and-error costs. (3) Opaque recommendation rationale. Many existing MCP server recommendation processes operate as black boxes, providing developers with little explanation of why a particular MCP server is selected for a given task. Without transparent rationales, developers cannot easily assess the suitability of recommended MCP servers or understand how to use them in their solutions. The lack of interpretability undermines developer trust and limits the effectiveness of human-in-the-loop development workflows. In practice, developers often need not only a recommendation result but also concrete reasoning and usage guidance to determine whether an MCP server truly meets their task requirements and to integrate it efficiently into their system implementation. In response to the above challenges, recent studies have begun to explore techniques to improve tool selection and use in LLM-based agent systems. Within the MCP ecosystem, prior work has systematically collected and analyzed MCP servers and tools from measurement and datasetconstruction perspectives [7, 8]. These efforts provide valuable insights into ecosystem scale, capability distribution, and potential security risks, thereby improving our understanding of what resources exist in the MCP landscape. However, existing studies primarily focus on ecosystem characterization rather than task-oriented decision making. While registries and metadata help catalog available MCP servers, they do not address the core question of which server or tool to invoke for a specific user task. In practice, selecting an appropriate MCP server requires aligning task intent with both semantic relevance and structural constraints, such as programming language compatibility, interface requirements, and execution environments. Current MCP ecosystem datasets often exhibit inconsistent granularity, incomplete capability descriptions, and non-standard naming conventions, further hindering reliable task-to-tool mapping. Despite the rapid growth of MCP infrastructure, LLM-based agent systems still lack principled recommendation mechanisms for automatically retrieving suitable MCP servers and generating executable invocation plans grounded in real-world

, Vol. 1, No. 1, Article . Publication date: April 2026.

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

task requirements. Bridging this gap requires moving beyond static ecosystem measurement toward task-aware, structurally constrained, and interpretable recommendation frameworks. To address this gap, this article introduces a large-scale, real-world dataset, Task2MCP, for the task-oriented MCP ecosystem, comprising 4,800 development tasks and 5,642 MCP servers. This dataset overcomes the limitations of existing MCP resources by offering a diverse and comprehensive collection of realistic development tasks paired with structured MCP metadata. On average, each task is annotated with a category, subcategory, programming language, theme, and a ranked list of compatible MCP servers. Task2MCP is poised to be an invaluable resource for advancing the development of task-aware MCP recommendation models. To construct this dataset, we first collect MCP server entries from three widely used online directories, which cover a diverse array of tool types, programming languages, and service domains. We then enrich these entries with GitHub repository metadata, perform deduplication and normalization, and apply a validation workflow to ensure data quality. This process results in a high-confidence dataset comprising 5,642 MCP servers and 4,800 realistic development tasks. To demonstrate the potential of Task2MCP for improving task-oriented MCP server recommendations, we train a recommendation model, T2MRec, designed to rank MCP servers for realistic development tasks and establish a benchmark for fair comparison. The model combines the semantic relevance of textual descriptions with the structural compatibility of engineering constraints. Specifically, T2MRec first ranks MCP server candidates based on their alignment with the given task, then refines possible recommendations through centroid-based expansion and LLM-based re-ranking. This robust, task-aware approach aims to enhance the accuracy and reliability of tool recommendations, reducing the trial-and-error process traditionally associated with tool selection. Furthermore, we also develop a recommender artificial intelligence (AI) agent for MCP servers designed to operationalize this approach in a real-world, interactive environment. This agent provides developers with dynamic, task-specific MCP server or tool recommendations, along with detailed usage guidelines and explanations for each recommendation. It supports multi-turn dialogue, allowing developers to adjust their tool selections as project requirements and constraints evolve. The agent also delivers a comparative analysis of multiple tool options, improving transparency (or interpretability) and helping developers make more informed decisions. For more details on our dataset and model, please refer to the open-source project available on GitHub1 . The main contributions of our work are as follows: • New Dataset for MCP Research. We construct the Task2MCP dataset, a large-scale, taskoriented benchmark dataset that systematically links realistic development tasks to curated MCP servers. The dataset covers thousands of MCP servers and tasks organized under a taxonomy-grounded hierarchy, providing high-quality supervision for MCP server recommendation and other MCP-related research fields. • New Benchmark for MCP Server Recommendation. We propose T2MRec, a task-to-MCP recommendation model that jointly models semantic relevance and structural compatibility, and further refines candidate rankings through a constrained LLM-based re-ranking module. The model is designed to be scalable, reproducible, and suitable for real-world engineering scenarios. We then conduct extensive experiments and analyses on the Task2MCP dataset, demonstrating that the T2MRec model consistently outperforms selected traditional, graphbased, and LLM-based baselines. Our results provide empirical insights into the roles of semantic modeling, structural constraints, and LLM refinement in task-oriented MCP server recommendation. 1 https://github.com/ssea-lab/Task2MCP

, Vol. 1, No. 1, Article . Publication date: April 2026.

He et al.

• New Recommender AI Agent for System Implementation. We design and implement an interactive recommender AI agent for MCP server recommendations, which operationalizes the T2MRec model in a real-world conversational environment for developing LLM-based agent systems. The agent recommends MCP servers, explains their suitability for a given task, and provides actionable guidance on tool usage and execution workflows, thereby supporting human-in-the-loop decision-making to facilitate system development. The remainder of this article is organized as follows. Section 2 reviews the related work. Section 3 introduces the construction of the Task2MCP dataset. Section 4 details the proposed T2MRec model. Section 5 presents the experimental studies we conducted and analyzes the experimental results in depth. Section 6 shows a case study demonstrating the recommender AI agent’s practical application. Section 7 discusses potential threats to the validity of our work. Finally, Section 8 concludes the article and outlines our future work. 2 2.1

Related Work MCP Ecosystem

MCP has recently emerged as a practical interface layer for connecting LLM-based agents with external tools, services, and data sources [9]. This standardization has accelerated the growth of the MCP ecosystem by simplifying the integration of agents and tools [10]. However, MCP servers are distributed across heterogeneous platforms, and there are significant variations in metadata structures, resource quality, and document consistency. To address these challenges, researchers are currently exploring the MCP ecosystem from three complementary perspectives: ecosystem measurement, tool management, and benchmark evaluation. Early studies mainly focus on understanding the MCP ecosystem itself. For example, MCPCorpus [7] constructs a large-scale, evolvable dataset of MCP servers and clients, enabling empirical analysis of protocol adoption, implementation diversity, and potential security risks. In a similar spirit, Guo et al. [8] presented a market-level measurement view of MCP by crawling and normalizing artifacts from multiple MCP marketplaces, thereby revealing the MCP ecosystem’s structural properties and growth patterns. Complementing these measurement-oriented efforts, MCPZoo [11] emphasizes the collection of runnable MCP servers and provides a resource that is better aligned with execution-level evaluation and agent-tool interaction studies. These previous studies established an important empirical foundation for understanding what kinds of MCP resources exist, how they are organized, and how they may support downstream LLM-based agent systems. A second line of work investigates MCP from the perspective of tool discovery and retrieval. Rather than treating MCP servers as static protocol endpoints, these efforts examine how LLM-based agents can identify, rank, and compose relevant tools under natural language intents. For example, MCP-Zero [12] formulates proactive toolchain construction as a retrieval-oriented problem and compiles an MCP tool corpus for evaluating server and tool discovery. Building upon such structured MCP resources, HumanMCP [13] further introduces human-like queries for evaluating tool retrieval, thereby moving beyond template-style requests toward more realistic user expressions. This line of research highlights that, in practical deployments, the bottleneck of LLM-based agents using MCP often lies not only in tool execution but also in accurately locating the right tool under ambiguity, scale, and heterogeneous descriptions. The third research direction is the construction of benchmarks for capability and robustness evaluation. For example, MCP-RADAR [14] and MCP-Universe [15] develop representative task suites over MCP-enabled environments to assess how well LLM-based agents can use tools in realistic scenarios. MCP-Bench [16] and MCPMark [17] further extend this paradigm by stressing , Vol. 1, No. 1, Article . Publication date: April 2026.

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

multi-step reasoning, cross-tool coordination, and more comprehensive MCP workflows. MCPAgentBench [18] further evaluates real-world agent performance with MCP-mediated tools. MCPTox [19] benchmarks the risks of tool poisoning on real-world MCP servers. In domain-specific settings, IoT-MCP [20] studies MCP-enabled interaction in Internet of Things environments and evaluates agent performance over both basic and complex tasks. In parallel with capability evaluation, MCPSecBench [21] shifts attention to the attack surface introduced by MCP and provides a systematic security benchmark that covers multiple threat categories, clients, and platforms. These efforts collectively suggest that MCP should be evaluated not merely as a protocol abstraction, but as an end-to-end agentic infrastructure involving discovery, planning, invocation, and defense. In brief, the above studies have primarily focused on ecosystem-wide collection, measurement, and characterization of MCP resources [22]. Although these studies improve our understanding of the resources available in the MCP ecosystem, they offer limited guidance on selecting task-time servers in practical development settings. In real-world use, developers and LLM-based agents must still identify appropriate servers from a large and noisy candidate pool, verify whether their capabilities align with the target task, evaluate interface and dependency compatibility, and endure substantial trial-and-error due to invalid, low-quality, or incompatible resources. Consequently, there remains a clear gap between ecosystem-level resource awareness and dependable, on-demand selection and invocation of MCP servers. 2.2 Task-Oriented Recommendation Task-oriented recommendation in software engineering concerns the automatic matching of task demands to candidate objects capable of fulfilling them, such as Web services, tools, application programming interfaces (APIs), or development resources [23]. In contrast to traditional recommender systems that primarily focus on estimating user-item preferences, task-oriented recommendations must account not only for task relevance but also for feasibility constraints [24, 25]. The central challenge lies in accurately aligning tasks with users’ needs to achieve precise, personalized task assignment [26]. Accordingly, the literature has evolved from early heuristic and collaborative filtering methods to graph-based representation learning and, more recently, to LLM-driven retrieval and reasoning. 2.2.1 Traditional Methods. Traditional task-oriented recommendation methods can be broadly categorized into content-based, collaborative filtering, and matrix factorization approaches. Contentbased methods rank candidate resources by estimating their relevance to a given task description, typically through textual or code-level matching [27, 28]. This paradigm has remained competitive due to its simplicity and interpretability, and recent studies have substantially strengthened its semantic modeling capabilities [29]. For example, Wei et al. [30] proposed CLEAR, which combines BERT-based sentence representations with contrastive learning to improve semantic matching for API recommendation. Li et al. [31] further developed PTM-APIRec, which leverages pretrained source-code models to encode program context and improve context-aware recommendation. Although these methods can improve the modeling of task semantics, they still depend heavily on the quality and explicitness of task descriptions and resource documentation. Their effectiveness, therefore, tends to degrade when the task goal (or intent) is under-specified or when resource capability boundaries are not clearly documented. To reduce this dependence on explicitly expressed task semantics, collaborative filtering and matrix factorization models recommend items based on historical interaction data [32, 33]. Collaborative filtering infers task relevance from users’ past behaviors and shared preference patterns. For instance, Pu et al. [34] proposed the RG2 loss for collaborative filtering, improving recommendation performance and convergence speed. Matrix factorization builds on this idea by decomposing , Vol. 1, No. 1, Article . Publication date: April 2026.

He et al.

sparse interaction matrices into low-dimensional latent factors, thereby uncovering implicit associations among users, tasks, and candidate resources [35]. For example, Yuen et al. [36] proposed a time-aware task recommendation framework that integrates probabilistic matrix factorization with temporal preference modeling and online updating. Such matrix factorization methods are particularly effective when explicit features are limited and sufficient interaction data are available [37]. However, both collaborative filtering and matrix factorization remain vulnerable to cold-start settings and sparse supervision, which restricts their applicability in open and fast-evolving task recommendation scenarios [38]. 2.2.2 Graph Neural Network-Based Methods. Graph-based methods have become an important paradigm for task-oriented recommendation because they capture structured dependencies among tasks, resources, and historical interactions [39]. Their advantages are particularly evident in sparse, heterogeneous, and cold-start settings. Prior studies have shown that graph-based modeling can improve crowdsourcing task matching [40], enhance few-shot recommendation [41], and strengthen strict cold-start recommendation [42], suggesting that relational context provides useful signals for inferring task-resource suitability. In software engineering, this line of research has evolved from semantic matching to explicit relational modeling [43]. Until now, neural graph collaborative filtering (NGCF) [44] and LightGCN [45] have established strong graph-based recommendation baselines through neighborhood propagation. In task-oriented settings, Req2Lib [46] performs requirement-to-library matching, but it does not model higher-order relations among tasks, libraries, and projects. MLTaskKG [23] advances this direction by organizing tasks, models, implementations, and repositories into a knowledge graph for task-conditioned recommendation. GRec [47] and HGNRec [48] further strengthen this line by learning graph representations over library interaction data, enabling more effective modeling of higher-order dependency signals. Compared with traditional feature-driven methods, graph-based approaches better exploit relational evidence underlying task-resource matching. However, their effectiveness still depends on informative and sufficiently complete graph structures [49]. In open and fast-changing development environments, graph construction may be noisy or incomplete, and graph propagation alone is often insufficient to resolve semantic ambiguity in task descriptions. 2.2.3 LLM-Based Methods. Recently, the rise of LLMs has significantly transformed the paradigm of recommending and selecting tools or developers for specific tasks in software development [50, 51]. LLMs can directly comprehend natural-language task objectives and parse contextual constraints, enabling their widespread application in tool use [52], function calling [53], development task allocation [26], and AI agent systems [4]. Unlike traditional methods that primarily rely on explicit task features or user behavior, LLMs can analyze the semantic information in task descriptions to generate more personalized and precise task recommendations [54, 55]. Moreover, LLMs can not only directly generate recommendations based on task-level linguistic descriptions but also optimize recommendation strategies using user feedback, thereby granting LLM-based approaches a greater advantage over traditional methods in dynamic development environments [56–58]. By integrating LLMs and fine-tuning strategies, task recommendation models or systems can process complex task descriptions and multi-modal information, delivering more precise and personalized recommendations to developers or LLM-based agents [59]. Generally speaking, this approach shows promising results by addressing the limitations of traditional methods in handling intricate task scenarios. Compared with earlier methods, LLM-based approaches are better suited to coarse-grained, compositional, and under-specified tasks because they can directly interpret task intent and perform , Vol. 1, No. 1, Article . Publication date: April 2026.

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

semantic decomposition during the recommendation process [60, 61]. However, their recommendations may remain unstable when candidate resources have ambiguous capability boundaries or when structural compatibility and execution constraints are not explicitly modeled. Therefore, although LLMs substantially strengthen task understanding, effective task-oriented recommendation still requires explicit grounding in resource structure and operational feasibility [62]. 2.3

MCP Server Recommendation

Research on task-oriented MCP server recommendations aims to address bottlenecks arising from the rapid expansion of the MCP ecosystem. The same task often corresponds to a large number of candidate MCP servers or tools, and their quality varies widely. Directly injecting a large-scale tool set into the prompt will incur significant token costs and introduce noise, thereby reducing the task success rate and increasing interaction latency. In this context, existing work generally models the selection process as a retrieval problem and proposes mechanisms from various perspectives, including dynamic retrieval, large-scale filtering, and system routing. For instance, ScaleMCP [63] transforms tool selection from a static configuration to runtime autonomy, equips the agent with an MCP tool retriever that dynamically retrieves and synchronizes tools across multiple task rounds, and emphasizes key fields of tool documentation through weighted embedding strategies to improve retrieval quality and call effectiveness. HGMF (short for hierarchical Gaussian mixture framework) [64] focuses on the scalability of retrieval and selection by using a two-stage probabilistic pruning strategy to reduce the candidate space, thereby decreasing context noise and invocation overhead while improving selection accuracy. Complementing this, NetMCP [65] further argues that semantic relevance alone is insufficient, as real-world deployments are also affected by network latency. Therefore, it performs routing optimization by jointly considering semantic relevance and quality of service to improve task completion rates. Some recent studies also focus on context inflation caused by the expansion of MCP-related tool ecosystems. For instance, JSPLIT [66], a taxonomy-based solution for prompt bloating in MCP, includes only the subset of MCP tools highly relevant to the current task, thereby reducing token overhead and mitigating interference from irrelevant tools or servers. In contrast, the engineeringoriented MCP Bridge [67] places greater emphasis on providing general-purpose infrastructure for task execution. It unifies access to multiple MCP servers through a proxy layer and introduces confirmation mechanisms and isolated execution to improve usability and facilitate secure deployment. In addition, several empirical studies provide task sets and evaluation methodologies [68–70]. Through experiments on real-world tasks, they show that MCP server-recommendation scenarios pose specific challenges, including multi-step reasoning, large tool spaces, and performance volatility. These findings indicate that task-driven MCP server recommendation is a promising approach and also provide a reproducible benchmark for comparing subsequent methods. Overall, existing studies have explored several strategies to improve MCP tool selection, such as retrieval optimization and routing mechanisms. However, research on task-oriented MCP server recommendation remains limited. Most previous approaches primarily rely on semantic matching between tasks and tool descriptions, while overlooking structural constraints in practical development environments. Consequently, effectively recommending suitable MCP servers for complex development tasks is still a challenging problem. 2.4

Summary

As shown in Table 1, prior work has advanced MCP ecosystem analysis, task-oriented recommendation, and MCP-specific tool retrieval, but these directions remain only partially connected. Ecosystem studies improve resource visibility but provide limited guidance on selecting an MCP , Vol. 1, No. 1, Article . Publication date: April 2026.

He et al.

Table 1. Overall comparison of related work on task-aware MCP server recommendation. Category

Representative Study

Key Approach

Description

MCPCorpus [7]; Provides large-scale ecosystem visibility, MCPCrawler [18]; but remains descriptive rather than MCPZoo [11]. decision-oriented for MCP selection. Improves discovery on realistic queries, Tool Retrieval MCP-Zero [12]; but lacks task-grounded supervision to Resources HumanMCP [13]. determine which retrieved servers are truly suitable. MCPCapability and Evaluates MCP-enabled tool use and RADAR [14]; Tool-Use workflows, but benchmarks execution MCP-Bench [16]; Benchmarks rather than recommendation quality. MCPMark [17]. Task-Specific MCP A task-oriented MCP dataset that Recommendation Task2MCP systematically aligns realistic Dataset development tasks with MCP servers. MCP Ecosystem Measurement

MCP Ecosystem Analysis

Traditional Recommendation Graph-Based Task-Oriented Recommendation Recommendation LLM-Based Recommendation

MCP Server Recommendation

CLEAR [30]; Models relevance effectively, but does PTM-APIRec [31]; not design for the heterogeneous NeuMF [71]. constraints of MCP ecosystems. LightGCN [45]; Uses relational signals, but depends on NGCF [44]; graph completeness, which is hard to Req2Lib [46]; guarantee in fragmented MCP settings. MLTaskKG [23]. OpenP5 [55]; Understands task intent well, but often DSKIPP [58]; produces semantically plausible yet DiMA [59]; operationally incompatible LiS [53]. recommendations.

MCP-Oriented Retrieval and Selection

ScaleMCP [63]; HGMF [64]; NetMCP [65]; JSPLIT [66].

Task-Aware Recommendation

T2MRec

Improves retrieval and routing, but does not fully integrate semantic relevance, structural feasibility, and supervised task–MCP alignment. A task-to-MCP recommendation model that jointly models semantic relevance and structural feasibility.

MCP Task Task–MCP Data Data Interaction

S&S

MCPGuidance Specific

✓

✗

✓

✗

✓

✗

✓

✗

✓

✗

✓

✗

△

✗

△

✗

△

✓

✗

✓

Note: S & S: Semantic & Structural; ✓: explicitly covered; △ : partially covered; ✗: not explicitly covered.

server for a specific development task. General task-oriented recommendation methods improve task-resource matching, but they are not tailored to the fragmented, noisy, and constraint-sensitive nature of MCP ecosystem environments. Existing MCP-specific recommendation methods improve retrieval efficiency and system usability, but most still treat MCP server recommendation as a semantic retrieval problem. Consequently, a clear gap remains in task-aware MCP server recommendation for realistic development workflows. What is still missing is a benchmark that systematically aligns real development tasks with curated MCP servers, together with a recommendation framework that jointly models semantic relevance and structural feasibility. This gap motivates the construction of Task2MCP and the design of T2MRec in this article. 3

Task2MCP Dataset

3.1 Data Source and Motivation LLMs demonstrate strong capabilities in instruction-following, problem-solving, and multi-step reasoning. However, in real-world software development, many tasks fail at the practical stage of selecting appropriate MCP tools. As the MCP ecosystem continues to expand, developers often lack clear guidance on which tools are most suitable for a given task, making tool selection increasingly difficult and error-prone. This challenge motivates the construction of a structured dataset for task-aware MCP tool recommendation and evaluation. In this work, the focus is placed on tool selection within the MCP ecosystem. MCP is a standard that enables LLM-based agents to access external tools via MCP servers in a consistent, interoperable manner. The number and diversity of MCP servers are growing rapidly; however, to the best of our knowledge, few public benchmarks currently provide a systematic mapping between real software , Vol. 1, No. 1, Article . Publication date: April 2026.

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation MCP Website

Common Crawler

Log Information

MCP Server Database Deduplication Correction

①

②

Structure Parsing

Information Extracting

http://

Author

Task Category Standards

③ Data Reviewing

URL Link

LLMs Generation

Task Database Logic Validating

AI Use Taxonomy: A Human-Centered Approach User Goals Task Complexity Human-AI Collaboration Patterns Socio-Technical Context

Task Tree

Rectification

Page Details Timestamp

①

②

Task matching

Data Reviewing

Extracting Attributes

Task

Task2MCP Database

Fig. 1. Overview of the construction pipeline of the Task2MCP dataset, including MCP set construction, task set construction, and task-MCP interaction set construction.

development tasks and suitable MCP servers. The absence of such data makes it difficult to train task-aware MCP server recommendation models and hinders fair, reproducible evaluation. To address this gap, we construct a large-scale, high-quality task–MCP interaction dataset that maps taxonomy-grounded development tasks to curated MCP server candidates. The following sections describe the dataset construction pipeline and the dataset content in detail. 3.2 Dataset Construction The Task2MCP dataset is designed to capture (i) the landscape of actively used MCP servers and their tool capabilities, (ii) a taxonomy-grounded collection of realistic development tasks, and (iii) supervised task–MCP relevance signals represented by the top-ranked MCP server candidates for each task. Specifically, this dataset comprises three components: raw MCP data, raw task data, and interaction data between tasks and MCP servers. Figure 1 illustrates the overall construction pipeline of this dataset. 3.2.1 MCP Set. The MCP server data collection includes the following three steps. (1) Crawling MCP directories. We crawled three widely used MCP directories, including Glama2 , mcp.so3 , and mcpservers.org4 , to build an initial candidate set of MCP servers. For each directory, we developed automated crawling scripts that download the relevant pages and parse their web structures into structured records. We then normalized the extracted fields across directories to ensure consistent formatting. For each directory entry, we extracted the MCP server name and, when provided, the corresponding GitHub repository’s uniform resource locator (URL). We also removed duplicate entries across directories and kept a single canonical record for each server. (2) Enriching metadata from GitHub. The MCP directories we crawled typically provide only short summaries and lack sufficient information to support reliable task-to-tool matching. Therefore, after extracting the GitHub repository URL for each MCP entry, we treated GitHub as the primary source of ground-truth metadata and capability descriptions. For each repository, we automatically 2 https://glama.ai 3 https://mcp.so 4 https://mcpservers.org

, Vol. 1, No. 1, Article . Publication date: April 2026.

He et al.

retrieved and normalized repository-level signals that reflect both usability and maintenance status, including the repository owner/author, project description, license information, primary programming language, and popularity indicators such as stars and watch counts. In addition to these high-level attributes, we extracted tool-relevant textual content crucial for recommendations, such as the stated functionality of an MCP server, the list of supported tools or endpoints (when available), and other usage-oriented descriptions in the repository documentation. When the same MCP server appeared across multiple directories, we consolidated records by using the GitHub URL as the canonical identifier and kept a unified representation of its metadata. This GitHub-based enrichment step substantially improved the completeness and consistency of MCP server profiles, providing richer signals for both semantic matching and structure-awareness in downstream recommendation models. (3) Cleaning and normalizing MCP entries. MCP entries collected from different directories are often noisy. The same server may appear multiple times under slightly different names, some records contain incomplete fields, and others point to outdated or invalid repositories. To ensure this corpus is reliable for model training and evaluation, we applied a unified cleaning and normalization workflow following GitHub enrichment. We first validated each record by resolving its GitHub URL and removing entries that could not be accessed or did not correspond to an MCP server implementation. We then deduplicated MCP servers across sources using their GitHub URLs as the canonical key, merging duplicate records while preserving the most informative text description and the most complete metadata. During this process, we corrected inconsistent fields by crosschecking directory entries against GitHub metadata and filled in missing values whenever they could be inferred from the corresponding repository (e.g., license, primary programming language, and owner). Finally, we normalized categorical attributes, such as category and subcategory, to a consistent schema and standardized tool/capability text to a unified format, reducing variance caused by different naming conventions across directories. After these steps, we obtained a cleaned MCP corpus with consistent identifiers and complete metadata. 3.2.2 Task Set. The task data collection includes the following three steps. (1) Task taxonomy definition. To avoid an ad hoc task set and to ensure that our dataset covers a broad range of realistic development scenarios, we grounded task set construction in an external, well-defined taxonomy. Specifically, we adopted the National Institute of Standards and Technology (NIST) specification5 of 16 human–AI activity types as our task subcategories, and we further organized them into four higher-level groups to form a two-level task hierarchy. As shown in Figure 2, this taxonomy serves two purposes. First, it provides a consistent way to define “task diversity” in our dataset, ensuring that tasks are not concentrated in a few popular areas. Second, it enables structured analysis and evaluation, as we can report results not only overall but also by category and subcategory. Throughout the dataset construction, each task is assigned a unique position in this concept hierarchy, which later supports task-tree–guided annotation and helps align tasks with MCP servers whose capabilities are most relevant to that part of the task space. (2) Task generation. After defining the above taxonomy, we populated each of the 16 activity types with concrete software development tasks that are specific enough to guide tool selection. We used an LLM as a drafting assistant to propose candidate tasks under each activity type, conditioned on the category definition and the requirement that tasks reflect realistic developer needs. For each activity type, the model generated task descriptions that included a clear objective, practical constraints, and an expected outcome; we also ensured coverage across different application themes and programming contexts. Each generated task was stored in a structured format with a short title and a detailed requirement specification, along with optional implementation constraints 5 https://www.nist.gov/publications/ai-use-taxonomy-human-centered-approach

, Vol. 1, No. 1, Article . Publication date: April 2026.

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

NIST Human-AI Activities

Cognitive & Information

Predictive & Decision

Personalization & Assistance

Automation & Execution

Content Creation

Prediction

Personalization

Process Automation

Content Synthesis

Decision Making

Digital Assistance

Monitoring

Information Retrieval

Recommendation

Discovery

Robotic Automation

Image analysis

Detection

Performance Improvement

Vehicular Automation

Fig. 2. Two-Level task taxonomy based on the NIST human–AI activity framework.

such as the intended programming language, a higher-level category label, a domain theme, and a unique identifier. This step produces a diverse pool of candidate tasks that are aligned with the NIST taxonomy while remaining close to real-world development workflows. (3) Quality control. Since the task set is intended to serve as a basis for supervising and evaluating MCP tool recommendations, we applied strict quality control to ensure that each task is clear, feasible, and correctly categorized. We first conducted automated checks to detect obvious issues, such as near-duplicate tasks across categories, overly generic descriptions that lack specific inputs or expected outputs, and mismatches between task descriptions and their assigned NIST activity types. Then, we performed manual reviews to verify that each task represents a realistic development request, contains sufficient detail to support tool selection, and does not rely on hidden assumptions or unclear requirements. The review process was conducted by multiple authors using shared criteria, achieving high inter-annotator agreement (Cohen’s 𝜅 > 0.8). We resolved disagreements in ambiguous cases through discussion. During this process, we revised or removed tasks that were ambiguous, redundant, or difficult to implement as stated. After filtering out unqualified tasks and further refinement, we retained only those that passed both automated screening and human validation, resulting in a high-quality task corpus with consistent labels and reliable task descriptions for downstream recommendation and analysis. 3.2.3 Task–MCP Interaction Set. To train and evaluate task-oriented MCP server recommendation models, we require supervised signals that indicate which MCP servers are most likely to be useful for implementing a given development task. To achieve this, we linked each task in the task set to a curated set of MCP server candidates, taking into account both the task’s semantics and its position within the structured task taxonomy. First, we categorized each task within a two-level hierarchy, consisting of four top-level groups and sixteen NIST human–AI activity subcategories. This classification helps narrow the search for relevant MCP servers that typically support the identified activity type. Next, we analyzed each task description, including its objective, constraints, and expected outcomes. We matched it with the descriptions of MCP server capabilities and the list of tools available in the corresponding repositories. This process produced a ranked list of MCP , Vol. 1, No. 1, Article . Publication date: April 2026.

He et al.

Table 2. Primary attributes of the MCP and task sets.

Data Set

Attribute

Field Type

Count

MCP

mcp_num license language subcategory category system official

Integer (numeric) Categorical (string) Categorical (string) Categorical (string) Categorical (string) Categorical / Boolean Boolean

5,642 21 95 52 22 3 2

Task

task_num language category subcategory theme

Integer (numeric) Categorical (string) Categorical (string) Categorical (string) Text / Categorical

4,800 21 4 16 606

servers for each task. These task-MCP links were established primarily according to whether an MCP server could realistically support task completion, while clearly inactive or outdated MCP servers had already been removed during preprocessing. They were selected not only for their direct functional relevance but also for their complementary nature, as they are often used together in a development workflow. To ensure the accuracy of these labels, we employed a human-in-the-loop verification system. This system used an LLM-based checker to identify mismatches, redundancies, and missing essential capabilities. Any ambiguous cases were reviewed manually, and the final curated, relevant set of MCP servers was confirmed. The task–MCP labels generated through this process can provide highquality, task-grounded supervision for training recommendation models and offer a reproducible benchmark for evaluating retrieval and ranking performance. 3.3 Ethical Compliance The Task2MCP dataset is constructed from publicly available MCP directories and GitHub repositories, as well as LLM-assisted task descriptions and task–MCP links. Only public metadata is utilized, and no additional personally identifiable information is intentionally collected beyond what is already accessible on GitHub. During the construction and annotation process, we ensure that the LLM used did not generate harmful, illegal, or discriminatory content. Additionally, tasks and labels that could potentially lead to misuse or contain explicit or offensive language were manually filtered out. The dataset is made available on GitHub for research on MCP tool recommendation and LLM-based agent systems, and we expect all users to adhere to relevant ethical and legal guidelines. 3.4

Dataset Characteristics

Our dataset comprises three key components: the MCP server corpus (denoted as MCP_raw), the task corpus (denoted as Task_raw), and the task–MCP interaction labels. In total, we have curated a collection of 5,642 MCP servers and 4,800 validated tasks. Each task is associated with a curated set of relevant MCP servers. Table 2 presents a summary of the primary attributes and their respective cardinalities. To facilitate understanding, Figure 3 also shows the dataset schema together with an illustrative example. , Vol. 1, No. 1, Article . Publication date: April 2026.

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation Field

Description

MCP_raw id

Unique identifier of the MCP server record

name

Name of the MCP server

description

Repository description of the MCP server

author

Repository owner or author name

url

GitHub repository URL of the MCP server

license

License type of the repository

language

Primary programming language of the MCP server

Category

Language

Theme

Ranker

Initial Ranking K1

Capability Neighborhood Expansion

MCP

K2 Expanded Candidates ∪

MCP

Expanded Pool Ck

Tasks

• MCP Text • Structured Attributes

Structured Distance Features (am ,at)

φcat

φlan Structural Scoring sstr Semantic Scoring ssem

dthe ∈{0,1}

φthe

MCP

Fused Score s (m, t) Weighted Combination

Recommendation List

1 2

Anchor Centroid Candidate Embeddings zm

MCP

dlan ∈{0,1}

MCP

Broader

Developer

dcat ∈{0,1,2}

Anchor Set Ck1

Task Embedding

B. Structural Compatibility Scoring

Task t = (xt ,at)

M1 candidates … Mk1 candidates

ssem

gt (·)

Final Result

MCP

m1 m2 … mi-1 mi

+ -

MCP

Anchor Candidate Retrieval + -

...

API

C. Candidate FinalRefinement Outputs

MCP Tower + -

...

MCP

A. Semantic Relevance Scoring Text Representation Sparse Lexical MCP Vectorization vm and vt xm

+ -

Input Data MCP m = (xm ,am)

From{K1 Anchors} and pool to{K candidates}

Constrained LLM Re-ranking Task Description MCP Metadata LLM Reranker

• • • •

MCP

...

MCP

Top-K MCP Servers

Prompt Tool Constraints Tool dependencies Tool composition Tool workflow

Fig. 4. Architecture of the T2MRec model. The model progressively narrows the search space of MCP servers through a multi-stage pipeline, including three core components: Semantic Relevance Scoring, Structural Compatibility Scoring, and Candidate Refinement.

Given a task description and its associated development constraints, T2MRec generates a ranked list of MCP server candidates likely to support task implementation. The model adopts a multi-stage retrieval-and-refinement paradigm, progressively narrowing the search space from coarse semantic matching to fine-grained capability reasoning. Specifically, T2MRec consists of three main components: (i) Semantic Relevance Scoring, which learns semantic correspondences between tasks and MCP server capabilities using a dual-tower encoder; (ii) Structural Compatibility Scoring, which evaluates engineering feasibility by aligning structured attributes such as category, programming language, and theme; (iii) Candidate Refinement, which improves coverage and ranking quality through centroid-based expansion and constrained LLM re-ranking. This multi-stage design ensures both retrieval efficiency and recommendation quality, while restricting computationally expensive reasoning to a small-scale candidate set. Figure 4 illustrates the overall architecture of the T2MRec model. 4.1

Problem Formulation

We formulate task-to-MCP server recommendations as a task-oriented top-𝐾 ranking problem. Given a task specification and associated engineering constraints, the objective is to generate a fixed-length candidate list from a rapidly evolving MCP ecosystem to assist developers or LLMbased agents in selecting and composing appropriate MCP servers. Let the MCP server set be 𝑁 M = {𝑚 𝑗 }𝑀 𝑗=1 and the task set be T = {𝑡𝑖 }𝑖=1 . Each MCP server 𝑚 ∈ M is represented as a two-tuple 𝑚 = (𝑥𝑚 , 𝑎𝑚 ), where 𝑥𝑚 contains the textual description (e.g., name, description, and tools) and 𝑎𝑚 contains the structured metadata (e.g., category, subcategory, language, license, system, and official). Similarly, each task 𝑡 ∈ T is represented as 𝑡 = (𝑥𝑡 , 𝑎𝑡 ), where 𝑥𝑡 denotes the concatenated textual description (e.g., name, description, and theme) and 𝑎𝑡 denotes a representation of structured attributes (e.g., category, subcategory, and language). For each task 𝑡, our dataset provides a curated list of MCP servers that are deemed relevant for implementing 𝑡. We denote the corresponding positive set as: M + (𝑡) = { 𝑚 ∈ M | 𝑚 is annotated as relevant to 𝑡 } , , Vol. 1, No. 1, Article . Publication date: April 2026.

(1)

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

where |M + (𝑡)| denotes the number of annotated relevant MCP servers for the task. For convenience, we define a binary relevance label as follows: ( 1, if 𝑚 ∈ M + (𝑡), 𝑦 (𝑚, 𝑡) = (2) 0, otherwise. Here, 𝑦 (𝑚, 𝑡) = 0 should be interpreted as an unlabeled or non-relevant instance in a weaksupervision setting, rather than a definitive claim that all remaining MCP servers are strictly irrelevant to the target task. Our research goal is to learn a scoring function: 𝑠 : M × T → R,

(𝑚, 𝑡) ↦→ 𝑠 (𝑚, 𝑡),

(3)

which estimates the suitability of MCP server 𝑚 for task 𝑡 based on both textual attributes (𝑥𝑚 , 𝑥𝑡 ) and structured attributes (𝑎𝑚 , 𝑎𝑡 ). During inference, given a new task 𝑡 ′ , our model ranks all MCP server candidates by 𝑠 (𝑚, 𝑡 ′ ) and outputs a top-𝐾 recommendation list. During training, we optimize this scoring function using the task–MCP server interaction set by contrasting positive pairs (𝑚, 𝑡) with negative MCP servers sampled from { 𝑚 ′ ∈ M | 𝑦 (𝑚 ′, 𝑡) = 0 }, encouraging the model to assign higher scores to annotated MCP servers than to the sampled negatives. In T2MRec, the scoring function integrates semantic relevance and structural compatibility, and is further refined through candidate expansion and LLM-based re-ranking. 4.2

Semantic Relevance Scoring

Semantic retrieval aims to identify MCP servers whose capabilities are semantically aligned with the task intent. Because task descriptions primarily express requirements, whereas MCP tool descriptions emphasize capabilities, a learned representation is needed to bridge the semantic gap between them. Recall from Section 4.1 that each MCP server 𝑚 and each task 𝑡 are represented as (𝑥𝑚 , 𝑎𝑚 ) and (𝑥𝑡 , 𝑎𝑡 ), respectively. Let 𝑥𝑚 denote the concatenated MCP text and 𝑥𝑡 denote the concatenated task text. In this section, we focus on the text pair (𝑥𝑚 , 𝑥𝑡 ) and learn a semantic scoring function such that MCP servers that are better aligned with the task intent will receive higher scores. 4.2.1 Text Representation. We first convert the concatenated task text 𝑥𝑡 and the concatenated MCP text 𝑥𝑚 into sparse lexical vectors: v𝑚 = v(𝑥𝑚 ) ∈ R𝑑 ,

v𝑡 = v(𝑥𝑡 ) ∈ R𝑑 ,

(4)

where v(·) denotes a vectorized operation, and 𝑑 is the shared vocabulary size. Tasks and MCP servers share the same vocabulary to ensure that they lie in a common feature space. We then apply an 𝐿2 normalization for text representations: v𝑚 v𝑡 ṽ𝑚 = , ṽ𝑡 = . (5) ∥v𝑚 ∥ 2 ∥v𝑡 ∥ 2 This step reduces sensitivity to document length and provides well-scaled inputs for the subsequent encoder. It also preserves informative lexical signals, such as tool names, APIs, and library identifiers, which frequently appear in both task requirements and MCP tool descriptions. 4.2.2 Semantic Encoder. Directly comparing the normalized sparse vectors (ṽ𝑚 and ṽ𝑡 ) is often insufficient for task-to-tool matching. Task texts typically encode intent, constraints, and desired outcomes, whereas MCP server descriptions enumerate capabilities, interfaces, and supported tools. Sparse lexical overlap is brittle under synonymy and paraphrasing, and it cannot adequately model the asymmetric correspondence between requirement expressions on the task side and capability , Vol. 1, No. 1, Article . Publication date: April 2026.

He et al.

expressions on the MCP tool side. Therefore, we introduce a learnable projection that maps both sides into a shared dense space, where the model can measure semantic compatibility more robustly. To capture semantic correspondences between tasks and MCP server capabilities, we employ a dual-tower encoder architecture with two side-specific encoders, 𝑔𝑚 (·) and 𝑔𝑡 (·). The task tower emphasizes requirement-bearing signals, and the MCP tower focuses on capability and tool descriptors. This design results in independent embeddings for tasks and MCP servers, enabling large-scale retrieval by pre-computing and indexing all MCP server embeddings offline and computing only the task embedding online at query time. This decoupling is essential when |M | is large, and the model must return top-𝐾 candidates under strict latency constraints. Given the normalized sparse inputs, we encode a task and an MCP server as: ′

z𝑚 = 𝑔𝑚 ( ṽ𝑚 ) ∈ R𝑑 ,

′

z𝑡 = 𝑔𝑡 ( ṽ𝑡 ) ∈ R𝑑 ,

(6)

where 𝑑 ′ is the output embedding dimension. In this study, each tower is implemented as an 𝐿-layer multilayer perceptron (MLP). The 𝐿 − 1 layers use affine transformations to compress the high-dimensional sparse input and learn weighted combinations of lexical features. Rectified linear unit (ReLU) introduces nonlinearity, allowing the encoder to capture cross-feature interactions beyond linear matching. Finally, dropout regularizes the hidden units, preventing co-adaptation and overfitting and improving robustness while keeping the architecture lightweight. The final layer outputs a 𝑑 ′ -dimensional embedding. We further normalize the tower outputs to unit length: z𝑚 z𝑡 ẑ𝑚 = , ẑ𝑡 = . (7) ∥z𝑚 ∥ 2 ∥z𝑡 ∥ 2 Normalization removes scale ambiguity in embeddings and ensures that scores are comparable across tasks. The semantic relevance score is then computed by: ⊤ 𝑠 sem (𝑚, 𝑡) = ẑ𝑚 ẑ𝑡 ,

(8)

which equals cosine similarity due to normalization. During inference, we pre-compute ẑ𝑚 for all MCP servers and store them in an embedding index. Given a new task 𝑡 ′ , we compute ẑ𝑡 ′ once and obtain possible candidates by ranking MCP servers according to 𝑠 sem (𝑚, 𝑡 ′ ). This design enables efficient large-scale retrieval because MCP server embeddings can be precomputed offline and indexed. Overall, the two-tower semantic encoder reduces the representation gap between requirementcentric task texts and capability-centric MCP tool texts. By learning semantic correspondences between tasks and MCP servers, the encoder improves robustness beyond raw lexical overlap. It also enables scalable retrieval by decoupling offline MCP server encoding from online task encoding. 4.2.3 Contrastive Alignment Training. The dual-tower encoder is trained using task–MCP interaction labels under a contrastive learning objective. Recall that for each task 𝑡, M + (𝑡) ⊆ M denotes the curated set of positive MCP servers. During model training, we construct a mini-batch of tasks 𝐵 , where 𝐵 = |B| is the batch size. For each task 𝑡 ∈ B, we sample one positive MCP B = {𝑡𝑖 }𝑖=1 𝑖 𝐵 . All MCP servers paired with other server 𝑚𝑖+ ∈ M + (𝑡𝑖 ), yielding 𝐵 matched pairs {(𝑚𝑖+, 𝑡𝑖 )}𝑖=1 tasks in the same batch are treated as implicit negatives for 𝑡𝑖 , enabling efficient contrastive learning without explicitly enumerating the full candidate set M. Since 𝑦 (𝑚, 𝑡) = 0 denotes unlabeled or non-relevant pairs rather than strictly irrelevant ones, some in-batch negatives may actually be useful MCP servers that are unlabeled. Nevertheless, the ′ contrastive objective provides a scalable and effective alignment signal in practice. Let ẑ𝑡𝑖 ∈ R𝑑 ′ and ẑ𝑚 𝑗 ∈ R𝑑 denote the 𝐿2 -normalized embeddings produced by the task tower and the MCP tower for task 𝑡𝑖 and MCP server 𝑚 𝑗 , respectively. We define the semantic relevance score used , Vol. 1, No. 1, Article . Publication date: April 2026.

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

for training as the dot product of normalized embeddings, as shown in Equation (8). For each task 𝑡𝑖 , let M B = {𝑚 +𝑗 | 𝑡 𝑗 ∈ B} denote the set of MCP servers appearing in the current mini-batch (including 𝑚𝑖+ ). The temperature-scaled contrastive loss for task 𝑡𝑖 is then defined as: exp 𝑠 sem (𝑡𝑖 , 𝑚𝑖+ )/𝜏 , (9) L𝑡𝑖 = − log Í 𝑚∈ M B exp(𝑠 sem (𝑡𝑖 , 𝑚)/𝜏) where 𝜏 > 0 is a temperature hyperparameter controlling the sharpness of the softmax distribution. This objective increases the semantic relevance score assigned to the matched pair (𝑡𝑖 , 𝑚𝑖+ ) relative to all other MCP servers in M B , thereby separating positives from in-batch negatives. A smaller 𝜏 yields a sharper distribution and places greater emphasis on hard negatives. The overall mini-batch loss is the average over tasks: 𝐵

1 ∑︁ L𝑡 . 𝐵 𝑖=1 𝑖

(10)

We optimize the parameters of the two towers by minimizing L using stochastic gradientbased optimization. Once the training process ends, the dual-tower encoder is frozen, and for any task–MCP server pair (𝑚, 𝑡), the cosine similarity 𝑠 sem (𝑚, 𝑡) in Equation (8) is used as the final semantic relevance score throughout inference. This objective encourages semantically relevant MCP servers to be closer to the task embedding than unrelated ones. 4.3

Structural Compatibility Scoring

While semantic similarity captures textual relevance between 𝑥𝑚 and 𝑥𝑡 , it may overlook engineering feasibility and ecosystem compatibility. For example, an MCP server may be semantically relevant to a task but still be unusable due to a mismatch in programming language, platform requirements, or taxonomy. To account for such factors, we define a structural compatibility score 𝑠 str (𝑚, 𝑡) based on the structured attributes (𝑎𝑚 and 𝑎𝑡 ). This score complements semantic retrieval by introducing lightweight constraint-aware signals beyond pure textual similarity. 4.3.1 Compatibility Feature Design. Recall that each MCP server and task are represented as 𝑚 = (𝑥𝑚 , 𝑎𝑚 ) and 𝑡 = (𝑥𝑡 , 𝑎𝑡 ), respectively. For notational convenience, we denote the structured attributes as: 𝑎𝑚 = (𝑐𝑚 , 𝑐˜𝑚 , ℓ𝑚 , 𝑝𝑚 ), 𝑎𝑡 = (𝑐𝑡 , 𝑐˜𝑡 , ℓ𝑡 , ℎ𝑡 ), (11) where 𝑐 denotes category, 𝑐˜ denotes subcategory, ℓ denotes programming language, 𝑝𝑚 denotes the operating system in which an MCP server runs, and ℎ𝑡 denotes the task theme. Our attribute design is inspired by prior multi-dimensional taxonomies of recommendation settings, which characterize tasks and methods from complementary perspectives [72]. The attributes 𝑝𝑚 and ℎ𝑡 characterize compatibility from the system and task sides, respectively. The former specifies the operating environment supported by an MCP server, whereas the latter describes the application setting of the given task. In practice, tasks within the same theme often share similar deployment assumptions and software interfaces. Hence, the compatibility between 𝑝𝑚 and ℎ𝑡 serves as a useful signal of practical deployability. By leveraging these attributes, we define a small set of interpretable compatibility features: 𝝓 str (𝑚, 𝑡) = 𝜙 cat (𝑚, 𝑡), 𝜙 lan (𝑚, 𝑡), 𝜙 the (𝑚, 𝑡) . (12) • Category feature 𝜙 cat (𝑚, 𝑡). This feature measures taxonomy proximity between an MCP server and the given task based on (𝑐𝑚 , 𝑐˜𝑚 ) and (𝑐𝑡 , 𝑐˜𝑡 ), reflecting whether they reside in a similar functional ecosystem. , Vol. 1, No. 1, Article . Publication date: April 2026.

He et al.

• Language feature 𝜙 lan (𝑚, 𝑡). This feature captures whether the task-side language constraint ℓ𝑡 is compatible with the MCP-side declared programming language ℓ𝑚 . • Theme feature 𝜙 the (𝑚, 𝑡). This feature measures whether the task theme ℎ𝑡 is compatible with the MCP-side system environment 𝑝𝑚 , serving as a lightweight signal of ecosystem-level feasibility beyond pure lexical similarity. Overall, these structured features provide a lightweight and interpretable complement to semantic matching. They inject signals of taxonomy consistency, language compatibility, and deployability into the ranking process of possible MCP servers, thereby improving constraint awareness while preserving inference efficiency. 4.3.2 Taxonomy-Based Distance Modeling. To quantify taxonomy proximity, we represent the category taxonomy as a rooted tree Tcat , whose root is the taxonomy root. In this tree, first-level nodes correspond to categories, and second-level nodes correspond to subcategories. For an MCP server 𝑚 and a task 𝑡, let 𝑣𝑚 and 𝑣𝑡 denote the subcategory nodes uniquely determined by (𝑐𝑚 , 𝑐˜𝑚 ) and (𝑐𝑡 , 𝑐˜𝑡 ), respectively. We define the taxonomy distance as the tree distance: 𝑑 cat (𝑚, 𝑡) = depth(𝑣𝑚 ) + depth(𝑣𝑡 ) − 2 × depth LCA(𝑣𝑚 , 𝑣𝑡 ) , (13) where 𝑑 cat (𝑚, 𝑡) denotes the tree distance between nodes 𝑣𝑚 and 𝑣𝑡 in Tcat , and LCA(𝑣𝑚 , 𝑣𝑡 ) denotes the lowest common ancestor of 𝑣𝑚 and 𝑣𝑡 . Since both 𝑣𝑚 and 𝑣𝑡 are depth-2 nodes in our two-level taxonomy, 𝑑 cat (𝑚, 𝑡) ∈ {0, 2, 4}. Specifically, 𝑑 cat (𝑚, 𝑡) = 0 when (𝑐𝑚 , 𝑐˜𝑚 ) = (𝑐𝑡 , 𝑐˜𝑡 ), 𝑑 cat (𝑚, 𝑡) = 2 when 𝑐𝑚 = 𝑐𝑡 but 𝑐˜𝑚 ≠ 𝑐˜𝑡 , and 𝑑 cat (𝑚, 𝑡) = 4 otherwise. We then normalize this distance to obtain a bounded proximity score: 𝑑 cat (𝑚, 𝑡) . (14) 4 This formulation preserves 𝑑 cat (𝑚, 𝑡) as a proper tree metric while separating distance measurement from proximity-score normalization. We further define two binary compatibility checks: 𝜙 cat (𝑚, 𝑡) = 1 −

𝜙 lan (𝑚, 𝑡) = I(ℓ𝑡 , ℓ𝑚 ),

(15)

𝜙 the (𝑚, 𝑡) = I(ℎ𝑡 , 𝑝𝑚 ),

(16)

where I(·, ·) denotes a binary indicator function defined on an attribute pair. It takes the value “1” if the two attributes are regarded as compatible under the predefined matching rules, and “0” otherwise. In particular, I(ℓ𝑡 , ℓ𝑚 ) = 1 indicates that the task language is compatible with the language specification of an MCP server. Similarly, I(ℎ𝑡 , 𝑝𝑚 ) = 1 indicates that the task theme is compatible with the platform or system environment. Finally, we calculate the structural compatibility score as a weighted combination of the above signals: 𝑠 str (𝑚, 𝑡) = 𝑤 cat × 𝜙 cat (𝑚, 𝑡) + 𝑤 lan × 𝜙 lan (𝑚, 𝑡) + 𝑤 the × 𝜙 the (𝑚, 𝑡), (17) where 𝑤 cat, 𝑤 lan, 𝑤 the > 0 are feature weights. In this work, we use a simple heuristic setting and assign equal weights to all structural signals. This choice keeps the method simple and reproducible, while avoiding extra hyperparameter tuning. 4.3.3 Final Matching Score. We combine semantic relevance and structural compatibility to obtain the final score for initial ranking: 𝑠 (𝑚, 𝑡) = 𝛼 sem 𝑠 sem (𝑚, 𝑡) + 𝛼 str 𝑠 str (𝑚, 𝑡)

𝛼 sem + 𝛼 str = 1,

(18)

where 𝛼 sem and 𝛼 str denote the fusion weights assigned to semantic relevance and structural compatibility, respectively. By default, we set 𝛼 sem > 𝛼 str , reflecting that semantic relevance is the primary signal, while structural compatibility provides complementary guidance on feasibility. The , Vol. 1, No. 1, Article . Publication date: April 2026.

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

resulting 𝑠 (𝑚, 𝑡) is used to generate the initial candidate list, which is further refined by candidate expansion and LLM-based re-ranking. 4.4

Candidate Refinement

After semantic retrieval and structural compatibility scoring, T2MRec obtains an initial ranking of MCP server candidates for a given task. However, this ranking may still be subject to two limitations. First, high-precision retrieval may miss complementary MCP servers that are not explicitly mentioned in the task description. Second, lightweight matching scores may not fully capture fine-grained alignment between task intent and MCP tool capabilities. To address these issues, we introduce a candidate refinement stage that transforms the learned semantic and structural signals into the final top-𝐾 recommendation list. Given a task 𝑡, we first compute the semantic score 𝑠 sem (𝑚, 𝑡) and the structural score 𝑠 str (𝑚, 𝑡) for each MCP server candidate 𝑚 ∈ M. We then apply a multi-stage pipeline comprising anchor retrieval, neighborhood expansion, and constrained LLM re-ranking. The initial ranking stage fuses semantic and structural evidence to produce a high-confidence anchor set C𝐾1 (𝑡). The centroid expansion stage enlarges this set to form an expanded candidate pool C𝐾2 (𝑡), thereby improving coverage of MCP servers near the task in the semantic space. Finally, the LLM re-ranking stage, which refines the candidate pool and selects the final recommendation set C𝐾 (𝑡) under practical constraints, aims to capture better fine-grained alignment between task intent and MCP tool capabilities. 4.4.1 Anchor Candidate Retrieval. Given the fused matching score 𝑠 (𝑚, 𝑡) defined in Section 4.1, we first construct a high-precision anchor set to support downstream expansion. Concretely, we sort all MCP server candidates 𝑚 ∈ M by 𝑠 (𝑚, 𝑡) and retain the top-𝐾1 MCP servers: C𝐾1 (𝑡) = TopK𝑚∈ M 𝑠 (𝑚, 𝑡), 𝐾1 . (19) This anchor set prioritizes precision. It retains MCP servers that are both semantically relevant and structurally feasible, serving as the reference set for the subsequent centroid expansion step. Using a small 𝐾1 helps concentrate the anchor set on high-confidence candidates, preventing later expansion from drifting toward noisy regions of the MCP server space. 4.4.2 Capability Neighborhood Expansion. Although the anchor set C𝐾1 (𝑡) typically contains highly relevant MCP servers, it may not cover complementary tools required in realistic development workflows. To improve candidate coverage without excessively increasing 𝐾1 , we design a centroidbased expansion mechanism. Let ẑ𝑚 denote the MCP server embedding and ẑ𝑡 denote the task embedding produced by the semantic encoder. We construct an expanded query representation by aggregating the task embedding with the embeddings of the anchor MCP servers: ∑︁ 1 © ª ẑ𝑚 ® . (20) ẑ𝑡 + 𝐾1 + 1 𝑚∈ C (𝑡 ) 𝐾 1 « ¬ Conceptually, c(𝑡) moves the query representation from the task text alone to a composite embedding that reflects both the task and its high-confidence capability neighborhood. This representation helps retrieve MCP servers that are not only semantically related to the task description but also coherent with the anchor set. We then retrieve additional candidates from the remaining MCP server pool by ranking them according to centroid similarity: ⊤ C𝐾2 (𝑡) = TopK𝑚∈ M\C𝐾 (𝑡 ) ẑ𝑚 c(𝑡), 𝐾2 − 𝐾1 , (21) c(𝑡) =

, Vol. 1, No. 1, Article . Publication date: April 2026.

He et al.

and form the expanded candidate set: C𝐾 ′ (𝑡) = C𝐾1 (𝑡) ∪ C𝐾2 (𝑡).

(22)

The centroid expansion step is computationally lightweight. It only requires a simple aggregation to form c(𝑡) and an additional similarity search over the existing embedding index, without any expensive cross-encoding. Despite this low cost, it broadens retrieval coverage by prioritizing MCP servers that are close to the anchor capability set in the embedding space. This bias tends to introduce MCP servers that are semantically coherent with the anchors and potentially complementary, thereby increasing candidate diversity without sacrificing consistency with the task intent. Take a simple task “read a PDF file and extract all tables” as an example. The anchor set retrieves MCP servers for PDF parsing and table extraction. Centroid expansion then introduces a complementary MCP server for optical character recognition (OCR), which is often needed when PDF files contain scanned pages. Although OCR may not be explicitly mentioned in the task text, it lies within the anchor neighborhood in the embedding space by our approach, thereby improving the completeness of the candidate set. 4.4.3 Constrained LLM Re-ranking. The final stage employs an LLM-based re-ranker to refine the ordering of the candidate pool C𝐾 ′ (𝑡). Crucially, we use an LLM as a constrained ranking component rather than an open-ended retriever. It operates on a small, pre-filtered candidate set and returns only a reordered list, thereby improving controllability and reducing bounds inference latency. We formalize the re-ranking operator as: 𝜋𝐾 (𝑡) = LLM-Rerank 𝑡, C𝐾 ′ (𝑡) , (23) where 𝜋𝐾 (𝑡) denotes the final top-𝐾 ranked tool list for task 𝑡. The LLM model receives the task text 𝑥𝑡 together with a compact summary of structural constraints derived from 𝑎𝑡 . For each candidate MCP server 𝑚 ∈ C𝐾 ′ (𝑡), it also receives a standardized candidate card containing the MCP server identifier, name, description, tool keywords, and key metadata. This LLM model then reorders MCP servers by assessing fine-grained intent-capability fit that is difficult to capture using lightweight similarity metrics and rule-based compatibility checks alone. We constrain the LLM model to re-rank only the retrieved set C𝐾 ′ (𝑡), ensuring deterministic and machine-parseable outputs. It must return exactly 𝐾 MCP server identifiers in ranked order; if replacement is allowed, at most two items may be substituted, and only with alternatives from C𝐾 ′ (𝑡). Decoding is deterministic, and any output violating length, membership, or formatting constraints is discarded in favor of the pre-LLM ordering. This constrained design allows T2MRec to leverage LLMs’ reasoning capabilities while maintaining stable inference latency and predictable behavior. To improve reproducibility and make the re-ranking stage auditable, we instantiate LLMs with a fixed prompt template rather than a free-form instruction. The template serializes the task context, structural constraints, candidate cards, ranking objective, and output schema into a deterministic input format. Figure 5 shows the prompt used in our design. This prompt operationalizes Equation (23) as a constrained list-wise ranking problem over a bounded candidate pool. The LLM model is therefore responsible only for reordering the retrieved MCP servers, not for expanding the search space or generating new candidates. This design allows the re-ranker to leverage the LLM model’s reasoning capabilities while keeping the recommendation process grounded in the upstream retrieval and compatibility-scoring stages. 4.5

Model Training

The T2MRec model is trained by optimizing the dual-tower semantic encoder on the task–MCP interaction data. In contrast, the structural compatibility scoring module and the constrained , Vol. 1, No. 1, Article . Publication date: April 2026.

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

Prompt Template for Constrained LLM Re-ranking

Role You are a constrained list-wise re-ranking module for task-oriented MCP server recommendation. Given one development task and a fixed candidate pool, rank the candidates by overall suitability. Select identifiers only from the provided candidate pool.

Input (a) Task Data. ID Name Description Attributes

{𝑖 } {𝑡𝑖 } {𝑥𝑡 } {𝑎𝑡 }

(b) Ranking Criteria. Rank candidates by overall suitability for the task. Prefer candidates that: • Closely match the task intent and required capabilities. • Satisfy the specified engineering constraints. • Fit naturally into the intended execution workflow. • Compose effectively with other candidates when coordination or dependencies are required. • Avoid redundancy, weak relevance, and overly generic functionality. (c) Candidate Card. For each candidate MCP server 𝑚 𝑗 ∈ C𝐾 ′ (𝑡 ), provide: ID Name Description Metadata

{𝑗} {𝑚 𝑗 } {𝑥𝑚 } {𝑎𝑚 }

(d) Rules. • Return exactly 𝐾 MCP server identifiers in ranked order. • Every identifier must appear in the provided candidate pool. • Preserve each identifier exactly as given. • Do not output any content outside the required JSON object. • Output must follow the JSON list format shown below.

Output JSON Object. Return a single JSON object of the form: { "Task": 𝑡𝑖 , "MCP_servers": [𝑚 1 , 𝑚 2 , ..., 𝑚𝑘 ], "Explanation": "Briefly explain why these MCP servers are ranked in this order." }

Fig. 5. Prompt template used by the constrained list-wise LLM re-ranker.

LLM-based re-ranking module are inference-stage components, which do not introduce additional trainable parameters. This design keeps model learning focused on the semantic alignment between task requirements and MCP server capabilities, while leaving structural compatibility filtering and fine-grained ranking refinement to later stages. For model training, we construct positive instances from annotated task–MCP pairs in the interaction set. For each task 𝑡, the MCP servers in M + (𝑡) are treated as positive instances. Since the dataset provides curated positive associations rather than exhaustive binary relevance labels, the remaining unlabeled MCP servers are not treated as guaranteed true negatives. Instead, negative , Vol. 1, No. 1, Article . Publication date: April 2026.

He et al.

supervision is introduced implicitly via mini-batch contrast, in which MCP servers paired with other tasks in the same batch serve as negatives for the current task, and vice versa. This strategy is well-suited to the weakly supervised nature of the recommendation setting and avoids imposing overly strong assumptions on unlabeled task–MCP pairs. The semantic encoder is optimized with the contrastive objective introduced in Equation (10) in Section 4.2.3, which encourages matched task–MCP pairs to be close in the shared embedding space while separating mismatched pairs. After model training, the encoder is frozen and used to produce semantic relevance scores for downstream retrieval and ranking. 4.6

Model Inference

Algorithm 1 summarizes the complete inference pipeline of T2MRec. The algorithm first performs semantic retrieval to identify MCP servers relevant to the task description, then incorporates structural compatibility scoring to filter out engineering-incompatible candidates. The candidate pool is subsequently expanded using centroid-based capability neighborhood exploration, and finally refined through constrained LLM re-ranking. The inference cost of T2MRec is dominated by corpus-level retrieval and candidate selection. Let 𝑀 denote the number of MCP servers and 𝑑 ′ the embedding dimension. Since MCP server embeddings are precomputed offline, online inference only requires encoding the input task once, followed by semantic relevance scoring over the MCP corpus, which costs 𝑂 (𝑀𝑑 ′ ) under exact retrieval. Structural compatibility scoring introduces only linear overhead, i.e., 𝑂 (𝑀), because each compatibility check is constant-time. The centroid-based refinement stage adds one additional retrieval pass of the same order, while candidate selection incurs 𝑂 (𝑀 log 𝑀) under exact sorting. Finally, the constrained LLM re-ranker is applied only to a bounded candidate pool and thus contributes an additional cost of 𝐶 LLM (𝐾) without changing the asymptotic dependence on 𝑀. Therefore, the overall inference complexity of T2MRec is 𝑂 (𝑀𝑑 ′ + 𝑀 log 𝑀 +𝐶 LLM (𝐾)) under exact retrieval, indicating that the model remains scalable with respect to corpus size while restricting expensive LLM reasoning to a small late-stage re-ranking step. 5 5.1

Experiments and Result Analysis Research Questions

To evaluate the proposed T2MRec model, we design experiments on the Task2MCP dataset to answer the following research questions (RQs). • RQ1: How does T2MRec perform compared with competitive baselines? • RQ2: How do the specific component designs of T2MRec affect model performance? • RQ3: How do the setting choices of different parameters affect model performance? 5.2

Baseline Methods

To assess the effectiveness of T2MRec, we compare it against selected competitive baselines from three categories: traditional, graph-based, and LLM-based recommendation methods. All baselines are adapted to the same Task2MCP formulation and evaluated under identical data splits, candidate spaces, and evaluation protocols. In this setting, each task is treated as an input query, and each MCP server is regarded as a candidate item. For methods originally designed for user-item recommendation, we reformulate each task–MCP association as an interaction and use the curated relevant MCP servers of each task as positive instances. For text-aware and LLM-based methods, we use the same task and MCP textual representations as model inputs and require all methods to produce a ranked list of MCP servers for each task under the same candidate pool. , Vol. 1, No. 1, Article . Publication date: April 2026.

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

Algorithm 1: Inference Algorithm of T2MRec Input: Task 𝑡 = (𝑥𝑡 , 𝑎𝑡 ), MCP server set M = {𝑚 𝑗 }𝑀 𝑗=1 with precomputed embeddings {ẑ𝑚 }𝑚∈ M Output: Ranked top-𝐾 MCP server list C𝐾 (𝑡) 1 Initialize hyperparameters 𝛼 sem, 𝛼 str, 𝑤 cat, 𝑤 lan, 𝑤 the, 𝐾1, 𝐾2 , and 𝐾 2 Compute sparse lexical vector v𝑡 ← v(𝑥𝑡 ) using Equation (4) 3 Normalize task input ṽ𝑡 ← v𝑡 /∥v𝑡 ∥ 2 using Equation (5) 4 Compute task embedding z𝑡 ← 𝑔𝑡 ( ṽ𝑡 ) using Equation (6) 5 Normalized task embedding ẑ𝑡 ← z𝑡 /∥z𝑡 ∥ 2 using Equation (7) 6 foreach 𝑚 ∈ M do ⊤ ẑ using Equation (8) 7 Compute semantic relevance score 𝑠 sem (𝑚, 𝑡) ← ẑ𝑚 𝑡 8 Compute structural compatibility score 𝑠 str (𝑚, 𝑡) using Equation (17) 9 Compute fused score 𝑠 (𝑚, 𝑡) using Equation (18) Select anchor set C𝐾1 (𝑡) using Equation (19) 11 Compute centroid c(𝑡) using Equation (20) 12 Retrieve expanded candidates C𝐾2 (𝑡) using Equation (21) 13 Construct final candidate pool C𝐾 ′ (𝑡) ← C𝐾1 (𝑡) ∪ C𝐾2 (𝑡) using Equation (22) 14 Compute C𝐾 (𝑡) ← LLM-Rerank(𝑡, C𝐾 ′ (𝑡)) using Equation (23) 15 return C𝐾 (𝑡) 10

5.2.1 Traditional Recommendation Models. Our experiment first includes traditional recommendation models (e.g., collaborative filtering and matrix factorization) as non-graph baselines. These methods provide strong reference points for assessing whether T2MRec’s performance gains in recommendation stem from improved modeling of task–MCP associations rather than from increased architectural complexity alone. • NeuMF6 (2017) [71] is a neural collaborative filtering model that combines generalized matrix factorization with a multi-layer perceptron to capture both linear and non-linear interaction patterns. • MSBPR7 (2023) [28] extends Bayesian personalized ranking for one-class collaborative filtering by partitioning non-interacted items with auxiliary feedback and item/user-based similarity and by learning multiple pairwise preferences over these partitions. • ReCAFR8 (2025) [57] is a review-centric contrastive alignment framework that aligns user, item, and review representations in a unified space to alleviate sparsity and improve robustness of personalized recommendations. 5.2.2 GNN-based Recommendation Models. We further consider graph-based recommendation methods, which are widely used for modeling relational dependencies in user-item interaction data. In our setting, these methods operate on the task–MCP interaction graph and serve as strong baselines for evaluating whether explicit graph modeling improves recommendation quality.

6 https://github.com/hexiangnan/neural_collaborative_filtering 7 https://github.com/Delta-zl/Code-MSBPR 8 https://github.com/smufang/ReCAFR.git

, Vol. 1, No. 1, Article . Publication date: April 2026.

He et al.

• NGCF9 (2019) [44] incorporates the bipartite interaction graph into representation learning through neural message passing, enabling the modeling of collaborative signals and higherorder connectivity. • LightGCN10 (2020) [45] is a simplified graph collaborative filtering model that retains only neighborhood aggregation, learning node representations by linearly propagating embeddings over the interaction graph. • GDSRec11 (2023) [73] is a graph-based distributed collaborative filtering recommendation method that models bias information in user and item representations. • CVH-REC12 (2026) [74] models mashup-API interactions and tag usage records as a multiview knowledge graph, and it employs cross-view hypergraph neural networks with contrastive learning and multi-task learning to learn entity representations. 5.2.3 LLM-based Recommendation Models. Our experiment also includes recent LLM-based recommendation methods, enabling a comparison of T2MRec against new approaches that exploit LLMs for recommendation and ranking. These baselines are particularly relevant to our problem setting because task and MCP server descriptions are both naturally expressed in text. • TiGER13 (2023) [54] formulates recommendation as a generative retrieval problem by assigning semantic identifiers to items and autoregressively generating the identifier of the target item. • OpenP514 (2024) [55] is an open-source platform for developing and evaluating LLM-based generative recommender systems, which we use as a representative generative recommendation baseline in this study. • LC-Rec15 (2024) [56] adapts LLMs to recommendation by integrating language semantics with collaborative semantics through learned item indices and alignment-oriented tuning. • MPT-Rec16 (2025) [75] improves task recommendation for emerging tasks through a twostage prompt-tuning framework, which disentangles shared and task-specific knowledge and transfers it to new tasks via task-aware prompts with reduced training cost. 5.3

Evaluation Metrics

We evaluate all methods using top-𝑘 ranking metrics with 𝑘 ∈ {5, 10}. Since the task-to-MCP recommendation aims to return a short list that both covers useful MCP servers and prioritizes the most suitable ones, we report Recall@𝑘, Precision@𝑘, F1@𝑘, and NDCG@𝑘. Among these metrics, Recall@𝑘 and NDCG@𝑘 are treated as the primary indicators for model performance comparison, and Precision@𝑘 and F1@𝑘 are reported as supplementary statistics. For a test task 𝑡 ∈ Ttest , let M + (𝑡) denote the annotated relevant MCP server set and C𝐾 (𝑡) = [𝑚 1, . . . , 𝑚𝑘 ] the top-𝑘 ranked list returned by a method. We first define the number of relevant MCP servers retrieved within the top-𝑘 positions as 𝐻𝑘 (𝑡) = C𝐾 (𝑡) ∩ M + (𝑡) . 9 https://github.com/huangtinglin/NGCF-PyTorch 10 https://github.com/kuandeng/LightGCN 11 https://github.com/MEICRS/GDSRec 12 https://zenodo.org/records/18346231 13 https://github.com/XiaoLongtaoo/TIGER.git 14 https://github.com/agiresearch/OpenP5 15 https://github.com/RUCAIBox/LC-Rec/ 16 https://github.com/BAI-LAB/MPT-Rec

, Vol. 1, No. 1, Article . Publication date: April 2026.

(24)

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

(25)

(26)

test

Recall@𝑘 measures the fraction of annotated relevant MCP servers covered by the top-𝑘 list, whereas Precision@𝑘 measures the proportion of returned MCP servers that are relevant. To jointly reflect coverage and exactness, we further report F1@𝑘, which is defined from the per-task precision and recall values: ( 2𝑃𝑘 (𝑡 )𝑅𝑘 (𝑡 ) , if 𝑃𝑘 (𝑡) + 𝑅𝑘 (𝑡) > 0, (27) 𝐹 1𝑘 (𝑡) = 𝑃𝑘 (𝑡 )+𝑅𝑘 (𝑡 ) 0, otherwise, and macro-averaged as 1 ∑︁ F1@𝑘 = 𝐹 1𝑘 (𝑡). (28) |Ttest | 𝑡 ∈ T test

Since Recall@𝑘 does not distinguish whether relevant MCP servers appear at the top or near the bottom of the ranked list, we additionally report NDCG@𝑘 to evaluate position-sensitive ranking quality. Under binary relevance, the discounted cumulative gain for task 𝑡 is DCG@𝑘 (𝑡) =

𝑘 ∑︁ I[𝑚𝑖 ∈ M + (𝑡)] 𝑖=1

log2 (𝑖 + 1)

(29)

and the corresponding ideal DCG is IDCG@𝑘 (𝑡) =

min(𝑘,|∑︁ M + (𝑡 ) | ) 𝑖=1

1 . log2 (𝑖 + 1)

The macro-averaged NDCG@𝑘 is then defined as 1 ∑︁ DCG@𝑘 (𝑡) . NDCG@𝑘 = |Ttest | 𝑡 ∈ T IDCG@𝑘 (𝑡)

(30)

(31)

test

A higher NDCG@𝑘 value indicates that MCP servers relevant to the target task are ranked the top on average, which is particularly important in practice because developers usually inspect only the first few recommendations. 5.4

Implementation and Settings

All the experiments for T2MRec17 were conducted on a server equipped with dual Intel Xeon Silver 4210 central processing units (2.20 GHz), two NVIDIA A6000 graphics processing units, and 128 GB of random access memory, using Python 3.8 and PyTorch 2.1.2 with CUDA Toolkit18 12.1. For baseline methods, we followed the settings reported in their original papers and reproduced them based on publicly available source code. Since different baseline implementations have their own dependency requirements, they were executed in the corresponding software environments specified by their original implementations or released code. The Task2MCP dataset was partitioned into training, validation, and test sets in a 60:20:20 ratio. The validation set was used for hyperparameter tuning, and all final results were reported on the held-out test set. 17 https://github.com/ssea-lab/Task2MCP/T2MRec 18 https://developer.nvidia.com/cuda/toolkit

, Vol. 1, No. 1, Article . Publication date: April 2026.

He et al.

Table 3. Overall performance comparison of T2MRec and selected baselines on the Task2MCP dataset. The best results are shown in bold, and the best baseline results are underlined. Gains indicate the relative improvement of T2MRec over the strongest baseline for each metric. Category

Model

NeuMF Traditional MSBPR ReCAFR

Recall@5 Recall@10 Precision@5 Precision@10 F1@5

F1@10 NDCG@5 NDCG@10

0.5375 0.5463 0.4803

0.6423 0.6489 0.5957

0.1101 0.1093 0.0966

0.0674 0.0654 0.0600

0.1902 0.2247 0.1609

0.1226 0.1190 0.1090

0.4377 0.3994 0.3539

0.4568 0.4346 0.3916

NGCF LightGCN GDSRec CVH-REC

0.4541 0.4047 0.3841 0.1550

0.5355 0.5047 0.6454 0.2853

0.0904 0.0809 0.1775 0.1384

0.0559 0.0505 0.1209 0.1165

0.1937 0.1349 0.1444 0.1165

0.1016 0.0918 0.1417 0.1682

0.3352 0.3035 0.0834 0.3359

0.3634 0.3358 0.0927 0.1572

TiGER OpenP5 LLM-based LC-Rec MPT-Rec T2MRec

0.5631 0.0081 0.4677 0.5838 0.5971

0.5827 0.0097 0.5076 0.6533 0.6702

0.1115 0.0017 0.0945 0.1804 0.4246

0.0585 0.0009 0.0508 0.1463 0.3952

0.1859 0.0029 0.1575 0.2690 0.4963

0.1064 0.0016 0.0924 0.2456 0.4973

0.5203 0.0061 0.3872 0.4440 0.5452

0.5267 0.0063 0.3985 0.5501 0.6633

2.23%↑

2.52%↑

57.51%↑

62.89%↑

45.80%↑ 50.61%↑

4.57%↑

17.07%↑

GNN-based

Gains

For text modeling, each task 𝑡 was represented as a unified text field 𝑥𝑡 by concatenating its task name, description, programming language, category, and theme. Similarly, each MCP server 𝑚 was represented by 𝑥𝑚 as a concatenation of its name, description, supported programming language, system type, tool list, category, and subcategory. To model semantic alignment between task requirements and MCP tool capabilities, we trained a dual-tower projection network over the textual representations of tasks and MCP servers. Both the task tower and the MCP tower were implemented as three-layer MLPs, with a hidden dimension of 512 and an output embedding dimension of 256. ReLU was used as the activation function between layers, and dropout with a rate of 0.2 was applied for regularization. The two towers were trained using a symmetric contrastive objective with in-batch negative samples. We optimized the T2MRec model with AdamW [76], using a learning rate of 1 × 10−3 , a batch size of 256, and a training duration of 200 epochs. The temperature parameter in the contrastive loss was set to 0.07. After model training, the task encoder and MCP encoder projected their inputs into a shared semantic embedding space. During model inference, semantic relevance and structural compatibility were optimally weighted at 0.9 and 0.1, respectively, and the fused scores were used to form the initial candidate set. For the final refinement stage, we adopted a constrained LLM-based re-ranking procedure with GPT-5.2. Given a task description and a structured list of candidate MCP servers, the LLM model was instructed to reorder the candidates by the degree of alignment between the task requirements and the MCP servers’ capabilities. To improve output consistency, we enforced a strict output format and used deterministic decoding during model inference. We further applied a validity check to verify the returned ranking before producing the final recommendation list. In our method, LLMs are used only as a constrained re-ranking component over a compact candidate set, which keeps inference overhead manageable. A more substantive role appears in the downstream interactive recommendation system, where it grounds recommended MCP servers in developers’ concrete requirements and produces practical guidance for adoption and use. 5.5

Experimental Results

5.5.1 Performance Comparison (RQ1). We analyzed the recommendation performance of T2MRec and selected competitive baselines on the Task2MCP dataset, using multiple evaluation metrics, to assess the overall effectiveness of the proposed method. We compared T2MRec with traditional, , Vol. 1, No. 1, Article . Publication date: April 2026.

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

graph-based, and LLM-based recommendation methods using the same data split, candidate space, and evaluation protocol. Table 3 presents the comparison results, where the underlined values denote the best results achieved by the baseline methods. Overall, T2MRec achieved the best overall performance over all baselines. Specifically, it obtained the highest Recall@5 and Recall@10, indicating a stronger ability to retrieve relevant MCP servers within a limited top-𝐾 recommendation budget. It also achieved the highest NDCG@5 and NDCG@10, suggesting that relevant MCP servers were ranked earlier in the recommendation list. Furthermore, T2MRec outperformed all baselines regarding Precision@5, Precision@10, F1@5, and F1@10, confirming its superiority in both retrieval effectiveness and ranking quality. These results demonstrate that T2MRec provides more accurate and better-ordered recommendations for task-oriented MCP tool selection in realistic development scenarios. First, traditional recommendation methods showed limited effectiveness in this setting. Methods such as MSBPR, NeuMF, and ReCAFR primarily model task–MCP associations from observed interaction patterns and therefore rely heavily on historical co-occurrence signals to infer relevance. Such modeling is useful for capturing coarse-grained matching regularities. Still, it is less effective for task-oriented MCP server recommendation, where the key challenge lies in aligning task intent with MCP tool capability. In our scenario, task descriptions usually emphasize user goals, functional requirements, constraints, and expected outcomes, whereas MCP server descriptions focus more on supported tools, interfaces, execution environments, and service capabilities. This mismatch makes it difficult for interaction-driven methods to establish accurate correspondences when semantic understanding is required. Among the traditional baselines, NeuMF and MSBPR were the most competitive, with NeuMF showing relatively stronger ranking quality and MSBPR achieving slightly better Recall and F1 values. However, both methods still consistently underperformed T2MRec across the overall evaluation. These results suggest that learning from interaction signals alone is insufficient for accurately recommending MCP servers for complex development tasks. Second, graph-based recommendation methods improved the modeling of structural relations, but their gains remained limited. NGCF and LightGCN extend traditional recommender systems by exploiting higher-order connectivity on the task–MCP interaction graph, allowing information to propagate through multi-hop neighborhoods. GDSRec and CVH-REC further incorporate graphbased structural information from different perspectives, reflecting the potential of graph modeling for recommendation. For recommendation problems with dense interactions and strong relational patterns, such graph propagation is often highly effective. However, the task–MCP server recommendation scenario considered in this study differs from these settings. The interaction graph is relatively sparse, and the connections between tasks and MCP servers depend not only on structural proximity but also on whether an MCP server semantically satisfies the target task’s functional requirements. Under these conditions, higher-order propagation may introduce noisy or weakly relevant signals, rather than providing reliable evidence for recommendations. In the experiment, although graph-based methods captured structural information more effectively than traditional interaction-driven approaches, they still failed to achieve competitive overall performance. This result suggests that structural modeling alone is insufficient for task-to-MCP recommendation. Third, LLM-based recommendation methods achieved stronger overall performance, highlighting the importance of semantic understanding in this specific recommendation scenario. Methods such as TiGER, LC-Rec, MPT-Rec, and OpenP5 are better equipped to capture the semantic relationships between task descriptions and MCP server descriptions. They are therefore generally more suitable for task-oriented MCP tool recommendation than traditional or graph-based methods. Among them, MPT-Rec was the strongest baseline on most evaluation metrics, and TiGER achieved the best baseline result on NDCG@5, showing that LLM-based methods can improve both retrieval and ranking quality when semantic matching is critical. However, their overall performance , Vol. 1, No. 1, Article . Publication date: April 2026.

He et al.

remained below that of T2MRec. Without an effective retrieval and filtering mechanism, LLM-based recommendations may suffer from instability, sensitivity to prompting strategies, and difficulty in maintaining consistent performance. These findings indicate that while LLMs exhibit strong semantic reasoning capabilities, relying on them alone is insufficient to produce stable, accurate MCP server recommendations in realistic, large-scale development-task settings. The overall comparison indicates the effectiveness of T2MRec for task-oriented MCP server recommendation. By consistently outperforming competitive baselines on both retrieval and ranking metrics, T2MRec demonstrates that accurate MCP tool recommendations for development tasks benefit from the joint use of semantic modeling, structural information, and ranking refinement. Answer to RQ1 T2MRec consistently outperforms selected competitive baselines in recommendation performance on the Task2MCP dataset. It demonstrates greater effectiveness in both retrieving relevant MCP servers and ranking them ahead of competing candidates, yielding the best overall performance across evaluation metrics. These results provide clear evidence that T2MRec is more effective than existing representative methods and can better capture the complex relationships between task requirements and MCP tool capabilities. 5.5.2 Ablation Study (RQ2). To answer RQ2, we conducted an ablation study to examine the contribution of each key component in T2MRec. Specifically, we evaluated four variants by removing or replacing one component at a time, including the lexical representation module, the two-tower encoder, contrastive learning, and the LLM-based re-ranking stage. Table 4 reports the ablation study results on the test set. Overall, removing any component degraded model performance, confirming that all components contribute to T2MRec’s overall performance. Among them, the largest performance drops were observed when the two-tower encoder or contrastive learning was removed, indicating that semantic representation learning is the core of our approach and that the re-ranking stage primarily contributes to finer-grained ordering. (1) Contrastive Learning. To examine the effect of the training objective, we constructed the w/o CL variant by replacing the contrastive loss with a binary cross-entropy (BCE) style point-wise objective. As shown in Table 4, this variant suffered the largest performance drop among all ablated versions. Specifically, Recall@10 decreased from 0.6702 to 0.4568, Precision@10 dropped from 0.3952 to 0.2514, F1@10 declined from 0.4973 to 0.3218, and NDCG@10 decreased from 0.6633 to 0.4413. Similar degradation was consistently observed across the top-5 metrics. These results show that contrastive learning is a critical component of T2MRec and makes an essential contribution to both retrieval and ranking performance. The effectiveness of contrastive learning is closely related to the characteristics of the task-toMCP server matching problem. In this task, unobserved task–MCP server pairs cannot be treated as reliable negative samples, since some unlabeled MCP servers may still be potentially relevant to Table 4. Performance comparison among T2MRec variants. Metric w/o CL w/o Two-tower w/o Lexical w/o Re-ranking T2MRec

Recall@5 Recall@10 Precision@5 Precision@10 F1@5 F1@10 NDCG@5 NDCG@10 0.3824 0.4726 0.5754 0.5932 0.5971

0.4568 0.5481 0.6482 0.6668 0.6702

0.2761 0.3423 0.4129 0.4213 0.4246

, Vol. 1, No. 1, Article . Publication date: April 2026.

0.2514 0.3148 0.3836 0.3925 0.3952

0.3220 0.3218 0.3979 0.3970 0.4805 0.4805 0.4946 0.4947 0.4963 0.4973

0.3527 0.4385 0.5287 0.5389 0.5452

0.4413 0.5297 0.6463 0.6574 0.6633

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

Grouped Violin Plot of Input vs. Learned Distances Across Task Categories Input Embedding

160 p<0.001

p<0.001

140

p<0.001

Task-to-MCP Distance

120

p<0.001

100

n nte

tio

rea

n nte

the

sis on

i cis

kin

tio

tec

al git

ist

ass

sco

g ma

lys

m for

o ati

iev

etr

c an

pro

m ei

rfo

rin

ito

rso

ati

liz

ict

ion

ati

ess

roc

tom

ati

c Re

tom

ati

tic

o ob

lar

u hic

tom

Task Category

Fig. 6. Effect of the two-tower encoder on task-to-MCP alignment in the shared embedding space using violin plots. Statistical significance is evaluated using paired one-sided Wilcoxon signed-rank tests with 95% confidence intervals, where 𝑝-value denotes the category-wise difference between the input and embedding distances. 𝑝 < 0.001 indicates a very significant statistical difference.

the target task. Under such conditions, a BCE-style point-wise objective may impose inaccurate negative supervision, thereby distorting the learned matching space. In contrast, the contrastive objective emphasizes the relative discrimination between positive and negative candidates within a mini-batch, making it better suited to handle label ambiguity. This component enables our model to learn a more discriminative compatibility space, thereby improving both candidate retrieval and ranking quality. (2) Two-tower Encoder. To examine the effect of the dual-tower architecture, we constructed the w/o two-tower variant, which removed the task and MCP projection networks and instead matched them directly in the original representation space. This variant consistently underperformed the complete model on all evaluation metrics. For example, Recall@10 decreased from 0.6702 to 0.5481, F1@10 decreased from 0.4973 to 0.3970, and NDCG@10 decreased from 0.6633 to 0.5297. These results show that the dual-tower architecture is important for model performance. To further examine whether the two-tower encoder improves alignment at the representation level, we compared the distributions of task-to-MCP distances before and after embedding learning across all task categories, as shown in Figure 6. For each category, statistical significance was evaluated using a paired one-sided Wilcoxon signed-rank test with a confidence interval of 95%, where 𝑝 measured whether the embedding space significantly reduced the distance of matched taskto-MCP pairs relative to the input space. Figure 6 shows that the embedding distance distributions were consistently far lower than the input distance distributions, and all categories satisfied 𝑝 < 0.001. This statistical result also confirms that the two-tower encoder learns a more effective shared space for aligning tasks with MCP servers. This improvement can be attributed to the heterogeneous semantics of tasks and MCP servers. Task descriptions focus on requirements and goals, whereas MCP tool descriptions emphasize capabilities and execution support. The dual-tower architecture helps bridge this gap by learning , Vol. 1, No. 1, Article . Publication date: April 2026.

He et al.

separate mappings and projecting both sides into a shared space for compatibility modeling. Without this projection process, our model cannot align task requirements with MCP server capabilities as effectively, which results in weaker recommendation performance. (3) Lexical Representation. To examine the effect of lexical-aware input representation, we designed the w/o lexical variant by replacing the lexical-aware sparse vectors defined in Equations (4) and (5) with SBERT-based dense semantic embeddings for both tasks and MCP servers, while keeping the dual-tower encoder unchanged. As shown in Table 4, this change caused a consistent performance drop across all evaluation metrics. Specifically, Recall@10 decreased from 0.6702 to 0.6482, Precision@10 decreased from 0.3952 to 0.3836, F1@10 decreased from 0.4973 to 0.4805, and NDCG@10 decreased from 0.6633 to 0.6463. Similar degradation was also observed in the top-5 metrics. These results indicate that lexical representation remains an important component in task-oriented MCP server recommendation. This finding is consistent with the characteristics of MCP tool descriptions. In many cases, MCP servers are described using capability-related lexical cues, such as tool names, library names, framework terms, API keywords, and other domain-specific identifiers, which often provide direct evidence for matching task requirements with MCP tool capabilities. Although dense semantic representations are effective in capturing overall semantic similarity, they may smooth over exact lexical signals when texts are mapped into a more abstract embedding space. Instead, lexical-aware representations explicitly preserve these distinctive cues, making them better suited to identifying concrete capability indicators in MCP server descriptions. Consequently, removing this component weakened our model’s ability to capture fine-grained task–MCP server alignment, leading to lower retrieval effectiveness and poorer ranking quality. (4) LLM Re-ranking. To examine the contribution of the final refinement stage, we constructed the w/o re-ranking variant by removing the constrained LLM-based re-ranking module and directly using the pre-reranking order for MCP server recommendations. As shown in Table 4, this variant performed slightly worse than the complete model across all evaluation metrics. In particular, Recall@5 decreased from 0.5971 to 0.5932, Recall@10 decreased from 0.6702 to 0.6668, NDCG@5 decreased from 0.5452 to 0.5389, and NDCG@10 decreased from 0.6633 to 0.6574. Similar declines were also observed in Precision and F1. These results indicate that although the re-ranking stage can consistently improve both retrieval and ranking performance, its effect is more moderate than that of the contrastive objective or the dual-tower architecture. This finding is consistent with the role of re-ranking in T2MRec. After the earlier stages have produced a strong candidate set, the remaining challenge is to more effectively distinguish among highly relevant MCP servers that differ subtly in their capability alignment. The constrained LLM-based re-ranking module provides this finer-grained refinement by leveraging more detailed semantic matching between task requirements and MCP server descriptions. Therefore, while its contribution is relatively small, it still yields more accurate and better-ordered recommendations, especially at the top of the ranked MCP server list. Answer to RQ2 All component designs of T2MRec contribute to its overall performance, but their effects differ in magnitude. The contrastive learning objective and the dual-tower architecture have the largest impact, the lexical-aware representation further improves performance by preserving capability-related cues in MCP tool descriptions, and the LLM-based re-ranking stage provides additional gains by refining the order of highly relevant candidates.

, Vol. 1, No. 1, Article . Publication date: April 2026.

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

From Language to Action: Enhancing LLM Task Efficiency with Task-Aware MCP Server Recommendation

Related documents