RadioMaster: Multi-Agent System for Autonomous Radio Signal Generation Jiazhen Lei, Tianze Cao, Yuxin Sha, Sihan Wang, Bingbing Wang, Fengyuan Zhu, Zeming Yang, Xiaohua Tian Traditional Manual Workflow
arXiv:2606.01862v1 [cs.MA] 1 Jun 2026
Abstract Translating user intents into physical radio signals represents the critical yet notoriously tedious final step in wireless prototyping, as it requires intricate knowledge of physical layer details and presents immense implementation challenges. Large Language Models (LLMs) and multi-agent systems have revolutionized conventional software engineering, raising the compelling question of whether they can resolve these formidable difficulties. However, our investigations reveal that current models experience significant limitations and fail to accomplish this task when applied to radio signal generation. This performance degradation primarily stems from severe domain ignorance and a fundamental insensitivity to physical hardware constraints. To bridge this gap, we introduce RadioMaster, a fully autonomous multi-agent framework designed to seamlessly translate user input into real-world wireless emissions. RadioMaster operates on three synergistic pillars: RadioWiki for domain-specific knowledge retrieval, RadioAgent for collaborative I/Q sample generation alongside hardware configuration, and RadioEmulator for closed-loop physical layer verification. Furthermore, we construct RadioBench, the first comprehensive benchmark tailored specifically for the radio signal generation domain. Extensive real-world evaluations demonstrate that RadioMaster significantly outperforms state-of-the-art (SOTA) baselines regarding configuration viability and signal fidelity.
CCS Concepts • Computing methodologies → Artificial intelligence.
Keywords Multi-Agent System, Radio Signal Generation, Large Language Model
1
Introduction
Radio signal generation represents the critical “last mile” for validating wireless communication paradigms in the physical world. Software-defined Radio (SDR) serves as the cornerstone technology for this phase, offering unprecedented prototyping flexibility by decoupling complex signal processing from dedicated hardware [24, 57]. In practice, engineers derive their implementation goals from diverse multimedia sources, such as textual descriptions, complex system block diagram images, or dense PDF protocol standards. Translating these intricate intents into executable baseband algorithms and hardware configurations demands substantial multidisciplinary expertise [14, 42]. As illustrated in the top portion of Fig. 1, the traditional manual workflow required to realize this translation is notoriously tedious and prone to errors. Human operators must manually parse thick protocol specifications, mathematically translate them into digital baseband algorithms, and subsequently write intricate hardware
2.Complex Baseband Coding & Debugging
1.Manual Protocol Study
3.Tedious Hardware Setup & Integration FAILED
Tedious and Error-Prone
RadioMaster Autonomous Workflow 2.Autonomous System Processing
1.User Input
Request: Send BLE Beacon on Channel 37 RadioWiki (Abstract Goal) RAG
RadioAgent
3.Automatic Implementation & Success RadioEmulator
OOK to Radio
Planner Operator Background Knowledge
Worker Modulator
Closed-Loop Verification
SUCCESS
Efficient and Robust
Figure 1: Comparison between the tedious traditional manual workflow and the efficient RadioMaster autonomous multiagent framework for radio signal generation.
execution code to configure SDR platforms. This disjointed process creates a severe bottleneck, significantly raising the barrier to entry and stifling rapid prototyping in wireless systems [16, 48]. LLMs [3, 5, 7, 10, 11, 33] and multi-agent systems [20, 23, 25, 29, 32, 41, 55] have revolutionized high-level software engineering and generic automation. Current SOTA foundation models and agentic frameworks, such as Claude Code [2], GPT Codex [7], Qwen3 Max [11], DeepSeek V3.2 [33], and Gemini3.1 Pro [5], excel at processing general semantic logic and writing standard computational code. This remarkable progress naturally raises a compelling question regarding whether these intelligent systems can resolve the formidable difficulties inherent in radio signal generation. However, our preliminary investigations reveal that their direct application to physical cyber-physical systems [34, 35, 52, 54] exposes profound limitations. Specifically, these generic models experience near-total failure when tasked with the complete lifecycle of radio signal generation. This catastrophic performance primarily stems from a fundamental lack of highly specialized wireless domain corpora and the physical layer grounding required to comprehend strict hardware constraints and complex baseband processing. Consequently, our empirical evaluations identify three critical limitations hindering their practical deployment: (1) the hallucination of internal APIs and functions, where models frequently generate syntactically plausible but nonexistent library calls; (2) the semantic misinterpretation of intricate protocols, as generic models struggle to maintain the tight parameter coupling and rigid data formatting demanded by complex wireless standards; and (3) the insensitivity of physical hardware constraints, leading to
Conference’17, July 2017, Washington, DC, USA
Jiazhen Lei, Tianze Cao, Yuxin Sha, Sihan Wang, Bingbing Wang, Fengyuan Zhu, Zeming Yang, Xiaohua Tian
naive configurations that fail to appropriately map digital parameters to actual physical transmission capabilities. Ultimately, these shortcomings produce unexecutable configurations, completely failing to bridge the gap between abstract multimodal requirements and actual physical deployment. To overcome these bottlenecks, we introduce RadioMaster, a fully autonomous multi-agent framework designed to seamlessly translate user inputs into verified physical radio signals. As depicted in Fig. 1, RadioMaster operates on three synergistic pillars. First, RadioWiki utilizes an adaptive routing Retrieval Augmented Generation (RAG) mechanism over a domain-specific knowledge base to ground the generation process and alleviate hallucinations. Second, RadioAgent orchestrates a collaborative architecture comprising a Planner, Worker, Modulator, and Operator. This team dynamically decomposes complex intents, performs baseband processing, and synthesizes precise execution scripts. Third, RadioEmulator establishes rigorous closed-loop verification within a virtual communication environment to validate signal integrity and protocol compliance before actual over-the-air transmission, ensuring reliable configurations reach the physical hardware. Furthermore, to provide a standardized quantitative evaluation for this novel domain, we propose RadioBench, the first comprehensive benchmark suite specifically designed to assess autonomous radio signal generation capabilities across various task complexities. Extensive evaluations on real-world hardware testbeds demonstrate that RadioMaster significantly outperforms existing baselines regarding configuration viability and physical signal fidelity. To summarize, we make the following contributions: • We propose RadioMaster, pioneering the first autonomous multi-agent framework to successfully realize end-to-end radio signal generation, seamlessly translating user intentions into executable physical waveforms without tedious manual intervention. • We design a comprehensive system architecture featuring RadioWiki for hallucination-suppressed knowledge retrieval, a collaborative RadioAgent for specialized task execution, and a simulation-based RadioEmulator for closed-loop verification, alongside the construction of RadioBench, the first rigorous domain-specific evaluation benchmark. • We conduct extensive empirical evaluations on real-world hardware testbeds against state-of-the-art baselines, demonstrating the profound superiority of RadioMaster in generating high-fidelity, executable radio signals. We hope this pioneering work can drive and accelerate future artificial intelligence-assisted rapid standardization and system verification within the wireless domain.
2 Related Work 2.1 Radio Signal Generation Traditional efforts to streamline radio signal generation have yielded specialized frameworks like MATLAB toolboxes [37], the GNU Radio ecosystem [13], and the Universal Hardware Driver (UHD) [4]. Recently, AI-assisted methods have been leveraged to enhance isolated stages of this pipeline, such as protocol comprehension [27, 36], baseband synthesis [45, 47] and signal recognition [15, 31,
Task Level 1
Task Level 2
Protocol Understanding QA (250) “In the IEEE 802.11b standard, what is the highest data rate supported, and which core modulation technique is used to achieve it?” a) 54Mpbs OFDM.
b) 11Mbps DSSS.
c) 2Mbps FHSS.
d) 300Mbps MIMO.
Signal Processing QA (250)
“Which transformation is used to convert discrete-time signals from the time domain to the frequency domain?” a) Continuous Fourier Transform.
b) Laplace Transform.
c) Discrete Fourier Transform.
d) Hilbert Transform.
Wi-Fi/BLE Signal Generation (200/200) “Develop an automated script utilizing MATLAB for PlutoSDR to transmit an IEEE 802.11b beacon frame. Operate the SDR at Channel 6. The physical layer must explicitly employ DSSS modulation to achieve a communication rate of 11 Mbps, with the digital baseband sampling rate configured strictly to 11 MS/s. The MAC layer configuration must broadcast the SSID ’RadioMaster‘ with a beacon interval of 100 TUs. Configure the hardware transmitter RF gain to exactly 10 dB.”
Communication Theory QA (250) “Which 5G NR usage scenario addresses pure machinecentric use cases characterized by a large number of connected devices?” a) eMBB. b) mMTC. c) URLLC. d) C-MTC.
Hardware Constraint QA (250) “What is the typical power consumption of the ADS7040 at 1 MSPS with a 1.8-V AVDD?” a) 171uW.
b) 555uW.
Metric
c) 56uW.
Question Answering Accuracy (QAA)
d) 1uW.
Task Level 3 Ambiguous Input (100) “……Furthermore, establish an arbitrary but technically sound sample rate supported by the hardware……”
Physical Constraints Input (100) “……The use case is optimized for close-proximity detection within a 5-meter radius……”
Configuration Pass Rate (CPR)
Hardware Deployability Rate (HDR)
Signal Integrity Rate (SIR)
Figure 2: Overview of the RadioBench.
46, 58]. However, these AI-assisted systems predominantly focus on fragmented sub-tasks, largely neglecting the holistic generation lifecycle. Consequently, a profound scalability gap persists between these isolated works, preventing unified and autonomous real-world deployment.
2.2
Multi-Agent System
Recent research highlights a paradigm shift from monolithic LLMs to multi-agent systems (MAS) for tackling complex challenges. MAS has demonstrated remarkable efficacy in modeling autonomous interactions across domains like LLM collaboration [21, 32, 51, 53], embodied AI [22, 44, 50], scientific problem-solving [18, 19, 38, 55], multimedia cognition [17, 26, 28, 39, 43] and multimedia generation [30, 40, 49, 56]. Despite these widespread successes in computational environments, extending MAS capabilities into the physical wireless domain remains significantly underexplored [52]. Specifically, autonomously translating abstract user intents into executable physical radio emissions via multi-agent orchestration persists as a critical, unresolved challenge.
3
Motivation
In this section, we measure the capabilities of several mainstream foundation models in the radio signal generation task, which motivated us to develop RadioMaster.
3.1
Benchmarking Autonomous Radio Signal Generation
To comprehend the capabilities of existing models throughout the full lifecycle of radio signal generation, we conduct a preliminary evaluation study. Motivated by the absence of an effective evaluation standard for this specific domain, we design RadioBench, the first domain-specific benchmark suite tailored for quantifying the reasoning and configuration capabilities of LLMs and mulitagent systems in radio signal generation tasks. We systematically structure the evaluation into three progressive task levels. • Task Level 1 focuses on foundational domain knowledge. It contains 1000 expert-curated question-answering (QA) pairs evenly
Conference’17, July 2017, Washington, DC, USA
Gemini3.1 Pro
Qwen3-Max
DeepSeek-V3.2
Claude-Opus-4.6
GPT-5.3-Codex
MiniMax-M2.5
Figure 3: Preliminary evaluation of state-of-the-art foundation models on RadioBench. distributed across four core dimensions, specifically protocol understanding (PU), signal processing (SP), communication theory (CT), and hardware constraints (HC). • Task Level 2 evaluates practical implementation. Models must translate explicit requirements into executable radio signals within real-world hardware boundaries. We consider the two most prevalent wireless standards, the Wi-Fi standards and Bluetooth Low Energy (BLE), alongside two dominant configuration paradigms, which are MATLAB-based and UHD-based approaches. We design 400 cases, which are further split by configuration paradigms, resulting in 100 cases each for MATLABbased Wi-Fi, denoted as W-M; UHD-based Wi-Fi, denoted as W-U; MATLAB-based BLE, denoted as B-M; and UHD-based BLE, denoted as B-U. • Task Level 3 assesses deployment adaptability under complex conditions. It includes 100 cases evaluating ambiguous user requirements denoted as AR-M and AR-U for MATLAB and UHD, respectively, alongside 100 cases resolving intricate demands under stringent physical constraints denoted as PC-M and PC-U. Crucially, both Task Level 2 and Task Level 3 require rigorous empirical validation on physical SDR platforms. This hardware-inthe-loop validation utilizes packet sniffers to capture and verify over-the-air signal fidelity. Metrics. We evaluate performance across four progressive metrics. First, Question Answering Accuracy (QAA) measures the factual correctness of theoretical responses. Second, Configuration Pass Rate (CPR) evaluates software viability by confirming script execution without errors. Third, Hardware Deployability Rate (HDR) assesses whether configurations successfully deploy onto actual hardware platforms without driver violations. Finally, Signal Integrity Rate (SIR) confirms if the emitted physical waveform is successfully captured and decoded by the target receiver.
3.2
Limitations
To empirically evaluate the capabilities of SOTA foundation models in autonomous radio signal generation, we conduct a preliminary study assessing six mainstream models using the RadioBench framework. We record QAA for Task Level 1 and CPR for Task Level 2 and Task Level 3 to measure execution viability. As shown in Fig. 3, while models exhibit proficiency in basic knowledge retrieval, their
“Please use PlutoSDR to implement BLE Beacon transmission, operating on Channel 37. Set the transmit gain to 10dBm.” (Abstract Goal) ——Task Level 2
User Input
😟
SOTAs
…… % Configure the BLE Physical Layer for 2 Mbps GFSK cfgLLPDU = bleAdvertisingPDUConfig; cfgLLPDU.PDUType = 'Advertising Indication'; cfgLLPDU.Payload = struct('LocalName', 'RadioMaster’); ……
(a)
>> Error： Unrecognized function or variable 'bleAdvertisingPDUConfig'. …… % Payload Generation (b) function payload = generateAdvertisingPayload(deviceName) deviceNameBytes = unicode2native(deviceName, ‘UTF-8’); nameLength = length(deviceNameBytes);
😣
SOTAs
😫
SOTAs
……
completeNameAD = [ … de2bi(nameLength + 1, 8, ‘left-msb’), … % length 0 0 0 0 1 0 0 0, … % AD Type: complete name (0x08) ];
>> Error： Misunderstanding of BLE protocol. Error in field ‘complete name.’ …… % Setup Pluto transmitter tx = sdrtx('Pluto', ... 'RadioID', 'usb:0', ... 'CenterFrequency', bleParams.ChannelFrequency, ... 'BasebandSampleRate', 2e6, ... 'Gain’, 10, ... 'ChannelMapping', 1);
(c)
>> Error： Hardware constraints: Unknown. Mismatched 'Gain' configuration.
Figure 4: Analysis of fundamental limitations in current models. (a) Hallucinated APIs/functions. (b) Semantic misinterpretations. (c) Insensitivity of physical hardware constraints.
practical performance degrades precipitously. They demonstrate poor understanding of hardware constraints in Task Level 1 and fail advanced implementation requirements in subsequent levels. Analyzing the generated scripts revealed three fundamental limitations hindering their application in complete radio configuration. • The hallucination of internal APIs and functions. Generative models frequently hallucinate library commands, undefined variables, and function conflicts. Without access to exact documentation, they synthesize syntactically plausible but nonexistent commands. For instance, as illustrated in Fig. 4(a), a generated BLE script erroneously invoked a non-existent function named ‘bleAdvertisingPDUConfig’, whereas the correct API
Conference’17, July 2017, Washington, DC, USA
Jiazhen Lei, Tianze Cao, Yuxin Sha, Sihan Wang, Bingbing Wang, Fengyuan Zhu, Zeming Yang, Xiaohua Tian
RadioWiki(§4.2 ) User Input “Please use PlutoSDR to implement BLE Beacon transmission, operating on Channel 37” (Abstract Goal)
RadioEmulator(§4.4 )
OOK to Send
Adaptive Routing RAG AWGN Rayleigh etc.
Task Protocol Specs
Signal Processing
Hardware Constraints
Communication Theory
Virtual Transmitter
PASS Feedback
Virtual I/Q Receiver
Background Knowledge
FAIL
Closed-Loop Verification OOK to Send
RadioAgent(§4.3 )
Tool Pool MATLAB Engine
UHD Help
GNU Radio
Spectrum Analyzer
Website
Code Debugger
Custom Protocol
Summary
Link Budget
Logger
MATLAB Help
Wireless Help
Feedback
Task Decomposition - Sub Task 1 - Reasoning ……
UHD-Based
- Sub Task N … - Reasoning ……
MATLAB-Based
I/Q Data
Tool Chain
Custom-Based
.bin files Tool Call
Protocol Split
Tool Call
Package Building
…
I/Q Samples Q I
…
Tool Call
Example Code
Tool Call
Summary
Modulation Header Core Context
PAYLOAD
CRC
Upper-layer Data
Figure 5: RadioMaster System Block Diagram. call within the MATLAB Toolbox is ‘bleLLAdvertisingChannelPDUConfig’. Such generative hallucinations immediately break the execution pipeline. • The semantic misinterpretation of intricate protocols. Due to a scarcity of specialized wireless communication training data, current models lack expert domain knowledge. This deficiency makes them susceptible to configuration errors within complex protocol fields. They struggle to maintain the tight parameter coupling and rigid formatting mandated by wireless standards, resulting in flawed protocol payloads that fail validation as shown in Fig. 4 (b). • The insensitivity of physical hardware constraints. These models exhibit a profound lack of physical cognition regarding hardware boundaries. When translating theoretical requirements into practical deployments, they often ignore the operational constraints of the target platform. For instance, assigning a raw integer to represent a transmitter gain of 10𝑑𝐵𝑚 in MATLAB scripts fails because the numeric hardware parameter does not map linearly to absolute physical power. This leads to severe deployment errors as depicted in Fig. 4 (c). Motivation: The critical bottlenecks in full-lifecycle radio signal generation stem from hallucinated functions, protocol misunderstandings, and unawareness of physical constraints. Generative models lack the essential contextual awareness for these tasks. Consequently, efficiently integrating external domain knowledge with internal reasoning to achieve robust and autonomous configuration remains a critical open challenge.
4 Design 4.1 Overview of RadioMaster In this section, we introduce RadioMaster, a fully autonomous multi-agent system engineered to automate the complete radio signal generation lifecycle. As illustrated in Fig. 5, the core architecture of RadioMaster comprises three synergistic modules. Explicit user requirements initially arrive at RadioAgent, elaborated in Section 4.3, which serves as the central reasoning engine to orchestrate protocol comprehension, baseband processing, and hardware configuration. During this execution phase, RadioAgent dynamically invokes RadioWiki, presented in Section 4.2, to supply highly relevant external expert insights utilizing an adaptive routing Retrieval Augmented Generation mechanism over a domain-specific knowledge base. The generated configuration is then passed to RadioEmulator, detailed in Section 4.4, which establishes a closedloop validation mechanism through virtual simulation. If anomalies are detected during verification, diagnostic feedback is routed back to RadioAgent for iterative refinement. Upon successful validation, the Operator module synthesizes a comprehensive pipeline configuration file, enabling users to execute the complete generation lifecycle via a single command.
4.2
Background Knowledge Construction
Current LLMs rely on generalized training corpora, lacking deep domain expertise and missing recent wireless protocol advancements. To bridge this gap, we introduce RadioWiki, an extensible
Conference’17, July 2017, Washington, DC, USA
knowledge base tailored for the radio frequency sector depicted in Fig. 6. We pair RadioWiki with a specialized RAG system that autonomously retrieves essential information, integrates real-time online data, and compiles findings into a structured format. This provides fact-grounded context to augment the reasoning and execution of downstream agentic modules. 4.2.1 Data Collection. To comprehensively cover the radio signal generation lifecycle, we curate authoritative multimodal materials. These include ten prevailing wireless protocols such as Wi-Fi, BLE, 3GPP 4G LTE, 3GPP 5G NR, LoRaWAN, Zigbee, SparkLink, RFID, UWB, and Ambient IoT. Hardware resources encompass mainstream SDR platforms like the USRP series and ADALM-Pluto SDR, alongside radio frequency front-end datasheets. We also include communication theory literature and software toolbox guidelines. We systematically partition this knowledge base to ensure only affected sections require reconstruction when integrating new materials. To process diverse texts and diagrams, multimodal embedding models transform this knowledge into high-dimensional vectors for database storage. Ultimately, this adaptable foundational database provides the granular context necessary for robust end-to-end radio signal generation, remaining easily upgradable alongside future standard evolutions and hardware iterations. 4.2.2 Adaptive Routing RAG. To support scalable and accurate knowledge grounding in our radio-domain multi-agent system, we instantiate an adaptive routing RAG. Given a query 𝑞, this RAG selects a small set of relevant repositories, retrieves highvalue evidence, and supplies structured context for downstream generation: 𝑦ˆ ∼ 𝑝𝜃 (𝑦 | 𝑞, R (𝑞)) , (1) where R (𝑞) is the retrieved context set. Let C = {1, . . . , 𝐾 } denote domain repositories (e.g., wireless protocols, baseband toolchains, SDR hardware manuals). Adaptive routing follows a two-stage design to balance efficiency and robustness. Stage 1: Lexical fast routing. We compute a domain-weighted lexical confidence: ∑︁ 𝑠 lex (𝑘 | 𝑞) = 𝑤𝑡 1[𝑡 ∈ K𝑘 ], (2) 𝑡 ∈𝜙 (𝑞)
where 𝜙 (𝑞) is normalized tokenization, K𝑘 is the keyword inventory of repository 𝑘, and 𝑤𝑡 upweights high-precision technical entities (e.g., chipset IDs, standard clauses, toolbox names). If max 𝑠 lex (𝑘 | 𝑞) ≥ 𝜏lex,
(3)
𝑘
routing is finalized directly, avoiding embedding-based routing. Stage 2: Semantic centroid routing. If lexical confidence 𝜏lex is insufficient, we compute query embedding e𝑞 and repository centroids c𝑘 , and score: 𝑠 sem (𝑘 | 𝑞) = cos(e𝑞 , c𝑘 ).
(4)
A single-repository decision is accepted only under confidence and margin constraints: 𝑠 (1) ≥ 𝜏abs,
𝑠 (1) − 𝑠 (2) ≥ 𝜏margin,
(5)
where 𝑠 (1) and 𝑠 (2) are the top-2 semantic scores. Otherwise, we use top-𝑀 multi-repository fallback. Here, the two hyperparameters 𝜏abs and 𝜏margin control reliability and uniqueness, respectively.
Knowledge Base Protocol Covered
Hardware Covered
Theory Covered
SDR Hardware
Books & Guide
Components Toolbox
User Manual
Adaptive Routing Layer Lexical Fast Routing User Input
……
𝑪𝒐𝒏𝒇. ≥ 𝝉𝒍𝒆𝒙
Protocol (𝑤𝑡−1 ) Configuration Key ( 𝑤𝑡 )
Query Enhancement
Category Selection
Direct Category Output Yes
Hardware (𝑤1 ) ToolBox (𝑤2 )
Semantic Centroid Routing
A.1, B.3, ……
Embedding Center Vector Model Matching
D.4, M.2.5, ……
No
𝒆𝒒
𝒄𝒌
Retrieval and Structured Output
Hybrid Retrieval and Fusion Dense Retriever
& Sparse Retriever
Fusion
Online Search
LLM
Structured Output
*Constraint* - [PHY Layer] … -[MAC Layer] … *Example Code* - [Code Block1] … - [Code Block2] … *Parameter* - [Carrier] … - [Sample Rate] … *Evidence* - ……
Figure 6: Architecture of RadioWiki.
Hybrid retrieval and fusion. Given the routed repository set, we perform dual-channel evidence acquisition by querying each repository with both a dense retriever over the vector index V𝑘 and a sparse retriever over the BM25 index B𝑘 , thereby coupling semantic coverage with exact lexical matching. Let 𝜋 𝑣 (𝑑 | 𝑞) and 𝜋𝑏 (𝑑 | 𝑞) denote normalized relevance scores induced by the dense (DC) and sparse channels (SC) for candidate chunk 𝑑, respectively. We aggregate them using 𝑆 (𝑑 | 𝑞) = 𝛼 𝜋 𝑣 (𝑑 | 𝑞) + (1 − 𝛼) 𝜋𝑏 (𝑑 | 𝑞) + 𝛽 1[𝑑 ∈ DC ∩ SC], (6) where 𝛼 ∈ [0, 1] controls the dense-sparse trade-off and the agreement bonus 𝛽 explicitly favors candidates jointly supported by both retrieval views, which empirically stabilizes ranking under terminology drift and query paraphrase. Intuitively, the first term emphasizes semantic proximity, the second term preserves high-precision entity-level matching (e.g., standard clauses, chipset identifiers, toolbox names), and the third term acts as a consistency prior that promotes cross-view corroboration. After score fusion, we apply near-duplicate suppression to collapse semantically redundant or overlapping chunks and then perform global ranking over the deduplicated pool to produce a compact, high-fidelity context set for downstream generation. 4.2.3 Structured Generation. After the retrieval phase, an LLM synthesizes the raw heterogeneous data into a standardized format to guide the downstream reasoning of RadioAgent. The model aggregates candidate contexts from the local hybrid retriever alongside dynamic online search results. Guided by engineered prompts, it filters irrelevant noise and reorganizes the multifaceted information into a highly cohesive structured output. As illustrated
Conference’17, July 2017, Washington, DC, USA
Jiazhen Lei, Tianze Cao, Yuxin Sha, Sihan Wang, Bingbing Wang, Fengyuan Zhu, Zeming Yang, Xiaohua Tian
in Fig. 6, this formatting concisely distills essential protocol constraints, hardware-specific parameters, authoritative evidence, and verified code snippets into a unified paradigm. RadioWiki effectively transforms fragmented domain knowledge into deterministic context, mitigating hallucination risks and equipping RadioAgent with precise specifications for robust SDR configuration.
4.3
Tx
Rx
USRP B210
MT7612U
Multi-Agent Collaboration
To orchestrate the complex execution of radio signal generation, we propose RadioAgent, an autonomous multi-agent collaborative framework. It comprises four synergistic roles, specifically the Planner, the Worker, the Modulator, and the Operator. These agents operate within a tightly coordinated pipeline, systematically dividing responsibilities to form a robust iterative generation and refinement loop. Furthermore, we deploy a comprehensive tool pool to facilitate dynamic tool invocation by each agent, significantly enhancing their operational capabilities across diverse tasks. 4.3.1 Planner. As illustrated in Fig. 5, the Planner serves as the cognitive reasoning center. Upon receiving user input and the structured context generated by RadioWiki, it decomposes the overarching task into a chronological sequence of atomic subtasks with explicit logical evidence. It then synthesizes a deterministic tool chain dictating the execution workflow for downstream modules. Crucially, the Planner adapts dynamically by seamlessly integrating diagnostic feedback from the closed-loop verification of RadioEmulator into its contextual memory. This enables intelligent replanning and self-correction in subsequent iterations. 4.3.2 Worker. The Worker systematically executes the tool chain formulated by the Planner. It dynamically invokes specialized utilities from the external tool pool to accomplish operations like protocol splitting, package building, and code debugging. To ensure deterministic execution, every tool invocation adheres to a rigidly structured formatting template encapsulated within ‘<Call></Call>’ tags. Three dedicated semantic delimiters define operational parameters where ‘<Tool></Tool>’ specifies the utility name, ‘<Query></Query>’ contains the explicit input payload, and ‘<Purpose></Purpose>’ articulates the underlying objective. The sequence terminates with an ‘<EndCall></EndCall>’ tag, and the synthesized results form the final summative response to guide the next stage. 4.3.3 Modulator. Leveraging the structured summary from the Worker, the Modulator executes the core digital baseband processing pipeline. It orchestrates the precise assembly of upper-layer protocol data packets and generates the corresponding bitstream. Guided by user requirements and established physical layer constraints, the Modulator performs appropriate digital modulation to map the bitstream into the complex domain. The resulting In-phase and Quadrature (I/Q) samples are finalized and saved into a .bin file, providing the exact baseband waveform required for physical transmission. 4.3.4 Operator. The Operator connects digital baseband synthesis with physical hardware deployment by generating executable configuration scripts tailored to the target platform, encompassing implementations utilizing UHD APIs, MATLAB environments, or
PlutoSDR
Figure 7: The real-world hardware testbed is deployed for physical signal verification, featuring USRP B210 and PlutoSDR platforms. Table 1: Quantitative Comparisons on RadioBench in Task Level 1. The best results are marked in bold. Method
PU
SP
CT
HC
Overall QAA
0.69 0.70 0.55 0.75 0.69 0.39
0.75 0.89 0.82 0.91 0.89 0.48
Foundation Models Gemini3.1 Pro [5] Qwen3-Max [11] DeepSeek-V3.2 [33] Claude-Opus-4.6 [3] GPT-5.3-Codex [7] MiniMax-M2.5 [10]
0.77 0.91 0.87 0.96 0.94 0.50
0.77 0.97 0.94 0.97 0.98 0.48
0.78 0.96 0.90 0.97 0.96 0.54
Open-Source Multi-Agent Systems AutoIoT [41] IoTPilot [20]
0.91 0.94
0.91 0.98
0.94 0.97
0.55 0.72
0.83 0.90
RadioMaster (Ours)
0.94
0.97
0.94
0.91
0.94
custom protocols. It ingests the .bin file from the Modulator and treats the contained samples as the explicit sequence destined for aerial transmission. To guarantee reliability, a strict gating mechanism ensures physical deployment is initiated exclusively after RadioEmulator completes its validation loop and returns a pass signal. Upon receiving authorization, the Operator synthesizes the final run_pipeline executable, empowering users to execute the entirely verified generation lifecycle via a single command.
4.4
The Closed-Loop Verification Chain
Unlike conventional software agents that rely on virtual compilation to debug syntax, physical radio deployment introduces profound complexities. Within the RadioMaster framework, the absence of software exceptions during baseband synthesis does not guarantee transmission fidelity. Emitted waveforms may still exhibit critical structural defects, such as inaccurate transmission rates or erroneously constructed protocol fields. To address this disparity, we design RadioEmulator as a rigorous closed-loop verification safeguard prior to hardware emission. This module constructs an end-to-end virtual communication system within MATLAB, featuring a virtual transmitter, realistic
Conference’17, July 2017, Washington, DC, USA
Table 2: End-to-end performance comparison across practical implementation challenges in Task Level 2 and Task Level 3. The best results are marked in bold.
Method
Level 2
CPR Level 3
Overall
Level 2
HDR Level 3
Overall
Level 2
SIR Level 3
Overall
0.32 0.06 0.13 0.20 0.04 0.01
0.01 0.01 -
0.01 -
0.01 0.01 -
Foundation Models Gemini3.1 Pro [5] Qwen3-Max [11] DeepSeek-V3.2 [33] Claude-Opus-4.6 [3] GPT-5.3-Codex [7] MiniMax-M2.5 [10]
0.35 0.06 0.21 0.20 0.08 0.01
0.36 0.08 0.19 0.23 0.03 -
0.35 0.07 0.20 0.21 0.06 0.01
0.34 0.06 0.16 0.19 0.06 0.01
0.29 0.07 0.10 0.22 0.02 -
Open-Source Multi-Agent Systems AutoIoT [41] IoTPilot [20]
0.04 0.03
0.04 0.03
0.04 0.03
0.01 0.01
0.03 0.03
0.02 0.02
-
-
-
RadioMaster (Ours)
0.88
0.79
0.83
0.85
0.77
0.81
0.79
0.64
0.71
channel models, and a virtual receiver to evaluate the synthesized I/Q samples. If anomalies emerge, RadioEmulator routes diagnostic feedback to the Planner to trigger iterative refinement. Upon successful validation, it authorizes the Operator to synthesize the final run_pipeline executable. This gating mechanism ensures that verified and high-fidelity configurations reach the physical radio platform.
5 Evaluation 5.1 Experimental Settings 5.1.1 Baselines. Building upon the comprehensive benchmark design detailed in Section 3.1, we evaluate the performance of our proposed RadioMaster framework against a diverse array of stateof-the-art methodologies utilizing the RadioBench framework. The baselines are categorized as follows: • Foundation Models: We select several SOTA LLMs for direct comparison, including Gemini3.1 Pro [5], Qwen3-Max [11], DeepSeek-V3.2 [33], Claude-Opus-4.6 [3], GPT-5.3-Codex [7], and MiniMax-M2.5 [10]. These models represent the vanguard of artificial intelligence, exhibiting exceptional generalized capabilities in complex task planning, sophisticated semantic comprehension of technical protocols, and advanced, context-aware code generation. • Open-Source Multi-Agent Systems: Recent years have witnessed a rapid proliferation of multi-agent systems tailored for the broader Internet of Things (IoT) ecosystem, within which reliable radio signal generation serves as the foundational pillar for network connectivity. We evaluate representative frameworks in this space, including AutoIoT [41] and IoTPilot [20]. These systems demonstrate robust proficiency in parsing ambiguous user requirements, autonomously orchestrating task workflows, and synthesizing configuration files for generic IoT deployments. 5.1.2 Evaluation Metrics. To ensure a rigorous and standardized assessment, all experimental evaluations are strictly conducted
within the RadioBench framework. We employ a tiered suite of evaluation metrics tailored to the specific complexity of each task level, as previously detailed and outlined in Section 3.1: i) For Task Level 1, which predominantly consists of foundational knowledge queries, we utilize Question Answering Accuracy (QAA) to quantitatively measure the precision of information retrieval and domain reasoning across all evaluated methods. ii) For the practical implementation challenges in Task Level 2 and Task Level 3, we introduce a progressive, three-tiered metric system to assess the depth and physical viability of task completion: Configuration Pass Rate (CPR), Hardware Deployability Rate (HDR) and Signal Integrity Rate (SIR). 5.1.3 Implementation Details. To evaluate successful over-the-air emission of real-world waveforms, we deploy generated configurations on physical hardware testbeds. As shown in Fig. 7, the PlutoSDR [1] and USRP B210 [12] serve as physical transmitters representing MATLAB-based and UHD-based methodologies, respectively. To validate signal integrity and protocol compliance, we employ commercial receivers. A MediaTek MT7612U [9] network interface paired with Omnipeek software decodes Wi-Fi packets, while the LightBlue application [8] parses BLE advertising packets. This end-to-end physical verification ensures complete interoperability with standard commercial devices. Further hardware setup details reside in Section A of the Appendix. 5.1.4 Experimental Details. RadioBench test cases are evenly distributed between the two configuration paradigms. We curate this dataset via LLM-assisted generation, followed by meticulous manual sanitization and expert calibration to guarantee strict protocol compliance. To establish a fair comparative baseline, we standardize the underlying reasoning engine. Both AutoIoT [41] and IoTPilot [20] multi-agent baseline systems utilize Qwen3-Max [11], maintaining architectural consistency with our proposed RadioMaster system. Finally, to manage autonomous code generation complexities, RadioAgent modules limit code debugger invocations to 10 local iterations. If closed-loop validation fails, RadioEmulator
Conference’17, July 2017, Washington, DC, USA
Jiazhen Lei, Tianze Cao, Yuxin Sha, Sihan Wang, Bingbing Wang, Fengyuan Zhu, Zeming Yang, Xiaohua Tian
Table 3: Ablation study on different structural variants. “w/o” means RadioMaster framework without a certain part. The best results are marked in bold. Method
QAA Level 1
Level 2
CPR Level 3
Overall
Level 2
HDR Level 3
Overall
Level 2
SIR Level 3
Overall
RadioMaster w/o RadioWiki RadioMaster w/o RadioAgent RadioMaster w/o RadioEmulator
0.89 0.94 0.93
0.76 0.70 0.80
0.64 0.63 0.81
0.70 0.66 0.80
0.67 0.59 0.78
0.53 0.46 0.78
0.60 0.52 0.78
0.45 0.44 0.36
0.31 0.29 0.23
0.38 0.36 0.29
RadioMaster
0.94
0.88
0.79
0.83
0.85
0.77
0.81
0.79
0.64
0.71
routes diagnostic feedback back to RadioAgent for comprehensive regeneration, constrained to a maximum of 5 global attempts to optimally balance signal fidelity and computational overhead.
5.2
Main Results
As detailed in Table 1 and Table 2, we evaluate the performance of RadioMaster against selected baselines by recording QAA for Task Level 1, alongside CPR, HDR, and SIR for Task Level 2 and Task Level 3. Detailed and additional results reside in Section B and Section C of the Appendix. We deduce three key findings from these quantitative results. i) First, while RadioMaster achieves overall QAA comparable to foundation models, it demonstrates profound superiority in comprehending strict hardware constraints. Baselines exhibit a critical deficiency in physical hardware cognition, creating a primary bottleneck against successful real-world deployment. The RadioWiki module empowers our system to accurately map complex interdependencies of hardware limitations to ensure physical viability. ii) Second, RadioMaster achieves substantial performance enhancements over all baselines across practical implementation challenges. Although baseline models frequently synthesize configuration files, empirical testing reveals these scripts remain overwhelmingly unexecutable. For example, the CPR of the strongest baseline reaches only 35%, yielding a near-zero SIR. In stark contrast, RadioMaster achieves an 83% overall CPR and a 71% overall SIR. This success stems from the highly relevant structured context retrieved by RadioWiki, synergized with the systematic division of labor and closed-loop reasoning within the RadioAgent framework. iii) Third, all evaluated methodologies experience performance degradation in Task Level 3 relative to Task Level 2, directly reflecting escalated task complexity. Specifically, the SIR of RadioMaster drops from 79% in Task Level 2 to 64% in Task Level 3. This advanced level challenges systems with ambiguous user requirements and exceptionally stringent physical constraints. In scenarios lacking explicitly defined parameters, generative models naturally become highly susceptible to structural hallucinations. Consequently, to maximize end-to-end performance and reliability, it is essential for users to provide explicit and comprehensive parameter specifications during the initial prompting phase.
5.3
Ablation Studies
To evaluate the individual contributions of internal core modules, we conduct comprehensive ablation studies on structural variants of our proposed framework. We independently ablate RadioWiki, RadioAgent, and RadioEmulator from the complete architecture. For
the variant lacking RadioAgent, we replace the collaborative multiagent architecture with Qwen3-Max [11] as the sole reasoning engine. As detailed in Table 3, benchmarking these variants across the RadioBench dataset yields three critical insights. i) Integrating highly relevant contextual retrieval with standardized output significantly mitigates generative hallucinations. Ablating RadioWiki precipitously declines overall system performance, plummeting the overall CPR from 83% to 70% and the SIR from 71% to 38%. Without deterministic grounding provided by domainspecific knowledge bases, the underlying model frequently hallucinates protocol specifications and hardware APIs, resulting in fatal syntax errors during baseband synthesis. ii) The systematic division of labor inherent to RadioAgent fundamentally enhances the entire system’s robustness. When condensing multi-agent orchestration into a single foundation model, the solitary reasoning engine struggles to manage the complexity required for end-to-end signal generation, reducing the overall SIR to 36%. Conversely, our collaborative paradigm effectively isolates subtask intricacies, preventing intermediate error compounding and ensuring highly stable execution across diverse SDR platforms. iii) RadioEmulator acts as an indispensable gating mechanism, guaranteeing final signal fidelity. We observe that the CPR and HDR of the variant without RadioEmulator are marginally higher than those of the full system. Since these two metrics primarily evaluate software executability and hardware deployment independent of the emulator, this minor fluctuation merely reflects normal probabilistic variance during the generative experiments. Without closed-loop validation, the framework blindly synthesizes executable scripts, causing the SIR to plummet to 29%. This highlights the vital necessity of iterative simulation-based feedback to verify true signal integrity prior to physical deployment.
6
Conclusion
In this paper, we present RadioMaster, a pioneering multi-agent framework that conquers the “last mile” of wireless prototyping by autonomously translating user intents into radio signals in the air. By synergizing adaptive knowledge retrieval, collaborative agentic execution, and rigorous closed-loop simulation, RadioMaster systematically overcomes the domain ignorance and physical insensitivity that plague current generative models. Evaluated on our newly proposed RadioBench, RadioMaster demonstrates profound superiority over SOTA baselines in real-world hardware deployments. Ultimately, RadioMaster successfully bridges the divide between digital reasoning and cyber-physical systems, serving as a catalyst to accelerate AI-driven standardization and rapid prototyping in next-generation wireless communications.
Conference’17, July 2017, Washington, DC, USA
References [1] ADALM-PLUTO. https://www.analog.com/en/resources/evaluation-hardwareand-software/evaluation-boards-kits/adalm-pluto.html. [2] Claude Code. https://code.claude.com/docs/en/overview. [3] Claude-Opus-4.6. https://platform.claude.com/docs/en/about-claude/models/ whats-new-claude-4-6. [4] Ettus USRP Hardware Driver. https://www.ettus.com/sdr-software/uhd-usrphardware-driver/. [5] Gemini3.1 Pro. https://ai.google.dev/gemini-api/docs/models/gemini-3.1-propreview?hl=zh-cn. [6] GLM-5-Turbo. https://docs.z.ai/guides/llm/glm-5-turbo. [7] GPT-5.3-Codex. https://openai.com/zh-Hans-CN/index/introducing-gpt-5-3codex/. [8] LightBlue. https://punchthrough.com/lightblue/. [9] MediaTek MT7612U. https://www.mediatek.com/products/broadband-wifi/ mt7612u. [10] MiniMax-2.5. https://www.minimax.io/news/minimax-m25. [11] Qwen3-Max. https://qwen.ai/blog?id=241398b9cd6353de490b0f82806c7848c5d2777d& from=research.latest-advancements-list. [12] USRP B210. https://www.ettus.com/all-products/ub210-kit/. [13] Eric Blossom. Gnu radio: tools for exploring the radio frequency spectrum. Linux journal, 2004(122):4, 2004. [14] Luis A Camunas-Mesa and Jos’e M de la Rosa. Combining software-defined radio learning modules and neural networks for teaching communication systems courses. Information, 14(11):599, 2023. [15] Shuai Chen, Yong Zu, Zhixi Feng, Shuyuan Yang, and Mengchang Li. Radiollm: Introducing large language model into cognitive radio via hybrid prompt and token reprogrammings. arXiv preprint arXiv:2501.17888, 2025. [16] Xiang Cheng, Weibo Wen, Haotian Zhang, Boxun Liu, Zonghui Yang, Jianan Zhang, and Xuesong Cai. Embodied intelligent wireless (eiw): Synesthesia of machines empowered wireless communications. arXiv preprint arXiv:2511.22845, 2025. [17] Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, and Qing Li. Videoagent: A memory-augmented multimodal agent for video understanding. In European Conference on Computer Vision, pages 75–92. Springer, 2024. [18] Zhihao Fan, Lai Wei, Jialong Tang, Wei Chen, Wang Siyuan, Zhongyu Wei, and Fei Huang. Ai hospital: Benchmarking large language models in a multi-agent medical interaction simulator. In Proceedings of the 31st International Conference on Computational Linguistics, pages 10183–10213, 2025. [19] Alireza Ghafarollahi and Markus J Buehler. Sciagents: automating scientific discovery through bioinspired multi-agent intelligent graph reasoning. Advanced Materials, 37(22):2413523, 2025. [20] Kaijie Gong, Wei Dong, Hao Wang, Yingqi Peng, and Yi Gao. Programming embedded iot applications in natural language with iotpilot. In Proceedings of the 23rd Annual International Conference on Mobile Systems, Applications and Services, pages 70–82, 2025. [21] T Guo, X Chen, Y Wang, R Chang, S Pei, NV Chawla, O Wiest, and X Zhang. Large language model based multi-agents: A survey of progress and challenges. In 33rd International Joint Conference on Artificial Intelligence (IJCAI 2024). IJCAI; Cornell arxiv, 2024. [22] Xudong Guo, Kaixuan Huang, Jiale Liu, Wenhui Fan, Natalia Vélez, Qingyun Wu, Huazheng Wang, Thomas L Griffiths, and Mengdi Wang. Embodied llm agents learn to cooperate in organized teams. IEEE Transactions on Computational Social Systems, 2026. [23] Junda He, Christoph Treude, and David Lo. Llm-based multi-agent systems for software engineering: Literature review, vision, and the road ahead. ACM Transactions on Software Engineering and Methodology, 34(5):1–30, 2025. [24] Mehrdad Hessar, Ali Najafi, Vikram Iyer, and Shyamnath Gollakota. Tinysdr: Low-power sdr platform for over-the-air programmable iot testbeds. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20), pages 1031–1046, Santa Clara, CA, February 2020. USENIX Association. [25] Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al. Metagpt: Meta programming for a multi-agent collaborative framework. In The twelfth international conference on learning representations, 2023. [26] Guanjie Huang, Danny HK Tsang, Shan Yang, Guangzhi Lei, and Li Liu. Cuedagent: A collaborative multi-agent system for automatic cued speech recognition. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 8313–8321, 2025. [27] Long Huang, Ming Zhao, Limin Xiao, Xiujun Zhang, and Jungang Hu. Chat3gpp: An open-source retrieval-augmented generation framework for 3gpp documents. In 2025 IEEE International Conference on Communications Workshops (ICC Workshops), pages 492–497. IEEE, 2025. [28] Rongjie Huang, Mingze Li, Dongchao Yang, Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong, Jiawei Huang, Jinglin Liu, et al. Audiogpt: Understanding and generating speech, music, sound, and talking head. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 23802–23804, 2024.
[29] Md Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. Mapcoder: Multi-agent code generation for competitive problem solving. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4912–4944, 2024. [30] Mingcheng Li, Xiaolu Hou, Ziyang Liu, Dingkang Yang, Ziyun Qian, Jiawei Chen, Jinjie Wei, Yue Jiang, Qingyao Xu, and Lihua Zhang. Mccd: Multi-agent collaboration-based compositional diffusion for complex text-to-image generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 13263–13272, 2025. [31] Taotao Li, Zhenyu Wen, Yang Long, Zhen Hong, Shilian Zheng, Li Yu, Bo Chen, Xiaoniu Yang, and Ling Shao. The importance of expert knowledge for automatic modulation open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(11):13730–13748, 2023. [32] Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. A survey on llm-based multi-agent systems: workflow, infrastructure, and challenges. Vicinagearth, 1(1):9, 2024. [33] Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, et al. Deepseek-v3. 2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556, 2025. [34] Junhua Liu, Fanfan Lin, Xinze Li, Kwan Hui Lim, and Shuai Zhao. Physicsinformed llm-agent for automated modulation design in power electronics systems. arXiv preprint arXiv:2411.14214, 2024. [35] Kaiwei Liu, Bufang Yang, Lilin Xu, Yunqi Guo, Guoliang Xing, Xian Shuai, Xiaozhe Ren, Xin Jiang, and Zhenyu Yan. Tasksense: A translation-like approach for tasking heterogeneous sensor systems with llms. In Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems, pages 213–225, 2025. [36] Ali Maatouk, Kenny Chirino Ampudia, Rex Ying, and Leandros Tassiulas. Telellms: A series of specialized large language models for telecommunications. arXiv preprint arXiv:2409.05314, 2024. [37] Cleve Moler and Jack Little. A history of matlab. Proceedings of the ACM on Programming Languages, 4(HOPL):1–67, 2020. [38] Qi Peng, Jialin Cui, Jiayuan Xie, Yi Cai, and Qing Li. Tree-of-reasoning: Towards complex medical diagnosis via multi-agent reasoning with evidence tree. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 1744– 1753, 2025. [39] Jiayuan Rao, Zifeng Li, Haoning Wu, Ya Zhang, Yanfeng Wang, and Weidi Xie. Multi-agent system for comprehensive soccer understanding. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 3654–3663, 2025. [40] Yan Rong, Jinting Wang, Guangzhi Lei, Shan Yang, and Li Liu. Audiogenie: A training-free multi-agent framework for diverse multimodality-to-multiaudio generation. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 8872–8881, 2025. [41] Leming Shen, Qiang Yang, Yuanqing Zheng, and Mo Li. Autoiot: Llm-driven automated natural language programming for aiot applications. In Proceedings of the 31st Annual International Conference on Mobile Computing and Networking, pages 468–482, 2025. [42] George Sklivanitis, Adam Gannon, Stella N. Batalama, and Dimitris A. Pados. Addressing next-generation wireless challenges with commercial software-defined radio platforms. IEEE Communications Magazine, 54(1):59–67, 2016. [43] Mingyang Su, Chao Liu, Jingling Zhang, WU Shuang, and Mingming Fan. Simviews: An interactive multi-agent system simulating visitor-to-visitor conversational patterns to present diverse perspectives of artifacts in virtual museums. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 6740–6750, 2025. [44] Sinan Tan, Weilai Xiang, Huaping Liu, Di Guo, and Fuchun Sun. Multi-agent embodied question answering in interactive environments. In European Conference on Computer Vision, pages 663–678. Springer, 2020. [45] Shailja Thakur, Baleegh Ahmad, Hammond Pearce, Benjamin Tan, Brendan Dolan-Gavitt, Ramesh Karri, and Siddharth Garg. Verigen: A large language model for verilog code generation. ACM Transactions on Design Automation of Electronic Systems, 29(3):1–31, 2024. [46] Jingwen Tong, Zijian Li, Fang Liu, Wei Guo, and Jun Zhang. Wirelessagent++: Automated agentic workflow design and benchmarking for wireless networks, 2026. [47] Jiazhao Wang, Wenchao Jiang, Ruofeng Liu, Bin Hu, Demin Gao, and Shuai Wang. Nn-defined modulator: Reconfigurable and portable software modulator on iot gateways. In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), pages 775–789, 2024. [48] Xinquan Wang, Fenghao Zhu, Zhaohui Yang, Chongwen Huang, Xiaoming Chen, Zhaoyang Zhang, Sami Muhaidat, and Mérouane Debbah. Bridging physical and digital worlds: embodied large ai for future wireless systems. arXiv preprint arXiv:2506.24009, 2025. [49] Zhenyu Wang, Aoxue Li, Zhenguo Li, and Xihui Liu. Genartist: Multimodal llm as an agent for unified image generation and editing. Advances in Neural Information Processing Systems, 37:128374–128395, 2024. [50] Di Wu, Xian Wei, Guang Chen, Hao Shen, Xiangfeng Wang, Wenhao Li, and Bo Jin. Generative multi-agent collaboration in embodied ai: A systematic review.
Conference’17, July 2017, Washington, DC, USA
Jiazhen Lei, Tianze Cao, Yuxin Sha, Sihan Wang, Bingbing Wang, Fengyuan Zhu, Zeming Yang, Xiaohua Tian
arXiv preprint arXiv:2502.11518, 2025. [51] Binfeng Xu, Zhiyuan Peng, Bowen Lei, Subhabrata Mukherjee, Yuchen Liu, and Dongkuan Xu. Rewoo: Decoupling reasoning from observations for efficient augmented language models. arXiv preprint arXiv:2305.18323, 2023. [52] Huatao Xu, Liying Han, Qirui Yang, Mo Li, and Mani Srivastava. Penetrative ai: Making llms comprehend the physical world. In Findings of the Association for Computational Linguistics: ACL 2024, pages 7324–7341, 2024. [53] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations, 2022. [54] Minghui Zhao, Junxi Xia, Kaiyuan Hou, Yanchen Liu, Stephen Xia, and Xiaofan Jiang. Flexifly: interfacing the physical world with foundation models empowered by reconfigurable drone systems. In Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems, pages 463–476, 2025.
[55] Weike Zhao, Chaoyi Wu, Yanjie Fan, Pengcheng Qiu, Xiaoman Zhang, Yuze Sun, Xiao Zhou, Shuju Zhang, Yu Peng, Yanfeng Wang, et al. An agentic system for rare disease diagnosis with traceable reasoning. Nature, pages 1–10, 2026. [56] Tianchen Zhou, Zhongjie Duan, Cen Chen, Wenmeng Zhou, Yanhao Wang, and Yaliang Li. Agentstory: A multi-agent system for story visualization with multi-subject consistent text-to-image generation. In Proceedings of the 2025 International Conference on Multimedia Retrieval, pages 1894–1902, 2025. [57] Fengyuan Zhu, Mingwei Ouyang, Luwei Feng, Yaoyu Liu, Xiaohua Tian, Meng Jin, Dongyao Chen, and Xinbing Wang. Enabling software-defined phy for backscatter networks. In Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services, pages 330–342, 2022. [58] Hang Zou, Yu Tian, Bohao Wang, Lina Bariah, Samson Lasaulce, Chongwen Huang, and Mérouane Debbah. Rf-gpt: Teaching ai to see the wireless world. arXiv preprint arXiv:2602.14833, 2026.
Conference’17, July 2017, Washington, DC, USA
(b)
(a)
TX
RX 8.2m
RX
(a)
RX
(b)
Rx Range
TX
TX
Tx
4m Figure 8: Real-world hardware testbed deployed in an Figure 9: Real-world hardware testbed deployed in a indoor conference room. corridor scenario. packet-level verification. As illustrated in Fig. 10, a trial is deemed successful only when the commercial Omnipeek network analyzer successfully captures and decodes the emitted packets containing the precisely configured parameters, such as the correct SSID.
B
Figure 10: Packet-level verification interface via the Omnipeek network analyzer.
A
Further Hardware Setup Details
To rigorously evaluate the efficacy of our proposed framework in real-world environments, we conduct extensive over-the-air experiments across two distinct physical scenarios. Specifically, the fundamental implementation challenges of Task Level 2 are evaluated within a typical indoor conference room (as depicted in Fig. 8), while the advanced physical constraints of Task Level 3 are validated in a long corridor environment (as shown in Fig. 9). At the transmitter (TX) side, the complete RadioMaster service is deployed on a host PC. To comprehensively cover the dominant SDR configuration paradigms, this host is interfaced with an ADALMPlutoSDR for MATLAB-based configurations and an Ettus USRP B210 for UHD-based deployments. At the receiver (RX) side, we establish a heterogeneous sniffing architecture to capture the emitted waveforms. For the Wi-Fi standards, a separate PC equipped with a MediaTek MT7612U network sniffer is utilized to capture over-the-air traffic. Concurrently, for BLE evaluations, we leverage the native radio frequency front-end of a commercial smartphone, utilizing the LightBlue application to seamlessly monitor and parse BLE broadcast signals. During the corridor-based experiments focusing on Task Level 3, we explicitly test the system’s adaptability to dynamic environmental factors. This is achieved by systematically varying the physical topologies, such as adjusting the spatial distance between the Tx and Rx antennas, to validate the specific edge cases defined in the benchmark. Ultimately, the successful execution of any given case in Task Level 2 or Task Level 3 is explicitly confirmed through
Detailed Results on RadioBench
We provide a granular analysis of the experimental results across Task Level 2 and Task Level 3 within the RadioBench framework, as detailed in Table 4 and Table 5. A detailed examination of the CPR and HDR reveals distinct proficiency biases among the evaluated foundation models across different configuration paradigms. Specifically, Gemini3.1 Pro [5] and DeepSeek-V3.2 [33] demonstrate a pronounced aptitude for UHD-based cases, achieving relatively higher CPR and HDR in these low-level API environments. Conversely, Claude-Opus-4.6 [3] exhibits superior performance in MATLAB-based tasks, seamlessly handling high-level scripting logic. However, despite these paradigm-specific advantages in generating syntactically executable code, all standalone foundation models fundamentally fail to complete the ultimate radio signal generation task during actual physical deployment. As demonstrated by the experimental results, their SIR remains consistently near zero across all evaluated scenarios. This discrepancy critically underscores the last mile bottleneck in wireless prototyping, highlighting that software-level executability does not equate to physical layer fidelity. Without domain-specific grounding and closed-loop verification, generic models cannot satisfy the rigid semantic formatting and strict physical constraints required for successful over-the-air transmission. In stark contrast, RadioMaster maintains a significantly higher and more stable SIR across all configuration paradigms, effectively bridging the gap between theoretical code generation and actual physical hardware deployment.
C
Results on Different Fundamental LLMs
To validate the architectural robustness and generalizability of the proposed framework, we conducted comparative experiments by substituting the fundamental LLM powering the RadioMaster reasoning engine. As summarized in Table 6, we evaluated the system using three distinct state-of-the-art models: Qwen3-Max [11], DeepSeek-V3.2 [33], and GLM-5-Turbo [6]. The empirical results indicate that while the inherent reasoning capabilities of the fundamental LLM introduce minor performance
Conference’17, July 2017, Washington, DC, USA
Jiazhen Lei, Tianze Cao, Yuxin Sha, Sihan Wang, Bingbing Wang, Fengyuan Zhu, Zeming Yang, Xiaohua Tian
Table 4: Detailed results on Task Level 2. Best results are in bold.
Method
W-M
CPR B-M W-U
B-U
HDR B-M W-U
W-M
SIR B-M W-U
B-U
W-M
B-U
0.75 0.01 0.37 0.14 0.08 0.01
0.01 -
-
-
0.01 -
Foundation Models Gemini3.1 Pro [5] Qwen3-Max [11] DeepSeek-V3.2 [33] Claude-Opus-4.6 [3] GPT-5.3-Codex [7] MiniMax-M2.5 [10]
0.09 0.04 0.03 0.23 0.02 0.01
0.01 0.06 0.05 0.38 0.11 0.01
0.54 0.10 0.35 0.05 0.10 -
0.76 0.02 0.41 0.15 0.10 0.01
0.09 0.04 0.02 0.22 0.01 -
0.01 0.06 0.04 0.37 0.08 -
0.49 0.10 0.18 0.04 0.06 -
Open-Source Multi-Agent Systems AutoIoT [41] IoTPilot [20]
0.04 -
0.06 0.07
0.03 0.01
0.03 0.04
0.01 -
0.02 0.02
0.01
0.01 0.02
-
-
-
-
RadioMaster (Ours)
0.90
0.93
0.88
0.83
0.89
0.85
0.88
0.79
0.86
0.83
0.79
0.71
SIR AR-U PC-M
PC-U
Table 5: Detailed results on Task Level 3. Best results are in bold. Method
AR-M
CPR AR-U PC-M
PC-U
HDR AR-U PC-M
AR-M
PC-U
AR-M
0.32 0.08 0.16 0.06 0.04 -
-
-
0.02 -
-
Foundation Models Gemini3.1 Pro [5] Qwen3-Max [11] DeepSeek-V3.2 [33] Claude-Opus-4.6 [3] GPT-5.3-Codex [7] MiniMax-M2.5 [10]
0.12 0.02 0.04 0.30 0.02 -
0.68 0.18 0.24 0.24 0.04 -
0.10 0.04 0.04 0.32 0.06 -
0.56 0.10 0.10 0.06 0.02 -
0.10 0.02 0.04 0.30 -
0.68 0.16 0.16 0.24 0.02 -
0.06 0.04 0.02 0.30 0.02 -
AutoIoT [41] IoTPilot [20]
0.02 -
0.04 0.08
0.06 0.02
0.02 0.02
0.02 -
0.04 0.08
0.06 0.02
0.02 0.02
-
-
-
-
RadioMaster (Ours)
0.90
0.76
0.78
0.74
0.90
0.72
0.76
0.70
0.72
0.60
0.62
0.62
Open-Source Multi-Agent Systems
Table 6: Performance robustness evaluation of the proposed RadioMaster framework across different fundamental LLMs. Method
QAA Level 1
Level 2
CPR Level 3
Overall
Level 2
HDR Level 3
Overall
Level 2
SIR Level 3
Overall
RadioMaster (Qwen3-Max) [11] RadioMaster (DeepSeek-V3.2) [33] RadioMaster (GLM-5-Turbo) [6]
0.94 0.93 0.95
0.88 0.84 0.89
0.79 0.76 0.75
0.83 0.70 0.77
0.85 0.79 0.81
0.77 0.73 0.73
0.81 0.76 0.77
0.79 0.71 0.70
0.64 0.59 0.63
0.71 0.65 0.67
fluctuations in the complex scenarios of Task Level 2 and Task Level 3, the overall system performance remains stable and robust. Regardless of the underlying LLM, RadioMaster consistently achieves an overall SIR exceeding 0.65, substantially outperforming any standalone foundation model or baseline multi-agent system. This resilience demonstrates that the efficacy of RadioMaster stems from its synergistic multi-agent workflow and domain-specific
retrieval-augmented architecture, rather than relying exclusively on the raw reasoning power of a specific LLM. Consequently, the proposed framework is highly generalizable and model-agnostic. As foundation models continue to evolve, RadioMaster can seamlessly integrate more advanced LLMs to incrementally push the boundaries of autonomous radio signal generation performance.