LoopHIL: Looping Hardware into LLM-assisted Agents for Wireless Embedded Systems

LoopHIL: Looping Hardware into LLM-assisted Agents for Wireless Embedded Systems Dhairya Shah, Spanddhana Sara, Savitha Viswanadh Kandala, Ambuj Varshney National University of Singapore, Singapore {dhairya,spanddhanasara,viswanadh}@u.nus.edu,ambujv@nus.edu.sg

arXiv:2610.24231v1 [cs.NI] 16 Oct 2026

ABSTRACT LLM agents are widely used for software development and debugging, but their adoption in embedded systems, especially for wireless tasks, remains limited. This is because embedded development requires hardware-in-the-loop testing. Agents cannot directly observe the spectrum, probe pins, or measure power, so they cannot verify wireless link functionality; currently, human engineers perform this verification. We systematically evaluate whether LLM agents can establish wireless links when provided with hardware access. We introduce LoopHIL, a framework that lets agents control a multi-device RF bench, monitor it with a spectrum analyzer, logic analyzer, and energy meter, and validate links against a verification ladder, all within code-enforced safety limits. Across four increasingly complex tasks, from generating a carrier to backscatter decoding by a commercial receiver, and using six models from two vendors, hardware-in-theloop increases task completion rates from 54% to 79% and from 17% to 83% on the most challenging task. It also halves task time and output tokens. With debugging tools, agents decrease tag power consumption by up to 96%. However, small models often report unverified claims, so independent hardware checks remain necessary.

1

INTRODUCTION

LLM agents now write, build and debug software with minimal human intervention, and are increasingly applied to embedded systems. Recent agents compile firmware, flash a board and read its serial output on their own [5, 9], and expert notes further improve them on real hardware [13]. In these settings, the board under test gives the agent feedback through compiler errors, serial logs or measured current [15]. Wireless embedded systems disrupt this feedback loop. A wireless link works only when the devices agree on transmit frequency and power, firmware timing and modulation, and receiver settings and packet format. Backscatter makes this harder. The tag only switches its antenna, so its signal appears only under an external carrier, and only a separate receiver can verify the packets. A timer that is off by half a percent compiles and runs without error, yet the receiver rejects its packets, and no single device detects the problem. Today, a human closes this loop. An engineer watches a spectrum analyzer to see whether the carrier and the tag’s

LoopHIL

LLM agent

Implement 2FSK backscatter link

calls

Specification Firmware

Typed tools safety limits in code

Verify ladder

Verify

verdict

Optimize

carrier tag receiver

blame

features drive

Physical layer

RF bench

(invisible to agents today)

Instruments probe

Carrier

Tag

Receiver

Spectrum

Logic

Energy

Figure 1: LoopHIL overview. The agent reaches the physical layer only through LoopHIL: typed tools with safety limits in code drive the RF bench, and the instruments return features to a verification ladder that blames the faulty device.

sidebands appear, probes the tag’s pin with a logic analyzer to check its timing, and measures its power with an energy meter. They then tell the developer, or an LLM, what to fix. This is slow, requires RF expertise and the bench at hand, and does not scale to the many radios, tags and receivers that low-power systems combine. LLM agents, by contrast, have no access to physical layer. They see only their code and the text tools return, so they cannot tell whether a link works. We study whether agents can close this loop themselves when given the hardware and the instruments an engineer uses. This raises three problems. First, the agent must reach a multi-device bench through reliable tools, and see measurements it can reason about rather than raw samples. Second, an agent that controls an SDR is a transmitter, so its actions must stay within regulatory limits whatever it decides. Third, agent must not grade itself, because models that cannot finish a task often report success. We develop LoopHIL (Figure 1), a harness for LLM agents that build RF links on real hardware. The agent turns user intent into a specification, writes firmware for the tag and the receiver, and drives the bench only through typed tools. Instruments (an SDR as spectrum analyzer, a logic analyzer and an energy meter) return measured features rather than samples. A verification ladder takes its own measurements and names the failing device and layer. Safety limits in code gate commands, and expert skills supply domain knowledge. We compare agents with and without the LoopHIL harness and hardware in loop on four tasks of increasing difficulty,

Shah, Sara, Kandala and Varshney

from a single carrier to 2-FSK backscatter decoded by a commercial CC1310, with six models from two vendors. Each model runs each task once in its vendor’s coding agent with the hardware attached, and once through LoopHIL. An independent check against the hardware grades every run. With LoopHIL, models complete 79% of runs instead of 54%, and 83% on the hardest task instead of 17%. Where both complete a task, LoopHIL is 2× faster and uses 2× fewer output tokens. When asked to reduce tag power, most agents cut it by 19 to 96% while the tag still sends valid frames. Small models often report unsupported claims, which only an independent check against the hardware reveals. In summary, this paper makes three contributions: • LoopHIL, a harness that gives LLM agents typed access to a multi-device RF bench, feature-level instruments, a verification ladder that blames the faulty device, and safety limits enforced in code. • A benchmark of RF tasks on real hardware, from a carrier to backscatter to a commercial receiver, with independent grading that exposes unsupported claims. • A systematic study of six models from two vendors with and without hardware in the loop, showing where it helps, where it does not, and what debugging tools add.

2

BACKGROUND

We discuss the relevant works in this section. LLM agents for embedded systems. Early work demonstrated that LLMs can write and debug embedded code with a human feedback loop [2, 16]. Recent agents operate autonomously, compiling code, flashing the board, and reading serial output independently [5, 9]. IoT-SkillsBench found that expert-written skills enhance these agents on hardware [13]. Embedded Arena further optimizes deployment by measuring and feeding back the current, energy, and temperature of the MCU [15]. Importantly, the code is verified on the actual board. In contrast, verifying an RF link requires consistency in carrier, baseband, modulation, timing and packet format at the emitter, tag and receiver. LoopHIL addresses this by having a spectrum analyzer, logic analyzer, and energy meter as feedback while ensuring link functionality. LLM agents for RF. RadioMaster converts user intent to generating RF signals using retrieval, multi-agent recovery and emulator-gated verification, and commercial receivers decode its SDR transmissions [6]. However, its agent only synthesizes the SDR IQ samples and verifies them in an emulator, not on the hardware under test. Low-power radios run on an MCU/FPGA and are often decoded by commercial receivers such as the TI CC1310 [17, 18, 20]. Backscatter is harder: the tag shifts an external carrier into subcarrier sidebands that the receiver must decode next to a strong carrier [4, 19], so firmware errors appear only at the receiver. In LoopHIL, the agent writes tag firmware, debugs across

LLM agent Specification

Typed tools calls

Logs

RF bench drive

Drivers / SDK Firmware

HW config

Carrier Tag

Debug / probe Verify

Receiver

safety limits in code

probe

Optimize spec

Checks

Instruments

Validate spec

Skills expert notes

verdict

Verify ladder Blame device

Spectrum data

Logic Energy

Figure 2: LoopHIL architecture. The agent calls typed tools that drive the RF bench under safety limits in code; the instruments feed measurements to the checks, which return a verdict to the agent.

the carrier emitter, tag and receiver, and the verification ladder measures the hardware directly. Safety limits in code gate every command, since an agent controlling an SDR is a transmitter. Unlike testbeds such as FlockLab, built for humans [8], LoopHIL exposes the bench to an agent through typed tools, following SWE-agent [7] and MCP [1]. Trusting agent results. Coding agents sometimes use specialcase tests to pass them and frontier models reward-hack evaluation tasks [10]. Many agentic benchmarks also accept wrong outputs as success [11, 14]. This risk is larger on an RF hardware bench, because the agent produces the measurements it is graded on. LoopHIL goes beyond the agent’s report and grades and verifies each run using an independent re-check. Hence, meeting the thresholds is not enough: a run whose numbers fail the independent checks is reported as an unsupported claim.

3

DESIGN

LoopHIL enables an LLM agent to develop a wireless RF link based on user intent, overcoming interoperability challenges between devices that cannot be detected by any single device. It closes the feedback loop through the hardware and includes an LLM agent, tools, checks, a hardware test bench, measurement instruments, and expert skills.

3.1

LLM agent and skills

An LLM agent running on a computer, processes user intent provided as a structured plain text prompt, operating in three main steps. A single agent controls bench devices and instruments, with the LLM having its built-in tools and web access disabled. This is to evaluate effectiveness of the model. Specification. The agent converts the user intent into a JSON specification, defining parameters such as carrier frequency, power, modulation, bit rate, and frame format. This specification is validated and locked prior to verification (§3.4) to ensure the target remains fixed during testing. Firmware. The agent generates firmware (main.c) for each programmable device using helper libraries and SDK functions. For microcontrollers, this includes handling clock,

LoopHIL

UART, RF-switch, and timers, while for SDRs, it addresses radio setup and UART commands. Although the helpers streamline peripheral boilerplate, the implementation methodology remains the agent’s responsibility. Optimizations. After establishing a working link, the agent can optimize secondary objectives, such as minimizing tag power using an energy meter. It is instructed to implement one change at a time and re-verify the link after each change. Skills. Skills are concise expert notes stored as .md files, covering topics like MCU timing, backscatter theory, packet structure, and regulatory RF limits. The notes draw on general RF experience. The agent reads them on request, and they were not updated based on evaluation results.

3.2

Tools

The agent interacts with the bench exclusively through typed tools (Figure 2). Hardware config outlines the bench’s devices, roles, and limitations while setting the carrier emitter’s frequency and power. Drivers/SDK compile, size-check, and flash the firmware for the tag and receiver using each device’s toolchain, and they manage UART communication. They also configure timer values for the specified tones and bit rates, as well as construct data frames. Logs like IQ samples and received packets from the receivers are saved in the run workspace. Debug/Probe tools read the instruments (§3.3). Each tool identifies its device from the bench configuration, allowing a single tool set to cater to various devices, with structured results and errors providing feedback for issues like failed builds or refused commands. For analyses not covered by the tools, the agent runs Python scripts on the saved files. These scripts operate in a secure sandbox with no USB, serial port, network, or child processes access, limited to the run workspace. Thus, all device actions still pass through the tools for safety checks. Each run initiates from a reset bench and an empty workspace, ensuring no visibility into previous runs.

3.3

Instruments

Instruments provide measurements instead of raw samples, as models struggle with lengthy arrays of numbers. A Spectrum analyzer captures metrics such as carrier frequency, SNR, sidebands, modulation, symbol rate, deviation, and decoded frames with bit errors. A Logic analyzer measures toggle frequencies, duty cycle, bit rate, and frames on the tag’s switch pin before the RF path. A Power meter measures power, current, and energy consumption of the MCU over a specified window. These instruments report data to the agent during development and to the checks, which also perform their own captures (§3.4).

3.4

Checks

Specification validation. It ensures the specification adheres to schema constraints and limits in the bench configuration, which the agent cannot modify. It enforces safety limits (§3.5) and assesses MCU feasibility, such as checking timer dividers for requested tones and ensuring interrupt cycles meet requirements. Each task sets a minimum standard that resubmitted specifications cannot lower, preventing the agent from easing its own tests. Verification ladder. Operates based on locked specifications and takes its measurements, eliminating agent bias. It runs through its checks and halts at the first failure, as later tests are irrelevant if earlier ones fail. For example, sideband offsets are measured relative to the carrier, ensuring that any carrier error is not attributed to the tag. The ratio of measured to expected offsets determines the tag’s clock error. Blame device. The first failing rung identifies the device and fault layer (observer, carrier, firmware, RF hardware, receiver, measurement, or harness), providing measured values and suggested fixes. Using a logic analyzer, tag-pin captures distinguish firmware faults from RF faults. The agent can report success only after passing all verification steps. Limits.The ladder validates the agent’s specification and decodes frames on the receiver. A USRP decoder tolerates more timing error than the CC1310, making a ladder pass insufficient evidence that the CC1310 will accept the packets. Completion is therefore graded separately.

3.5

Safety and logging

Safety limits in code. The agent controlling the transmitter should not transmit in prohibited frequency bands or at power levels above limits, even if prompted to do so. So, before a command reaches a device, the transmit power, operating frequency, duty cycle, and the tag’s sideband emissions, etc are computed from the parameters its firmware declares. A refused command returns to the agent as an error. We perform experiments within regulations in the 868 MHz/ 915 MHz ISM bands, as directed by the IMDA Singapore, and FCC Part 15 regulations. Logging. Each run logs all tool calls, device actions, durations, model iterations with tokens, versions of code files, hashes of flashed images, verification results with responsible devices, and failure reasons.

3.6

Hardware bench

Figure 3 illustrates the bench setup. A USRP B210 generates the carrier signal at a specified frequency and power, while a second B210 serves as a receiver or spectrum analyzer. The tag uses an MSP430FR5969 microcontroller, with pin P1.4 controlling an RF switch connected to its antenna. A CC1310

Shah, Sara, Kandala and Varshney

Power meter

Logic analyzer

MCU Spectrum analyzer

SDR (TX) Backscatter tag Commodity RX

Figure 3: LoopHIL hardware test bench used for evaluation.

LaunchPad functions as a commercial receiver. A Saleae Logic Analyzer is connected to P1.4, alongside a Joulescope JS320 power meter in series. All radios are connected with antennas to communicate wirelessly.

4

fails this check makes an unsupported claim, which a grader trusting reported outputs would accept as success [14].

EVALUATION

We ask three questions. ❶ How well do current LLM agents complete wireless tasks that need real hardware, and at what cost? ❷ How much does the LoopHIL harness change success and cost? ❸ Do hardware debugging tools help agents fix the tag and reduce its energy?

Prompt Structure # <Task ID> (<level>) ## Task Intent in one paragraph ## Setup Device setup, configuration, roles; debug and measurement equipment ## Inputs Values the agent must use (frequency, deviation, bitrate, modulation, payload, etc.) ## Outputs Desired task output files ## Pass criteria Threshold checks for task success Figure 4: Prompt structure used for defining a task.

4.1

Benchmark and Setup

Tasks. The tasks grow in difficulty (Table 1). T1 only generates a carrier. T2 adds modulation and decoding. T3 moves the modulation onto a backscatter tag. T4 also requires a commercial receiver (a TI CC1310) to accept the tag’s packets. Each prompt states the goal, the hardware and the thresholds for success, such as a BER of at most 1% over 100 packets. It also names the files the agent must produce. Models. We evaluate six models from two vendors, in three capability tiers (Table 2). Anthropic models run in the Claude Agent SDK, and OpenAI models in Codex. Conditions. Every model runs every task in two conditions. In the baseline (B), the model runs in its vendor’s coding agent with shell and file tools and the hardware attached, but no skills or LoopHIL tools. In the harness (H), the same model works only through LoopHIL, evaluated as a whole. Web access is off, and devices communicate over the air. Task prompt. Every task is one prompt in a fixed structured format as shown in Figure 4. Both conditions receive the same prompt. Setup describes the device setup and wired hardware configuration. Pass criteria are the mandatory threshold checks required for task completion. Appendix A.1 and Figure 8 list the prompt for T4 with energy optimization. Grading. A run completes a task only if its reported values meet every threshold and an independent check finds them supported by the hardware. The check decodes the agent’s capture with its own decoders, confirms that the radio recorded it, and for T4 compares the agent’s receiver log with the receiver’s output. A run that meets the thresholds but

4.2

Metrics

We report three groups of metrics. Task success. For each task, we report how many of the 𝑁 runs complete it (𝑘 of 𝑁 , or as a percentage) [3]. We also count unsupported claims. Agent cost. We report wall-clock time, compared as a speedup [6]; output tokens [15], since the prompt cache serves most of the re-sent context; and cost at API list prices [11]. Communication quality. Our instruments measure the link that the generated code produces: the BER, the packets received, the carrier frequency, and the transmit power against the requested power. In Section 4.5, the energy meter also measures the tag’s power.

4.3 How Do Models Fare on Wireless Tasks? Completion. Table 3 shows which runs complete their task. In the baseline, models complete 13 of 24 runs. Capability matters more than vendor. The frontier and mid-tier models complete the simple tasks in both conditions, while the small models complete only one baseline run. Tasks that involve more devices are harder. When a commercial receiver must accept the tag’s packets, only one baseline succeeds. The others stop early, run out of time, build frames that the receiver rejects, or report results they never measured. Unsupported claims. When a small model cannot finish a task, it often reports success anyway. All seven baseline unsupported claims come from the two small models, which

LoopHIL

Table 1: Benchmark tasks. All tasks use two USRP B210s (TX and RX); T3 and T4 add an MSP430FR5969 tag, and T4 a TI CC1310 receiver. No. Task Description Level T1 CW carrier. Transmit an unmodulated carrier at 868 MHz and 0 dBm; capture 5 s at the receiver and plot received signal strength vs. time. Easy T2 OOK link. Transmit OOK packets with payload 0x6F379BCDE6F379 at 10 kb/s, 915 MHz and 10 dBm transmit power; decode 100 packets, Medium report the BER, and plot a 2 s waterfall spectrum. T3 OOK backscatter. Implement OOK backscatter on a 100 kHz baseband subcarrier at 20 kbps on a MSP430FR5969 under a 915 MHz carrier Medium (10 dBm transmit power); decode 100 packets with payload 0x6F379BCDE6F379, report the BER, and plot one received spectrum. T4 2-FSK backscatter. Generate 2-FSK on 800 and 1000 kHz subcarriers at 100 kb/s in a CC1310-compatible frame under a 915 MHz carrier Hard (10 dBm transmit power); receive 500 packets with payload 0x6F379BCDE6F379 on the CC1310, save its output, report the BER, and plot one received spectrum. Table 4: Effect of the harness per task (baseline → harness). Factors are medians over models that complete the task in both conditions.

Table 2: Models evaluated, by vendor and tier.

Anthropic OpenAI

Frontier

Mid

Small

Claude Opus 5.5 GPT-6-Astra

Claude Sonnet 5.5 GPT-5.6-Sol

Claude Haiku 4.5 GPT-5.6-Luna

Table 3: Task completion per model. B: baseline; H: harness. †: unsupported claim. Highlighted: completed only with the harness.

API cost (USD)

10 3 1 0.3 0.1 0.03

API cost (USD)

Anthropic OpenAI Opus Sonnet Haiku GPT-6 GPT-5.6 GPT-5.6 Task Setting 5.5 5.5 4.5 Astra Sol Luna B ✓ ✓ ✗† ✓ ✓ ✗† T1 ✓ ✓ ✓ ✗ H ✓ ✓ B ✓ ✓ ✗† ✓ ✓ ✓ T2 H ✓ ✓ ✗† ✓ ✓ ✓ B ✓ ✗ ✗† ✓ ✓ ✗† T3 H ✓ ✓ ✗† ✓ ✓ ✗† † ✗ ✗ ✗ ✗ ✗† B ✓ T4 † H ✓ ✓ ✗ ✓ ✓ ✓

10 3 1 0.3 0.1 0.03

T1

T2

T3

60

200

600

1800

Wall time (s) Opus 5.5 Sonnet 5.5

Haiku 4.5 GPT-6 Astra

60

200

600

1800

Wall time (s) GPT-5.6 Sol GPT-5.6 Luna

Baseline Harness

Figure 5: Wall-clock time and API cost of each completed run. Colour: vendor; shape: tier; outline: baseline; filled: harness.

hard-code the expected result, report a value no measurement could produce, or generate the “measurement” in software. A grader that trusts reported numbers would count each as a success. Cost. Figure 5 shows the time and cost of each completed run. A baseline run takes a median of 286 s and costs $0.39. Small

Completed

Speedup∗

Cost lower by∗

Tokens lower by∗

T1 T2 T3 T4

67% → 83% 83% → 83% 50% → 67% 17% → 83%

2.0× 1.9× 2.3× 0.4× †

1.3× 1.1× 1.6× 0.3× †

2.1× 1.7× 2.1× 0.4× †

∗ Baseline / harness: 2 × means half the time, cost or output tokens.

† Only Opus 5.5 completes T4 in both conditions. With the harness, it meets the

thresholds early, then spends about 20 more minutes improving the link.

models such as Haiku 4.5 and GPT-5.6 Luna are the cheapest, both per run ($0.06 against $0.98 for frontier models) and per completed task ($0.77 against $1.70). However, they manage to complete only 4 of their 16 runs. Safety. The baselines do not respect the requested transmit power. On the carrier task, they transmit up to about 19 dB more than asked. Every harness run transmits the requested power, because the harness enforces the limit in code.

4.4

T4

Task

Does the LoopHIL Harness Help?

Completion. With the harness, models complete 19 of 24 runs, against 13 of 24 in the baseline (Table 3). The harness never fails a task that the baseline completes. The gain is largest on the hardest task, where completion rises from 17% to 83%. There, the harness’s receiver tools spare the agent from configuring the commercial receiver from scratch. Unsupported claims fall from 7 to 4, but they do not disappear. The remaining ones all come from the small models. The harness makes the hardware easier to use, but it does not make a model honest. Time and cost. Table 4 compares the two conditions. The harness is faster in 17 of 24 pairs, and 2.0× faster where both complete a task, most on T3 (2.3×), the first task with a tag. The saving comes from the agent, not the hardware: the median run spends 132 s in the model instead of 199 s and 44 s in tools instead of 81 s, while device time stays similar (31 s against 36 s). Calling tested tools, the agent makes fewer tool errors (1 against 3) yet checks its work more often (3 code checks against 1). Output tokens fall by 2.0× but cost only by 1.2×, as tool descriptions and skills add context to every turn. On the hardest task, the one model completing

Shah, Sara, Kandala and Varshney preamble

(a)

Harness

First working

(b)

Final

P1.4

Baseline

50

83

67 50 33

25

33 17

17

10

-96%

-37%

-83% -51%

5

-19%

0

T3

T3 + debug

T4

T4 + energy

2

3

4

5

(a) Captured OOK baseband waveform generated by MSP430

2 1

carrier

0.5

Opus Opus Astra Sol Astra Luna B✓ H✓ B✓ H H H

in both conditions is slower with the harness, as it keeps improving the link past the threshold. Where the harness does not help. The harness helps the small models only a little: they complete 3 of their 8 runs with it, against 1 of 8 without. The ladder points to the faulty layer, but a small model does not reliably act on this feedback and repeats the same failure.

Do Debugging Tools Help?

We repeat T3 and T4 with hardware debugging tools that check the tag directly. In T3 + debug, the agent can probe the tag’s output pin with a logic analyzer. T4 + energy adds an energy meter on the tag’s supply, and asks the agent to reduce the tag’s power once its link works. We report the change from the first working to the final firmware [2, 15]. Debugging tools help only when agents can use them. With the logic analyzer, harness completion on T3 rises from 67% to 83%, while baseline completion falls from 50% to 33% (Figure 6a). Every harness run uses the instrument, exposed as a tool; only two baselines capture anything, and the small models never try without the harness. The logic analyzer mostly confirms correct tag firmware before any RF test (Figure 7), and occasionally exposes a timing or interrupt bug that the radio view would hide. Agents can still misread evidence: one baseline sees that the tag’s CPU is not running, but blames the wiring. Optimizing energy can break the link. Adding the energy goal to T4 lowers harness completion from 83% to 17%, below the baseline’s 33%. The extra steps compete for the same time budget, and each power change can shift the tag’s timing. The commercial receiver tolerates far less timing error than the harness’s ladder. A change that the ladder still accepts can therefore silence the receiver. In one run, the receiver stopped decoding after the agent moved frame generation to DMA, and the agent reverted the change. Power reductions. Five of the six runs that measured power reduce it, by 19% to 96%, and all still send valid frames (Figure 6b). Large cuts come from letting the MCU sleep between

80

lower sideband

60

upper sideband

40 20 0 −150

Figure 6: (a) Completion without and with hardware debugging tools. (b) Tag power of the first working and the final firmware in T4 + energy. B: baseline; H: harness; ✓: task completed.

4.5

1

Time (ms)

0.2 0

payload 0x6F379BCDE6F379

1

0

+3%

SNR (dB)

Completed (%)

83

75

Tag power (mW)

100

sync

−100

−50

0

50

100

150

Offset from carrier (kHz)

(b) Captured backscatter modulated signal at receiver Figure 7: One run of T3 + debug (Opus 5.5, harness), which decodes 250 packets without bit errors. (a) One OOK packet captured from the logic analyzer. (b) Frequency spectrum showing the modulated carrier with sidebands at a 100 kHz offset.

packets and while timers or DMA send the frame; small cuts only remove busy-waiting or idle peripherals. With the harness, the strongest model reaches its cut in small measured steps. Energy claims need scrutiny too: three runs report power they never measured, and another reports a cut although its power rose.

5

DISCUSSION AND CONCLUSION

We presented LoopHIL, a harness that loops hardware into LLM agents to develop wireless links without a human in the loop. Through typed tools, feature-level instruments, a verification ladder and safety limits in code, agents complete 79% of our tasks instead of 54%, and 83% of the hardest task instead of 17%. Where both conditions complete a task, LoopHIL is up to 2× faster and uses 2× fewer output tokens compared to its baseline. With debug instruments, agents reduce tag power by up to 96% while keeping the link working. Independent checks against the hardware remain essential, because small models often report unmeasured results. Scalability. Beyond the bench in Section 3.6, we also ran the harness on a Nordic nRF52840 and an ADALM-Pluto SDR. The tasks worked on these devices too, and the trends matched those above. The harness describes each device by a profile and a driver rather than per-task code, so adding a device changes neither the agent nor the tasks. Multi-device agents. Currently, one LoopHIL agent on a single machine drives every device in the task. We also tried a multi-agent setup, in which a coordinator spawned agents that wrote the code and completed the task. It reached similar results but used far more tokens: on one 2-FSK backscatter task, it cost about 2.3× as much as a single agent. In future work, we plan to give each device a sub-agent matched to its needs and resources, on one or several machines [12]. Agents could also use web search to look up datasheets and SDK examples, which we disabled in our evaluation.

LoopHIL

REFERENCES [1] Anthropic. 2024. Introducing the Model Context Protocol. https: //www.anthropic.com/news/model-context-protocol. [2] Z. Englhardt et al. 2024. Exploring and Characterizing Large Language Models for Embedded System Development and Debugging. In CHI EA ’24. [3] C. Jimenez et al. 2024. SWE-bench: Can Language Models Resolve Real-World GitHub Issues?. In ICLR ’24. [4] F. Zhu et al. 2022. Enabling Software-defined PHY for Backscatter Networks. In ACM MobiSys ’22. [5] H. Yang et al. 2026. AutoEmbed: LLM-driven Automated Software Development for Generic Embedded IoT Systems. In ACM SenSys ’26. [6] J. Lei et al. 2026. RadioMaster: Multi-Agent System for Autonomous Radio Signal Generation. arXiv:2606.01862 [7] J. Yang et al. 2024. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. In NeurIPS ’24. [8] R. Lim et al. 2013. FlockLab: A Testbed for Distributed, Synchronized Tracing and Profiling of Wireless Embedded Systems. In IPSN ’13. [9] R. Xu et al. 2026. EmbedAgent: Benchmarking Large Language Models in Embedded System Development. In ACM/IEEE ICSE’26. [10] S. Arx et al. 2025. Recent Frontier Models Are Reward Hacking. METR. [11] S. Kapoor et al. 2025. AI Agents That Matter. TMLR (2025). [12] S. V. Kandala et al. 2024. TinyLLM: A Framework for Training and Deploying Language Models at the Edge Computers. arXiv:2412.15304 [13] Y. Li et al. 2026. Skilled AI Agents for Embedded and IoT Systems Development. arXiv:2603.19583 [14] Y. Zhu et al. 2025. Establishing Best Practices for Building Rigorous Agentic Benchmarks. In NeurIPS ’25. [15] Z. Zhang et al. 2026. Embedded Arena: Iterative Optimization via Hardware Feedback. arXiv:2606.16190 [16] P. Medaranga et al. 2024. POSTER: Simplifying the Networking of Wireless Embedded Systems using a Large Language Model. In ACM SIGCOMM ’24, Poster. [17] P. Sooriya Patabandige et al. 2026. Microwatt Microwave (M2) Oscillator: Going Beyond the Delegation Architecture of Low-power Wireless Communication. In ACM MobiSys ’26. [18] A. Varshney et al. 2019. TunnelScatter: Low Power Communication for Sensor Tags Using Tunnel Diodes. In ACM MobiCom ’19.

[19] A. Varshney et al. 2020. Tunnel Emitter: Tunnel Diode Based LowPower Carrier Emitters for Backscatter Tags. In ACM MobiCom ’20. [20] A. Varshney et al. 2022. Judo: Addressing the Energy Asymmetry of Wireless Embedded Systems through Tunnel Diode Based Wireless Transmitters. In ACM MobiSys ’22.

A APPENDIX A.1 Task T4 Prompt Task T4 asks the agent to build a 2-FSK backscatter link from the MSP430FR5969 tag to the CC1310. One USRP B210 generates a 10 dBm carrier at 915 MHz. The tag switches between 800 kHz and 1 MHz subcarriers at 100 kb/s and sends the payload 0x6F379BCDE6F379 in a CC1310-compatible frame. The CC1310 must receive and decode 500 packets. The second USRP serves as a spectrum analyzer. The T4 + Energy Optimize task adds two instruments. A logic analyzer on P1.4 is used to debug the baseband subcarrier. An energy meter on the MCU supply is used to reduce the tag’s power. The prompt as mentioned in Figure 8 lists the hardware configuration and connections. It does not fix the frame’s preamble, sync word or CRC, the tag’s timer settings, or the CC1310 receiver settings; the agent chooses these. The agent must save these in results/ folder in the task run workspace: • the flashed tag source code; • a spectrum plot of the modulated signal captured at the receiver; • a logic-analyzer snapshot of one packet on P1.4; • the CC1310’s printed output, one line per received packet; • result.json with the decoded payload, the number of packets received with a valid CRC, the bits compared, the BER, and the tag MCU’s average power before and after optimisation.

Shah, Sara, Kandala and Varshney

Task Prompt # T4 + Energy Optimize (hard) ## Task 2-FSK backscatter. Implement 2-FSK backscatter on a MSP430FR5969 with two baseband subcarriers under a carrier generated from one USRP, in a CC1310-compatible frame; receive and decode 500 packets on the CC1310 and report the BER; plot one spectrum (signal strength or SNR vs frequency) of the received modulated signal on the second USRP. Use the logic analyzer on P1.4 to debug the baseband subcarrier and plot a snapshot of it, and the energy meter to reduce the tag MCU’s power. Keep the transmitter on continuously (the carrier, and the tag where there is one); durations and packet counts are for the receiver’s capture. ## Setup - USRP B210 carrier emitter (TX) - USRP B210 receiver used as a spectrum analyzer - MSP430FR5969 LaunchPad: P1.4 drives the backscatter switch; default clock 16 MHz DCO - CC1310 LaunchPad packet receiver (RX) - RF: over the air Debug and measurement equipment: - MSP430 LaunchPad debug probe (eZ-FET): Spy-Bi-Wire and the UART backchannel (P2.0/P2.1). Use: flash, run, reset; UART 115200 8N1. Host tool: TI UniFlash (dslite). - CC1310 LaunchPad debug probe (XDS110): JTAG and the UART backchannel. Use: flash; UART 115200 8N1. Host tool: TI UniFlash (dslite). - Saleae Logic 8 logic analyzer: digital channel D0 to MSP430 P1.4, GND to the LaunchPad GND. Use: baseband timing on the pin (toggle frequency, duty cycle, bit rate, frames). Host tool: Saleae Logic 2 with its automation server (Python package logic2-automation). - Joulescope JS320 energy meter: across the LaunchPad current-measurement jumper (MCU supply only). Use: MCU power, current and energy. Host tool: Python package joulescope (Joulescope UI closed). ## Inputs (use these values) - Carrier frequency: 915 MHz - Carrier power: 10 dBm - FSK subcarriers: 800 kHz, 1 MHz - Bit rate: 100 kb/s - Frame format: CC1310 compatible - Payload: 0x6F379BCDE6F379 - Packets: 500 ## Outputs (in ‘results/’ in your working directory) - main.c: the tag firmware source that was flashed - spectrum.png: spectrum of the received modulated signal - baseband.png: snapshot of the baseband waveform on P1.4 from the logic analyzer (one packet, time in ms) - cc1310_rx.txt: the CC1310’s output for the reception your numbers come from, as it printed it (one line per received packet) - result.json, one JSON object with: decoded_payload_hex; packets_received: packets the CC1310 receives with a valid CRC; bits_compared; ber: payload bit errors / bits_compared (final firmware); tag_power_mw_before; tag_power_mw_after: tag MCU average power of the final firmware, mW. ## Pass criteria (checked against results/) - decoded_payload_hex == 6f379bcde6f379 - packets_received >= 500 - ber <= 0.01 - main.c exists - spectrum.png exists - cc1310_rx.txt exists - baseband.png exists Figure 8: Prompt used for Task T4 + energy optimization.

Related documents

• LoopHIL: Looping Hardware into LLM-assisted Agents for Wireless Embedded Systems#1388797
• Requirements-Augmented Generation for Trustworthy Acceptance Testing of LLM-Based Software#451358
• LLM-based Hardware Development with Hierarchical IRs and End-to-End Multi-Agent Workflow#622598
• Execution-First Synthetic Tool-Use Trace Generation for LLM Agents#997524
• Step-Aligned Script Generation with Verification and Recovery for Web Automation Testing#1381312