arXiv CSopen access

IoT-Brain: Grounding LLMs for Semantic-Spatial Sensor Scheduling

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

networking, internet, protocols, distributed systems

IoT-Brain: Grounding LLMs for Semantic-Spatial Sensor Scheduling Zhaomeng Zhou1 , Lan Zhang1,2,* , Junyang Wang1 , Mu Yuan3 , Junda Lin1 , Jinke Song4 1 University of Science and Technology of China 2 Institute of Artificial Intelligence, Hefei Comprehensive National Science Center

arXiv:2604.08033v1 [cs.AI] 9 Apr 2026

3 The Chinese University of Hong Kong

4 The Hong Kong University of Science and Technology

{zhouzhm, iswangjy, linjunda}@mail.ustc.edu.cn, [email protected] [email protected], [email protected]

ABSTRACT

KEYWORDS

Intelligent systems powered by large-scale sensor networks are shifting from predefined monitoring to intent-driven operation, revealing a critical Semantic-to-Physical Mapping Gap. While large language models (LLMs) excel at semantic understanding, existing perception-centric pipelines operate retrospectively, overlooking the fundamental decision of what to sense and when. We formalize this proactive decision as Semantic–Spatial Sensor Scheduling (S3) and demonstrate that direct LLM planning is unreliable due to inherent gaps in representation, reasoning, and optimization. To bridge these gaps, we introduce the Spatial Trajectory Graph (STG), a neurosymbolic paradigm governed by a "verify-beforecommit" discipline that transforms open-ended planning into a verifiable graph optimization problem. Based on STG, we implement IoT-Brain, a concrete system embodiment, and construct TopoSense-Bench, a campus-scale benchmark with 5,250 natural-language queries across 2,510 cameras. Evaluations show IoT-Brain boosts task success rate by 37.6% over the strongest search-intensive methods while running nearly 2× faster and using 6.6× fewer prompt tokens. In real-world deployment, it approaches the reliability upper bound set by reducing 4.1× network bandwidth, providing a foundational framework for LLMs to interact with the physical world with unprecedented reliability and efficiency.

Large Language Models, IoT Networks, Sensor Scheduling

CCS CONCEPTS • Computing methodologies → Artificial intelligence; • Computer systems organization → Sensor networks. * Lan Zhang is the corresponding author.

This work is licensed under a Creative Commons Attribution 4.0 International License. MobiCom ’26, Austin, TX, USA © 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2505-0/2026/10 https://doi.org/10.1145/3795866.3796695

ACM Reference Format: Zhaomeng Zhou1 , Lan Zhang1,2,* , Junyang Wang1 , Mu Yuan3 ,, Junda Lin1 , Jinke Song4 . 2026. IoT-Brain: Grounding LLMs for Semantic-Spatial Sensor Scheduling. In The 32nd Annual International Conference on Mobile Computing and Networking (MobiCom ’26), October 26–30, 2026, Austin, TX, USA. ACM, New York, NY, USA, 16 pages. https://doi.org/10.1145/3795866.3796695

INTRODUCTION

The proliferation of large-scale sensor networks in smart cities and industries is catalyzing a paradigm shift towards intelligent automation[39, 46, 53]. This leap from traditional coverage-driven optimization[10, 30, 56, 73] to real-time semantic goal satisfaction introduces a profound, yet underexplored challenge. A simple request like "Can you help me check my wallet between the library and the gym?" exposes a stark divide between the vagueness of human language and the precise physical operations a sensor network must execute. We term this the Semantic-to-Physical Mapping Gap (Fig. 1(a)), a fundamental hurdle that renders conventional task-specific models and rigid scripts ineffective. The advent of large language models (LLMs) has provided a powerful engine for semantic interpretation [17, 24, 47, 50], offering a promising path forward. Recent work on "Penetrative AI" further shows that LLMs can reason directly over raw sensor data [16, 57, 74, 79]. Yet these advances largely remain within operates retrospectively, assuming the relevant sensor streams are already available, what we term Reactive Perception (Fig.1(c)). This perspective overlooks a more fundamental upstream challenge, Proactive Scheduling, that is paramount in large-scale deployments. Before any meaningful perception can occur, LLMs must decide what to sense and when to attend. The decision precedes and enables all subsequent perception but has received limited systematic study [5, 33, 40]. We formalize this pivotal challenge as the Semantic–Spatial Sensor Scheduling (S3) problem.

MobiCom ’26, October 26–30, 2026, Austin, TX, USA Semantic Intent Scheduling

Conventional Scheduling Tasks

Area Coverage

Predefined Trajectory Tracking Zzz

2 Zzz

3 4

Multi-Object Tracking

Zhou et al.

Zzz

Resource Optimization

Goal: Maximize Coverage / Minimize Cost

？

Find path: Cafe→Lab

12:00 7.25

Query: Find my keys between the library and the gym.

Find path: Lab→Club

Library

Cafe

Store

User

Goal: Fulfill Ad-hoc Human Intent

(a) The Semantic-to-Physical Mapping Gap.

sp it

Clu

No Match Found

(b) Limitations of conventional approaches.

Gym

The Semanticto-Physical Mapping Gap

Perception Engine

What is happening in these feeds? A person is walking and talking.

User

Find my keys between the library and the gym. Library

Gym

Detected near parking lot!

Figure 1: The challenge of the S3 problem. Solving the S3 problem with off-the-shelf LLMs is far from this disciplined approach "verify-before-commit", a princistraightforward. Our preliminary study (§2.2) tasks an LLM ple requiring that all semantic hypotheses be fully validated with end-to-end scheduling in a real-world topological enviagainst reality before they are deemed executable. ronment, revealing three fundamental challenges: STG design. The "verify-before-commit" principle’s pro(1) Symbol-to-Semantic Chasm. LLMs’ native shortcomgressive translation of high-level intent into concrete acing in comprehending raw, machine-oriented symbolic topolotion inherently embodies a search for an optimal topologgies prevents them from building an effective world model, ical pathway within semantic constraints. This complex slashing their planning success by over 5× compared to when search process mirrors the established paradigm of graph provided with structured, human-readable knowledge. construction[13, 68, 76], which we formalize as the Spa(2) Inferential Leap from Points to Paths. LLMs’ protial Trajectory Graph (STG), a neurosymbolic paradigm that found difficulty in inferring topological relationships like operates through a systematic, multi-stage workflow. The connectivity from disconnected symbols causes even a perparadigm first mandates the structuring of ambiguous infectly informed model to achieve merely 26% trajectory covtent into a verifiable, hypothesized graph. It then requires erage, leading to fragmented and unsound paths. grounding of the graph through an iterative validation loop (3) Optimization Shortfall in LLM Planning. The inagainst a physical world model, methodically transforming herent "satisficing" nature of LLMs leads to resource-heavy uncertainty into verified facts. Finally, the paradigm conplans, exhibiting up to 45% redundant sensor overlap even cludes with the optimization of the now-verified graph into with structured guidance and compounding to a staggering a resource-aware, dynamic execution plan. This principled 48× token overhead when processing raw symbolic data. decomposition systematically separates semantic interpretaCore idea: verify-before-commit. This work aims to tion from deterministic validation and scheduling. bridge the gap between an LLM’s high-level semantic reasonIoT-Brain Implementation. We implement the STG paring and the need for resource-efficient, physically grounded adigm in IoT-Brain, a system engineered for robust, realsensor scheduling. Our insight stems from observing LLMs’ world interaction. Instead of a monolithic pipeline, IoT-Brain native behavior on the S3 problem, showing that their plans adopts a modular architecture that leverages an LLM’s adare often ungrounded and glaringly misaligned with sensorvanced tool-calling and programming capabilities[32, 55]. It network constraints. By contrast, when supplied with preemploys a reactive, tool-based loop to continuously ground vetted, task-relevant topological knowledge, LLM-derived its semantic reasoning against our physical world model, plans become both reliable and efficient. Such oracle-like effectively transforming abstract hypotheses into verifiable access, however, is infeasible in dynamic deployments. In facts. This design strategically decouples the LLM’s highresource-sensitive environments, executing a speculative, unlevel semantic inference from the deterministic, resourceverified plan is prohibitively expensive. We therefore adopt conscious tasks of scheduling and sensor control. The entire a paradigm in which the LLM must proactively and auworkflow is further accelerated by shared memory modules tonomously discover and validate the necessary groundthat cache executable reasoning evidence, amortizing the ing knowledge through direct interaction with the physicost of interaction across sessions. To evaluate IoT-Brain, we cal world before any operational decision is taken. We term also constructed TopoSense-Bench, a large-scale benchmark

Human/LLM-Readable Structured Knowledge

<node id='-235' lat='...' lon='...'> Scenario: duty_room <tag k='man_made' v='surveillance' /> Type: Office Room <tag k='name' v='duty_room_camera_1' /> </node> Boundary: Vertices [-450, -451, ...] <node id='-450' lat='...' lon='...'> Feature Points: <tag k='door' v='duty_room_door' /> <tag k='hall_1F_vertex' v='...' /> Doors: </node> The ... duty_room_door (ID: -450) Oracle Process Connects to: hall_1F

Cameras: duty_room_camera_1 (ID: -235) ......

Position: (lat, lon)

100 75 50 25 0 100 75 50 25 0

(a) From raw symbols to structured knowledge.

Naive Oracle

Single-Scenario

Multi-Scenario

Scenario Coverage

Naive Oracle

Single-Scenario

Multi-Scenario

Coverage Overlap

Single-Scenario Avg. Tokens (×103)

Raw Symbolic Representation (Machine-Oriented)

MobiCom ’26, October 26–30, 2026, Austin, TX, USA Avg. Overlap Ratio (%) Avg. Coverage (%)

IoT-Brain: Grounding LLMs for Semantic-Spatial Sensor Scheduling

Multi-Scenario

Trajectory Coverage Naive Oracle

Single-Scenario

48× Multi-Scenario

Inference Cost

(b) Empirical planning failures.

Figure 2: The representation gap and resulting planning failures. tailored to the S3 problem featuring a university-scale digital brittle fixed-topology scripts [9, 28]. As illustrated in Fig. 1(b), twin with 2,510 cameras and 5,250 real-world queries. both approaches break down in large-scale, dynamic settings [62, 65, 72]. Manual operation does not scale [18], and Contributions of our work are summarized as follows: • We identify and formalize the S3 problem, a critical yet hard-coded scripts fail to generalize to the fluid and unpreoverlooked challenge in intent-driven sensor networks, dictable nature of human intent [48, 67]. and introduce the Spatial Trajectory Graph (STG), a neuThe emergent semantic understanding of LLMs offers rosymbolic paradigm that grounds LLM reasoning in a a transformative and promising route beyond the historiverifiable, physical-world structure. cal constraints of traditional sensor networks. Pioneering • We design and implement IoT-Brain, our concrete syssystems show LLMs can parse complex sensor data and tem instantiation of STG, and we construct and release interact with the physical world, opening a new frontier TopoSense-Bench, a large-scale benchmark with 5,250 realfor AIoT [16, 21, 57, 60, 79]. Yet, the current LLM-for-AIoT world queries to catalyze future research in this domain. landscape is structurally imbalanced. As shown in Fig. 1(c), • We conduct evaluations of IoT-Brain on our TopoSensemost work concentrates on Reactive Perception, where models Bench and a physical testbed. Experimental results demonpassively analyze pre-collected sensor streams. We instead strate the superiority of STG. On the benchmark’s most tackle Proactive Scheduling, the upstream problem of decidcomplex tasks, IoT-Brain boosts task success by 37.6% over ing precisely which sensors to activate and when. This shift the strongest search-intensive methods[12, 54] while runfrom passive interpretation to active, goal-directed participaning nearly 2× faster and using 6.6× fewer prompt tokens. tion is essential for truly autonomous AIoT systems, and the Furthermore, in live deployments, our system approaches transition is nontrivial, exposing fundamental challenges in the reliability of a resource-agnostic upper-bound approach reliably grounding LLM reasoning in the physical world. while consuming 4.1× less network bandwidth.

BACKGROUND & MOTIVATION

We first survey the sensor-scheduling landscape, contrasting coverage-driven methods with emerging LLMs’ capabilities to expose a critical gap (§2.1). Then we present a preliminary study that decomposes LLM-based proactive scheduling into three fundamental gaps (§2.2). Next, we motivate a new planning paradigm that addresses these gaps (§2.3).

2.1

Background

Classical research in sensor networks has centered predominantly on coverage-driven optimization tasks, such as maximizing sensor coverage or tracking objects within a static, pre-defined field-of-view [10, 11, 35]. While effective for welldefined engineering objectives, this paradigm is fundamentally ill-suited for intent-driven queries specified in ambiguous natural language. In practice, such ad-hoc semantic tasks are often relegated to rudimentary solutions, typically relying on laborious human-in-the-loop operations [26, 44, 52] or

2.2

Preliminary Study

To dissect why the leap from perception to scheduling is challenging for LLMs, we conducted a preliminary study designed to answer a core question. Can LLMs effectively plan routes and schedule sensors when given raw, symbolic topological data? To investigate this, we constructed a dedicated testbed using real-world topological data from OpenStreetMap (OSM) [25], encompassing five multi-scenario buildings and a set of 373 manually curated queries that require both spatial understanding and path planning. We compared two approaches. In the Naive setting, the LLM is prompted directly with machine-oriented OSM text (e.g., XML-style nodes and ways), and is asked to produce a sensor activation plan. In the Oracle setting, which serves as an upper bound, the same symbolic data is preprocessed into a structured, human-readable knowledge base that makes locations, connections, and salient features explicit (see Fig. 2(a)) for which the LLM then uses for reasoning. We assessed each approach on its ability to produce correct and efficient

MobiCom ’26, October 26–30, 2026, Austin, TX, USA

schedules, measuring scenario coverage, trajectory coverage, resource overlap, and token consumption. The results, summarized in Fig. 2(b), reveal three tightly coupled gaps that together impede reliable LLM-based scheduling. Gap 1: Symbol-to-Semantic Chasm. The disparity between the Naive and Oracle settings reveals a fundamental representation mismatch. The quantitative impact of this chasm is stark. When the LLM is provided with structured, human-readable knowledge, scenario coverage boosts by over 2.1× on single-scenario tasks and a remarkable 5× on multi-scenario tasks compared to the baseline Naive approach. Because LLMs are trained on natural language rather than machine-oriented symbolic topologies, they struggle to effectively parse OSM-style structures [19, 20, 58] and therefore cannot assemble a usable world model. Planning performance consequently collapses, underscoring the need for a dedicated symbol-to-semantic translation layer. Gap 2: Inferential Leap from Points to Paths. Trajectory coverage lays bare a deeper reasoning failure. In the Naive setting, it is essentially zero. Even with an explicit, intelligible map, the Oracle reaches only 26% on multi-scenario tasks. This shortfall reflects a deficit in multi-step path formation across large topologies. LLMs can reference named places and operate over simple pre-specified graphs, yet they rarely assemble the connectivity constraints that turn local doorways into a valid end-to-end route. Rather than simply providing structured data and expecting a correct plan, a practical system must actively scaffold reasoning by constructing connectivity, verifying reachability, and validating long-horizon trajectories within the spatial graph. Gap 3: Optimization Shortfall in LLM Planning. Beyond representation and reasoning, planning quality falters at the optimization level. Even with perfect topological semantics, the Oracle still yields inefficient schedules, with overlap reaching 45%. Without structured guidance, the inefficiency compounds dramatically. On multi-scenario tasks, the Naive setting expends a staggering 48× more tokens than the Oracle. These patterns clearly indicate LLMs inherently tend to satisfice rather than optimize, producing plausible yet resource-heavy plans [41, 71]. Therefore, a practical system should capitalize on LLM’s semantic strengths while delegating global resource efficiency to deterministic algorithms[6].

2.3

Motivation & Core Ideas

The preceding findings yield a crucial insight that constructing a reliable and efficient semantic scheduling system cannot simply treat the LLM as an unconstrained, end-to-end black-box planner. These inherent, fundamental gaps in representation, reasoning, and optimization necessitate a novel paradigm to explicitly structure and guide the LLM’s role. This motivates our core idea to replace monolithic opaque planning with a structured and verifiable workflow centered

Zhou et al.

on the Spatial Trajectory Graph (STG). Our neurosymbolic STG paradigm decomposes the intractable scheduling problem into a principled three-stage process, each instantiated by a corresponding phase in our IoT-Brain system. 1) Intent Formalization (§4.2) first leverages an LLM’s semantic competence to translate a user’s ambiguous query into a hypothesized STG, a structured but unverified blueprint of intent. 2) Feasibility Grounding (§4.3) then enters an iterative "verify-before-commit" loop where each element of the blueprint is systematically validated against the physical world model until a fully consistent and grounded STG is produced. 3) Optimal Synthesis (§4.4) finally compiles the now-verified blueprint into a resource-optimal sensor activation plan and executes it adaptively using a perception-in-the-loop mechanism to handle unfolding real-world dynamics. The multistage process provides a systemically verifiable solution for LLM-based grounding, making a significant step towards enabling truly intelligent AIoT systems.

S3 AND STG FORMULATION

We first formally define the Semantic–Spatial Sensor Scheduling (S3) problem as a principled mapping from high-level user intent to a resource-aware dynamic activation plan (§3.1). Building on this formulation, we introduce the Spatial Trajectory Graph (STG), a paradigm that grounds free-form language into an optimization-ready spatial graph to systematically address this fundamental challenge (§3.2).

3.1

The S3 Problem

Environment and Query. We consider a large, densely instrumented site where a user issues a natural language query 𝑄 𝑁 𝐿 . The environment is represented by a comprehensive world model 𝑊 = (Gspatial, Gsensor, Φ) where Gspatial is a labeled spatial graph of locations and traversability, Gsensor is a device graph of sensors and their capabilities, and Φ links the two by encoding sensor visibility and geometry. Semantic Compilation. Given 𝑄 NL and 𝑊 , semantic compilation yields a set of verifiable spatiotemporal predicates Ω(𝑄 NL, 𝑊 ) specifying geographical anchors, spatial regions, relational constraints, and explicit temporal bounds (e.g., "11:30 a.m."). Let Π(Ω) denote the admissible spatiotemporal witnesses, representing the trajectories that must be observed within a specific timeframe to satisfy the query. Plans and Objective. Answering the query requires generating a dynamic activation plan 𝑃 (𝑡), a time-varying set of sensors intended to reconstruct a witness trajectory 𝜏 ∈ Π(Ω). A plan is feasible if its collective observation over time, denoted Ð as P = 𝑡 𝑃 (𝑡), completely covers at least one witness 𝜏. The objective is to find an optimal plan 𝑃 ∗ (𝑡) that maximizes a fidelity-cost trade-off, formalized as: ∫ 𝑃 ∗ (𝑡) = arg max F (P; Π(Ω), Φ) −𝜆 Cost(𝑃 (𝑡))𝑑𝑡 , (1) 𝑃 (𝑡 ) ⊆ Gsensor

IoT-Brain: Grounding LLMs for Semantic-Spatial Sensor Scheduling

MobiCom ’26, October 26–30, 2026, Austin, TX, USA

of grounded spatial paths (G★) to find an optimal 𝜏𝑉∗ . Second, it determines the optimal temporal scheduling (𝜎𝑡∗ ) along that path. The objective is formalized as: (𝜏𝑉∗ , 𝜎𝑡∗ ) = arg max F (𝜏𝑉 , 𝜎𝑡 ) − 𝜆 Cost(𝜏𝑉 , 𝜎𝑡 ) , (2)

where fidelity F measures the quality of the reconstructed trajectory, monotonically increasing with its completeness and accuracy. The Cost(𝑃 (𝑡)) function captures all resource expenditures, including the number of active sensors, activation duration, and redundant spatial overlap. Inherent Requirements. While Eq. 1 specifies the optimization objective, solving it directly with off-the-shelf LLMs is intractable given the model’s intrinsic computational limits. Informed by our preliminary study (§2.2), a viable LLM-based approach must satisfy three fundamental requirements to align model capability with rigorous problem demands. 1) Semantic Grounding. The solution must anchor the LLM’s linguistic outputs to the physical world. Fuzzy descriptions like "near the main entrance" must be unambiguously resolved to concrete spatial entities in 𝑊 to become actionable. 2) Topological Feasibility. The solution must enforce topological validity on the LLM’s generated plans. Any long-horizon trajectory 𝜏 must be explicitly verified as physically traversable within G𝑠𝑝𝑎𝑡𝑖𝑎𝑙 , precluding the LLM from simply hallucinating impossible paths. 3) Resource Efficiency. The solution must decouple planning from optimization. Given that LLMs are satisficers, not optimizers, a practical system must delegate the selection of a resource-efficient schedule to specialized deterministic algorithms. These requirements motivate a paradigm that structures and constrains the LLM’s role, moving beyond brittle end-to-end planning.

where the optimal dynamic plan 𝑃 ∗ (𝑡) is derived from (𝜏𝑉∗ , 𝜎𝑡∗ ). This reframing is pivotal because it separates the verifiable, static spatial planning from the adaptive, online temporal scheduling, making the problem tractable. Principled Inference Workflow. The reframing supports a three-stage workflow guided by the "verify-before-commit" principle. i) Intent Formalization. From the user query, construct an ungrounded graph 𝐺 0 populated with candidate locations and a hypothesized spatial path 𝜏𝑉 ,0 . ii) Feasibility Grounding. Enter an iterative loop that validates spatial hypotheses in 𝜏𝑉 ,0 against the world model 𝑊 , disambiguates semantics, and enforces topological feasibility until a fully grounded spatial path 𝐺★ emerges. iii) Optimal Synthesis. On the grounded path 𝐺★, compute the optimal dynamic scheduling function 𝜎𝑡∗ maximizes the objective in Eq. 2. This yields a verifiably resource-aware dynamic activation plan 𝑃 ∗ (𝑡). Within this workflow, the LLM proposes and refines spatial hypotheses, while the correctness of the spatial path and the optimality of the spatiotemporal schedule are secured by explicit checks and deterministic solvers.

3.2

The STG Paradigm

End-to-end sensor selection couples semantics, topology, resources, and timing, hindering verification of intermediate assumptions. To make decisions verifiable and cost-aware, we introduce STG. STG decouples semantic interpretation, topological grounding, and resource optimization by inserting an explicit spatiotemporal trajectory graph where candidate hypotheses are checked before commitment. This blueprint transforms a user’s intent into a structured trajectory, the trajectory into verified spatiotemporal facts, and these facts into an optimized, dynamic activation schedule. STG Definition. An STG is a quadruple 𝐺 = (𝑉 , 𝐸, 𝜏𝑉 , 𝜎𝑡 ) that provides a verifiable specification for a dynamic plan. It comprises: (i) a connected subgraph (𝑉 , 𝐸) of relevant locations; (ii) a verified spatial witness path 𝜏𝑉 = (𝑣 1, . . . , 𝑣𝑚 ) that satisfies the spatial predicates; and (iii) a dynamic sensor scheduling function 𝜎𝑡 , which maps each spatial node 𝑣𝑖 ∈ 𝜏𝑉 to a set of sensors to be activated at a dynamically determined time 𝑡𝑖 . The induced dynamic plan is thus 𝑃 (𝑡) = 𝜎𝑡 (𝑣𝑖 ) for 𝑡 = 𝑡𝑖 . An STG is hypothesized (𝐺 0 ) when its spatial path 𝜏𝑉 ,0 is unverified, and becomes grounded (𝐺★) only after 𝜏𝑉 is validated against the world model 𝑊 . Objective Reparameterization. The STG structure recasts the original optimization over dynamic plans 𝑃 (𝑡) as a more tractable, two-stage process. First, it searches over the space

(𝜏𝑉 ,𝜎𝑡 )

IOT-BRAIN SYSTEM ARCHITECTURE

To instantiate the STG paradigm, we implemented IoT-Brain, a modular framework that turns high-level semantic queries into verifiable, resource-aware sensor activation plans. As depicted in Fig. 3, IoT-Brain follows a three-stage pipeline comprising Semantic Structuring (§4.2), Symbolic Grounding (§4.3), and Adaptive Execution and Perception (§4.4).

4.1

System Workflow

IoT-Brain takes two inputs, a natural language query 𝑄 NL and a world model 𝑊 combining detailed spatial knowledge with a sensor-network map, and processes them through a structured and verifiable three-stage pipeline. Semantic Structuring. The workflow employs three LLM agents as one-shot planners using system prompts. First, the Topological Anchor ➊ extracts geographical entities to map textual mentions in 𝑄 NL to spatial graph nodes, seeding a scaffold. Building on this, the Semantic Decomposer ➋ factors the goal into a logical witness walk of atomic sub-tasks. To bridge high-level plans and physical constraints, the Spatial Reasoner ➌ analyzes transitions to formulate verifiable hypotheses regarding topological connectivity and attributes. Symbolic Grounding. To validate these hypotheses, the Grounding Verifier ➍ operates as an agent in an iterative Thought-Action-Observation loop [78]. It translates abstract

MobiCom ’26, October 26–30, 2026, Austin, TX, USA

Zhou et al.

Figure 3: The system overview and workflow of IoT-Brain. hypotheses into concrete checks by invoking our customcrafted deterministic Verifying Toolkit, a Python library designed to query the explicit geometry and sensor coverage within 𝑊 . This rigorous loop prunes topologically infeasible branches until a consistent grounded graph emerges. Verified facts are cached as system-wide topological consensus in Spatial Memory ➎ to accelerate future lookups. Adaptive Execution and Perception. With the verified graph, the Scheduling Synthesizer ➏ acts as a graph-to-code compiler, generating a Python script that invokes specialized heuristic solvers from the Execution API Pool for resourceoptimal scheduling. Successful scripts are cached in Programming Memory ➐ for efficient in-context reuse [42, 81]. The Perception Aligner ➑ then executes this schedule through a seamlessly coordinated pipeline: a VLM first grounds the text description to a visual target, passing the visual embedding to a Re-ID network for cross-camera association, while a Kalman-ETA filter [3] predicts arrival times to trigger downstream sensors just-in-time precisely.

4.2

Intent-to-Blueprint Structuring

The initial phase of IoT-Brain converts the unstructured user query 𝑄 NL into a structured representation of intent and context. This process, termed Semantic Structuring, progressively builds a hypothesized STG, which is a machine-interpretable blueprint derived from the query but not yet verified against the world model. Fig. 4 depicts the three-agent pipeline, illustrated using the "lost backpack" query. ➊ Topological Anchor. The pipeline begins with the Topological Anchor, a coarse-grained parser that identifies the spatial scope of the user’s intent. By leveraging topological

Figure 4: Dataflow of the Semantic Structuring phase. priors, the agent instantiates a candidate subgraph containing both explicitly mentioned locations and implicitly required transition points. For the running example, the agent identifies the defined starting public communication area and the target destination testing laboratory, while simultaneously deducing necessary intermediate connectors, such as the Library elevator (shown as node_2 in Fig. 4), required to bridge the vertical floor transition. These explicit and implicit geographical entities collectively form the initial, unverified vertices 𝑉0 and edges 𝐸 0 of a nascent STG, effectively representing the initial structural hypothesis of the user’s underlying intended spatial context. ➋ Semantic Decomposer. While the Anchor provides disconnected spatial candidates, the Semantic Decomposer imposes order by factoring high-level intent into a sequence of

IoT-Brain: Grounding LLMs for Semantic-Spatial Sensor Scheduling

MobiCom ’26, October 26–30, 2026, Austin, TX, USA

Iteration 1 Ambiguity Resolution

Grounding Verifier Verifying Toolkit Doors_verify() Cameras_verify() Elevators_verify() Road_paths_verify() Facilities_verify()

Spatial Memory public communication area：2 camera,15 study-desks Library：3 exit points, library_main_gate, library_side_door, library_back_door ...

STG Update

Apply Heuristic Strategy Thought Interactive Refinement Strategy (Optional)

I sat in ... (position)

Iteration 2 Iteration 3 Iteration ... 4

"Verify facilities before choosing cameras." { "status":"Success", "facilities":

Observation{ "study_desk":15,..

facilities_verify(building ="library", floor="4F", scenario=... Action

Resulting Thought

"Facilities confirmed; camera-count hypothesis...

node_1.state = Verified node_1.verified_facilities = ["study_desk", "bookshelf","sofa","atrium-side bench"] Spatial Memory

spatial_memory["facility_map | library | 4F | public_communication_area"] = ["study_desk", "bookshelf", "sofa", "atrium-side bench"]

Ending thought: all hypothses have been verified, now we got a full grounded STG

Figure 5: Workflow of the Hypothesis-Verification Loop. atomic operations mapped to these nodes. This step ensures topological coherence by constructing a valid traversal that chains these entities, such as planning a continuous route from the starting Library 4F through inferred connectors like the Elevator to the Hospital 1F. This process defines the spatial witness walk 𝜏𝑉 ,0 = (𝑣 1, . . . , 𝑣𝑚 ), enriching the graph with a hypothesized trajectory and task annotations. ➌ Spatial Reasoner. However, this planned trajectory remains speculative as it relies on high-level topological priors rather than physically grounded facts. Implicit assumptions arise wherever the semantic logic of the planner might diverge from strict physical availability (e.g., assuming a specific door is unlocked or a pathway is currently traversable). The Spatial Reasoner systematically bridges this gap by scrutinizing the entire generated trajectory to convert these implicit assumptions into explicit, deterministic, and verifiable hypotheses. In the context of the library-to-hospital transition, it detects potential ambiguity and hypothesizes that a specific, valid exit must be confirmed against the world model. Similarly, for the area covering task, it formulates hypotheses regarding the existence of specific facilities (e.g., "study desks") to narrow the intended sensing scope. Each hypothesis is cast as a concrete, verifiable proposition regarding the world model, defined with an explicit scope and a wide range of diverse admissible evidence sources. This collaborative three-agent pipeline collectively returns a comprehensive hypothesized spatial path, encapsulated in the initial STG, 𝐺 0 = (𝑉0, 𝐸 0, 𝜏𝑉 ,0, ∅), that is now fully ready for the critical and rigorous Symbolic Grounding phase.

4.3

Hypothesis-to-Fact Grounding

A detailed hypothesized graph remains non-actionable as long as its elements are unverified. The Symbolic Grounding phase converts the speculative STG, 𝐺 0 into a grounded STG, 𝐺★ by rigorously testing each hypothesis against the world model 𝑊 . As shown in Fig. 5, a "verify-before-commit" loop promotes facts, prunes contradictions, and resolves ambiguities until the graph is verifiably consistent with 𝑊 . ➍ The Hypothesis-Verification Loop. Grounding hinges on a rigorous verification loop executed by the Grounding

Verifier, operating as a tool-using agent capable of querybased interaction with the physical world model. Its primary responsibility is to eliminate ambiguity by converting semantic assumptions into verifiable topological facts. Through a Thought-Action-Observation cycle [63, 78], the agent addresses specific node hypotheses, such as identifying a valid exit in the Library, translating them into designing a corresponding topological verification query. It executes this design via invoking the crafted Verifying Toolkit (e.g., doors_verify()) to retrieve concrete spatial data from the environment. The resulting observation serves as ground truth, enabling the agent to define the precise sensor scheduling logic for that location and progressively transform the speculative skeleton into a fully grounded STG. Resolving Ambiguity. A critical challenge emerges when the retrieved observation is not definitive, such as the toolkit returning multiple valid exits for the Library. To resolve the uncertainty deterministically, IoT-Brain employs a recursive reasoning strategy based on topological consistency. Upon detecting multiple potential exits, the agent initiates a secondary check to evaluate the connectivity of each candidate relative to the subsequent waypoint (the Hospital). By filtering out exits that lack a valid traversable path to the destination, the system autonomously identifies the topologically sound option. This heuristic ensures the final plan is physically executable without human intervention, though an optional interactive strategy is available for extreme cases where user clarification is preferred. ➎ Amortized Verification via Caching. Throughout the iterative process, successfully verified facts are cached according to their associated locations in a Spatial Memory module, a technique inspired by classic indexing and memoization [37]. This mechanism effectively amortizes the computational overhead of repeated queries for the same entities (e.g., retaining the validated Library exit details for future requests), thereby significantly accelerating future verification. The verification loop terminates once all the structural hypotheses in 𝐺 0 are resolved. The final output 𝐺★ represents a verified spatial blueprint where every node, edge, and

MobiCom ’26, October 26–30, 2026, Austin, TX, USA

Zhou et al.

Input Objective

Appearance Description

...white backpack...

...wear a black shirt...

IDb IDa

IDc

...

Target Initialization VLM

First Frame

IDa

Object Found!

Mission Complete (Early Exit)

Target BBox

Object Detector

Candidate BBox

ReID Network

Target Tracking

feature vector fj Matched Target BBox

Loop

tarrive twakeup & VLM

feature vector fref

IDi

Kalman-ETA Estimation

v^k

Figure 6: Workflow of the Perception Aligner. the contained spatial path 𝜏𝑉 are consistent with the physical world, thereby providing a solid and reliable foundation for the subsequent scheduling optimization.

4.4

From Plan to Optimized Action

With a fully grounded spatial blueprint 𝐺★ in place, the final phase of IoT-Brain translates it into dynamic action and perception in the physical world. The workflow comprises two components to achieve resource optimization synergistically. The first is a deterministic compiler that synthesizes an executable and resource-aware plan. The second is an adaptive executor that manages real-time operation. ➏ Scheduling Synthesizer. With the verified spatial blueprint 𝐺★ established, the Scheduling Synthesizer translates the plan into optimized action. Functioning as a graph-tocode compiler, it transforms the grounded nodes and edges into an executable Python script. This stage operationalizes the resource optimization objective in Eq. 2 by mapping the verified sub-tasks to specialized functions within our Execution API Pool. Consequently, the validated path segment inside the Library is processed by generating a call to indoor_path_camera_search(), which utilizes a deterministic set-cover algorithm [85] to select the minimal sensor set required for full visibility. By embedding these solvers within a deterministic API layer, the compiler ensures the final schedule strictly honors the topological constraints verified in the previous phase while maximizing resource efficiency. Successful query–script pairs are cached in Programming Memory ➐ to accelerate future compilations on semantically-related tasks. ➑ Perception Aligner. A static script is insufficient for dynamic, real-world execution. The Perception Aligner therefore operates as an online executor, tightly coupling perception with control to keep only necessary sensors active. As depicted in Fig. 6, the process commences with target grounding, where a VLM anchors the user’s textual description of

the "white backpack" to a specific visual instance in an initial video frame [31, 64]. To solve the association problem of maintaining the target’s identity across a distributed camera network, we employ a robust feature extractor (e.g., a Re-ID network) to match visual appearances [27]. Concurrently, a predictive model (e.g., a Kalman filter) estimates arrival times at downstream viewpoints along the planned route, enabling just-in-time sensor activation to optimize resource usage [8, 75, 80]. Finally, the same VLM performs continuous verification, conducting frame-level reasoning to evaluate task predicates. This allows the system to intelligently decide when the query is satisfied, triggering early termination to conserve resources [4]. This unified loop reduces overhead by activating sensors only when needed and deactivating them once sufficient proof is obtained.

EVALUATION ON TOPOSENSE-BENCH

We conduct a comprehensive evaluation on our large-scale benchmark, TopoSense-Bench, to empirically validate the STG paradigm and quantify the performance of IoT-Brain. Our highlights are as follows: • IoT-Brain delivers superior reliability and efficiency. On complex tasks, it boosts task success by up to 7.4× over the classical Hierarchical planner[22, 61] and runs nearly 2× faster with 6.6× fewer prompt tokens than the searchintensive Backtracking planner[12, 54] (§5.2). • IoT-Brain scales and generalizes well. Its verification overhead grows near-linearly with query complexity, not exponentially. Furthermore, its performance gains persist across diverse foundation models, confirming the benefits stem from our architecture, not a specific backbone (§5.3). • IoT-Brain derives strength from the full STG pipeline. Ablations confirm the synergy of its core components and reveal that unverified hypotheses are actively harmful, reinforcing "verify-before-commit" as a prerequisite for reliable physical-world planning (§5.4).

5.1

Experimental Setup

The TopoSense-Bench Benchmark. To rigorously evaluate our systems, we constructed TopoSense-Bench, a largescale benchmark for the S3 problem. Its central design principle is semantic textualization, transforming raw OSM data into a structured knowledge base. We normalize ontological tags [2], generate hierarchical human-readable names [1, 36], and project geodetic coordinates into a site-local Cartesian frame, creating a unified substrate where physical topology and sensor capabilities are jointly considered. The benchmark instantiates a realistic campus spanning 33 buildings, 161 floor plans, and a dense network of 2,510 cameras. Layered on this environment is a suite of 5,250 natural language queries grounded in the operational realities of large-scale sensor networks. Since providing spatiotemporal

IoT-Brain: Grounding LLMs for Semantic-Spatial Sensor Scheduling Semantic Query

Semantic Query Thinker

Global Task Parser

Path is Physically infeasible

Program Synth -esizer Unexecutable Script

(a) Hierarchical Planner

Single -Step Object

Verifier

Loop

Semantic Query

Program Synthesizer

Sequential Verified, Locally Optimal Plan

Ungrounded Global Plan

Executable yet Incomplete Script

① ②

API Calling

Observer

(b) Reactive Agent

Lack of Global View

Task Manager Ungrounded Subtask-Set

Semantic Query

Semantic Graph Constructer

Success！

Spatial Hypothesizer

Global Graph With Hypotheses

Ungrounded Global Graph

②

③

③ ④

Environment Response

MobiCom ’26, October 26–30, 2026, Austin, TX, USA

Globally Suboptimal Path

Fail, backtrack to …

Worker Scripts with misaligned task topology semantics

Committed to a locally optimal but globally inefficient solution. (c) Backtracking Planner

Success: A single, globally consistent, and progressively verified blueprint

Executable & Complete Script

Global Consistent, Grounded Graph Program Synth -esizer

Grounding Verifier Grounding Knowledge Supplement

Verify toolset Loop calls Physical Knowledge Observer

(d) STG Paradigm

Figure 7: Conceptual workflow comparison of agentic paradigms. We contrast the brittle Hierarchical approach, the myopic Reactive agent, and the costly Backtracking planner with our structuring verifiable STG paradigm. Table 1: Statistics & Taxonomy of TopoSense-Bench. It produces a complete, high-level plan upfront without intermediate verification, yielding speed at the cost of brittleness. Knowledge Base Statistics Value (2) The Reactive planner [15, 78] instantiates the ThoughtBuildings / Floor Plans / Outdoor Segments 33 / 161 / 53 Action-Observation loop. It operates step-by-step, making it Total Topological Scenarios / Deployed Cameras 7,832 / 2,510 highly adaptive but often myopic on long-horizon tasks. (3) Query Dataset Statistics Count (%) The Backtracking planner [12, 54] is inspired by depth-first Tier 1: Intra-Zone Perception search over a decision tree. It improves reliability by explorT1.a: Focal Scene Awareness 1,433 (27.3%) ing multiple branches, but this expanded search typically e.g., "verify activity near the door to room 5F 1" incurs prohibitive inference costs and can commit to locally T1.b: Panoramic Coverage 1,129 (21.5%) optimal yet globally inefficient paths. Fig. 7 contrasts these e.g., "how many people are in the conference hall?" workflows with our STG approach. Tier 2: Intra-Building Coordination Evaluation Metrics. We assess each paradigm’s perforT2: Intra-Building Coordination 988 (18.8%) mance from the dual perspectives of reliability and efficiency. e.g., "I lost a notebook between lab-8 and lab-7" We use a suite of standardized metrics: For reliability, we Tier 3: Inter-Building Coordination measure (1) task success rate (TSR), the percentage of queries T3.a: Open-Space Coordination 946 (18.0%) yielding a functionally correct final answer, and (2) blueprint e.g., "track my path from the west gate to the tennis court" correctness (BC), which evaluates if a pre-execution plan T3.b: Hybrid Indoor-Outdoor 754 (14.4%) is logically sound and topologically feasible, scored via an e.g., "I walked from building-1 to building-2..." LLM-as-a-Judge protocol [82]. For efficiency, we quantify (3) inference cost, measured in total LLM tokens; (4) interaction anchors is a realistic prerequisite for initiating tasks, the core rounds, the number of agent–LLM turns; and (5) end-to-end 𝑆 3 challenge focuses on reasoning over the extensive topolatency, the total time from query submission to final answer. logical knowledge base to infer the complete, optimal sensor 5.2 End-to-End Performance path between these points. To ensure consistently high data quality, we employed a rigorous three-stage construction Reliability Analysis. Fig. 8(a-b) presents the TSR and BC pipeline. Domain experts first authored ∼200 base templates across all task categories and difficulty tiers. A key observaper tier to cover a wide spectrum of realistic scenarios. Using tion is that while baseline planners often struggle to translate these as seeds, GPT-o3 [51] synthesized thousands of dissymbolically correct plans into real-world success, IoT-Brain tinct queries by instantiating diverse semantic intents across consistently maintains high TSR, especially as task comthe complex campus topology. Crucially, every resulting plexity increases to long-horizon settings. This empirically query underwent comprehensive, rigorous manual expert demonstrates the STG paradigm’s effectiveness in resolving verification and ground-truth annotation to guarantee logithe fundamental gaps that plague end-to-end LLM planning. cal soundness and topological solvability, which ultimately A notable paradox emerges on simple Tier 1 tasks. The ensures the benchmark’s high fidelity. plan-less Reactive planner achieves a strong TSR, demonCompared Paradigms. We benchmark IoT-Brain against strating for simple tasks, adaptive tool-use can suffice in easy agentized implementations of three influential LLM plansettings. The Hierarchical planner, by contrast, highlights ning paradigms. To ensure comparability, all systems use the the critical gap between symbolic planning and physical exesame foundation LLMs and API toolkits, differing only in cution. It frequently produces plausible blueprints with high their reasoning and execution policies. (1) The Hierarchical BC scores, yet achieves markedly lower TSR. This discrepplanner [22, 61] follows a decompose-then-execute strategy. ancy reflects the representation and reasoning gaps, where

MobiCom ’26, October 26–30, 2026, Austin, TX, USA Hi. (71.45%) Re. (0.00%)

Ba. (71.88%) Io. (71.88%)

Token Count (×103)

100

100 BC (%)

TSR (%)

90 50

0 T1. F T1. P

T3. O T3. H

(a) Task Success Status

80 70 60 50 40 T1. F T1. P

Hi. P/C (518.89/426.50) Re. P/C (4983.76/1104.94)

T3. O T3. H

(b) Planning Success Status

Ba. P/C (7704.60/2076.68) Io. P/C (1168.43/2126.55)

Prompt Completion

Hi. R/L (3.00/64.61) Re. R/L (11.93/201.48)

10 5 3.5 2.5 1.5 0.5 T1. F

T1. P

T3. O

T3. H

Ba. R/L (19.19/378.27) Io. R/L (12.50/210.65)

Rounds Latency

600

400

200

T1. F

T1. P

T3. O

0 T3. H

End-to-End Latency (s)

Ba. (61.58%) Io. (68.51%)

Interaction Rounds

Hi. (39.13%) Re. (50.35%)

Zhou et al.

(d) Execution Time Consumption

Figure 8: The overall performance on TopoSense-Bench. We evaluate Hierarchical (Hi.), Reactive (Re.), Backtracking (Ba.), and IoT-Brain (Io.) across five task categories: T1.F (Focal Scene), T1.P (Panoramic), T2 (Intra-Building), T3.O (Open-Space), and T3.H (Hybrid). Metrics include task success rate, blueprint correctness, token usage, iteration rounds, and end-to-end latency. The legend displays the average performance for each paradigm. a plan is logically sound on paper proves physically unrealwe evaluate IoT-Brain’s performance across four foundation izable without grounding in the real world. This confirms a LLMs. As expected, more powerful models yield higher endcorrect blueprint is necessary but not sufficient for success. to-end reliability (Fig. 9(b)), confirming that the quality of the As complexity escalates to Tier 2 and 3, the necessity underlying LLM is a significant factor. More importantly, the of structured grounding becomes indisputable. The perforefficiency profile shows no orders-of-magnitude variation in mance of all baseline planners degrades sharply, which is a latency and token cost across models (Fig. 9(c)). While difclear manifestation of the reasoning gap. Their inability to ferent APIs exhibit distinct latency characteristics, the token construct coherent, long-horizon plans leads to systemic failconsumption remains relatively stable. These results indicate ure. In stark contrast, on the most complex T3.Hybrid task, STG supplies a strong scaffold that channels reasoning into IoT-Brain achieves a 46.1% TSR. This not only represents a verifiable structure, allowing less capable models to perform more than 7.4× improvement over the Hierarchical planner’s competitively and confirming that observed gains arise from 6.2% but also surpasses the strongest search-intensive alternaour architecture, not from any specific LLM’s capabilities. tive, the Backtracking planner, by 37.6%. This sustained perScalability with Query Complexity. We investigate how formance is a direct result of STG’s "verify-before-commit" IoT-Brain’s computational verification overhead scales with process, which constructs a globally coherent blueprint and query complexity, a key determinant of real-world usability. systematically closes the reasoning gap. For cached scenarios, verification overhead grows sublinEfficiency Analysis. Reliability must also be delivered efearly, as the system intelligently reuses entries from Spaficiently. The computational overhead for each framework tial Memory to amortize costs (Fig. 9(e)). In stark contrast, is reported in Fig. 8(c–d) show that the cost escalates with for novel, unverified scenarios, the number of verification task complexity for unstructured planners, exposing an optirounds increases in a near-linear fashion purely dependent mization gap. The Hierarchical planner is fast yet unreliable, on the count of distinct locations referenced in the query, while Reactive and Backtracking exhibit rapidly increasing rather than the global environment size (Fig. 9(f)). This pretoken consumption and latency as complexity grows. In purdictable and bounded growth stands in sharp contrast to the suit of reliability through exhaustive search, Backtracking exponential combinatorial blowup typical of unconstrained reaches nearly 600 s on the most complex Tier 3 task. planning and demonstrates the inherent scalability of our IoT-Brain closes this gap by casting planning as constrained structured, hypothesis-driven verification process. optimization on a verified graph, yielding a markedly differ5.4 Ablation Study ent efficiency profile. Its resource use grows more smoothly Impact of Individual Components. Fig. 9(a) quantitatively with complexity. The principled verification loop is more shows the indispensable role of each component. Notably, focused than the brute-force exploration of Backtracking removing the Grounding Verifier is catastrophic, causing and less myopic than the trial-and-error of Reactive. While the multi-scenario TSR to plummet below 10%, confirming delivering superior reliability, IoT-Brain is nearly 2× faster the necessity of a "verify-before-commit" loop to prevent on the most complex tasks and uses, on average, 6.6× fewer hallucination-prone planning. Removing the Spatial Reasoner prompt tokens than Backtracking. These results show that also severely degrades performance, especially on complex the STG paradigm not only improves reliability but also protasks, as the Verifier is consequently forced into computavides a scalable path to computational efficiency, achieving tionally expensive exhaustive checks without the Reasoner’s a win-win in both correctness and cost. focused hypotheses. Similarly, omitting the Topological An5.3 Scalability and Sensitivity Analysis chor nearly halves multi-scenario TSR, underscoring its imSensitivity to Foundation Models. To demonstrate that portance in providing the initial scaffolding for long-horizon the benefits of STG are not tied to a single proprietary model, plans. Finally, eliminating the Memory modules, while not

IoT-Brain S-TSR

w/o A. w/o R. w/o V. (a) Single Component Ablation M-TSR

S-Lat.

w/o M.

E2E Latency (s)

TSR (%)

100 w/o A.+V.

25 GL. Q. D. Ge. (b) Reliability (TSR & BC) Veri. Rounds

200

w/o A.+R.

300

IoT-Brain

M-Lat.

100

w/o R.+V. w/o A.+R.+V.

3.5 GL. Q. D. Ge. (c) Efficiency (Latency & Tokens) Veri. Rounds

500 20

(d) Combined Component Ablation

4.0

180

750

250 0 1

4.5

200

Veri. Time

4 7 10 13 Scenario Count (e) Cached Scenarios

Token Cost (k)

220

Token Cost (k)

100

Latency (s)

100

Veri. Time

Verification Time (s)

200

BC (%)

E2E Latency (s)

TSR (%)

300 50

TSR (%)

100

Verification Rounds

M-Lat.

Verification Time (s)

S-Lat.

TSR (%)

M-TSR

100

Verification Rounds

S-TSR

MobiCom ’26, October 26–30, 2026, Austin, TX, USA

BC (%)

IoT-Brain: Grounding LLMs for Semantic-Spatial Sensor Scheduling

750

500 20

250 0 1

4 7 10 13 Scenario Count (f) Unverified Scenarios

Figure 9: (a) & (d): Ablation study of IoT-Brain’s core components on single and multi scenario tasks. (A.: Anchor, R.: Reasoner, V.: Verifier, M.: Memory, S: Single-Scenario, M: Multi-Scenario). (b) & (c): Sensitivity to different LLMs. (GL.: GLM-4, Q.: Qwen-Max, D.: DeepSeek-V3, Ge.: Gemini-2.5-Flash). (e) & (f): Scalability with query complexity. impacting single-run TSR, nearly doubles latency in comTeaching Stadium Building Dining Area plex settings, demonstrating their critical value in efficiently Playground 84 No.1 150 Cameras 6 Cameras Cameras 194 Cameras amortizing repetitive verification costs. Lab Outdoor Building Area Library Synergy of Combined Components. Fig. 9(d) underscores 268 160 410 Cameras Cameras Cameras the components’ tight complementarity. A variant lacking all Student Teaching Apartments Teaching Activity Building 912 Cameras Building core modules performs as poorly as the Hierarchical baseline, Center No.2 No.3 40 135 Cameras 151 Cameras with its multi-scenario TSR collapsing to 6.4%, confirming Cameras Campus (2510 Cameras) Lab Building (268 Cameras) that the full pipeline is indispensable for robust behavior. Figure 10: The real-world testbed environment. More tellingly, a variant that isolates the Spatial Reasoner by removing its structuring and verification counterparts 6.1 Testbed Configuration performs even worse, with its multi-scenario TSR dropping Physical Environment. Our real-world testbed is a largeto a mere 5.1%. This result reveals a crucial, counter-intuitive scale university campus instrumented with 2,510 Hikvision insight that without validity checks, the Reasoner’s unveriIP cameras distributed across 11 major areas. This diverse fied hypotheses are not merely neutral but actively harmful, environment presents significant heterogeneity, where covintroducing strong, misleading biases that derail the entire erage ranges from sparse outdoor road networks to dense planning process. Collectively, these results powerfully valiindoor deployments, with the Lab Building alone hosting date our philosophy that a structured, verifiable grounding 268 cameras covering complex topologies (see Fig. 10). process is not an add-on, but the fundamental prerequisite System Deployment. The IoT-Brain1 system runs on a cenfor safe and reliable physical-world planning. tralized server equipped with two NVIDIA A100 GPUs. Its

6 EVALUATION ON A PHYSICAL TESTBED

We complement our benchmark results with an end-to-end evaluation of IoT-Brain in a large-scale, physical testbed. The experiments confirm the STG paradigm’s effectiveness in a real-world deployment, yielding a near-optimal balance of reliability and resource efficiency. Our highlights are: • IoT-Brain demonstrates strong resource efficiency, achieving a 49.84% TCR that approaches the reliability upper bound while using 4.1× less bandwidth and processing 4.2× fewer frames than the resource-agnostic method (§6.2). • IoT-Brain’s architecture is both efficient and robust. Latency profiling confirms its planning engine is swift, with dominant costs arising from task-intrinsic complexity. Being model-agnostic, the framework allows practitioners to flexibly balance performance, cost, and privacy by integrating diverse VLM backbones (§6.3).

core planning engine utilizes the Gemini-2.5-Flash API[23] for efficient reasoning, while the Perception Aligner executes on-premises using a compact and efficient visual stack that includes YOLOv8[66] for real-time detection and PersonViTB/16[29] for robust image re-identification. All components communicate over the standard campus Wi-Fi infrastructure, reflecting a practical deployment setting constrained by realistic bandwidth fluctuations.

6.2

Real-World Deployment Results

Evaluation Protocol. We evaluated end-to-end performance on 587 annotated real-world trajectories across three paradigms. In addition to our IoT-Brain framework, we considered two baselines to benchmark against both common practice and a theoretical optimum: (1) Static Scheduling, which mirrors 1 Project page: https://github.com/houqiii/IoT-Brain

MobiCom ’26, October 26–30, 2026, Austin, TX, USA

Zhou et al.

Latency (s)

Bandwidth (GB)

TFP (frames)

Static Scheduling Naive Parallel IoT-Brain (Ours)

3.61 65.64 49.84

403.42 927.99 413.69

0.138 0.540 0.131

179 704 166

conventional security practice by triggering downstream sensors using a constant-velocity pedestrian model[59], and (2) Naive Parallel Scheduling, a resource-agnostic upper bound that activates all potentially relevant cameras simultaneously. To ensure a controlled comparison, all vision–language queries were handled by a locally deployed Qwen-VL-Chat model[7]. Performance was measured using a comprehensive suite of metrics, including task completion rate (TCR), end-to-end latency, network bandwidth, and total frames processed (TFP). Performance Analysis. Tab. 2 highlights the trade-off between reliability and resource use. Static Scheduling is frugal yet brittle, reaching only 3.61% TCR and failing systematically whenever a target deviates from its pre-defined coverage. At the other extreme, Naive Parallel establishes an empirical upper bound on reliability at 65.64% TCR, but does so at untenable cost, consuming 4.1× more bandwidth and processing 4.2× more frames than our system. IoT-Brain strikes an optimal balance within this trade-off. Driven by the STG paradigm for intelligent planning and the Perception Aligner for just-in-time execution, it achieves a high TCR of 49.84%, approaching the empirical upper bound, while keeping a resource footprint comparable to the far less reliable static strategy that uses the least bandwidth and processing the fewest frames among all paradigms. Its sequential, plan-informed activation prevents the heavy data and processing burden of parallel operation. The results confirm that verifiable planning coupled with perception-aligned execution yields a near-optimal balance, delivering robust reliability with exceptional efficiency. The remaining performance gap stems primarily from inherent semantic ambiguities, where vague user descriptions preclude deterministic grounding to the static topology. Furthermore, perception limitations contribute to sporadic failures, as the underlying VLM faces challenges in identifying targets under complex real-world lighting or occlusion.

6.3

System Analysis and Sensitivity

Latency Breakdown. We profiled the end-to-end latency for the IoT-Brain pipeline to characterize its temporal behavior. The breakdown in Fig. 11(a) clarifies the cost structure of complex semantic scheduling. The two dominant phases are Symbolic Grounding and V–L Inference, which account for most of the execution time on challenging Tier 2 & 3 tasks. The high cardinality of scenarios necessitates extensive verification cycles during grounding, while a larger scheduled

(a) Latency Breakdown

100

Task Completion Rate (%) End-to-End Latency (s)

End-to-End Latency (s) of V-L Inference

Paradigm

TCR (%)

Pipeline Phases Semantic Structuring Symbolic Grounding Program Synthesis Physical Execution Perception Alignment V–L Inference

Task Completion Rate (%)

Table 2: Real-World Scheduling Paradigm Comparison.

400

60 40

200

20 0

Q-Max

Q-Chat

G-4.5V G-4V-Plus

(b) Sensitivity to VLM Foundation Models. (Q: Qwen-VL, G: GLM)

Figure 11: System performance analysis. sensor set increases the volume of frames for VLM inference. By contrast, the initial Semantic Structuring stage is remarkably efficient. Overall, this analysis reveals that the primary latency drivers are not inefficiencies in our planning engine, but rather the intrinsic complexity of the tasks, which demands substantial verification and perception effort. Sensitivity to VLM Foundation Models. The framework’s end-to-end performance is naturally influenced by the specific capabilities of the VLM selected within the Perception Aligner. To demonstrate architectural generality, we evaluate our framework with a diverse spectrum of both local open-source and proprietary API-based VLMs. As shown in Fig. 11(b), the results reveal a clear and quantifiable performance–latency trade-off. Cloud-hosted API-based models such as Qwen-VL-Max achieve higher TCR but introduce significant network communication latency, whereas the lightweight open-source Qwen-VL-Chat offers lower latency with a corresponding reduction in TCR. This empirical outcome underscores our framework’s inherent robustness. The STG paradigm, by decoupling planning from perception, is fundamentally model-agnostic, providing a stable planning scaffold that functions effectively regardless of the underlying VLM. The choice of foundation model thus becomes a configurable trade-off for deployers, allowing them to flexibly balance performance, cost, and data privacy.

DISCUSSION

Principles of Generalizability. Although our current instantiation centers on a camera network, the STG paradigm operates on a robust sensor-agnostic abstraction layer where nodes represent generic spatial coverage requirements rather than specific hardware interfaces. The LLM reasoning engine deals exclusively with logical predicates (e.g., is_covered(location)), while the adaptation to specific sensing modalities occurs entirely within the deterministic Verifying Toolkit, which encapsulates the physical constraints. For instance, replacing a camera with a microphone array or a thermal sensor only requires updating the toolkit’s geometric calculation function to validate attributes like an acoustic detection radius or thermal sensing range instead of a visual field-of-view, while the high-level semantic planning logic remains structurally identical. This modularity empowers the framework to extend to diverse IoT scenarios (e.g., audio

IoT-Brain: Grounding LLMs for Semantic-Spatial Sensor Scheduling

sensing [69] or thermal sensing [49]) without expensive retraining or fine-tuning of the core reasoning engine, though deployment in highly dynamic settings with frequent sensor outages would necessitate continual state updates [34]. System Deployment Effort. IoT-Brain is explicitly designed to streamline practical deployment and minimize engineering overhead through three strategic design choices. First, regarding topological knowledge construction, our pipeline parses standard OSM data to semi-automatically generate the sensor-integrated topological representation (encoding attributes like FOV and sensing radius), limiting manual intervention primarily to the binding of specific sensor IDs. Second, regarding model tuning, the framework is fundamentally training-free by leveraging the one-shot or few-shot prompting strategies detailed in §4, thereby eliminating the need for expensive data collection or fine-tuning. Third, regarding algorithmic integration, the system adopts a "composition over creation" approach by wrapping existing off-the-shelf solutions. Specifically, it invokes standard optimization algorithms (e.g., ILP solvers) via the API pool and integrates established models (e.g., YOLO, ReID) into the perception module, significantly reducing the overall development and maintenance complexity. Architectural Privacy-by-Design. Beyond simple policy compliance, privacy in IoT-Brain is an inherent result of its decoupled architecture. First, we enforce a principle of symbolic isolation where the planning LLM, even if cloud-based, operates solely on abstract text symbols such as Sensor_01. Crucially, the model never accesses raw privacy-sensitive sensor streams like video, audio, or thermal data, effectively creating a structural privacy air-gap. Second, the system ensures rigorous data minimization through its optimization objective. This mechanism mathematically enforces that sensors are activated only for the strictly necessary spatiotemporal windows verified by the graph, rather than performing invasive persistent monitoring. This structural guarantee remains valid regardless of the sensing modality, providing a robust blueprint for privacy-preserving intelligent sensing aligned with strict organizational requirements [43, 45].

RELATED WORK

LLMs for Sensor System Control. Recent pioneering works have begun to explore using LLMs to translate high-level human intent into executable sensor actions, typically operating within strictly constrained smart-home settings [5, 33, 40]. To mitigate the model’s unpredictability, these approaches often rely on formal grammars or predefined API templates to enhance plan robustness. However, while such methods ensure syntactic validity, they fail to address the complexity of large-scale spatial reasoning. Our work differs fundamentally by tackling the twin challenges of campusscale scalability and physical-world grounding. We shift the

MobiCom ’26, October 26–30, 2026, Austin, TX, USA

research focus from ensuring small-scale execution robustness to achieving verifiable, resource-optimal scheduling in massive deployments, where resource contention and topological constraints are paramount. LLM-based Agentic Planning. The advent of LLMs has catalyzed the development of sophisticated agentic planning frameworks, most notably including the Hierarchical [22, 61], Reactive [63, 78], and Backtracking [54, 77] paradigms. These approaches excel at reasoning within reliable digital substrates, which are typically exemplified by software APIs or coding environments where execution feedback is immediate and deterministic. However, such methods lack the mechanisms to handle the ambiguity inherent in the physical world. Our work centers on grounding language-based reasoning in this noisy reality. To achieve this, STG functions as a rigorous neurosymbolic scaffold that constrains the LLM’s reasoning process through verifiable graph construction. This architecture supplies the essential guarantees needed for reliability in high-stakes embodied settings. Language-Guided Perception. Historically, research in person re-identification has centered primarily on metric learning for image-to-image retrieval [14, 83, 84]. The recent introduction of large language models has enabled powerful cross-modal alignment, facilitating text-to-image reidentification and language-guided tracking in continuous video streams [38, 70]. Yet, most such studies presume a predefined, passive sensor stream or an exhaustively broad search space. Our Perception Aligner fundamentally challenges this passive paradigm by operating within a proactively scheduled, on-demand feed. It provides just-in-time verification to drive the next sensor activation, ensuring the system performs efficient, closed-loop active perception rather than relying on persistent, resource-intensive, and often unscalable open-domain tracking mechanisms.

CONCLUSION

We formalize Semantic-Spatial Sensor Scheduling (S3) and reveal critical gaps in LLM planning. To address these, we introduce STG, a "verify-before-commit" neurosymbolic paradigm that decouples semantic inference from deterministic scheduling. Our implementation, IoT-Brain, evaluated on our TopoSense-Bench benchmark and real-world deployment, demonstrates significant gains in both reliability and efficiency over representative planners. These contributions provide a practical blueprint for LLMs to translate high-level intent into correct physical action robustly.

ACKNOWLEDGEMENTS We thank the anonymous MobiCom reviewers and shepherd for their constructive comments. This research was supported by the China National Natural Science Foundation with No. 623B2093, No. 62441228 and Science and Technology Tackling Program of Anhui Province No.202423k09020016.

MobiCom ’26, October 26–30, 2026, Austin, TX, USA

REFERENCES [1] 2025. Nominatim API Manual (latest). https://nominatim.org/releasedocs/latest/api/Overview/. [2] 2025. OpenStreetMap Taginfo. https://taginfo.openstreetmap.org/. [3] Avinash Achar, Dhivya Bharathi, B. Anil Kumar, and Lelitha Devi Vanajakshi. 2020. Bus Arrival Time Prediction: A Spatial Kalman Filter Approach. IEEE Transactions on Intelligent Transportation Systems 21 (2020), 1298–1307. https://api.semanticscholar.org/CorpusID: 182478323 [4] Aishwarya Agrawal, Jiasen Lu, Stanislaw Antol, Margaret Mitchell, C. Lawrence Zitnick, Devi Parikh, and Dhruv Batra. 2015. VQA: Visual Question Answering. International Journal of Computer Vision 123 (2015), 4 – 31. https://api.semanticscholar.org/CorpusID:3180429 [5] Harith Al-Safi, Harith S. Ibrahim, and Paul Steenson. 2025. Vega: LLMDriven Intelligent Chatbot Platform for Internet of Things Control and Development. Sensors (Basel, Switzerland) 25 (2025). https://api. semanticscholar.org/CorpusID:279531727 [6] Raghav Arora, Shivam Singh, Karthik Swaminathan, Ahana Datta, Snehasis Banerjee, B. Bhowmick, Krishna Murthy Jatavallabhula, Mohan Sridharan, and Madhava Krishna. 2024. Anticipate & Act: Integrating LLMs and Classical Planning for Efficient Task Execution in Household Environments†. 2024 IEEE International Conference on Robotics and Automation (ICRA) (2024), 14038–14045. https: //api.semanticscholar.org/CorpusID:271799905 [7] Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-VL: A Frontier Large Vision-Language Model with Versatile Abilities. ArXiv abs/2308.12966 (2023). https://api.semanticscholar.org/CorpusID: 263875678 [8] Stefano Basagni, Federico Ceccarelli, Chiara Petrioli, Nithila Raman, and Abhimanyu Venkatraman Sheshashayee. 2019. Wake-up Radio Ranges: A Performance Study. 2019 IEEE Wireless Communications and Networking Conference (WCNC) (2019), 1–6. https: //api.semanticscholar.org/CorpusID:202548980 [9] Will Brackenbury, Abhimanyu Deora, Jillian Ritchey, Jason Vallee, Weijia He, Guan Wang, Michael L. Littman, and Blase Ur. 2019. How Users Interpret Bugs in Trigger-Action Programming. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (2019). https://api.semanticscholar.org/CorpusID:140242523 [10] Jiarui Cai, Mingze Xu, Wei Li, Yuanjun Xiong, Wei Xia, Zhuowen Tu, and Stefan 0 Soatto. 2022. MeMOT: Multi-Object Tracking with Memory. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022), 8080–8090. https://api.semanticscholar. org/CorpusID:247839756 [11] Ai Chen, Santosh Kumar, and Ten-Hwang Lai. 2007. Designing localized algorithms for barrier coverage. In ACM/IEEE International Conference on Mobile Computing and Networking. https://api.semanticscholar. org/CorpusID:2864152 [12] Junzhi Chen, Juhao Liang, and Benyou Wang. 2024. Smurfs: MultiAgent System using Context-Efficient DFSDT for Tool Planning. In North American Chapter of the Association for Computational Linguistics. https://api.semanticscholar.org/CorpusID:269635774 [13] Liyi Chen, Panrong Tong, Zhongming Jin, Ying Sun, Jieping Ye, and Huixia Xiong. 2024. Plan-on-Graph: Self-Correcting Adaptive Planning of Large Language Model on Knowledge Graphs. ArXiv abs/2410.23875 (2024). https://api.semanticscholar.org/CorpusID:273707190 [14] Yanbei Chen, Xiatian Zhu, and Shaogang Gong. 2017. Person Reidentification by Deep Learning Multi-scale Representations. 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) (2017), 2590–2600. https://api.semanticscholar.org/CorpusID:4729614

Zhou et al. [15] Ziyang Chen, Zhangli Zhou, Lin Li, and Zheng Kan. 2024. Active Inference for Reactive Temporal Logic Motion Planning. 2024 IEEE International Conference on Robotics and Automation (ICRA) (2024), 2520–2526. https://api.semanticscholar.org/CorpusID:271798591 [16] Ye Cheng, Minghui Xu, Yue Zhang, Kun Li, Ruoxi Wang, and Lian Yang. 2024. AutoIoT: Automated IoT Platform Using Large Language Models. IEEE Internet of Things Journal 12 (2024), 13644–13656. https: //api.semanticscholar.org/CorpusID:274131336 [17] DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948 https://arxiv. org/abs/2501.12948 [18] Fiona M. Donald, Craig H. M. Donald, and Andrew Thatcher. 2015. Work exposure and vigilance decrements in closed circuit television surveillance. Applied ergonomics 47 (2015), 220–8. https: //api.semanticscholar.org/CorpusID:25574518 [19] J. Feng, Yuwei Du, Tianhui Liu, Siqi Guo, Yuming Lin, and Yong Li. 2024. CityGPT: Empowering Urban Spatial Cognition of Large Language Models. ArXiv abs/2406.13948 (2024). https://api.semanticscholar.org/ CorpusID:270619725 [20] J. Feng, Shengyuan Wang, Tianhui Liu, Yanxin Xi, and Yong Li. 2025. UrbanLLaVA: A Multi-modal Large Language Model for Urban Intelligence with Spatial Reasoning and Understanding. ArXiv abs/2506.23219 (2025). https://api.semanticscholar.org/CorpusID: 280010693 [21] Yi Gao, Kaijie Xiao, Fu Li, Weifeng Xu, Jiaming Huang, and Wei Dong. 2024. ChatIoT: Zero-code Generation of Trigger-action Based IoT Programs. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8 (2024), 1 – 29. https://api.semanticscholar. org/CorpusID:272563565 [22] Yingqiang Ge, Wenyue Hua, Kai Mei, Jianchao Ji, Juntao Tan, Shuyuan Xu, Zelong Li, and Yongfeng Zhang. 2023. OpenAGI: When LLM Meets Domain Experts. ArXiv abs/2304.04370 (2023). https://api. semanticscholar.org/CorpusID:258049306 [23] Google DeepMind. 2025. Gemini 2.5 Flash: Model Card. Technical Report. Google. https://storage.googleapis.com/model-cards/documents/ gemini-2.5-flash.pdf [24] Google DeepMind. 2025. Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities. arXiv abs/2507.06261 (2025). https: //arxiv.org/abs/2507.06261 [25] Mordechai (Muki) Haklay and Patrick Weber. 2008. OpenStreetMap: User-Generated Street Maps. IEEE Pervasive Computing 7 (2008), 12–18. https://api.semanticscholar.org/CorpusID:16588111 [26] Helen M. Hodgetts, François Vachon, Cindy Chamberland, and Sébastien Tremblay. 2017. See No Evil: Cognitive Challenges of Security Surveillance and Monitoring. Journal of applied research in memory and cognition 6 (2017), 230–243. https://api.semanticscholar. org/CorpusID:261257329 [27] Bin Hu, Xinggang Wang, and Wenyu Liu. 2024. PersonViT: Large-scale Self-supervised Vision Transformer for Person Re-Identification. Mach. Vis. Appl. 36 (2024), 32. https://api.semanticscholar.org/CorpusID: 271854919 [28] Justin Huang and Maya Cakmak. 2015. Supporting mental model accuracy in trigger-action programming. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (2015). https://api.semanticscholar.org/CorpusID:207225561 [29] HUST Vision and Learning Group. 2025. PersonViT. https://github. com/hustvl/PersonViT. GitHub repository. [30] Riheng Jia, Jinhao Wu, Xiong Wang, Jianfeng Lu, Feilong Lin, Zhonglong Zheng, and Minglu Li. 2023. Energy Cost Minimization in Wireless Rechargeable Sensor Networks. IEEE/ACM Transactions on Networking 31 (2023), 2345–2360. https://api.semanticscholar.org/

IoT-Brain: Grounding LLMs for Semantic-Spatial Sensor Scheduling CorpusID:257298122 [31] Jiayu Jiang, Changxing Ding, Wentao Tan, Junhong Wang, Jin Tao, and Xiangmin Xu. 2025. Modeling Thousands of Human Annotators for Generalizable Text-to-Image Person Re-identification. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025), 9220–9230. https://api.semanticscholar.org/CorpusID:276961622 [32] Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. A Survey on Large Language Models for Code Generation. ArXiv abs/2406.00515 (2024). https://api.semanticscholar.org/CorpusID: 270214176 [33] Evan King, Haoxiang Yu, Sangsu Lee, and Christine Julien. 2024. Sasha: Creative Goal-Oriented Reasoning in Smart Homes with Large Language Models. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 8, 1, Article 12 (2024), 38 pages. doi:10.1145/3643505 [34] Tomavs Krajnik, Jaime Pulido Fentanes, Marc Hanheide, and Tom Duckett. 2016. Persistent localization and life-long mapping in changing environments using the Frequency Map Enhancement. 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2016), 4558–4563. https://api.semanticscholar.org/CorpusID: 4969989 [35] Santosh Kumar, Ten-Hwang Lai, and Anish Arora. 2005. Barrier coverage with wireless sensors. Wireless Networks 13 (2005), 817–834. https://api.semanticscholar.org/CorpusID:565989 [36] Carolin Lawrence and Stefan Riezler. 2016. NLmaps: A Natural Language Interface to Query OpenStreetMap. In COLING 2016 System Demonstrations. [37] Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Kuttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. RetrievalAugmented Generation for Knowledge-Intensive NLP Tasks. ArXiv abs/2005.11401 (2020). https://api.semanticscholar.org/CorpusID: 218869575 [38] Yunhao Li, Xiaoqiong Liu, Luke Liu, Heng Fan, and Libo Zhang. 2024. LaMOT: Language-Guided Multi-Object Tracking. ArXiv abs/2406.08324 (2024). https://api.semanticscholar.org/CorpusID: 270391880 [39] Jie Lin, Wei Yu, Nan Zhang, Xinyu Yang, Hanlin Zhang, and Wei Zhao. 2017. A Survey on Internet of Things: Architecture, Enabling Technologies, Security and Privacy, and Applications. IEEE Internet of Things Journal 4 (2017), 1125–1142. https://api.semanticscholar.org/ CorpusID:31245252 [40] Kaiwei Liu, Bufang Yang, Lilin Xu, Yunqi Guo, Guoliang Xing, Xian Shuai, Xiaozhe Ren, Xin Jiang, and Zhenyu Yan. 2025. TaskSense: A Translation-like Approach for Tasking Heterogeneous Sensor Systems with LLMs. Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems (2025). https://api.semanticscholar.org/ CorpusID:278326090 [41] Zhaoyang Liu, Zeqiang Lai, Zhangwei Gao, Erfei Cui, Xizhou Zhu, Lewei Lu, Qifeng Chen, Yu Qiao, Jifeng Dai, and Wenhai Wang. 2023. ControlLLM: Augment Language Models with Tools by Searching on Graphs. ArXiv abs/2310.17796 (2023). https://api.semanticscholar.org/ CorpusID:264555643 [42] Huan Ma, Changqing Zhang, Yatao Bian, Lemao Liu, Zhirui Zhang, Peilin Zhao, Shu Zhang, H. Fu, Qinghua Hu, and Bing Wu. 2023. Fairness-guided Few-shot Prompting for Large Language Models. ArXiv abs/2303.13217 (2023). https://api.semanticscholar.org/ CorpusID:257687840 [43] Tinashe Magara and Yousheng Zhou. 2024. Internet of Things (IoT) of Smart Homes: Privacy and Security. J. Electr. Comput. Eng. 2024 (2024), 1–17. https://api.semanticscholar.org/CorpusID:269065642 [44] Alexandre Marois, Daniel Lafond, Alexandre Williot, François Vachon, and Sébastien Tremblay. 2020. Real-Time Gaze-Aware Cognitive

MobiCom ’26, October 26–30, 2026, Austin, TX, USA Support System for Security Surveillance. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 64 (2020), 1145 – 1149. https://api.semanticscholar.org/CorpusID:231876226 [45] Francesca Meneghello, Matteo Calore, Daniel Zucchetto, Michele Polese, and Andrea Zanella. 2019. IoT: Internet of Threats? A Survey of Practical Security Vulnerabilities in Real IoT Devices. IEEE Internet of Things Journal 6 (2019), 8182–8201. https://api.semanticscholar. org/CorpusID:201889124 [46] Hamid Menouar, Ismail Guvenc, Kemal Akkaya, Arif Selcuk Uluagac, Abdullah Kadri, and Adem Tuncer. 2017. UAV-Enabled Intelligent Transportation Systems for the Smart City: Applications and Challenges. IEEE Communications Magazine 55 (2017), 22–28. https://api.semanticscholar.org/CorpusID:38330180 [47] Meta AI. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv abs/2307.09288 (2023). https://arxiv.org/abs/2307.09288 [48] Phillip L. Morgan, Emily Collins, Tasos Spiliotopoulos, David J. Greeno, and Dylan M. Jones. 2022. Reducing risk to security and privacy in the selection of trigger-action rules: Implicit vs. explicit priming for domestic smart devices. Int. J. Hum. Comput. Stud. 168 (2022), 102902. https://api.semanticscholar.org/CorpusID:251341078 [49] Naser Hossein Motlagh. 2021. How Low Can You Go? Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5 (2021), 1 – 22. https://api.semanticscholar.org/CorpusID:248245897 [50] OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL] doi:10.48550/arXiv.2303.08774 [51] OpenAI. 2025. o3 and o4-mini System Card. System Card. OpenAI. https://cdn.openai.com/pdf/2221c875-02dc-4789-800be7758f3722c1/o3-and-o4-mini-system-card.pdf [52] Serge Pelletier, Joel Suss, François Vachon, and Sébastien Tremblay. 2015. Atypical Visual Display for Monitoring Multiple CCTV Feeds. Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (2015). https: //api.semanticscholar.org/CorpusID:304177 [53] Kai Peng, Hualong Huang, Muhammad Bilal, and Xiaolong Xu. 2023. Distributed Incentives for Intelligent Offloading and Resource Allocation in Digital Twin Driven Smart Industry. IEEE Transactions on Industrial Informatics 19 (2023), 3133–3143. https://api.semanticscholar. org/CorpusID:249911526 [54] Yujia Qin, Shi Liang, Yining Ye, Kunlun Zhu, Lan Yan, Ya-Ting Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Runchu Tian, Ruobing Xie, Jie Zhou, Marc H. Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2023. ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs. ArXiv abs/2307.16789 (2023). https://api.semanticscholar.org/CorpusID:260334759 [55] Changle Qu, Sunhao Dai, Xiaochi Wei, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Jun Xu, and Jirong Wen. 2024. Tool Learning with Large Language Models: A Survey. ArXiv abs/2405.17935 (2024). https: //api.semanticscholar.org/CorpusID:270067624 [56] Brian Reily, Terran Mott, and Hao Zhang. 2020. Adaptation to Team Composition Changes for Heterogeneous Multi-Robot Sensor Coverage. 2021 IEEE International Conference on Robotics and Automation (ICRA) (2020), 9051–9057. https://api.semanticscholar.org/CorpusID: 229297612 [57] Zhiwei Ren, Junbo Li, Minjia Zhang, Di Wang, Xiaoran Fan, and Longfei Shangguan. 2025. Toward Sensor-In-the-Loop LLM Agent: Benchmarks and Implications. Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems (2025). https://api. semanticscholar.org/CorpusID:278326126 [58] Stefano De Sabbata, Stefano Mizzaro, and Kevin Roitero. 2025. Geospatial Mechanistic Interpretability of Large Language Models. ArXiv abs/2505.03368 (2025). https://api.semanticscholar.org/CorpusID: 278339325

MobiCom ’26, October 26–30, 2026, Austin, TX, USA [59] Christoph Schöller, Vincent Aravantinos, Florian Samuel Lay, and Alois Knoll. 2019. The Simpler the Better: Constant Velocity for Pedestrian Motion Prediction. ArXiv abs/1903.07933 (2019). https: //api.semanticscholar.org/CorpusID:83458829 [60] Leming Shen, Qian Yang, Xinyu Huang, Zijing Ma, and Yuanqing Zheng. 2025. GPIoT: Tailoring Small Language Models for IoT Program Synthesis and Development. Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems (2025). https: //api.semanticscholar.org/CorpusID:276742179 [61] Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yue Ting Zhuang. 2023. HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face. ArXiv abs/2303.17580 (2023). https://api.semanticscholar.org/CorpusID:257833781 [62] Kyujin Shim, Sungjoon Yoon, Kangwook Ko, and Changick Kim. 2021. Multi-Target Multi-Camera Vehicle Tracking for City-Scale Traffic Management. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2021), 4188–4195. https://api.semanticscholar.org/CorpusID:235632675 [63] Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learning. In Neural Information Processing Systems. https://api.semanticscholar.org/CorpusID:258833055 [64] Wentao Tan, Changxing Ding, Jiayu Jiang, Fei Wang, Yibing Zhan, and Dapeng Tao. 2024. Harnessing the Power of MLLMs for Transferable Text-to-Image Person ReID. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024), 17127–17137. https: //api.semanticscholar.org/CorpusID:269626531 [65] Zheng Tang, Milind R. Naphade, Ming-Yu Liu, Xiaodong Yang, Stan Birchfield, Shuo Wang, Ratnesh Kumar, D. Anastasiu, and Jenq-Neng Hwang. 2019. CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019), 8789–8798. https://api.semanticscholar.org/CorpusID:85459559 [66] Ultralytics. 2025. Ultralytics YOLOv8. https://github.com/ultralytics/ ultralytics. GitHub repository. [67] Blase Ur, Melwyn Pak Yong Ho, Stephen Brawner, Jiyun Lee, Sarah Mennicken, Noah Picard, Diane Schulze, and Michael L. Littman. 2016. Trigger-Action Programming in the Wild: An Analysis of 200,000 IFTTT Recipes. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (2016). https://api.semanticscholar.org/ CorpusID:10883440 [68] Hanqing Wang, Wenguan Wang, Wei Liang, Caiming Xiong, and Jianbing Shen. 2021. Structured Scene Memory for Vision-Language Navigation. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021), 8451–8460. https://api.semanticscholar. org/CorpusID:232135021 [69] Jiang Wang, Yuanzheng He, Daobilige Su, Katsutoshi Itoyama, Kazuhiro Nakadai, Junfeng Wu, Shoudong Huang, Youfu Li, and He Kong. 2024. SLAM-Based Joint Calibration of Multiple Asynchronous Microphone Arrays and Sound Source Localization. IEEE Transactions on Robotics 40 (2024), 4024–4044. https://api.semanticscholar.org/ CorpusID:270123565 [70] Dongming Wu, Wencheng Han, Tiancai Wang, Xingping Dong, Xiangyu Zhang, and Jianbing Shen. 2023. Referring Multi-Object Tracking. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2023), 14633–14642. https://api.semanticscholar.org/ CorpusID:257365320 [71] Duo Wu, Jinghe Wang, Yuan Meng, Yanning Zhang, Le Sun, and Zhi Wang. 2024. CATP-LLM: Empowering Large Language Models for Cost-Aware Tool Planning. ArXiv abs/2411.16313 (2024). https://api. semanticscholar.org/CorpusID:274234379

Zhou et al. [72] Minghu Wu, Yeqiang Qian, Chunxiang Wang, and Ming Yang. 2021. A Multi-Camera Vehicle Tracking System based on City-Scale Vehicle Re-ID and Spatial-Temporal Information. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2021), 4072–4081. https://api.semanticscholar.org/CorpusID:235702604 [73] Sixu Wu, Haipeng Dai, Linfeng Liu, Lijie Xu, Fu Xiao, and Jia Xu. 2024. Cooperative Scheduling for Directional Wireless Charging With Spatial Occupation. IEEE Transactions on Mobile Computing 23 (2024), 286–301. https://api.semanticscholar.org/CorpusID:253343259 [74] Huatao Xu, Liying Han, Qirui Yang, Mo Li, and Mani B. Srivastava. 2023. Penetrative AI: Making LLMs Comprehend the Physical World. Proceedings of the 25th International Workshop on Mobile Computing Systems and Applications (2023). https://api.semanticscholar.org/ CorpusID:264145826 [75] Antoine Yang, Antoine Miech, Josef Sivic, Ivan Laptev, and Cordelia Schmid. 2022. Learning to Answer Visual Questions From Web Videos. IEEE Transactions on Pattern Analysis and Machine Intelligence 47 (2022), 3202–3218. https://api.semanticscholar.org/CorpusID:248570072 [76] Fan Yang, Dung-Han Lee, John Keller, and Sebastian A. Scherer. 2021. Graph-based Topological Exploration Planning in Large-scale 3D Environments. 2021 IEEE International Conference on Robotics and Automation (ICRA) (2021), 12730–12736. https://api.semanticscholar.org/ CorpusID:232428138 [77] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of Thoughts: Deliberate Problem Solving with Large Language Models. ArXiv abs/2305.10601 (2023). https://api.semanticscholar.org/CorpusID:258762525 [78] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. ReAct: Synergizing Reasoning and Acting in Language Models. ArXiv abs/2210.03629 (2022). https: //api.semanticscholar.org/CorpusID:252762395 [79] Xiaofan Yu, Lanxiang Hu, Benjamin Z. Reichman, Dylan Chu, Rushil Chandrupatla, Xiyuan Zhang, Larry Heck, and Tajana Rosing. 2025. SensorChat: Answering Qualitative and Quantitative Questions during Long-Term Multimodal Sensor Interactions. ArXiv abs/2502.02883 (2025). https://api.semanticscholar.org/CorpusID:276116215 [80] Zhou Yu, D. Xu, Jun Yu, Ting Yu, Zhou Zhao, Yueting Zhuang, and Dacheng Tao. 2019. ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering. ArXiv abs/1906.02467 (2019). https://api.semanticscholar.org/CorpusID:69645185 [81] Tony Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate Before Use: Improving Few-Shot Performance of Language Models. In International Conference on Machine Learning. https://api. semanticscholar.org/CorpusID:231979430 [82] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Haotong Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLMas-a-judge with MT-Bench and Chatbot Arena. ArXiv abs/2306.05685 (2023). https://api.semanticscholar.org/CorpusID:259129398 [83] Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, and Tao Xiang. 2019. Learning Generalisable Omni-Scale Representations for Person ReIdentification. IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (2019), 5056–5069. https://api.semanticscholar.org/ CorpusID:204575830 [84] Kaiyang Zhou, Yongxin Yang, Andrea Cavallaro, and Tao Xiang. 2019. Omni-Scale Feature Learning for Person Re-Identification. 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (2019), 3701–3711. https://api.semanticscholar.org/CorpusID:145050804 [85] Xiaojian Zhu, Mengchu Zhou, and Abdullah M. Abusorrah. 2022. Optimizing Node Deployment in Rechargeable Camera Sensor Networks for Full-View Coverage. IEEE Internet of Things Journal 9 (2022), 11396– 11407. https://api.semanticscholar.org/CorpusID:243947717

Record · ID 2492 · SHA-256 2018b12dc3b12982
Conceptio Open Knowledge Archive — every document is proof-bundled with source, license, and retrieval metadata.