arXiv:2609.21103v1 [cs.CR] 17 Sep 2026
NetInspector: Measuring and Improving LLM Capabilities for Reliable Intent-Based Networking Policy Generation Yuxuan Zhang
Hongxin Hu
Guofei Gu
Texas A&M University [email protected]
University at Buffalo [email protected]
Texas A&M University [email protected]
Abstract—Modern networks are large in scale and heterogeneous in configuration, making manual policy management increasingly impractical. Intent-Based Networking (IBN) addresses this by automating the translation of high-level operator goals into low-level network configurations. Yet existing IBN systems rely on static heuristics and fixed-feature classifiers that generalize poorly to distribution shifts such as new service definitions or evolving phrasing in operator requests. This brittleness resulted in security threats where intents that conflict with existing security policies can silently pass the resolution check and reach the network, producing misconfigurations with real operational consequences. Large Language Models (LLMs), with strong reasoning and translation capabilities demonstrated across many domains, are a natural candidate for IBN policy generation. However, it is unclear whether LLMs can be reliably applied to this task, nor whether their use mitigates or worsens the underlying security risk. In this work, we bridge this gap with a systematic measurement study of LLMs in the IBN policy generation pipeline. Our study shows that while fine-tuned LLMs excel at intent translation, they exhibit false negative rates of 35–48% when checking whether a proposed intent violates an existing security policy. The root cause is not a lack of logical reasoning capability, but a lack of persistent grounding in network topology and group hierarchy: LLMs cannot reliably navigate the large network state to retrieve the facts their reasoning depends on. Motivated by this finding, we introduce NetInspector, a three-layer agentic framework that enforces a verify-then-act protocol, decoupling information retrieval from reasoning so that the LLM focuses on symbolic reasoning while every policy decision is grounded in verifiable network facts retrieved from a live Environment Layer before approval. On NetInspectorBench, a 2,224-sample synthetic benchmark spanning campus, enterprise, and WAN topologies, NetInspector reduces FNR by over 30% relative to ungrounded baselines and remains robust under linguistic distribution shifts.
1. Introduction The increasing scale and heterogeneity of modern networks have made manual configuration a primary source of security vulnerabilities[9, 8, 11]. Traditional manage-
ment relies heavily on human experts to translate highlevel network management and security goals (e.g., isolate guest traffic) into low-level enforcement primitives (e.g., ACLs, VLAN tags). This manual workflow is not only time-consuming but fundamentally error-prone; a single misconfiguration can lead to open attack surfaces, routing loops, or service outages [8]. To mitigate these risks, IntentBased Networking (IBN) has emerged as a paradigm shift, moving the control plane from imperative configuration to declarative policy management. By allowing operators to specify “what” is needed rather than “how” to implement it, IBN promises to automate the enforcement of security invariants and reduce the vulnerabilities caused by human error. 1. Intent Verification 2. Conflict Detection
Submit Intent Administrative Practitioners
Intent Translation
Deployable Intents Intent Resolution
Policy Deployer
Figure 1. Illustration of the IBN Policy Generation process
However, the security assurance of an IBN system is entirely dependent on the integrity of its policy generation pipeline. As defined by Leivadeas et al. [21], realizing IBN involves complex translation stages, converting natural language intent into formal specifications (e.g., Nile [36]) before deployment. As illustrated in Figure 1, this process hinges on two critical functions: Intent Translation (parsing the request) and Intent Resolution (verifying the requested policy against the existing security policies in deployment). While Translation ensures the system understands the user, Intent Resolution acts as the security enforcement layer—the last automated checkpoint before deployment—responsible for blocking any intent that would violate a standing security policy. A missed violation can silently bypass an access restriction, redirect traffic across untrusted segments, or over-commit shared resources, and the resulting misconfigurations can lead to real operational and security incidents in production networks [9]. If this stage fails, the automation engine becomes a direct vector for committing unsafe policy changes to the network. To secure this pipeline, prior work has relied largely on static heuristics and rigid rule-based parsers. In Intent Translation, systems typically extract entities into predefined
1
F1 Score
0.9
Lumi-Bench NetInspector-Bench
0.8 0.7 0.6 0.5 0.4
(a) Visualization of data distribution shift on services included.
Lumi F1 Score
(b) Lumi performance with and without distribution shift.
Figure 2. Existing state-of-art approach (Lumi[15]) shows limited robustness under data distribution shift. Figure(a) shows the distribution shift between our benchmark and Lumi’s benchmark. Figure(b) presents Lumi’s performance in F1 score on both datasets.
templates using regular expressions [33]. For Intent Resolution, conflict detection often depends on extracting static numerical features to train classifiers. While early Machine Learning (ML) approaches [14, 38] and RNN-based models proposed by Jacobs et al. [15] improved flexibility, they heavily rely on models trained on fixed feature sets. From a security perspective, this reliance on static heuristics represents a significant vulnerability: these models are brittle and fail to generalize under distribution shifts. As shown in Figure 2 (a), a shift in service definitions or phrasing, common in evolving network environments, can render these static detectors obsolete. Figure 2 (b) demonstrates the consequence: the failure to detect such conflicts increases drastically, allowing unsafe intents to bypass the resolution check and reach the network as real misconfigurations. To overcome this brittleness, a growing body of work [7, 5, 24, 6, 12] has explored Large Language Models (LLMs) as policy generators in IBN, leveraging their semantic flexibility to map diverse natural-language intents into structured policies without rigid templates. These efforts, however, primarily showcase that LLMs are capable of producing syntactically valid policies; they do not investigate whether the policies LLMs generate can reliably avoid violating any existing security policies—the central concern once LLMs are placed on the security enforcement path. In this paper, we seek to address the central research question: Can LLMs reliably catch intents that violate an existing security policy before deployment? To this end, we conduct a systematic measurement study of LLMs across the IBN policy generation pipeline. On a synthetic benchmark constructed from real network topologies, we compare representative prompting and fine-tuning strategies against established non-LLM baselines on both Intent Translation and Intent Resolution. The results show that although finetuned LLMs translate natural-language intents into structured policies with high accuracy, their false negative rate on detecting security-policy violations remains at 35–48%. A closer examination indicates that this failure does not stem from limitations in logical reasoning. Rather, LLMs lack
persistent grounding in the network state and cannot reliably navigate the topology and group hierarchy to retrieve the facts on which their reasoning depends. Building on this insight, we propose NetInspector, a three-layer agentic framework that sits between operator intent and policy deployment. NetInspector enforces a verifythen-act protocol that decouples information retrieval from reasoning: a dedicated Environment Layer holds a live, queryable representation of the network state, and the LLM only commits to a policy decision after explicitly retrieving and reasoning over the relevant facts. This decoupling lets the LLM concentrate on the symbolic reasoning it is wellsuited for, while removing the burden of navigating the large network state. Across campus, enterprise, and WAN topologies in our benchmark, NetInspector reduces FNR by over 30% relative to ungrounded baselines and remains robust under linguistic distribution shifts, closing the security gap left by prior approaches. We summarize our contributions as follows: 1)
2)
3)
We conduct the first systematic evaluation of LLMs as security-policy checkers in IBN policy generation, revealing that their inability to reason over network topology and group hierarchies produces FNR of 35–48%, leaving the majority of unsafe intents undetected before deployment. We propose NetInspector, a three-layer agentic framework built on a verify-then-act design principle. By grounding LLM reasoning in verifiable network state, NetInspector reduces the FNR for catching policy-violating intents by over 30 percentage points versus ungrounded baselines, while maintaining robustness under distribution shifts. We open-source NetInspector-Bench, a 2,224sample benchmark spanning campus, enterprise, and WAN topologies with six categories of security-policy violations—such as privilege escalation, isolation bypass, and denial of service— enabling reproducible evaluation of securityoriented IBN policy enforcement.
2. Problem Statement and Related Work System Model. We consider an Intent-Based Networking (IBN) system through which network operators (e.g., administrators, lab managers, or service owners) submit naturallanguage intents inew ∈ Σ∗ to manage the underlying network, modeled as a state tuple σ = (G, H, c) that captures the physical topology G = (V, E), the groupcontainment hierarchy H (a DAG over user and resource groups), and the link-capacity function c : E → R+ . A typical intent expresses a high-level goal (e.g., “allow Engineering hosts to reach the project file server during business hours”). A translation function fT maps this utterance into a candidate policy Pnew = fT (inew ), expressed as a tuple (s, a, o, τ ) encoding a subject, an action (e.g., ALLOW, DENY, RATE_LIMIT), an object, and a condition such as a temporal scope or priority. Before Pnew is committed to the
network as low-level enforcement primitives (ACLs, VLAN tags, quality-of-service reservations), it must pass through the Intent Resolution stage—the last automated checkpoint that verifies whether the requested change is compatible with the standing security-policy library L = {P1 , . . . , PN } maintained by the security team. From the operator’s perspective, the interaction surface is the natural-language intent alone: the burden of materializing fT , accessing σ , and cross-checking Pnew against the full library L is delegated entirely to the system. Threat Model. We assume a benign but fallible insider: the operator is a legitimate user whose intent is not necessarily malicious, but whose knowledge of L and σ is necessarily incomplete in any realistically sized deployment. The threats we consider therefore arise not from adversarial intent injection, but from unintentional conflicts between Pnew and one or more standing policies—conflicts that the operator is in no position to foresee at submission time. Formally, we say Pnew violates L under σ whenever ∃ Pi ∈ L : Conflict(Pnew , Pi ; σ), where Conflict(·) is a class-specific predicate capturing the semantics of a particular violation type. Representative instantiations include privilege escalation [30, 39, 34, 23], where an over-broad access rule subsumes a previously restricted subgroup; isolation bypass [16, 17], where a forwarding rule traverses an untrusted segment; and denial of service [26, 3], where a reservation combined with existing allocations exceeds physical link capacity. The predicate set is open by construction: additional violation classes can be admitted without changing the surrounding framework. When the Intent Resolution stage fails to detect such a violation, the conflicting policy is silently committed to the network. Section 5.7 presents a detailed example of each representative class. Detection Objective. A reliable security-policy verification system for IBN can be modeled as a function v : (Pnew , L, σ) → {C ONFLICT, S AFE}, and a deployable instance of v must satisfy three requirements. First, it should accept natural-language intents in the form operators actually submit, without forcing them to enumerate or know the contents of L—that is, the system must materialize the translation function fT end-to-end. Second, every decision must be grounded in σ —topology, group membership, deployed access rules, and current resource allocations—rather than reasoning over surface syntax alone. Third, v should approach two ideal correctness properties: soundness, that v(·) = C ONFLICT implies ∃ Pi ∈ L : Conflict(Pnew , Pi ; σ), so that benign intents are not unnecessarily rejected; and completeness, that ∃ Pi ∈ L : Conflict(Pnew , Pi ; σ) implies v(·) = C ONFLICT, so that violating intents are reliably blocked before deployment. These requirements jointly motivate the design of NetInspector, presented in Section 4.3. Related Work. Over the past decade, a wide range of research has investigated how to implement IBN in practice. Existing approaches can be broadly categorized into nonLLM-based methods and LLM-based solutions, each with distinct focuses and limitations. Policy Generation in IBN. Earlier works in IBN primarily relied on rule-based systems, traditional natural lan-
guage processing, and machine learning techniques. Kim et al. proposed IBCS, an intent-based framework tailored for cloud security services, emphasizing policy translation and automation for secure service provisioning [18]. Similarly, Han et al. developed an intent-based virtualization platform for SDN, focusing on transforming user-defined intents into virtual network configurations [13]. While these systems provided a degree of automation, they typically required domain-specific expertise and lacked generalizability. Tuncer et al. introduced a northbound interface for softwaredefined networks, aiming to bridge the gap between user policies and low-level configurations using predefined templates [33]. Jacobs et al. explored refining network intents to adapt to evolving conditions, enhancing system flexibility [14]. However, these approaches often depended on rigid schemas and static rules, which limited their scalability. Yang et al. extended IBN concepts to optical networks by incorporating AI for automated operation and maintenance [38], while Savi and Siracusa proposed serviceaware provisioning for multi-layer transport networks [28]. Although effective in specialized domains, these systems did not generalize well across heterogeneous networks. Li et al. provided a comprehensive survey on deep learning for named entity recognition (NER), a foundational technique in parsing user intents [22]. Scheid et al. proposed the use of controlled natural languages for intent specification, improving interpretability at the cost of flexibility [29]. Singh et al. applied IBN to vehicular edge computing [31], and Anand et al. developed an intent-driven telemetry framework for in-band monitoring [1]. In parallel, Du et al. formalized intent policy formats through the IETF’s ANIMA working group [4], contributing to standardization efforts. Despite these contributions, non-LLM approaches generally require handcrafted rules, offer limited support for ambiguous or diverse inputs, and lack adaptability when applied to new scenarios. LLM in IBN. The advent of large language models (LLMs) has led to new opportunities for automating IBN tasks with greater flexibility. Mekrache and Ksentini proposed an LLM-enabled service configuration framework capable of converting natural language intents into network configurations [24]. Tu et al. similarly demonstrated the use of LLMs to translate user intent into configuration commands, offering improved handling of diverse input formats [32]. Dzeparoska et al. introduced LLM-based policy generation for IBN [5], while Habib et al. incorporated attention-based reinforcement learning with LLMs for network optimization and decision-making [12]. Fuad et al. proposed a comprehensive framework integrating LLMs into intent-driven SDN control systems [10]. More recently, Dzeparoska et al. investigated LLM-driven intent assurance guided by intent drift detection [6]. While these studies have demonstrated the potential of LLMs in enhancing intent translation and assurance, they are generally limited to task-specific evaluations, often focusing on translation alone. They lack a comprehensive analysis of LLM capabilities in reliably perform the tasks across the full IBN policy generation lifecycle, and offer limited guidance
on how to design effective LLM prompts, handle edge cases, or interpret LLM output reliability.
3. Understanding LLM’s Capability in Policy Generation 3.1. Overview To systematically explore whether large language models (LLMs) can reliably assist network maintainers in tasks for network policy generation, we further decompose this overarching goal into four focused research questions. • Q1: How does LLM perform compared to SOTA approaches in the tasks for policy generation? • Q2: How does prompt engineering affect LLM’s performance? • Q3: How does fine-tuning affect LLM’s performance? • Q4: In what tasks/scenarios does LLM struggle?
Zero-shot Few-shot Training Dataset
Human Expert
Base LLM
Testing Dataset
Results
RAG
Fine-tuned LLM
Results
Figure 3. Measurement Study Pipeline
Figure 3 presents the overall workflow of our evaluation. Our evaluation pipeline starts with a carefully crafted benchmark dataset constructed from open-source datasets from previous work[15]. With our created benchmark, we use a split training set to fine-tune models and build the retrieval base for RAG and few-shot prompting. We design task-specific prompts with a description of the task, step-bystep instructions, and optionally examples. Prompt design details are presented in Table 9. We assess four model types: zero-shot prompting, few-shot prompting, retrievalaugmented generation (RAG), and fine-tuned LLMs. We compare results from LLMs to the baseline approach using standard evaluation metrics such as accuracy, precision, recall, and F1 score.
3.2. NetInspector-Bench Existing IBN datasets [15, 35] were designed for generic conflict detection and intent translation, not for evaluating security invariant enforcement. They are limited to campusscale topologies, rely on rigid syntactic templates that allow models to overfit to surface patterns, and do not organize violation types into security threat classes. We constructed NetInspector-Bench to address both gaps — first by extending the coverage and linguistic diversity of prior benchmarks, and second by grounding the evaluation in concrete security threat categories.
Extension of Prior Benchmarks. NetInspector-Bench builds on the violation taxonomy of Jacobs et al. [15] and extends it in two dimensions. Topological coverage: we expand from campus-only to Campus, Enterprise, and WAN domains using real-world graphs from Topology Zoo [19], requiring models to reason about path-dependent constraints at different operational scales. Linguistic diversity: we inject state-modifying verbs and shuffle clause orders to prevent keyword overfitting, and synthesize scope-shadowing pairs where violation detection hinges entirely on resolving hierarchical group containment. Violation Types. We include six violation types: negation (allow/deny clash on the same flow); hierarchical (specific-scope intent overrides a group-level block via ancestor–descendant containment); synonym (lexically distinct but semantically equivalent service names, e.g., ssh vs. secure shell, hiding a collision); path (incompatible middlebox-chain requirements for the same flow); time (overlapping temporal windows with conflicting effects on the same group); and QoS (incompatible bandwidth directives that overcommit link capacity). Security Threat Framing. Beyond expanding existing conflict detection data, we map six violation types to three security threat classes: Privilege Escalation (PE) = negation + hierarchical + synonym (broad access overrides a specific block); Isolation Bypass (IB) = path (traffic routed through untrusted segments); Denial of Service (DoS) = time + QoS (resource overcommitment). This mapping, detailed in Table 8, transforms NetInspector-Bench from a conflict detection benchmark into a security-oriented evaluation suite. Of the 437 total violations across 2,224 samples, 173 are PE, 87 are IB, and 140 are DoS.
3.3. Evaluated Approaches
Baseline. For Intent Translation, we use DialogFlow [14], a commercial natural language understanding (NLU) platform that supports intent classification and entity extraction using deep learning, specifically sequence-to-sequence RNN models. For Intent Resolution, we employ Lumi [15], a rule-based system that integrates a machine learning model, specifically a Random Forest classifier, to detect conflicting or redundant intents within multi-user systems. LLM-Based. To answer Q2, we evaluated different prompt tuning techniques including zero-shot, few-shot w/o RAG, and few-shot w/ RAG. Each prompt defines the LLM’s role as a network assistant and supplies the relevant network context as structured JSON, alongside task-specific instructions. The detailed structure of each prompt template is presented in Table 9. We also consider fine-tuning, which is the stateof-the-art continuous learning approach. For fine-tuning, we selected GPT-4o after carefully balancing performance and cost considerations. The training parameters are listed in Table 7.
TABLE 1. I NTENT T RANSLATION R ESULTS : BASELINE USES AN ENCODER - DECODER RNN WITH LSTM AND WORD - LEVEL TOKENIZATION .
Method
Model
TABLE 2. I NTENT R ESOLUTION R ESULTS : I N BASELINE , RF REPRESENTS R ANDOM F OREST, LR REPRESENTS L INEAR R EGRESSION , SVM REPRESENTS S TATE V ECTOR M ACHINE . BASELINE MODELS RELY ON NUMERICAL FEATURES MANUALLY DEFINED AND EXTRACTED FROM THE INPUT INTENT PAIRS .
ROUGE-1 Precision Recall F1
Method
Model
FPR TPR FNR TNR F1
Zero-Shot
gpt-4o-mini gpt-4o claude-3.7
0.35 0.41 0.38
0.06 0.08 0.07
0.10 0.13 0.12
Zero-Shot
Llama 3.2-3B 0.47 0.55 0.45 0.55 0.54 gpt-4.1-nano 0.29 0.48 0.52 0.71 0.54 gpt-4o-mini 0.47 0.68 0.32 0.53 0.63
Few-Shot
gpt-4o-mini gpt-4o claude-3.7
0.42 0.47 0.45
0.09 0.12 0.11
0.15 0.19 0.18
Few-Shot
Llama 3.2-3B 0.46 0.54 0.46 0.54 0.53 gpt-4.1-nano 0.14 0.47 0.53 0.86 0.59 gpt-4o-mini 0.12 0.47 0.53 0.88 0.59
RAG
gpt-4o-mini gpt-4o claude-3.7
0.79 0.82 0.81
0.40 0.44 0.43
0.53 0.57 0.56
RAG
Llama 3.2-3B 0.41 0.48 0.52 0.59 0.51 gpt-4.1-nano 0.20 0.46 0.54 0.80 0.55 gpt-4o-mini 0.06 0.42 0.58 0.94 0.57
Fine-tuning
gpt-4.12025-04-14
0.96
0.78
0.86
Baseline[14]
RNN
0.85
0.47
0.60
3.4. Results 3.4.1. Intent Translation. The intent translation results in Table 1 reveal a stark performance hierarchy. Zero-shot and Few-shot approaches struggle significantly, yielding low F1 scores (0.10–0.19) and poor recall (0.06–0.12), underscoring the difficulty of unguided domain-specific translation. Retrieval-Augmented Generation (RAG) marks a major inflection point, jumping to F1 scores of 0.53–0.57. High precision (0.79–0.82) confirms that dynamically retrieving relevant examples effectively grounds the model, though recall remains a bottleneck. Crucially, only Fine-tuned GPT-4.1 achieves superior performance (F1 0.67), surpassing the traditional LSTM baseline (0.60). With high precision (0.89) and balanced recall (0.53), it demonstrates that while RAG offers improvement, domain-specific weight adaptation is essential for practical utility. These findings indicate that generalpurpose LLMs require substantial customization to match specialized non-LLM baselines in Intent-Based Networking. Finding I. LLMs demonstrate strong performance after fine-tuning on tasks grounded in natural language understanding, such as intent translation. This aligns with their pretraining on large-scale textual corpora and reinforces their strength in semantic parsing and generation.
3.4.2. Intent Resolution. Table 2 presents the performance of LLM-based approaches compared to traditional baselines on the security invariant enforcement task, revealing a distinct trade-off between precision and recall across prompting strategies. Fine-tuning, few-shot prompting, and RAG exhibit a strong conservative bias. While these approaches achieve high precision and minimal false positive rates, indicating high trustworthiness in their positive predictions, they
Fine-tuning
gpt-4.1 -2025-04-14
0.05 0.43 0.57 0.95 0.58
Baseline[15]
RF LR SVM
0.16 0.93 0.06 0.83 0.88 0.25 0.89 0.10 0.74 0.82 0.80 1.00 0.00 0.19 0.70
consistently suffer from low recall. This suggests that incorporating domain context, whether through weight updates or in-context examples, successfully suppresses false alarms but constrains the models from identifying more complex or subtle security violations. Conversely, zero-shot prompting demonstrates the inverse behavior; lacking these constraints, it achieves higher recall but succumbs to poor precision and a high false positive rate. Ultimately, all LLM-based methods are significantly outperformed by the traditional Random Forest baseline, which leverages labeled data to learn structured patterns, achieving a superior balance of accuracy and recall that the generative models fail to match. We attribute the underperformance of LLMs to a deficit in spatial and topological reasoning within the network context. For example, a security violation often arises when two intents compete for the same critical link in their routing paths. To detect this, a model must deduce the specific path for each intent based on the source, destination, and network topology. This requires a structured, multi-step reasoning process that standard LLMs struggle to execute spontaneously. Consequently, while fine-tuning and prompting strategies can improve precision, they do not fundamentally resolve the lack of deep topological understanding required for robust security invariant enforcement. We provide a more detailed analysis of the scenarios where LLMs fail in the following section. Finding II. Even fine-tuned models, while more consistent, struggle to detect indirect constraint violations, revealing persistent reasoning bottlenecks across prompting and training strategies.
3.5. Failed Case Analysis We manually examined the cases where LLMs failed to detect security invariant violations, and summarized the recurring scenarios where LLMs tend to miss. We believe these failure modes are beneficial to the community and should be considered by anyone integrating LLMs into security-critical networking tasks. Availability Invariant Violations (DoS Risk). LLMs consistently fail to enforce Availability Invariants, treating resource constraints as abstract numbers rather than physical limitations. As illustrated in Figure 11(a), Intent A mandates a minimum guarantee (40 Mbps) while Intent B sets a maximum cap (60 Mbps) for the same group. A naive LLM deems these compatible (40 ≤ 60), blindly approving the configuration. However, from a security perspective, this ignores the physical link capacity. If the underlying link supports only 50 Mbps, the LLM has inadvertently authorized a configuration that guarantees resource exhaustion, effectively creating a precondition for Denial-of-Service (DoS). This state amnesia leaves the network control plane vulnerable to availability attacks where authorized policies cannibalize critical resources. Privilege Escalation via Scope Blindness. Figure 11(b) demonstrates a critical Access Control Bypass vulnerability rooted in the LLM’s inability to resolve topological group membership. In this scenario, a group-level policy (e.g., Block ’dorms’) dictates a security boundary, while a specific flow intent (e.g., Allow endpoint X) requests access. The LLM, lacking access to the runtime state of endpoint X, assumes no explicit overlap equals no conflict. This violates the fail-safe defaults principle. By defaulting to a non-conflicting judgment without verifying membership, the LLM permits the specific flow to bypass the group-level restriction, resulting in unauthorized privilege escalation or isolation breach. Enforcement Granularity Evasion. LLMs struggle to maintain policy integrity across different layers of abstraction (Entity vs. Group), creating Inconsistent Enforcement vulnerabilities. As shown in Figure 11(c), an attacker might exploit the ambiguity between a specific IP quota and a broad group quota. Without explicit grounding, the LLM treats these as independent variables rather than hierarchical constraints. This failure allows conflicting security postures to coexist on the same traffic flow, depending on which rule takes precedence in the enforcement engine. Such nondeterministic behavior degrades the network’s security posture, making it impossible to formally verify that isolation or consumption limits are being enforced reliably. Analysis of recurring failure modes reveals that LLMs lack a structured internal understanding of network topology, intent scope, and policy semantics. This limitation is particularly pronounced when correct adjudication requires reasoning about physical feasibility, resource allocation, or complex policy interactions rather than surface-level syntax. Absent explicit modeling of network relationships and constraints result in failures that require topological awareness and reasoning over implicit policy dynamics.
Finding III. LLMs lack structured internal representations of network topology and state, which limits their ability to reason about feasibility, policy enforcement, or intent interactions that go beyond surfacelevel text patterns.
4. NetInspector 4.1. System Overview Figure 4(a) illustrates the overall pipeline of our system, which functions as a secure gateway for transforming unreliable user intents into verified network configurations. The workflow begins with the Intent Translation phase, where a fine-tuned LLM converts natural language requests into structured network policies. In our implementation, we utilize the Lumi [36] schema as the intermediate representation due to its deterministic grammar and well-established ecosystem. However, unlike traditional automation pipelines that might blindly deploy this output, we treat the translated policy as an untrusted object that must be vetted against the network’s security invariants. Consequently, the core of our architecture is NetInspector, which enforces a verify-then-act loop before any policy reaches the network. NetInspector is organized into three layers, each with a distinct role: the Environment Layer maintains a live, queryable store of network state; the Translation Layer exposes a typed tool set through which the agent retrieves topology, group membership, and resource constraints deterministically; and the LLM Layer drives the verification loop as a general-purpose reasoning engine. A policy is approved for deployment only after the agent has grounded its decision in observations retrieved from the Environment Layer, explicitly confirming that no security invariants are violated, such as isolation boundaries or availability constraints.
4.2. Design Rationale Section 3 identified a consistent failure root cause: LLMs lack persistent grounding in network state, causing them to approximate and frequently miss violations that require traversing topology, resolving group containment, or comparing physical resource limits. Three design requirements follow directly. First, every policy decision must be conditioned on live, verifiable network facts rather than the LLM’s parametric memory; this motivates the Environment Layer as an authoritative, queryable store of topology, accesscontrol rules, and bandwidth quotas. Second, group containment and entity scope must be resolved deterministically rather than inferred from surface text—the root cause of Privilege Escalation, Isolation Bypass, and DoS misses in Section 3; this motivates the Translation Layer’s typed tool set, which forces the agent to retrieve ground-truth membership, path data, and physical capacity before issuing any verdict. Thirdly, naively checking Pnew against every invariant in L incurs N LLM calls per intent; since policies with disjoint entity sets cannot conflict under any topology,
We need to protect our labs from data infiltration
Alice
Bob
We should no longer monitor the uploaded file on Bio Lab after the experiments are done
Layer
Policy A: for group(‘lab’) add middlebox('traffic monitor')
Policy B: from endpoint('192.16.1.1') to endpoint('172.16.11.39') remove middlebox('traffic monitor')
NetInspector LLM Layer
Intent Translation
Input B: Add traffic Environment monitor on all labs
Input A: Remove traffic monitor on Bio Lab
Check if ’172.16.11.39’ belong to ‘lab’ group
Verify-then-Act Loop
Training Vector Data Database
[\"172.16.11.39\", \"172.16.11.40\“…]
Intent Resolution
Translation Layer Translation Layer
functions.resolve_entity {\"handle\":\“labs\"}
Topology
Access Control Rules
Low-level Queries
Quota Rules Requested Info
get_shortest_path
get_service_rule
middlebox_on_path
get_bandwidth
resolve_entity
time_overlap
Translation Layer
Summarized Info
High-level Actions
Environment LLM Layer Layer
CONFLICT DETECTED
Environment Layer
LLM Layer Base LLM
Vector Database
‘’172.16.11.39 belongs to the ‘lab’ group. Policy A tries to add traffic monitor while Policy B tries to remove it.
(a) Overview of the pipeline. NetInspector enforces a verify-then-act loop to ground the LLM reasoning.
(b) NetInspector three-layer architecture Framework
Figure 4. Overview of the system and NetInspector Framework.
an entity-overlap pre-filter can safely prune the majority of pairs before any LLM invocation, keeping verification latency practical as L grows. Finally, we enforce the verifythen-act protocol by including a Verdict Node that ensures the LLM can only emit a terminal verdict after at least one grounded observation, preventing hallucinated or malformed output from reaching the network.
4.3. NetInspector Figure 4(b) details the architecture of NetInspector, a three-layer agentic framework designed to secure the interface between probabilistic planning and deterministic network enforcement. The design strictly enforces a separation of concerns principle where the Environment Layer acts as the immutable ground truth, the Translation Layer functions as a secure tool-use interface, and the LLM Layer serves as the symbolic reasoning engine. This layered architecture is critical for security assurance because it effectively creates a verify-then-act control loop. By placing a deterministic translation mechanism between the open-ended reasoning of the LLM and the concrete state of the network, we eliminate the model’s ability to guess topological facts. Instead, the framework forces the agent to plan a query, execute a validated tool, and receive a confirmed observation before it can issue any policy verdict. This ensures that every decision is causally linked to verifiable network evidence rather than probabilistic inference.
Environment Layer. The Environment Layer captures all runtime state relevant to policy generation: the physical topology, service-level access-control rules, and bandwidthquota policies. Unlike the parametric memory of an LLM, which is static and prone to obsolescence, this layer provides a live, queryable view E over the network state σ and the standing policy library L defined in §2. We model the physical topology as a directed graph in NetworkX[25], where nodes represent entities such as switches, hosts, and middleboxes, each annotated with immutable security roles and logical memberships. Edges within this graph store physical properties like capacity and latency, enabling the system to definitively validate path-dependent constraints that purely semantic parsers inevitably miss, such as isolation boundaries or waypoint enforcement. Specifically, we represent every switch, host, and middle-box as a node annotated with its IP prefix, functional role (e.g., edge-fw, coresw), and logical group membership (students, labs, dorms). Each vertex, therefore, records an immutable identifier, role tag, and group set, while each edge stores link capacity and latency as weights. This graph supports path-based queries such as reachability and middle-box placement without embedding device-specific semantics. Beyond topology, security policies are maintained as a priority-ordered JSON list whose entries have the schema 〈src, dst, service, action, enforced at, start, end〉. A rule binds a logical source (group label or host IP) to a destination scope ( for any or a group label), names the application service, specifies the decision, and locates the enforcement
TABLE 3. N ET I NSPECTOR S YSTEM P ROMPT D ESIGN . T HE PROMPT ENFORCES A STRUCTURED VERIFY- THEN - ACT PROTOCOL , ENSURING SEMANTIC PARSING PRECEDES TOPOLOGICAL DATA RETRIEVAL .
Component
Key Instructions & Content
Role & Objective
Act as NetInspector, an expert in network policy conflict detection. Your task is to decide whether two high-level intents conflict. Always understand each intent precisely (Subject, Action, Resource, Scope) before deciding.
Phase 1: Semantic
Step 0 (Parse): Extract Subject, Action, Resource, Scope in plain language. Step 1 (Logic): Compare parsed records using text only. Declare conflict=true if: • • •
Phase 2: Topology
Step 2 (Network-Aware Check): If text analysis is inconclusive, use deterministic tools: • • •
Few-Shot Examples
Same subject receives incompatible directives (e.g., Add vs Remove). Numeric limits clash (e.g., quota(’any’) overrides specific directions). Time windows overlap while prescribing contradictory behaviors.
Shared Path: Use get_shortest_path to check if separate flows share compromised links. Scope Expansion: Use resolve_entity to check if a group label subsumes a specific host. Capacity: Use get_effective_bandwidth. If min + max ≥ capacity, declare conflict.
Scope Hierarchy: Specific upload quota vs. Global ‘any‘ quota → Verdict: CONFLICT. Shared Path: Add IPS on Flow X vs. Remove IPS on Flow Y (sharing links) → Verdict: CONFLICT. Resource Exhaustion: Min 40Mbps + Max 60Mbps vs. 100Mbps Capacity → Verdict: CONFLICT.
point (e.g., border or core firewall). Time windows are optional and recorded in local wall-clock time. Bandwidth controls follow an analogous tuple format 〈endpoint, traffic type, limit mbps, start, end〉. Here, the endpoint may reference a logical group or subnet, the traffic type labels the application class, and the limit is expressed in megabits per second. Temporal scopes are likewise optional, enabling both permanent and time-bounded quotas. This structure allows the system to deterministically resolve first-match logic, which is essential for detecting policy shadowing where a broad rule might inadvertently override a specific block. Distinct from the policy list, bandwidth and usage limits are stored as tuple records, creating a dedicated store that prevents the agent from conflating independent constraints based on loose linguistic similarities. To maintain data integrity, this layer is managed by a dedicated state manager that applies updates atomically, ensuring that the agent always queries a consistent view of the network and preventing race conditions during the verification process. We would also like to highlight that the environment layer is maintained as a live, mutable store rather than a static snapshot. To do so, a dedicated state manager mediates all updates originating from network controllers or administrative tools: new service rules are appended to the JSON policy list, bandwidth quotas are patched in place, and topology changes trigger incremental edits to the NetworkX graph. Each mutation is versioned and applied atomically, so that every query to E is deterministic in the observed snapshot and side-effect-free—properties on which the verification pipeline below depends. Translation Layer. The Translation Layer acts as the secure API gateway that mediates all interactions between the LLM Layer and the Environment Layer. As shown in Figure 4(b), it exposes a typed tool set T = {t1 , . . . , t6 }, where each tj : Ij → Oj specifies an input schema Ij for admissible
arguments and an output schema Oj for the canonical reply. From a security perspective, tj executes only on inputs x ∈ Ij ; any malformed action is rejected before reaching E . This input sanitization, combined with the deterministic dispatch to E , is the architectural defense that bounds what the LLM Layer can observe. The six handles instantiate the operations illustrated in Fig. 4(b): resolve entity expands logical groups into concrete host or subnet identifiers; get shortest path and middlebox on path query the NetworkX topology for reachability and inline devices; get service rule returns the highest-priority access-control entry applicable to a source–destination–service triple; get bandwidth retrieves any bandwidth quota governing a given endpoint and traffic class; and time overlap checks whether two temporal scopes intersect. This mechanism prevents scope-related vulnerabilities by ensuring the agent reasons about the actual endpoints involved rather than just their semantic labels. Furthermore, the layer processes the often voluminous raw data from network queries, such as the output of Dijkstra’s algorithm, and returns a canonical JSON reply containing both the raw result and a compact textual summary. LLM Layer. At the reasoning tier, we use an off-theshelf LLM, prompted to behave as NetInspector, as the agent M. By stripping this layer of the burden of memorizing network state, we allow M to focus on symbolic planning and semantic reasoning. Concretely, M is a function (Pnew , hk ) → ak+1 , where the transcript hk = ((t(1) , o(1) ), . . . , (t(k) , o(k) )) records the sequence of tool actions and observations so far, and the action space A = T ∪ {C ONFLICT, S AFE} partitions every possible output into either a typed tool call or a terminal verdict. The system prompt initializes M with a rigorous security decision protocol. It supplies two critical ingredients: (i) domain knowledge—five conflict categories and a two-stage decision protocol, and (ii) the exact invocation syntax for the
six translation-layer tools. Table 3 shows the excerpt that is embedded in the main text. By grounding the model in this structured specification, we eliminate the need for finetuning and instead rely on prompt engineering plus a small in-context library of worked examples stored in the vector database. Upon receiving Pnew , M first performs semantic parsing to identify obvious linguistic contradictions. If none are found, it proceeds to an Information Retrieval Plan, identifying exactly what state information is missing, such as link utilization or group membership. It then enters a tool execution loop: at each step k , M(Pnew , hk ) either selects a new tool action (tj , x) ∈ T × Ij that extends hk with a fresh (tj , oj ) observation, or emits a terminal verdict in {C ONFLICT, S AFE}. The Verdict Node enforces that only the latter case terminates the loop; any non-verdict output is rerouted through T , so a malformed or hallucinated action never escapes. This disciplined “Thought / Action / Observation” loop effectively neutralizes the risk of ungrounded inference: the model serves as the logic engine that connects the dots, but the dots themselves are provided by the trusted Environment Layer. This architectural decision shifts the system’s failure mode from silent policy violation, where the model confidently guesses incorrectly, to explicit uncertainty, where the model pauses to request data, significantly enhancing the system’s fail-safe properties. NetInspector as a Verifier. Together, the three layers compose into the NetInspector verifier vNI (Pnew , L, σ), which realizes the abstract verifier v of §2. The verifier iterates M over each candidate invariant, routing every tool action through T to E , and emits C ONFLICT as soon as any pairwise check returns a conflicting verdict. Every verdict is therefore conditioned on a transcript hk whose observations come deterministically from E ; soundness and completeness of vNI relative to the ideal v are measured empirically in §5. Entity Pre-filter. To scale verification against a large policy library, we observe that all six violation types require at least one shared entity as a prerequisite (negation and synonym require the same service and scope, path requires the same flow endpoints, hierarchical requires overlapping groups, and QoS and time require the same group or traffic class). Therefore, we apply an entity-overlap pre-filter before any LLM invocation: ϕ(Pnew , L) = {Pi ∈ L : entities(Pnew ) ∩ entities(Pi ) ̸= ∅}, where entities(P ) denotes the set of entity labels appearing in P ; the verifier then passes only ϕ(Pnew , L) to M. In doing so, we safely pruned disjoint Pi from consideration and reduce the LLM calls by around 90%, as illustrated in Section 5.6.
5. Evaluation To evaluate our proposed framework, we focus on the following research questions. • RQ1: (Effectiveness) How effectively does NetInspector detect security invariant violations compared to existing methods? • RQ2: (Robustness) Does NetInspector maintain security coverage under linguistic distribution shifts?
•
•
•
RQ3: (Component Contribution) How does each component contribute to NetInspector’s security enforcement capability? RQ4: (Model Sensitivity) How does the choice of base LLM affect security enforcement performance? RQ5: (Scalability) How does NetInspector scale as the standing security invariant library grows?
5.1. Experiment Setup We implemented our framework utilizing LangGraph[20] for agentic workflow orchestration and ChromaDB[2] as the vector database for efficient context retrieval. To evaluate the robustness of our approach across varying model capabilities, we experimented with five different LLMs as the underlying base (gpt-4.1-mini, gpt5-mini, gemini-2.5-flash-lite, claude-haiku-4.5, qwen3-8b). This selection comprises four state-of-the-art frontier models and one representative open-source model. We compared our method against three existing baselines: Lumi[15], PGA[27], and a heuristic-based approach[40]. These selected baselines represent the spectrum of prior methodologies, covering both traditional machine learning (Lumi) and non-ML-based techniques (graph-based and heuristicbased). We assess performance using three primary metrics: False Positive Rate (FPR), False Negative Rate (FNR), and F1-score. These metrics were chosen to rigorously evaluate the trade-off between the system’s sensitivity to real conflicts and its ability to suppress false alarms. All experiments were conducted on a workstation running Ubuntu 24.04.3 LTS, equipped with an Intel(R) Core(TM) i7-14700F (2.10 GHz) CPU and 32 GB RAM. For evaluations involving local LLMs, we used an NVIDIA GeForce RTX 5070 Ti GPU with 16 GB VRAM.
5.2. Effectiveness of NetInspector (RQ1) Figure 5 presents the comparative performance of NetInspector against the Lumi, PGA, and Heuristic baselines. From a security perspective, FNR is the most critical metric: each undetected violation is a securitycompromising intent silently committed to the network. The results reveal a significant enforcement gap in existing methods, with baselines exhibiting FNRs between 35% and 48%, indicating that they fundamentally struggle to catch violations that depend on physical topology or group membership. In contrast, NetInspector reduces the FNR to approximately 3% by grounding its reasoning in a verify-then-act loop, successfully catching the topologydependent violations that standard classifiers consistently miss. Furthermore, NetInspector minimizes operational alert fatigue by maintaining a near-zero false positive rate. While the Heuristic baseline achieves low noise through excessive conservatism, the ML and graph-based models suffer from significant noise due to spurious text-level correlations. Consequently, NetInspector attains an F1 score exceeding 0.95, significantly outperforming all non-LLM baselines.
False Positive Rate
0.2
False Negative Rate
0.7
1
0.6
0.9
0.5
0.1
0.8
F1 Score
FNR
FPR
0.15
0.4 0.3
0.5
0.1 0
NetInspector Lumi
PGA
Heuristic
0
0.7 0.6
0.2
0.05
F1 Score
NetInspector Lumi
PGA
Heuristic
0.4
NetInspector Lumi
PGA
Heuristic
Figure 5. Security invariant enforcement performance (FNR, FPR, F1) comparing NetInspector against baselines [15, 27, 40]. Lower FNR means fewer security-violating intents reach deployment.
These results confirm that our three-layer agentic framework provides the semantic flexibility required for diverse intents while retaining the rigorous verification needed to enforce security invariants before deployment. 1 w/o distribution shift w/ distribution shift
0.9
component. Conversely, NetInspector maintains superior robustness (F1 >0.95) via its three-layer agentic framework and verify-then-act architecture. Instead of memorizing surface patterns, NetInspector utilizes the Translation Layer to dynamically query live intent properties. By grounding decisions in operational network state rather than historical text statistics, the framework generalizes to unseen inputs and maintains security coverage regardless of the linguistic surface form of the intent.
F1 Score
0.8
5.4. Component Contribution (RQ3) 0.7 0.6 0.5 0.4 NetInspector Lumi
PGA
Heuristic
Figure 6. F1 Score under original and shifted service distributions (shift defined in Figure 2). Lumi’s coverage collapses from 0.84 to 0.56 under shift; NetInspector is unaffected.
5.3. Robustness under Distribution Shift (RQ2) Network invariants must hold across evolving operational patterns and shifting usage behaviors. To evaluate this resilience, Figure 6 analyzes NetInspector’s performance under a data distribution shift, representing novel traffic patterns that deviate from historical training data. As shown in Figure 2(a), the test set introduces significant changes in service composition, mimicking real-world scenarios where new protocols and applications are introduced. The impact on security coverage is quantified in Figure 6(b), where traditional baselines like Lumi suffer catastrophic degradation, with F1 scores collapsing from 0.85 to 0.55. This confirms that static, feature-based classifiers function as brittle pattern-matchers that overfit to training keywords — a dangerous property for a security enforcement
We investigate the contributions of individual framework components by comparing three design variants: (i) a Simple Prompt baseline; (ii) a Structured Prompt providing raw network state and Chain-of-Thought guidance; and (iii) the full NetInspector three-layer agentic framework. Results in Table 4 indicate that while raw context improves performance, the specific presentation of that data is paramount. The “w/o Translation Layer” variant still suffers from high error rates as the LLM must simultaneously process lowlevel network details and high-level intent logic. This cognitive overload leads to over-sensitive decisions, proving that prompt engineering alone cannot bridge the gap between raw data and intent semantics. Conversely, the full NetInspector framework significantly reduces both false positives and negatives by utilizing the Translation Layer to decouple information retrieval from reasoning. This architecture allows the LLM to operate as a focused planner within a verify-then-act loop, querying structured properties rather than ingesting raw state. This structural decoupling is essential for security enforcement: LLMs serve as high-level decision engines supported by deterministic retrieval components, ensuring that every verdict is grounded in verifiable network facts rather than probabilistic pattern matching. Table 4 reports the per-sample inference time across the three design variants: while architectural complexity increases latency, the incremental cost of the full design over the Translation-Layer-only variant is modest, indicating that the LLM’s internal reasoning dominates the overhead rather than our system components.
TABLE 4. P ERFORMANCE AND PER - SAMPLE INFERENCE TIME ACROSS LLM S AND DESIGN APPROACHES . S IMPLE MEANS SIMPLE PROMPT DESIGN . W / O T RANSLATION L AYER MEANS WITHOUT THE TRANSLATION LAYER . F ULL D ESIGN MEANS THE FULL THREE - LAYER FRAMEWORK .
Model GPT-4.1-mini GPT-5-mini Gemini-2.5-Flash-Lite Claude-Haiku-4.5 Qwen3-8B
FN 146 133 72 172 131
Simple Prompt FP F1 Latency (s) 11 0.79 2.46 5 0.81 6.19 573 0.53 1.65 57 0.70 3.87 83 0.74 4.66
FN 87 49 105 23 76
TABLE 5. N- POLICY VERIFICATION LATENCY ( S ) VS . LIBRARY SIZE .
Model
N=10 N=20 N=50 N=100
Gemini-2.5-Flash Claude-Haiku-4.5 GPT-4.1-mini GPT-5-mini Qwen3-8B
2.8 6.0 6.7 18.9 10.7
6.1 10.7 12.0 34.8 21.5
9.3 24.5 26.8 56.8 53.7
22.1 48.7 50.4 98.3 107.3
TABLE 6. E NTITY PRE - FILTER EFFECTIVENESS ( CAMPUS DATASET, 1,000 TESTS ). “C HECKS W / FILTER ” = AVG . LLM CALLS PER INTENT.
N 10 20 50 100
Checks (no filter) 10 20 50 100
Checks (w/ filter) 1.1 2.1 5.0 9.8
Reduction 89.0% 89.5% 90.0% 90.2%
5.5. Impact of Different Base Models (RQ4)
We further analyze how different LLM backbones perform to isolate the impact of model capacity from our architectural design. As shown in Table 4, several consistent trends emerge across the evaluated models. While more capable models generally achieve more stable performance and exhibit lower sensitivity to contextual noise, they still experience significant degradation when operating under the Simple Prompt and raw-state settings. This demonstrates that model strength alone cannot overcome the inherent challenges of reasoning over ungrounded data or unformatted network state. However, under the full NetInspector design, performance gaps between models narrow significantly as all backbones benefit from the structural guidance provided by our three-layer agentic framework. By confining the LLM to high-level planning within a verify-then-act loop, the system effectively stabilizes security enforcement across different backbones. These results suggest that while model choice influences baseline accuracy, a well-designed system architecture is the dominant factor in reliably catching security invariant violations — regardless of which foundation model is used.
w/o Translation Layer FP F1 Latency (s) 72 0.82 5.68 68 0.87 14.07 154 0.72 3.07 144 0.83 5.05 135 0.77 5.34
FN 66 53 116 19 84
FP 56 24 83 29 80
Full Design F1 Latency (s) 0.86 6.96 0.91 16.86 0.76 2.00 0.95 7.97 0.81 10.52
[Input Policy A] ] for group('dorms') set bandwidth('min', '40', 'mbps')
[Input Policy B] for group('dorms') set bandwidth('max', '60', 'mbps') LLM Reasoning
LLM Reasoning
Intent A requires a minimum of 40 Mbps and Intent B caps at 60 Mbps for the same group; since 40 \u2264 60 (same units), there is a feasible bandwidth range (40\u201360 Mbps), so no contradiction.
Need to know group's effective capacity to determine if min(40) and max(60) can co-exist (rule uses capacity C).\nAction: functions.get_effective_bandwidth\nAction Input: {\"endpoint\":\"dorms\"}
Translation Layer Execution {\"limit_mbps\": 100}
Output NO_CONFLICT. The two intents do not conflict with each other because they impose compatible quota values.
Output CONFLICT.\nJustification: Intents both target group('dorms') \u2014 set bandwidth min=40 Mbps and max=60 Mbps; effective capacity C=100 Mbps and min+max=100 \u2265 C, so no guardband.\nVerdict: CONFLICT
Figure 7. Summarized example of Denial of Service. The left shows the execution of LLM reasoning without NetInspector, while the right showcases how LLM interacts with NetInspector and mitigates this threat through capacity-aware reasoning.
5.6. Scalability under N-Policy Libraries (RQ5) We evaluate NetInspector’s scalability when checking a new intent against a library of N standing security invariants—the operational mode faced by organizations as their policy sets grow. Two properties are essential for practical deployment: (1) the entity pre-filter introduced in §4.3 must substantially reduce the number of pairwise LLM calls, and (2) end-to-end latency must remain acceptable as N grows. Pre-filter effectiveness. Table 6 shows that the entity prefilter passes only ≈10% of library pairs for LLM evaluation across all library sizes. A new intent typically references specific network entities (hosts, groups, prefixes) that overlap with only a small fraction of standing invariants, yielding a consistent 10× reduction in LLM invocations. This efficiency is orthogonal to library size: as N grows, actual LLM calls grow at ≈10% of N , not at N itself. Latency vs. library size. Table 5 reports end-to-end verification latency for five LLMs as the library size N grows from 10 to 100. For context, without pre-filtering, total
latency would scale as per-pair latency ×N (from Table 4): e.g., 200 s for Gemini-2.5-Flash, 797 s for Claude-Haiku-4.5, at N =100. The pre-filter reduces actual latency to 8–10× below that unfiltered cost across all models. Even at N =100, the fastest model (Gemini-2.5-Flash) completes verification in 22 s, and all measured models remain within practical bounds for control-plane operations. [Input Policy A] from endpoint('19.16.1.1') to endpoint('172.16.1.3') set quota('any', '8', 'gb/wk')
[Input Policy B] for group('students') set quota('any', '1', 'gb/wk') LLM Reasoning
LLM Reasoning
Intent A specifies a quota for a specific IP address, while Intent B specifies a quota for a group that may include that IP address. If the endpoint '19.16.1.1' is part of the 'students' group, then the two intents would impose conflicting quota limits on the same traffic."
Need to know whether endpoint 19.16.1.1 is part of group('students') to decide if the per-group quota conflicts with the per-flow quota.\nAction: functions.resolve_entity\nAction Input: {\"handle\":\"students\"}
Output CONFLICT. The two intents conflict with each other because the first intent may impose conflicting quota limits on the same traffic.
Translation Layer Execution [\"172.16.2.4\", \"172.16.2.5\", \"172.16.2.6\", \"172.16.2.7\", \"172.16.2.8\", \"172.16.2.9\", \"172.16.2.10\", \"172.16.2.11\", \"172.16.2.12\", \"172.16.2.13\", \"172.16.2.14\", \"172.16.29.75\", \"172.16.29.76\", \"172.16.29.77\", \"172.16.29.78\", \"172.16.29.79\", \"172.16.29.80\", \"172.16.29.81\", \"172.16.29.82\", \"172.16.29.83\", \"172.16.29.84\", \"172.16.29.85\"] Output NO_CONFLICT\nJustification: Intent A applies an 8 GB/wk \"any\" quota to the specific flow 19.16.1.1\u2192172.16.1.3; Intent B applies a 1 GB/wk \"any\" quota to group('students') \u2014 resolved members do not include 19.16.1.1 or 172.16.1.3, so subjects do not overlap.\nVerdict: NO_CONFLICT
Figure 8. Summarized example of Privilege Escalation via scope disambiguation. The left shows the execution of LLM reasoning without NetInspector, while the right showcases the mitigation of LLM interacting with NetInspector.
5.7. Illustrative Case Study To concretely demonstrate how LLMs interact with NetInspector to enforce security invariants, we present the following illustrative case studies, showing how each of the three threat classes is either missed or caught depending on whether NetInspector is in the loop. Case Study 1: Preventing DoS via Capacity-Aware Reasoning. Figure 7 illustrates a critical defense against unintentional Denial of Service (DoS), where abstractly valid policies threaten network stability due to resource exhaustion. In this scenario, two intents target the dorms group: one demanding a minimum bandwidth guarantee, and another setting a maximum usage cap. A naive LLM, treating this purely as a symbolic logic problem, incorrectly concludes NO_CONFLICT because the numeric bounds form a valid mathematical range. However, this failure to account for physical constraints creates a high risk of oversubscription, where the network commits to guarantees it cannot physically support. NetInspector neutralizes this threat by enforcing state-grounded reasoning. Instead of making a
[Input Policy A] from endpoint('19.16.1.1') to endpoint('172.16.11.43') allow service('listserver') traffic('any')" [Input Policy B] from endpoint('19.16.1.1') to endpoint('172.16.7.32') block traffic('social media') protocol('nat')" LLM Reasoning
LLM Reasoning
They govern different flows: Intent A allows traffic to 172.16.11.43 (from 19.16.1.1), while Intent B blocks social media traffic to 172.16.7.32 (from 19.16.1.1). Destinations differ, so there is no overlapping intent on the same traffic to the same endpoint and thus no contradiction."
Need to see if the two flows share network hops so a block on social media/protocol nat could intercept traffic before the allow for listserver \u2014 check shortest paths for both source-destination pairs.\nAction: functions.get_shortest_path\nAction Input: {\"src\":\"19.16.1.1\",\"dst\":\"172.16.11.43\"}“ Action: functions.get_shortest_path\nAction Input: {\"src\":\"19.16.1.1\",\"dst\":\"172.16.7.32\"}"
Output NO_CONFLICT. The two intents do not conflict with each other because they are targeting different endpoints.
Translation Layer Execution {\"path\": [\"19.16.1.1\", \"172.16.1.2\", \"172.16.4.26\", \"172.16.11.39\", \"172.16.11.43\"]} {\"path\": [\"19.16.1.1\", \"172.16.1.2\", \"172.16.4.26\", \"172.16.7.27\", \"172.16.7.32\"]} Output CONFLICT.\nJustification: Intent A allows listserver (any) 19.16.1.1\u2192172.16.11.43; Intent B blocks social-media (protocol nat) 19.16.1.1\u2192172.16.7.32. Their shortest paths share hops (19.16.1.1 \u2192 172.16.1.2 \u2192 172.16.4.26), so the block could be enforced upstream of the allow, causing inconsistent enforcement.\nVerdict: CONFLICT
Figure 9. Summarized example of Isolation Bypass. The left shows the execution of LLM reasoning without NetInspector, while the right showcases how LLM interacts with NetInspector and mitigates this threat through topology-aware reasoning.
premature judgment based on incomplete assumptions, the agent recognizes that validity depends on the group’s effective capacity. It invokes the Translation Layer to retrieve the concrete bandwidth limit from the network state. Armed with this ground truth, the LLM determines that the combined constraints exceed available resources and correctly identifies the conflict. This structured verify-thenact workflow effectively bridges the gap between highlevel intent and physical reality, preventing the cumulative resource allocation errors that lead to service degradation. Case Study 2: Mitigating Privilege Escalation via Scope Disambiguation. Figure 8 addresses the fundamental vulnerability behind unintentional Privilege Escalation: the failure to accurately resolve hierarchical set containment relationships. In this scenario, the system must arbitrate between a flow-level quota (specific endpoint-to-endpoint) and a group-level quota. A naive LLM, lacking access to the directory state, cannot verify if the specific endpoint belongs to the target group. Consequently, it resorts to conservative speculation, assuming a potential overlap and flagging a false conflict. While this specific instance results in a false alarm, this same blindness is what allows broad permissions to inadvertently override specific restrictions in adversarial contexts. NetInspector prevents this boundary confusion through explicit entity resolution. Instead of guessing, the agent invokes the Translation Layer to query the group’s concrete membership. Upon confirming that the endpoint is not a member of the group, the system correctly concludes that the scopes are disjoint. By replacing proba-
bilistic assumptions with grounded evidence, the framework ensures that policies are applied strictly to their intended targets, enforcing the precise logical boundaries necessary to prevent privilege escalation. Figure 10 presents another more complicated example of Privilege Escalation where NetInspector performs multi-step reasoning to identify the security violation. Case Study 3: Preventing Isolation Bypass via TopologyAware Reasoning. Figure 9 demonstrates the necessity of topological grounding to prevent Isolation Bypass, where ignorance of the physical network allows traffic to violate segmentation boundaries. In this example, the system evaluates two intents governing different source–destination pairs. A naive LLM, restricted to an endpoint-centric view, incorrectly concludes NO_CONFLICT because the logical endpoints appear disjoint. This flat reasoning fails to account for the physical reality that logically distinct flows may traverse shared infrastructure. NetInspector overcomes this blindness by enforcing path-level verification. Recognizing that logical isolation does not guarantee physical separation, the agent queries the Translation Layer to retrieve the concrete forwarding paths for both flows. The returned data reveals that the flows converge on a shared intermediate hop, meaning a restrictive policy applied to one flow would inadvertently intercept the other. By exposing these hidden topological dependencies, the framework ensures that the LLM detects the violation, preventing the very class of routing errors that lead to traffic leakage and isolation bypass.
6. Limitations and Future Directions Adversarial threat model. Our threat model targets unintentional misconfigurations by benign insiders; adversarial prompt injection and intentionally crafted unsafe intents are out of scope. NetInspector’s architecture provides partial defenses—a Verdict Node enforces a strict LangGraph state machine, every approved verdict must be backed by Environment Layer observations, and the Translation Layer validates tool arguments against typed schemas before execution. A deployment exposed to adversarial users would additionally require guardrail models (e.g., LlamaGuard) and standard hardening such as RBAC and rate limiting. Securing IBN against adversarial inputs is complementary to, not subsumed by, the verification NetInspector provides. Coverage of violation classes and toolset. The three representative violation classes used throughout this paper and the six tools in our current implementation map onto the identity, topology, and capacity primitives that underlie network policy. We do not formally prove this taxonomy is exhaustive; production environments may surface violation classes outside our current coverage, such as service-chaining conflicts, multi-tenant policy layering, or rule-precedence interactions. The architecture is modular— adding a new tool requires only a JSON schema update and a small set of in-context examples—but identifying the relevant networking primitive for each new class still requires expert judgment.
Synthetic benchmark. NetInspector-Bench is constructed by synthesizing intents on top of three TopologyZoo graphs; we do not validate NetInspector on production IBN deployments with evolving state, complex rule-precedence stacks, or service chaining. Real environments are likely to surface failure modes not represented in our benchmark, and the FNRs we report should be treated as a lower bound on what production deployments will encounter. Closing this gap requires partnerships with operators willing to share anonymized policy logs. Scalability at very large libraries. Current evaluation characterizes NetInspector’s latency up to libraries of N =100 standing invariants. Production deployments may carry thousands. The entity pre-filter scales cheaply (its pass rate is independent of N ), but residual LLM-call latency continues to grow with N , and at very large scales the bottleneck may shift to retrieval-side I/O against the Environment Layer. One further improvement to the scaling problem is to compare input intent with existing security policies in parallel. Future Directions. Beyond closing the limitations above, several directions extend NetInspector’s reach. First, intents that are individually safe against each invariant but collectively unsafe (e.g., cumulative bandwidth across approved reservations) require cross-invariant aggregation outside our current pairwise scope; the Environment Layer already tracks per-link capacity, so extending it with a running admission ledger turns each capacity check into a query against accumulated reservations, capturing multi-intent resource exhaustion without new architectural components. Second, the LLM’s generative capabilities could be leveraged for counterfactual reasoning—analyzing why a violation occurred and proposing safe alternatives: for a bandwidth conflict the agent could propose reducing the requested rate to residual capacity; for a group violation it could rewrite scope to exclude the protected boundary, turning NetInspector from a binary verdict oracle into a repair engine. Third, the IBN lifecycle includes Intent Activation and Intent Assurance stages [21] beyond translation and resolution, which the verify-then-act pattern can naturally extend to. Finally, compiling the agent’s verified logic into programmable data plane primitives [41, 37] would enable line-rate enforcement of validated invariants. A natural scalability improvement is to parallelize the post-filter pairwise checks: since each is independent, latency drops from O(N · t) to O(t), bounded by the slowest single check.
7. Conclusion In this work, we seek to answer if LLM can be reliably used for Intent-Based Networking policy generation, which includes both translation of high-level intent and detect any violations to existing security invariants. Through our measurement study, we discovered that ungrounded models exhibit high FNR as security invariant enforcers, allowing unsecured intents to reach deployment undetected. To close this gap, we introduced NetInspector, a three-layer agentic framework that acts as a security enforcement layer between
operator intent and policy deployment. By decoupling highlevel planning from deterministic state retrieval, NetInspector enforces a strict verify-then-act protocol that grounds every policy decision in verifiable network facts. Our evaluation demonstrates that this architecture reduces the FNR by over 30 percentage points compared to state-of-the-art baselines and maintains robust security coverage even under operational distribution shifts.
[14] [15]
References [16] [1]
[2] [3] [4] [5]
[6]
[7]
[8] [9] [10]
[11]
[12]
[13]
M. Anand, R. Subrahmaniam, and R. Valiveti. “POINT: An intent-driven framework for integrated packet-optical in-band network telemetry”. In: Proc. IEEE Int. Conf. on Communications (ICC). 2018, pp. 1–6. Chroma Vector Database. URL: https : / / www . trychroma.com/. Idris Cinmere et al. “Direct-Conflict Resolution in Intent-Driven Autonomous Networks”. In: arXiv preprint arXiv:2401.08341 (2024). Z. Du et al. ANIMA Intent Policy and Format. Tech. rep. Fremont, CA, USA: IETF ANIMA Working Group, 2016. Kristina Dzeparoska et al. “LLM-based policy generation for intent-based management of applications”. In: Proc. 19th Int. Conf. on Network and Service Management (CNSM). 2023. Kristina Dzeparoska, Ali Tizghadam, and Alberto Leon-Garcia. “Intent Assurance using LLMs guided by Intent Drift”. In: arXiv preprint (2024). eprint: 2402.00715. Kristina Dzeparoska et al. “LLM-based policy generation for intent-based management of applications”. In: 2023 19th International Conference on Network and Service Management (CNSM). IEEE. 2023, pp. 1–7. Facebook outage reported by CNN. URL: https://en. wikipedia.org/wiki/2021 Facebook outage. Facebook, Whatsapp and Instagram back after outage. URL: https://www.bbc.com/news/technology58793174. Ahlam Fuad et al. “An intent-based networks framework based on large language models”. In: Proc. IEEE 10th Int. Conf. on Network Softwarization (NetSoft). 2024. Gartner Data reported financial lose due to network outage caused by human error. URL: https : / / www. popsci . com / technology / facebook - outage explanation/. Md Arafat Habib et al. “LLM-Based Intent Processing and Network Optimization Using AttentionBased Hierarchical Reinforcement Learning”. In: arXiv preprint (2024). eprint: 2406.06059. Y. Han et al. “An intent-based network virtualization platform for SDN”. In: Proc. 12th Int. Conf. on Network and Service Management (CNSM). 2016, pp. 353–358.
[17]
[18] [19] [20] [21] [22]
[23]
[24]
[25] [26]
[27]
[28]
A. S. Jacobs et al. “Refining network intents for selfdriving networks”. In: Proc. Afternoon Workshop on Self Driving Networks (SelfDN). 2018, pp. 15–21. Arthur S. Jacobs et al. “Hey, Lumi! Using Natural Language for Intent-Based Network Management”. In: 2021 USENIX Annual Technical Conference (USENIX ATC 21). USENIX Association, July 2021, pp. 625–639. ISBN: 978-1-939133-23-6. URL : https : / / www . usenix . org / conference / atc21 / presentation/jacobs. Peyman Kazemian, George Varghese, and Nick McKeown. “Header Space Analysis: Static Checking for Networks”. In: Proceedings of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 2012, pp. 113–126. Ahmed Khurshid et al. “VeriFlow: Verifying Network-Wide Invariants in Real Time”. In: Proceedings of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 2013, pp. 15–27. J. Kim et al. “IBCS: Intent-based cloud services for security applications”. In: IEEE Communications Magazine 58.4 (Apr. 2020), pp. 45–51. Simon Knight et al. “The internet topology zoo”. In: IEEE Journal on Selected Areas in Communications 29.9 (2011), pp. 1765–1775. LangGraph Framework. URL: https://www.langchain. com/langgraph. Aris Leivadeas and Matthias Falkner. “A survey on intent-based networking”. In: IEEE Communications Surveys & Tutorials 25.1 (2022), pp. 625–655. J. Li et al. “A survey on deep learning for named entity recognition”. In: IEEE Transactions on Knowledge and Data Engineering 34.1 (Jan. 2022), pp. 50– 70. X. Li, Y. Wang, and H. Zhang. “Deep Analysis and Detection of Firewall Anomalies Using Knowledge Graph”. In: Proceedings of the 18th International Conference on Software Technologies (ICSOFT). SciTePress. 2023. Abdelkader Mekrache and Adlen Ksentini. “LLMenabled intent-driven service configuration for next generation networks”. In: Proc. IEEE 10th Int. Conf. on Network Softwarization (NetSoft). 2024. NetworkX Framework. URL: https://networkx.org/en/. Phillip Porras et al. “A Security Enforcement Kernel for OpenFlow Networks”. In: Proceedings of the 19th Network and Distributed System Security Symposium (NDSS). Internet Society. 2012. Chaithan Prakash et al. “Pga: Using graphs to express and automatically reconcile network policies”. In: ACM SIGCOMM Computer Communication Review 45.4 (2015), pp. 29–42. M. Savi and D. Siracusa. “Application-aware service provisioning and restoration in SDN-based multilayer transport networks”. In: Optical Switching and Networking 30 (Nov. 2018), pp. 71–84.
[29]
[30]
[31]
[32]
[33] [34]
[35] [36]
[37]
[38]
[39]
[40]
[41]
E. J. Scheid et al. “A controlled natural language to support intent-based blockchain selection”. In: Proc. IEEE Int. Conf. on Blockchain and Cryptocurrency (ICBC). 2020, pp. 1–9. Ehab Al-Shaer and Hazem Hamed. “Discovery of policy anomalies in distributed firewalls”. In: Proceedings of IEEE INFOCOM 4 (2004), pp. 2605– 2616. A. Singh, G. S. Aujla, and R. S. Bali. “Intent-based network for data dissemination in software-defined vehicular edge computing”. In: IEEE Transactions on Intelligent Transportation Systems 22.8 (Aug. 2021), pp. 5310–5318. Nguyen Tu, Sukhyun Nam, and James Won-Ki Hong. “Intent-Based Network Configuration Using Large Language Models”. In: International Journal of Network Management 35.1 (2025), e2313. D. Tuncer et al. “A northbound interface for softwarebased networks”. In: Proc. 14th Int. Conf. on Network and Service Management (CNSM). 2018, pp. 99–107. Fulvio Valenza and Antonio Lioy. “Atomizing Firewall Policies for Anomaly Analysis and Resolution”. In: IEEE Transactions on Network and Service Management 19.4 (2022), pp. 4358–4374. Changjie Wang et al. “Netconfeval: Can llms facilitate network configuration?” In: Proceedings of the ACM on Networking 2.CoNEXT2 (2024), pp. 1–25. Avishai Wool. “How to Determine Your Firewall Rule Set Complexity”. In: eWeek (2011). Accessed July 2025. URL: https://www.eweek.com/security/how-todetermine-your-firewall-rule-set-complexity/. Jinzhu Yan et al. “{Brain-on-Switch}: Towards advanced intelligent network data plane via {NNDriven} traffic analysis at {Line-Speed}”. In: 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24). 2024, pp. 419–440. Hui Yang et al. “Intent defined optical network with artificial intelligence-based automated operation and maintenance”. In: Science China Information Sciences 63.6 (2020), p. 160304. Lihua Yuan et al. “FIREMAN: A Toolkit for Firewall Modeling and Analysis”. In: Proceedings of the 2006 IEEE Symposium on Security and Privacy (S&P). IEEE. 2006, pp. 199–213. Xiaoang Zheng, Aris Leivadeas, and Matthias Falkner. “Intent based networking management with conflict detection and policy resolution in an enterprise network”. In: Computer Networks 219 (2022), p. 109457. Guangmeng Zhou et al. “An efficient design of intelligent network data plane”. In: 32nd USENIX Security Symposium (USENIX Security 23). 2023, pp. 6203– 6220.
Appendix A: Additional NetInspector-Bench
Details
About
In this section, we provide more details of our benchmark. NetInspector-Bench extends and reframes prior conflict detection datasets [15] in two ways. First, the original Jacobs et al. dataset covers only campus-scale networks with a limited set of syntactic conflict patterns. We expand coverage to three network domains (Campus, Enterprise, WAN) and introduce new violation scenarios that require multi-hop topological reasoning and hierarchical group resolution — scenarios that the original benchmark does not test. Second, and more importantly, we reframe the task: rather than treating the six violation types as abstract “conflict classes,” we map them to three concrete security threat classes — Privilege Escalation (PE: negation, hierarchical, synonym), Isolation Bypass (IB: path), and Denial of Service (DoS: time, QoS) — enabling security-oriented evaluation of enforcement systems. Table 8 provides a full breakdown. The dataset spans 2,224 samples across the three topologies. To reflect realistic deployment conditions, only ∼20% of samples (437) are security-violating intents and 80% (1,787) are benign — a class imbalance that challenges systems to maintain high sensitivity without generating excessive false alarms. The threat class distribution across all topologies is: 173 PE violations, 87 IB violations, and 140 DoS violations. TABLE 7. F INE - TUNING PARAMETERS AND METRICS
Parameter Trained tokens Epochs Batch size LR multiplier Training time Training cost
Value 1,709,187 3 2 2 30 minutes $15
Appendix B: Prompt Engineering and FineTuning Configuration To comprehensively evaluate the capabilities of Large Language Models in network intent translation and security invariant enforcement, we designed a set of prompt templates ranging from simple instructions to complex, contextaware structures. As detailed in Table 9, our experimental design covers three primary strategies: •
•
Zero-Shot: This baseline approach provides the model with a concise task description and the input query directly, testing the model’s innate ability to generalize without prior examples. Few-Shot: We augment the zero-shot template by prepending multiple randomly selected demonstration examples. This in-context learning strategy aims to guide the model toward the expected output format and logic by providing immediate references.
TABLE 8. N ET I NSPECTOR -B ENCH BREAKDOWN . S IX VIOLATION TYPES MAP TO THREE SECURITY THREAT CLASSES : P RIVILEGE E SCALATION (PE) = NEGATION + HIERARCHICAL + SYNONYM ; I SOLATION B YPASS (IB) = PATH ; D ENIAL OF S ERVICE (D O S) = TIME + QOS . Topology Campus Enterprise WAN Total
•
#Samples 1224 500 500 2224
#Violation 237 100 100 437
#Benign 987 400 400 1787
negation 37 5 15 57
Few-Shot w/ RAG: To address the complexity of domain-specific security invariant reasoning, we implemented a hybrid Retrieval-Augmented Generation (RAG) template. This approach establishes a system role of “network intent translator” and dynamically retrieves semantically relevant examples using the text-embedding-3-large model with a Maximal Marginal Relevance (MMRE) retrieval strategy. The final user prompt incorporates the top-k retrieved examples—ranked by cosine similarity and filtered for contextual relevance—alongside domainspecific guidelines and entity-annotated queries.
Violation Type hierarchical synonym 33 28 15 10 15 15 63 53
Appendix C: Examples of LLM Failures Figure 11 illustrates three critical failure modes where naive LLM-based solutions struggle due to a lack of domain grounding. First, Figure 11(a) demonstrates a False Negative caused by Resource Oversubscription. In this scenario, the model analyzes two bandwidth intents—one setting a minimum guarantee of 40 Mbps and another setting a maximum cap of 60 Mbps. The LLM incorrectly outputs NO_CONFLICT by relying solely on abstract numerical logic (reasoning that 40 ≤ 60 creates a valid range), thereby failing to account for physical network constraints where satisfying the minimum guarantee might be impossible due to capacity oversubscription. Second, Figure 11(b) depicts a False Negative caused by Implicit Membership Dependency. Here, one intent applies to a specific endpoint (19.16.1.1) while the conflicting intent applies to a logical group (’dorms’). Lacking access to the underlying network state or directory services, the LLM fails to resolve whether the endpoint belongs to the group. It treats the scopes as disjoint and erroneously predicts NO_CONFLICT, missing the contradiction inherent in the endpoint’s actual membership. Conversely, Figure 11(c) highlights a False Positive
time 30 10 10 50
qos 30 35 25 90
Threat Class PE / IB / DoS 98 / 42 / 60 30 / 25 / 45 45 / 20 / 35 173 / 87 / 140
[Input Policy A] ] from endpoint('19.16.1.1') to endpoint('172.16.1.3') add middlebox('traffics monitor’) [Input Policy B] for group('dorms') remove middlebox('traffics monitor') LLM Reasoning
LLM Reasoning
Intent A targets a specific endpoint-to-endpoint flow, and Intent B targets a group\u2014there is no direct conflict unless the endpoint '19.16.1.1' or '172.16.1.3' is part of the 'dorms' group. Without that information, the intents do not inherently contradict each other.
I need to know whether the endpoints in Intent A belong to group('dorms') so I can determine if add vs remove of the same middlebox targets the same subject;\nAction: functions.resolve_entity\nAction Input: {\"handle\":\"dorms\"} Translation Layer Execution
[\"172.16.11.39\", \"172.16.11.40\", \"172.16.11.41\", \"172.16.11.42\", \"172.16.11.43\", \"172.16.11.44\", \"172.16.11.45\", \"172.16.11.46\", \"172.16.11.47\", \"172.16.11.48\", \"172.16.11.49\"] LLM Reasoning
Output
Complementing our prompt engineering efforts, we also evaluated the efficacy of fine-tuning. We adopted a lightweight training configuration to balance performance gains with computational efficiency. As summarized in Table 7, the model was trained on a dataset of 1,709,187 tokens over 3 epochs. We utilized a batch size of 2 and a learning rate multiplier of 2. The entire fine-tuning process was notably efficient, completing in just 30 minutes with a total compute cost of $15. This low resource footprint demonstrates the feasibility of adapting frontier models for specialized network tasks without prohibitive infrastructure requirements.
path 42 25 20 87
NO_CONFLICT The two intents do not conflict with each other because there is no evidence that the first traffic belongs to the second group.
I need shortest paths to see if the flow (19.16.1.1 \u2192 172.16.1.3) shares any network hops with traffic to/from dorm endpoints (so the add vs remove of 'traffics monitor' would conflict on shared hops). Translation Layer Execution {\"path\": [\"19.16.1.1\", \"172.16.1.2\", \"172.16.1.3\"]} {\"path\": [\"19.16.1.1\", \"172.16.1.2\", \"172.16.4.26\", \"172.16.11.39\"]} …… {\"path\": [\"19.16.1.1\", \"172.16.1.2\", \"172.16.4.26\", \"172.16.11.39\", \"172.16.11.49\"]} Output CONFLICT\Intent A adds \"traffics monitor\" for flow 19.16.1.1\u2192172.16.1.3 while Intent B removes \"traffics monitor\" for group('dorms'); shortest-paths show both use shared hops (19.16.1.1 and 172.16.1.2), so the same infrastructure would receive opposing add vs remove directives.\nVerdict: CONFLICT
Figure 10. Summarized example of a multi-step reasoning process to mitigate Privilege Escalation. The left shows the execution of LLM reasoning without NetInspector, while the right showcases the multi-step execution of LLM with NetInspector, including more than one round of interaction between the LLM and NetInspector.
caused by Hallucinated Scope Conflict. The reasoning that an endpoint might be part of a group leads the model to hallucinate an overlap, resulting in an unjustified CONFLICT prediction despite the absence of evidence linking the two scopes.
Appendix D: Additional Case Study Case Study 4: Multi-Step Reasoning to Prevent Security Regression. In this seciton we present another example of how NetInspector detects potential Privilege Escalation. In this example, the input intents actually requires NetInspector to perform multi-step reasoning to identify the security violation. Figure 10 illustrates a sophisticated defense against unintentional Privilege Escalation, where broad administrative actions inadvertently dismantle granular security controls. In this scenario, the system must reconcile opposing operational semantics across different abstraction levels: one intent explicitly adds a traffic monitor middlebox to a specific flow, while a second intent removes the same middlebox for an entire user group. A naive LLM, struggling
Template Name
Description
Template
0-shot
Describes the task and directly gives the input query.
USER ⟨task description⟩ ⟨input⟩
few-shot
Describes the task and provides multiple random-selected demonstration examples before the query.
USER ⟨task description⟩ ⟨demonstration example 1⟩ ⟨demonstration example 2⟩ ⟨demonstration example 3⟩ ⟨demonstration example 4⟩ ⟨input⟩
Few-shot w/ RAG
employs a hybrid retrieval-augmented generation approach, where semantically relevant examples are dynamically retrieved from a vector database based on input query embedding similarity. The retrieved examples are ranked by cosine similarity and filtered to maximize contextual relevance while maintaining prompt diversity.
SYSTEM ⟨role: network intent translator⟩ ⟨context: embedding model=text-embedding3-large, retrieval strategy=MMRE⟩ USER ⟨top-k semantically retrieved examples with similarity scores⟩ ⟨domain-specific guidelines⟩ ⟨task description⟩ ⟨query with entity annotations⟩
TABLE 9. T EMPLATES FOR SYSTEM PROMPTS USED IN THE MEASUREMENT STUDY.
[Input Policy A] ] for group('dorms') set bandwidth('min', '40', 'mbps')
[Input Policy A] ] from endpoint('19.16.1.1') to endpoint('172.16.1.3') add middlebox('traffics monitor’)
[Input Policy A] ] from endpoint('19.16.1.1') to endpoint('172.16.1.3') set quota('any', '8', 'gb/wk')
[Input Policy B] for group('dorms') set bandwidth('max', '60', 'mbps')
[Input Policy B] for group('dorms') remove middlebox('traffics monitor')
[Input Policy B] for group('students') set quota('any', '1', 'gb/wk')
LLM Reasoning
LLM Reasoning
LLM Reasoning
Intent A requires a minimum of 40 Mbps and Intent B caps at 60 Mbps for the same group; since 40 \u2264 60 (same units), there is a feasible bandwidth range (40\u201360 Mbps), so no contradiction.
Intent A targets a specific endpoint-to-endpoint flow, and Intent B targets a group\u2014there is no direct conflict unless the endpoint '19.16.1.1' or '172.16.1.3' is part of the 'dorms' group. Without that information, the intents do not inherently contradict each other.
Intent A specifies a quota for a specific IP address, while Intent B specifies a quota for a group that may include that IP address. If the endpoint '19.16.1.1' is part of the 'students' group, then the two intents would impose conflicting quota limits on the same traffic."
Output
Output
NO_CONFLICT The two intents do not conflict with each other because they impose compatible quota values.
NO_CONFLICT The two intents do not conflict with each other because there is no evidence that the first traffic belongs to the second group.
(a) False Negative caused by Resource Oversubscription
(b) False Negative caused by Implicit Membership Dependency
Output CONFLICT. The two intents conflict with each other because the first intent may impose conflicting quota limits on the same traffic.
(c) False Positive caused by Hallucinated Scope Conflict
Figure 11. Summarized example of cases where naive LLM-based solutions fail.
with both hierarchical set containment and topological physicalities, perceives these as non-conflicting, distinct tasks. This oversight creates a dangerous vulnerability where a high-level maintenance command could silently disable essential monitoring for sensitive flows. NetInspector resolves this ambiguity through a multi-step grounding process that chains entity resolution with topological verification. First, the agent invokes the Translation Layer to verify group membership, confirming that the specific flow is indeed a subset of the target group. Second, it retrieves the shortest path data to determine if the directives physically converge on the same infrastructure. By confirming that the add and remove operations target the same shared hops, the LLM correctly identifies the security invariant violation. This capability to chain reasoning steps is crucial for maintaining security coverage in complex environments, ensuring that broad policy changes do not accidentally regress the security posture of specific critical assets.