Semantic Identification of IoT Devices from Behavioral Primitives
arXiv:2606.12793v1 [cs.CR] 11 Jun 2026
SAMUEL WITT, School of EE&T, UNSW Sydney, Australia HASSAN HABIBI GHARAKHEILI, School of EE&T, UNSW Sydney, Australia Accurate identification of Internet of Things (IoT) devices, such as cameras, lightbulbs, and voice assistants, is important for security management and policy enforcement. Existing approaches typically learn device signatures from packets or flow records. These methods operate on low-level communication observations whose traffic patterns may vary across deployments, software versions, and user interactions. This paper studies device identification using Manufacturer Usage Description (MUD) profiles. MUD profiles describe device behavior using Access Control Entries (ACEs), where each ACE represents a behavioral primitive consisting of protocol, endpoint, direction, and port semantics derived from device communication policy. Our specific contributions are threefold. (1) Using 28 publicly available MUD profiles containing 1,023 ACE instances, we construct ACE-level semantic representations from compact behavioral text and evaluate how well they separate device behavior in the embedding space. We show that ACE-level representations preserve device-level behavioral distinctions more effectively than whole-profile embeddings and remain effective after whitening calibration. (2) We then evaluate whether this representation preserves device identity under controlled runtime variations, including unseen ACEs, drifted hostnames, and partial runtime observation. Exact ACE matching performs well when the runtime ACE overlap remains high, but it degrades sharply when the overlap becomes sparse or disappears. In contrast, semantic ACE matching preserves useful identification evidence across these stress-test conditions. (3) Finally, we evaluate the same identification approaches on real IoT traffic traces comprising more than 800,000 observed flows by converting runtime flows into ACElike behavioral primitives and progressively accumulating runtime observations. Exact overlap remains the strongest signal when stable overlap exists, while semantic ACE matching provides stronger identification evidence during the early stages of observation, frequently retaining the correct device among the highestranked candidates and remaining effective under sparse-overlap runtime traffic. These results show that semantic ACE matching can complement exact overlap matching when runtime behavior deviates from canonical device profiles. CCS Concepts: • Networks → Network management; Network measurement; Network monitoring; • Security and privacy → Network security. Additional Key Words and Phrases: IoT device identification, Manufacturer Usage Description (MUD), semantic matching, runtime traffic, network management
1
Introduction
IoT devices are increasingly integrated into enterprise, industrial, and home networks, creating new challenges for network management and security [14, 19, 31]. Accurate device identification is important for policy enforcement, access control, and anomaly detection [23]. Because IoT devices continuously communicate with cloud services and local infrastructure, their runtime traffic exposes recognizable communication patterns that can be used for device fingerprinting and classification [1, 19, 21, 22]. Firmware updates, endpoint changes, and third-party integrations can alter runtime behavior over time [9, 14, 17, 24]. As a result, device identification becomes difficult when runtime communication differs from previously observed behavior. Many existing identification approaches use packet traces or flow records to learn device signatures from statistical traffic features, endpoint information, or protocol sequences [7, 14, 16, 22, 26]. These methods can achieve high accuracy in controlled environments. However, packet- and flowlevel observations capture low-level communication activity that may vary across deployments, Authors’ Contact Information: Samuel Witt, School of EE&T, UNSW Sydney, Australia, [email protected]; Hassan Habibi Gharakheili, School of EE&T, UNSW Sydney, Australia, [email protected].
2
S. Witt and H. Habibi Gharakheili
software versions, and network environments [9, 24]. Firmware updates, runtime activity, and differences in user interaction can alter fine-grained traffic patterns even when the high-level behavior of the device remains largely unchanged [17]. As a result, traffic signatures learned in one runtime condition may become unreliable as exact communication overlap decreases. Manufacturer Usage Description (MUD) profiles [11] provide a standardized representation of device behavior. A MUD profile describes device communication behavior using Access Control Entries (ACEs), where each ACE specifies protocol, endpoint, direction, and port semantics. Unlike packets or statistical flow records, ACEs represent communication behavior as structured policy-level abstractions aligned with network management and security policy [5, 15]. Recent work showed that transport protocol and port combinations already form a useful behavioral vocabulary for IoT devices [2]. ACEs extend this abstraction by incorporating endpoint semantics, communication direction, and policy structure within a standardized representation. Runtime traffic can therefore be converted into ACE-like behavioral primitives without deep packet inspection or payload analysis. This representation does not replace packet- or flow-level classifiers. Instead, it provides a complementary macroscopic view of device behavior. Prior MUD-based identification methods compare profiles using exact ACE overlap [8, 9]. These methods work well when runtime observations closely match the canonical MUD profile. In practice, however, runtime behavior evolves over time. Devices may contact new endpoints, cloud services may change hostnames, and runtime observations may expose only part of the intended communication behavior [9]. Under these conditions, exact ACE overlap becomes sparse or disappears entirely, even when the underlying communication behavior remains similar. This creates a need for representations that remain useful when runtime behavior deviates from canonical device profiles. Such representations should preserve useful identification evidence across runtime evolution while remaining lightweight and compatible with existing MUD-based network management frameworks. This paper studies the semantic identification of IoT devices under evolving runtime behavior using ACE-level behavioral abstractions. We convert ACEs into compact behavioral text and use semantic encoders to construct ACE-level representations that remain comparable even when exact ACE overlap changes over time. We evaluate the approach under controlled runtime variations, including unseen ACEs, drifted hostnames, and partial runtime observation, and then validate it on real IoT traffic traces by converting observed flows into ACE-like behavioral primitives. The specific contributions of this paper are as follows: • We construct ACE-level embeddings from 28 publicly available MUD profiles, which contain 1,023 ACE instances. To address the token overhead of raw MUD JSON files, we compact ACEs into behavioral text before generating 1,024-dimensional embedding vectors. We then use geometric analysis to evaluate whether ACE-level representations separate device behavior in the embedding space. The results show that ACE-level embeddings preserve device-level behavioral distinctions more effectively than whole-profile MUD embeddings (§2). • We evaluate semantic ACE matching under controlled runtime conditions, including unseen ACEs, lexical endpoint drift, and partial runtime observation. These experiments examine how semantic ACE representations behave when runtime observations deviate from canonical MUD profiles and when the exact ACE overlap becomes limited (§3). • We evaluate the same identification methods on real IoT traffic traces comprising more than 800,000 observed flows by converting runtime flows into ACE-like behavioral primitives and progressively accumulating runtime observations. This evaluation examines how semantic ACE matching behaves under operational runtime traffic, including early-stage observations and varying levels of exact overlap (§4). We publicly release our data and code [30].
Semantic Identification of IoT Devices from Behavioral Primitives
2
3
ACE-Level Semantic Representation
Embedding models, such as BERT [6], are widely used in natural language processing to transform texts into numerical vectors, allowing semantically related texts to be compared using vector similarity. In our setting, ACEs describe communication behavior using structured text consisting of a transport-layer protocol, a port number, an endpoint (IP address or domain name), and a direction. ACE embeddings, therefore, allow related communication behaviors to remain comparable even when ACE text differs lexically. Unlike packet payloads or statistical flow records, ACEs already encode communication behavior as structured policy text. Therefore, general semantic encoders are a natural fit for ACE-level representations without requiring traffic-specific retraining [3] or packet tokenization [13]. This section aims to determine whether ACE embeddings carry enough behavioral semantics to support downstream device identification. A useful representation should distinguish devices while remaining robust to small variations in ACE text. Let us begin with a public dataset containing the MUD profiles of 28 commercial IoT devices [8], spanning cameras, sensors, smart plugs, hubs, and voice assistants. The MUD profiles are available in JSON format and collectively contain 1023 ACE instances, of which 710 are unique. Many nonunique ACEs correspond to shared infrastructure behavior such as DNS (UDP/53), NTP (UDP/123), DHCP, and gateway communication, which are found in multiple device profiles. We employ BGE-M3 [4] to transform text-based inputs into numerical embedding vectors. BGE-M3 is designed for semantic similarity tasks and supports structured text inputs, making it suitable for MUDlike behavioral text representations. BGE-M3 natively produces 1024-dimensional embeddings. However, the evaluated MUD files consume between 1346 and 23490 BGE-M3 tokens (dominated by repeated JSON syntax), making direct embedding computationally expensive. We therefore compact each ACE into a single line of behavioral text. For example: “egress ipv4 tcp (direction-initiated:from-device) dst:tech.carematix.com dst-port:8777” This transformation removes JSON boilerplate while preserving core communication semantics. It reduces the average whole-profile token count from 4,852 to 933 (i.e., an 83% reduction). In what follows, we analyze the BGE-M3 embeddings at three levels of granularity: (i) raw whole-file JSON, (ii) compact whole-file ACE text, and (iii) individual ACE embeddings. 2.1
Embedding Geometry
Table 1 summarizes three metrics we use to characterize the geometry of embeddings across the three granularity levels mentioned above. Mean pairwise cosine measures concentration: if the mean cosine is close to one, most vectors point in almost the same direction, and the cosine similarity has little room to separate distinct behavior representations. Effective rank estimates how many independent directions the embedding matrix actually uses. The 90% variance column reports how many principal components are needed to explain most of the variation in the embedding. Together, these metrics indicate whether the representation spreads behavior across a usable semantic space. The raw JSON representa- Table 1. Intrinsic geometry of BGE-M3 embeddings across three repretion is highly concentrated, sentations for evaluated MUD profiles. with a mean pairwise cosine Metric JSON MUD Compact MUD Per-ACE of 0.936. This means that most whole-profile vectors are nearly Mean pairwise cosine 0.936 0.865 0.698 collinear. The likely cause is that Effective rank 23.3 24.2 197.6 the encoder sees a large amount Dims for 90% var. 17 18 45 of repeated JSON syntax before it perceives the behavioral differences between devices. Compacting ACEs into behavioral text improves inter-device separation. Mean pairwise cosine decreases from 0.936 to 0.865, and the most
4
S. Witt and H. Habibi Gharakheili
Cosine Similarity
dissimilar device pairs become farther apart. Per-ACE embeddings preserve substantially greater behavioral variability. The effective rank increases from 24.2 for compact whole-file embeddings to 197.6 for ACE-level embeddings, and the number of dimensions required to explain 90% of the variance increases from 18 to 45. Because the whole-profile matrices contain only 28 device vectors, their mean-centered rank is capped at 27. The important point is not the absolute rank alone, but that ACE-level representation removes this device-level rank limitation and exposes many more rule-level directions. We note that per-ACE embeddings preserve distinctions between individual communication behaviors. However, device identification ultimately requires representing the collective behavior of a device rather than individual ACEs in isolation. Therefore, a natural baseline is to embed each ACE separately and then mean-pool the resulting vectors into a single device representation. Fig. 1 shows that mean-pooling reverses 1.0 much of the separation gained by the compact ACE representations. The pairwise co0.9 sine distribution shifts back toward a more concentrated range observed in raw JSON 0.8 embeddings. This occurs because shared Median 0.7 infrastructure ACEs dominate the resultMean IQR (Q1-Q3) ing device embedding, reducing the influJSON Compact Mean Pool ence of less frequent behavioral primitives. We observed similar qualitative trends in Fig. 1. Distribution of pairwise cosine similarities for deviceembeddings obtained from the OpenAI level embeddings across three representations. text-embedding-3-large model, suggesting that the observed geometry reflects properties of ACE text representations rather than artifacts of a specific encoder. Due to space constraints, we omit the OpenAI results. 2.2
Whitening Decorrelation
The concentration observed in whole-file and mean-pooled embeddings is a known anisotropy effect in embedding models [12], where a small number of dominant dimensions capture most of the variance and cause embeddings to cluster into a narrow region of the space. To reduce this concentration before downstream identification, we apply whitening decorrelation [27]. Details of the whitening procedure are provided in Appendix B.1. Whitening recenters and decorrelates the embedding vectors by projecting them onto principal components and rescaling each component by its variance. Whitening is applied uniformly to all three representations studied in § 2.1: raw JSON MUD profiles, compact whole-profile embeddings, and ACE-level embeddings. In all three cases, whitening reduces cosine concentration and increases effective rank. For example, raw mean-pooled ACE signatures have a mean pairwise cosine of 0.948 and an effective rank of 23.1; whitening before pooling reduces the mean cosine to 0.073 and raises the effective rank to 26.5, close to the maximum supported by the 28 device signatures. Whitening, therefore, acts as cosine calibration without adding behavioral information. Although whitening improves the geometry of all three representations, it does not make them equivalent. Raw JSON embeddings still spend most of their token budget on repeated syntax, while compact whole-profile embeddings still collapse an entire device profile into a single vector. In contrast, ACE-level embeddings preserve the same behavioral unit that appears at runtime. This distinction is important for downstream identification. ACE-level embeddings can therefore be aggregated into a device signature or preserved individually for ACE-level matching. The following section
Semantic Identification of IoT Devices from Behavioral Primitives
5
evaluates both retrieval strategies under controlled runtime conditions, including unseen ACEs, endpoint drift, and partial runtime observation. 3
Semantic Identification Under Evolving Runtime Behavior
The previous section showed that ACE-level embeddings preserve meaningful behavioral variability across IoT devices. We now evaluate whether this representation improves device identification under evolving runtime behavior. We assume access to a repository of known device profiles represented as MUD profiles. At runtime, a monitor observes one or more ACEs generated by a device and attempts to identify the device by matching the observed behavior against the known profiles. When runtime ACEs exactly match ACEs in a stored MUD profile, identification is straightforward because repeated matches reinforce the correct device profile as more ACEs are observed. In practice, however, runtime observations may differ from the canonical MUD profile in several ways. Observed ACEs may be completely unseen, hostnames may drift while protocol and port semantics remain unchanged, or runtime observations may expose only part of the intended communication behavior. We evaluate these conditions using synthetically generated runtime-observation episodes designed to isolate specific forms of runtime variation. The goal is not to reproduce operational traffic exactly, but to determine whether semantic ACE representations preserve useful identification evidence when exact ACE overlap is sparse or absent. 3.1
Runtime Observation Settings
We evaluate three runtime conditions: (1) unseen ACEs, where no exact ACE overlap exists in the reference repository, (2) lexical endpoint drift, where hostnames change while protocol and port semantics remain unchanged, and (3) mixed partial observation, where runtime queries contain varying combinations of exact matches, unseen ACEs, and drifted ACEs. The first two settings are controlled semantic stress tests in which exact ACE overlap is intentionally removed. The third setting models more realistic runtime observations in which some ACEs remain unchanged. Across these settings, we vary the number of observed ACEs, drifted ACEs, and exact ACE matches retained. Details of query generation, hostname perturbation, and ACE-family construction are provided in Appendix C. 3.2
Retrieval Methods
We compare three retrieval approaches that differ in how runtime ACEs are matched against the reference MUD profiles. Exact ACE matching. The first approach uses exact ACE overlap. Runtime observations are matched against reference MUD profiles using either Jaccard similarity (i.e., the size of the ACE intersection divided by the size of the ACE union) or exact ACE hit count (i.e., the number of ACEs shared by the runtime observation and the candidate profile). These methods are effective when unchanged ACEs remain in the runtime observation, but they cannot assign similarity to semantically related ACEs whose text differs lexically. Aggregated semantic matching. The second approach represents each device profile using a single embedding vector derived from ACE embeddings. The embeddings are first whitened to reduce anisotropy and cosine concentration, as discussed in §2.1. Device-level signatures are then constructed by aggregating the whitened ACE embeddings into a single vector representation. The second approach represents each device profile using a single vector obtained by mean-pooling whitened ACE embeddings. Construction details for mean-pooled signatures are provided in Appendix B.2. Aggregated signatures allow semantically related ACEs to contribute to similarity
6
S. Witt and H. Habibi Gharakheili
even when exact ACE overlap is absent. However, aggregating ACE embeddings into a single vector may still suppress fine-grained behavioral distinctions. ACE-level semantic matching. The third approach preserves individual ACE embeddings and performs matching directly at the ACE level using an asymmetric MaxSim formulation inspired by ColBERT-style late interaction [10]. Given a query ACE set 𝑄 and candidate reference profile 𝑅𝑑 , the candidate score is: 1 ∑︁ 𝑠 (𝑄, 𝑅𝑑 ) = max cos(𝑒𝑞 , 𝑒𝑟 ), |𝑄 | 𝑞 ∈𝑄 𝑟 ∈𝑅𝑑 where 𝑒𝑞 and 𝑒𝑟 denote ACE embeddings. ACE-level matching preserves individual communication behaviors and avoids the information loss introduced by aggregation. It is also naturally incremental because candidate scores can be updated as new runtime ACEs are observed. 3.3
Evaluation Methodology and Results
Each retrieval method assigns a similarity score between the runtime query and each candidate MUD profile, ranks the candidate devices by that score, and predicts the highest-ranked device. We report Top-1 accuracy and mean reciprocal rank (MRR). Top-1 measures the fraction of runtime queries for which the correct device is ranked first. MRR captures how highly the correct device is ranked on average. Exclusively unseen behavior and drifted endpoints. Table 2 reports two stress-test conditions in which exact ACE overlap is intentionally weakened or removed. Exclusively unseen behavior removes exact ACE overlap entirely, while drifted endpoints preserve related communication semantics despite lexical changes in ACE text. Under exclusively un- Table 2. Identification results (Top-1 accuracy) under exclusively unseen beseen behavior, Exact ACE havior and exclusively drifted endpoints. Δ columns report raw-to-whitened matching reduces to de- Top-1 gain in percentage points for the same method. terministic tie-breaking beEval Variant 𝑁 Jaccard Mean Pool MaxSim cause no exact ACE overlap Top-1 Δ Top-1 Δ remains. As a result, Top-1 Single unseen ACE 1,023 0.0371 0.6393 +14.7 0.6549 +9.6 accuracy collapses to nearUnseen ACE family 103 0.0291 0.8447 +13.6 0.8447 +4.9 Unseen ACE set 240 0.0417 0.7792 +25.4 0.7958 +25.0 chance performance across all unseen variants. In conDomain-full 280 0.0357 0.9393 +38.2 0.9143 −0.7 Domain-high 140 0.0714 0.9643 +15.0 0.9857 −0.7 trast, both the mean-pooled semantic baseline and ACElevel semantic matching retain high identification accuracy. For single unseen ACEs, MaxSim achieves 0.6549 Top-1 accuracy compared to 0.0371 for Jaccard matching. Accuracy further increases for unseen ACE families and unseen ACE sets, reaching 0.8447 and 0.7958, respectively. The whitened mean-pooled baseline performs similarly, reaching 0.6393, 0.8447, and 0.7792 across the same settings. Under drifted endpoints, MaxSim reaches 0.9143 Top-1 accuracy across all devices and 0.9857 for high-domain devices, while whitened mean pooling reaches 0.9393 and 0.9643. The Δ columns show that whitening substantially improves mean-pooled retrieval by reducing the influence of shared infrastructure ACEs in aggregated device signatures. MaxSim shows smaller gains because individual ACEs remain separate during retrieval. Semantic matching succeeds because related ACEs often preserve communication semantics even when the exact ACE text changes. In a single-unseen-ACE episode, the held-out ACE for the HP Printer device was a local IPv4 UDP multicast to 224.0.0.252/32 on port 5355. This ACE was removed from every candidate profile (reference), so exact matching had zero overlap. MaxSim nevertheless ranked this device first because the retained HP Printer profile contained a
Semantic Identification of IoT Devices from Behavioral Primitives
7
Table 3. Identification results under mixed partial runtime observation over 5, 772 runtime queries. Mean Pool reports mean-pooled ACE signatures after whitening. The four rightmost columns report Top-1 after grouping queries by the number of ACE matches retained against the correct reference profile. Method Jaccard Exact ACE-hit count MaxSim Mean Pool
Global Top-1
Global MRR
0 hits
1–2 hits
3–5 hits
> 5 hits
0.6532 0.7230 0.9189 0.9425
0.7223 0.7729 0.9350 0.9585
0.0088 0.0088 0.7093 0.9075
0.3555 0.3721 0.7831 0.8416
0.5894 0.6892 0.9273 0.9324
0.8809 0.9641 0.9945 0.9978
corresponding local IPv6 multicast behavior on the same UDP service. A harder unseen-ACE-set episode shows the same effect across multiple unrelated ACEs. The held-out HP Printer query combined cloud communication (xmpp009.hpeprint.com:80), local service discovery on port 5353, and DHCPv4 behavior. After all three exact ACEs were removed from every reference profile, MaxSim still ranked the HP Printer profile first by matching the query to retained ACEs with similar protocol-port behavior and communication roles (i.e., HP ePrint traffic on port 443, IPv6 local discovery, and DHCPv6). Mixed partial observation. Table 3 reports a more realistic runtime setting in which observed ACEs may contain a mixture of exact matches, unseen behavior, and drifted endpoints. The Global Top-1 and Global MRR columns report overall identification performance across all runtime queries. The remaining columns group queries by the number of exact ACE matches retained against the correct reference profile. Exact ACE matching performs well when many unchanged ACEs remain in the runtime observation. In the > 5 hit bin, Jaccard reaches 0.8809 Top-1 accuracy, and exact ACE-hit count reaches 0.9641. Performance degrades rapidly as the exact overlap decreases. In the zero-hit bin, both exact baselines collapse to near-chance performance (0.0088 Top-1), while MaxSim and mean pooling achieve 0.7093 and 0.9075, respectively. These synthetic stress tests isolate the impact of unseen ACEs, lexical endpoint drift, and partial runtime observation under controlled conditions. In the following section, we evaluate our identification approaches on real IoT traffic traces to assess how semantic matching behaves under operational conditions. 4
Runtime Identification on Real Flow Traces
The previous section evaluated identification methods under controlled runtime conditions using independent retrieval episodes with limited ACE observations. In this section, we evaluate the same identification methods using real IoT traffic traces and progressively increasing amounts of observed runtime behavior. Unlike §3, the runtime observations in this section are not synthetically generated from the canonical MUD profiles. The MUD profiles remain unchanged, while runtime behavior is derived directly from observed traffic traces. This allows us to evaluate the identification methods under operational traffic conditions while preserving the natural ordering of observed communications. We use two public datasets: traffic traces from [29] and canonical MUD profiles from [8]. The evaluation contains 26 runtime traces matched against the full 28-device MUD-profile repository; the Chromecast device is the only reference profile without a corresponding traffic trace. For each device trace, we retain flows that contain an IP transport protocol, as well as source and destination port information. Non-IP traffic and flows without transport-layer port information, such as ARP, are therefore excluded. DHCP, multicast discovery, and other local transport communications are
8
S. Witt and H. Habibi Gharakheili
retained when they satisfy these criteria. From the remaining flows, we keep up to the first 50,000 observations ordered by arrival time, or all available flows if fewer are present. In total, 810,490 flows are retained across the 26 device traces. Identification performance is evaluated progressively. At episode 𝑘, the first 𝑘 flows of each device trace are treated as the observed device behavior. Each retained flow is converted into the same compact behavioral representation used for the canonical MUD profiles. Flows are first oriented as ingress or egress, and remote IP addresses are mapped to hostnames whenever corresponding DNS information is available in the packet traces. This process resolves 3,286 of 3,487 unique device-remote IP pairs (94.2%). By flow volume, 284,851 of 810,490 flow instances (35.1%) are hostname-resolved since many repeated flows target local, broadcast, multicast, gateway, or otherwise unresolved endpoints. We evaluate three identification methods. The first uses exact ACE hit count, where every exact ACE match contributes evidence, including repeated ACE observations. The score is the normalized hit count obtained by dividing the total number of matches by the number of observed runtime ACEs. The second uses Jaccard similarity over unique ACE observations, as described in §3.2. The third uses the whitened ACE-level MaxSim formulation from §3.2. The two exact methods measure overlap directly, whereas MaxSim assigns similarity between semantically related ACEs even when exact overlap is absent. 4.1
Runtime Identification Results
1.0
Top-1 accuracy
We evaluate matching as a runtime identifica0.8 tion primitive instead of a complete device classifier. Our goal is to understand how exact and 0.6 semantic matching contribute identification ev0.4 idence as runtime observations accumulate. The exact ACE-hit count method achieves Hit-Count (exact matching) 0.2 Jaccard (exact matching) higher final identification accuracy by leveragMaxSim (semantic matching) 0.0 ing repeated observations that are discarded by 101 104 100 102 103 Jaccard matching. At the end of the observaNumber of observed flows tion period, exact hit count matching correctly Fig. 2. Top-1 accuracy across the 26 device traces as a identifies 25/26 runtime traces, while semantic function of the number of observed runtime flows. matching using MaxSim identifies 24/26. The 24 traces correctly identified by MaxSim are a subset of those correctly identified by exact matching. Therefore, exact hit-count matching remains the strongest identification signal, particularly when stable exact evidence is repeatedly observed in runtime traffic. The performance trace in Fig. 2 highlights the dynamics of the three approaches we evaluated. During the early stages of observation, semantic matching (i.e., MaxSim) provides stronger identification evidence than exact matching. After a single observed flow, MaxSim ranks the correct device (Top-1) for 13/26 traces, compared to 8/26 for exact hit-count matching. The gap slightly widens after observing 10 flows, where MaxSim reaches 25/26 correct predictions compared to 18/26 for exact hit-count matching. As repeated exact overlap accumulates, exact hit-count matching catches up after the arrival of about 50 flows and eventually overtakes semantic matching. We also note that although Jaccard-based matching approaches the accuracy of semantic matching near the end of the trace, it remains consistently weaker than both hit-count matching and MaxSim. The results above focus on Top-1 accuracy. However, matching is fundamentally a ranking mechanism, and the correct device may still appear among the highest-ranked candidates even when it is not the Top-1 prediction. To examine this, Fig. 3 reports the cumulative distribution of the rank assigned to the correct device across all the evaluated episodes. To reduce the influence of a small number of highly active devices, the analysis is restricted to the first 10,000 observed flows
Semantic Identification of IoT Devices from Behavioral Primitives
9
Table 4. Identification results for 9,023 random disjoint 50-flow runtime windows. Values in brackets report the mean margin of Top-1 score over the second-ranked candidate. Window set
Windows
Devices
Exact Top-1
MaxSim Top-1
Average Exact score
Average MaxSim score
All windows Exact < 0.50 Exact < 0.10 Exact = 0
9,023 3,287 1,841 1,152
25 14 8 7
5,707/9,023 893/3,287 40/1,841 0/1,152
6,823/9,023 2,099/3,287 702/1,841 457/1,152
0.602 [0.403] 0.126 [0.060] 0.036 [0.023] 0.013 [n/a]
0.570 [0.292] 0.468 [0.120] 0.459 [0.041] 0.458 [0.057]
CDF: P(rank n)
per device. MaxSim frequently retains the correct device among the highest-ranked candidates. Although exact exact hit-count achieves higher Top-1 accuracy (≈ 94% versus 83%), the correct device appears within the top three candidates for approximately 95% of MaxSim episodes. Thus, the correct device typically remains among the highest-ranked candidates even when semantic matching is not the Top-1 prediction. To reduce dependence on the starting point 1.0 of each traffic trace, we next evaluate short runtime observations drawn from throughout the 0.8 trace. For each device, we sample up to 500 0.6 random disjoint windows containing 50 consecutive flows1 , yielding 9023 runtime windows 0.4 in total. Unlike the cumulative episodes used Hit-Count (exact matching) 0.2 above (Fig. 2), each window represents an indeJaccard (exact matching) MaxSim (semantic matching) pendent short runtime observation drawn from 0.0 1 2 3 4 5 >5 a different point in the trace. Table 4 summaN: Top-n rank rizes the results. Fig. 3. CDF of the rank assigned to the correct device Across all windows, MaxSim achieves 6823 across runtime identification episodes. correct Top-1 predictions, compared to 5707 for hit-count matching. Although MaxSim identifies substantially more windows correctly, exact matching produces larger average score margins (i.e., margin of Top-1 score over the second-ranked candidate) when it succeeds. The average Top-1 score of MaxSim is 0.570 with an average margin of 0.292 (for windows with correct Top-1), compared to 0.603 and 0.403 for exact hit-count. The advantage of semantic ACE matching becomes increasingly apparent as exact overlap decreases (second–fourth rows in Table 4). For windows with exact score below 0.50, MaxSim identifies 2,099 cases correctly compared to 893 for exact hit-count matching. The gap widens further when the exact score falls below 0.10 (702 vs. 40), and in the 1,152 zero-overlap windows, where hit-count fails completely, MaxSim still identifies 457 windows correctly with an average score of 0.458 and margin of 0.057. The score behavior exhibits the same trend. As exact overlap decreases, the normalized hit-count score collapses from 0.602 across all windows to 0.013 in the zero-overlap setting. In contrast, the average MaxSim score remains relatively stable (0.570 to 0.458), indicating that semantic similarity continues to provide useful identification evidence even when exact overlap becomes sparse. These results are consistent with the controlled runtime experiments in §3. In the real traces, while exact matching is effective when stable overlap exists, semantic ACE matching provides complementary evidence for identification when exact overlap becomes sparse or disappears. For example, in one HP Printer window, multiple flows target “xmpp006.hpeprint.com:5222”, whereas the canonical profile contains the related endpoint “xmpp009.hpeprint.com:5222”. Exact matching assigns little evidence to the correct profile, while MaxSim still ranks HP Printer first by recognizing the similarity between the two ePrint communication behaviors. A similar effect occurs in a 1 The Blipcare BP Meter trace contained fewer than 50 eligible flows and was therefore excluded from this experiment.
10
S. Witt and H. Habibi Gharakheili
Triby Speaker window, where runtime flows contact “sip.invoxia.com:5228” instead of the canonical “sip.aws.invoxia.io:5228” endpoint. In both cases, semantic matching succeeds by preserving protocol, port, and endpoint family semantics despite reduced exact overlap. 5
Related Work
IoT device identification from network traffic. Prior work has shown that IoT devices expose recognizable communication patterns that can support device fingerprinting and classification [22, 24–26]. Existing approaches typically learn signatures from packets or flow records using statistical traffic features, protocol sequences, or endpoint information [7, 16]. More recent work applies representation learning to encrypted traffic flows and packet traces using transformer-style architectures and self-supervised pre-training [13, 28, 32, 33]. These approaches operate on finegrained traffic dynamics and require packet-level [18] or flow-level [26] representations. Our work studies a different level of abstraction. Instead of modeling fine-grained packet or flow dynamics, we represent device behavior using Access Control Entries (ACEs), which encode protocol, endpoint, direction, and port semantics. ACEs provide a standardized behavioral abstraction that can be derived from runtime traffic and compared against canonical MUD profiles. MUD-based monitoring and runtime verification. The Manufacturer Usage Description (MUD) standard [11] provides a machine-readable specification of intended IoT communication behavior. Work in [8] operationalized MUD for runtime monitoring and policy verification using the exact set overlap between observed runtime flows and canonical MUD profiles. Subsequent work explored policy enforcement and firewall-style verification using MUD semantics [5, 15]. These approaches treat ACEs primarily as exact policy rules for runtime verification. In contrast, our work studies semantic similarity between ACEs under evolving runtime behavior. We explicitly evaluate conditions where exact ACE overlap becomes sparse due to unseen ACEs, lexical endpoint drift, or partial runtime observation. Semantic encoders and late interaction retrieval. Embedding models such as BERT and SBERT are widely used for semantic retrieval [6, 20]. Recent encoders such as BGE-M3 support longer structured texts and cosine-based retrieval objectives [4]. Whitening-based decorrelation has also been shown to improve retrieval geometry and downstream classification [3, 12, 27]. We use semantic encoders to compare ACE-level behavioral representations and employ whitening as a retrieval calibration step. Our ACE-level matching method further adopts the late-interaction idea of ColBERT [10] by matching individual ACE embeddings directly instead of aggregating all runtime behavior into a single device representation. 6
Conclusion
IoT device identification becomes challenging when runtime behavior evolves and exhibits patterns slightly different from known device profiles. This paper studied the semantic identification of IoT devices using ACE-level behavioral abstractions derived from MUD profiles. We constructed ACE-level semantic representations from compact behavioral text and analyzed their embedding geometry across multiple representation granularities for the MUD profiles of 28 different consumer IoT devices. We then evaluated the efficacy of semantic ACE matching under controlled degradation of runtime overlap, including unseen ACEs, lexical endpoint drift, and partial runtime observation. Finally, we evaluated the same retrieval methods on real IoT traffic traces by converting runtime flows into ACE-like behavioral primitives and performing real-time runtime identification. We showed that semantic ACE matching complements exact overlap matching by providing useful evidence for identification under sparse-overlap conditions and enabling earlier candidate identification during runtime observation.
Semantic Identification of IoT Devices from Behavioral Primitives
11
References [1] Tadani Nasser Alyahya, Leonardo Aniello, and Vladimiro Sassone. 2024. ScaNeF-IoT: Scalable Network Fingerprinting for IoT Devices. In Proc. ACM ARES. Vienna, Austria. [2] Shayan Azizi et al. 2025. From Flows to Functions: Macroscopic Behavioral Fingerprinting of IoT Devices via Network Services. arXiv:2512.16348 [cs.NI] https://arxiv.org/abs/2512.16348 [3] Sylee Beltiukov, Satyandra Guthula, Wenbo Guo, Walter Willinger, and Arpit Gupta. 2025. Demystifying Network Foundation Models. In Proc. NeurIPS. Atlanta, Georgia, USA. [4] Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. In Findings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 2318–2335. doi:10.18653/v1/2024.findings-acl.137 [5] François De Keersmaeker, Ramin Sadre, and Cristel Pelsser. 2024. Supervising Smart Home Device Interactions: A Profile-Based Firewall Approach. In Proc IFIP Networking. Thessaloniki, Greece. [6] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proc. NAACL. Minneapolis, MN, USA. [7] Hang Guo et al. 2018. IP-Based IoT Device Detection. In Proc. ACM Workshop on IoT S&P. Budapest, Hungary. [8] Ayyob Hamza et al. 2022. Verifying and Monitoring IoTs Network Behavior Using MUD Profiles. IEEE Transactions on Dependable and Secure Computing 19, 1 (2022), 1–18. [9] Weijia He, Kevin Bryson, Ricardo Calderon, Vijay Prakash, Nick Feamster, Danny Yuxing Huang, and Blase Ur. 2024. Can Allowlists Capture the Variability of Home IoT Device Network Behavior?. In Proc. IEEE EuroS&P. [10] Omar Khattab and Matei Zaharia. 2020. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. In Proc. ACM SIGIR. Virtual Event, China. [11] Eliot Lear, Ralph Droms, and Dan Romascanu. 2019. Manufacturer Usage Description Specification. RFC 8520. doi:10.17487/RFC8520 [12] Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020. On the Sentence Embeddings from Pre-trained Language Models. In Proc EMNLP. Online. [13] X. Lin, G. Xiong, G. Gou, Z. Li, J. Shi, and J. Yu. 2022. ET-BERT: A Contextualized Datagram Representation with Pre-training Transformers for Encrypted Traffic Classification. In Proc. WWW. Lyon, France. [14] Eman Maali, Omar Alrawi, and Julie McCann. 2025. Evaluating Machine Learning-Based IoT Device Identification Models for Security Applications. In Proc. NDSS. San Diego, CA, USA. [15] Luca Morgese Zangrandi, Thijs Van Ede, Tim Booij, Savio Sciancalepore, Luca Allodi, and Andrea Continella. 2022. Stepping Out of the MUD: Contextual Threat Information for IoT Devices with Manufacturer-Provided Behavior Profiles. In Proc. ACSAC. Austin, TX, USA. [16] A. Pashamokhtari et al. 2022. Combining Stochastic and Deterministic Modeling of IPFIX Records to Infer Connected IoT Devices in Residential ISP Networks. IEEE Internet of Things Journal 10, 6 (Nov 2022), 5128–5145. [17] Arman Pashamokhtari, Norihiro Okui, Masataka Nakahara, Ayumu Kubota, Gustavo Batista, and Hassan Habibi Gharakheili. 2023. Dynamic Inference From IoT Traffic Flows Under Concept Drifts in Residential ISP Networks. IEEE IoT Journal 10, 17 (Apr 2023), 15761–15773. [18] Aleks Pasquini et al. 2025. Robust and Lightweight Modeling of IoT Network Behaviors from Raw Traffic Packets. IEEE Transactions on Machine Learning in Communications and Networking 3 (2025), 98–116. [19] Roberto Perdisci, Thomas Papastergiou, Omar Alrawi, and Manos Antonakakis. 2020. IoTFinder: Efficient Large-Scale Identification of IoT Devices via Passive DNS Traffic Analysis. In Proc. IEEE EuroS&P. Genoa, Italy. [20] Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. arXiv:1908.10084 [cs.CL] https://arxiv.org/abs/1908.10084 [21] Miraqa Safi, Sajjad Dadkhah, Farzaneh Shoeleh, Hassan Mahdikhani, Heather Molyneaux, and Ali A. Ghorbani. 2022. A Survey on IoT Profiling, Fingerprinting, and Identification. ACM TIOT 3, 4, Article 26 (Sep 2022), 39 pages. [22] Said Jawad Saidi et al. 2020. A Haystack Full of Needles: Scalable Detection of IoT Devices in the Wild. In Proc. IMC. Virtual Event, USA. [23] Rahul Anand Sharma, Elahe Soltanaghaei, Anthony Rowe, and Vyas Sekar. 2022. Lumos: Identifying and Localizing Diverse Hidden IoT Devices in an Unfamiliar Environment. In Proc. USENIX Security. Boston, MA, USA. [24] Arunan Sivanathan et al. 2020. Detecting Behavioral Change of IoT Devices Using Clustering-Based Network Traffic Modeling. IEEE Internet of Things Journal 7, 8 (Mar 2020), 7295–7309. [25] A. Sivanathan, H. Habibi Gharakheili, F. Loi, A. Radford, C. Wijenayake, A. Vishwanath, and V. Sivaraman. 2019. Classifying IoT Devices in Smart Environments using Network Traffic Characteristics. IEEE Transactions on Mobile Computing 18, 8 (2019), 1745–1759. [26] Arunan Sivanathan, Deepak Mishra, Sushmita Ruj, Natasha Fernandes, Quan Z. Sheng, Minh Tran, Ben Luo, Daniel Coscia, Gustavo Batista, and Hassan Habibi Gharakaheili. 2026. Real-Time and Trustworthy Classification of IoT
12
S. Witt and H. Habibi Gharakheili
Traffic Using Lightweight Deep Learning. IEEE Transactions on Network Science and Engineering 13 (2026), 3256–3273. [27] Jianlin Su, Jiarun Cao, Weijie Liu, and Yangyiwen Ou. 2021. Whitening Sentence Representations for Better Semantics and Faster Retrieval. arXiv:2103.15316 [cs.CL] https://arxiv.org/abs/2103.15316 [28] T. Wang, X. Xie, W. Wang, C. Wang, Y. Zhao, and Y. Cui. 2024. NetMamba: Efficient network traffic classification via pre-training unidirectional Mamba. arXiv:2405.11449 [29] Savindu Wannigama, Arunan Sivanathan, and Hassan Habibi Gharakheili. 2025. Descriptor: UNSW IoT Traffic Data with Packets, Flows, and Protocols (UNSW-IoTraffic). IEEE Data Descriptions 2 (Aug 2025), 311–323. [30] Samuel Witt. 2026. Semantic IoT Behavior. https://github.com/gonzow9/Semantic-IoT-Behavior. GitHub repository. Accessed: 2026-06-11. [31] Lingjing Yu, Bo Luo, Jun Ma, Zhaoyu Zhou, and Qingyun Liu. 2020. You Are What You Broadcast: Identification of Mobile and IoT Devices from (Public) WiFi. In Proc. USENIX Security. Boston, MA, USA. [32] Ruijie Zhao, Mingwei Zhan, Xianwen Deng, Yanhao Wang, Yijun Wang, Guan Gui, and Zhi Xue. 2023. Yet Another Traffic Classifier: A Masked Autoencoder Based Traffic Transformer with Multi-Level Flow Representation. Proceedings of the AAAI Conference on Artificial Intelligence 37 (06 2023), 5420–5427. doi:10.1609/aaai.v37i4.25674 [33] Jiawei Zhou, Woojeong Kim, Zhiying Xu, Alexander M. Rush, and Minlan Yu. 2024. NetFlowGen: Leveraging Generative Pre-training for Network Traffic Dynamics. arXiv:2412.20635 [cs.LG] https://arxiv.org/abs/2412.20635
Semantic Identification of IoT Devices from Behavioral Primitives
A
13
Generative AI Usage Statement
The authors developed, verified, and interpreted all scientific content, experimental design, analysis, figures, tables, and conclusions presented in this paper. ChatGPT 5.5 was used during manuscript preparation for language refinement, editing assistance, and text compaction. All AI-assisted output was reviewed, verified, and validated by the authors. B
Embedding Post-Processing Details
B.1
Whitening and Embedding Normalization
All ACE embeddings produced by BGE-M3 are whitened before downstream analysis and identification. Whitening parameters are estimated using only the reference ACE corpus to avoid query leakage. Let Σ = 𝑈 Λ𝑈 ⊤ denote the covariance eigendecomposition of the reference embeddings. Given a raw embedding 𝑥 ∈ R1024 , the whitened embedding is 𝑥˜ = Λ𝑘−1/2𝑈𝑘⊤ (𝑥 − 𝜇), where 𝜇 is the reference mean embedding and 𝑘 = 256 principal components are retained. All whitened embeddings are subsequently 𝐿2 normalized. B.2
Mean-Pooled Signatures
The aggregated-signature baseline represents an ACE set using a single vector obtained by meanpooling its whitened ACE embeddings. Let 𝑥˜𝑎 denote the whitened embedding of ACE 𝑎, and let 𝜇 (𝐴) =
1 ∑︁ 𝑥˜𝑎 |𝐴| 𝑎∈𝐴
denote the mean embedding of ACE set 𝐴. The corresponding mean-pooled signature is 𝜇 (𝐴) . ∥𝜇 (𝐴) ∥ 2 Candidate-device signatures are constructed from reference MUD profiles, and runtime signatures are constructed from observed ACE sets. 𝑣 (𝐴) =
C
Runtime Query Generation
This appendix describes the generation of runtime queries used in the controlled experiments of §3. The objective is to evaluate identification performance under three forms of overlap degradation: unseen ACEs, lexical endpoint drift, and mixed partial runtime observation. C.1
Exclusively Unseen Behavior
The unseen-behavior setting evaluates identification performance when exact ACE overlap is completely removed. We evaluate three variants. Single unseen ACE: Each runtime query contains one ACE that is removed from all candidate reference profiles before scoring. This produces 1023 evaluation episodes. Unseen ACE family: Semantically related ACEs are grouped into ACE families using reciprocal-nearest-neighbor clustering on whitened ACE embeddings (top-5 neighbors, cosine threshold 0.75). Runtime queries contain up to three ACEs drawn from the same ACE family. Entire families are removed from all candidate profiles before scoring, yielding 103 evaluation episodes. Unseen ACE set: Runtime queries contain three ACEs drawn from distinct ACE families whenever
14
S. Witt and H. Habibi Gharakheili
possible. The selected ACEs are removed from all candidate profiles before scoring. Ten random seeds per eligible device yield 240 evaluation episodes. In all three variants, runtime ACEs share no exact overlap with any ACE in the reference repository. C.2
Exclusively Drifted Endpoints
The drifted-endpoint setting evaluates identification performance under hostname variation while preserving protocol and port semantics. Only ACEs containing domain names (rather than IP addresses) are modified. The goal is to mimic common operational changes in cloud services, such as endpoint renaming, regional deployment differences, or domain migration. Hostname perturbations include: • numeric modification (e.g., time1.google.com → time2.google.com), • regional suffix or token insertion (e.g., api.vendor.com → api-eu.vendor.com), • and top-level domain substitution (e.g., service.example.com → service.example.net). Any perturbation that recreates an ACE already present in the reference repository is discarded and regenerated. We evaluate two variants: All devices, where perturbable ACEs are sampled from the full device corpus, and High-domain devices, where ACEs are restricted to devices containing at least ten domain-name ACEs. C.3
Mixed Partial Observation
The mixed partial-observation setting models incomplete runtime visibility together with optional endpoint drift and unseen ACEs. Each runtime query is generated by varying: • the retained fraction 𝑟 ∈ {0.10, 0.25, 0.50} of ACEs observed at runtime, • the fraction 𝑝 ∈ {0.00, 0.25, 0.50, 1.00} of retained ACEs whose hostnames are perturbed, • and whether one ACE is made unseen relative to the reference profile. Ten random seeds are generated per device per configuration cell. Invalid cells are skipped when no perturbable ACEs remain or when all ACEs become unseen simultaneously. The resulting grid contains 5,772 valid runtime queries across 24 configuration cells.