JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015
1
Multimodal Reasoning with LLM for Encrypted Traffic Interpretation: A Benchmark
（a）Traditional classification methods
Malicious (Tor)
45 00 3c 00 1c 46 40 06 00 c0 a8 01 c0 a8 01 02 00 50
Traditional Classifier "Offsets 42 has a weight of 0.8" (meaning it has no practical operational value)
Raw Traffic Bytes （b）ours Evidence-Grounded Forensic Report
45 00 3c 00 1c 46 40 06 00 c0 a8 01 c0 a8 01 02 00 50
Raw Traffic Bytes
Cognitive LLM
Abstract—Network traffic, as a key media format, is crucial for ensuring security and communications in modern internet infrastructure. While existing methods offer excellent performance, they face two key bottlenecks: (1) They fail to capture multidimensional semantics beyond unimodal sequence patterns. (2) Their “black box” property, i.e., providing only category labels, lacks an auditable reasoning process. We identify a key factor that existing network traffic datasets are primarily designed for classification and inherently lack rich semantic annotations, failing to generate human-readable evidence report. To address data scarcity, this paper proposes a Byte-Grounded Traffic Description (BGTD) benchmark for the first time, combining raw bytes with structured expert annotations. BGTD provides necessary behavioral features and verifiable chains of evidence for multimodal reasoning towards explainable encrypted traffic interpretation. Built upon BGTD, this paper proposes an end-to-end traffic-language representation framework (mmTraffic), a multimodal reasoning architecture bridging physical traffic encoding and semantic interpretation. In order to alleviate modality interference and generative hallucinations, mmTraffic adopts a jointly-optimized perception-cognition architecture. By incorporating a perceptioncentered traffic encoder and a cognition-centered LLM generator, mmTraffic achieves refined traffic interpretation with guaranteed category prediction. Extensive experiments demonstrate that mmTraffic autonomously generates high-fidelity, human-readable, and evidence-grounded traffic interpretation reports, while maintaining highly competitive classification accuracy comparing to specialized unimodal model (e.g., NetMamba). The source code is available at Traffic-Reasoning-Project.
Perception Traffic Encoder
arXiv:2604.08140v1 [cs.CR] 9 Apr 2026
Longgang Zhang, Xiaowei Fu, Fuxiang Huang, and Lei Zhang, Senior Member, IEEE
"class" Malicious (Tor) "traits" TLS 1.3 "evidence" high entropy, low ASCII ratio
"description"
"Automated Tor malware C2 beaconing with high-entropy, lowthroughput payloads
"notes"
Understandable classification results and reasoning details
Persistent and low throughput
Byte-Grounded Knowledge
Fig. 1. Comparison of traffic analysis paradigms. (a) Traditional classification methods that act as a “black box”, providing only a label and low-level feature weights that lack operational value. (b) Our proposed multimodal reasoning framework, composed of a Traffic Perception Encoder and a Cognitive LLM, instructed by Byte-Grounded Knowledge, generating an evidence-grounded report with human-understandable reasoning and executable insights.
relied on statistical features (e.g., packet size distribution and arrival time intervals, etc.) and machine learning techniques, but struggled to adapt to the highly-dimensional and dynamic Index Terms—Encrypted traffic classification, network traffic adversarial nature of modern network traffic. In contrast, interpretation, large language model, multimodal learning. deep learning (DL) models achieved significant performance improvements by automatically extracting hierarchical representations from raw byte sequences. In recent years, inspired by the I. I NTRODUCTION success of self-supervised pre-training in large models, traffic ETWORK traffic analysis is a core pillar for ensuring analysis models based on Transformers [33] and state space network security, implementing intrusion detection, and models (SSMs) [13] are emerged. For example, ET-BERT [19] conducting traffic engineering. With the widespread deployment introduced a masked burst flow model, MPAF [6] proposed of Transport Layer Security (TLS 1.3), Quick UDP Connections a multi-phase attribute fingerprint, YaTC [38] proposed a (QUIC), and anonymous routing networks such as Tor [9], end- multi-level flow representation (MFR) matrix, NetMamba [34] to-end encryption has made payload content extremely opaque. achieved ultra-fast inference using the linear-time complexity of This evolution has rendered traditional Deep Packet Inspection the Mamba architecture, FlowletFormer [22] further optimized (DPI) mechanisms, relying on plaintext signature matching, alignment capabilities by introducing behavior-semantic-aware largely ineffective. Facing this challenge, encrypted traffic Flowlet units, and WF-Transformer [39] further proposed a classification techniques have emerged. These methods heavily Transformer-based temporal feature extraction method. Despite the empirical success of deep representation learning This work was partially supported by National Natural Science Fund of China under Grants 92570110 and 62271090, Chongqing Natural Science models, contemporary cryptographic traffic analysis models Fund under Grant CSTB2024NSCQ-JQX0038, and National Youth Talent remain constrained by two key bottlenecks: (1) Semantic Void in Project. (Corresponding author: Lei Zhang) Unimodal Representations. Existing models essentially perform L. Zhang, X. Fu and L. Zhang are with the School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400044, China. nonlinear boundary partitioning in a high-dimensional space, (E-mail: [email protected], [email protected], [email protected],) directly mapping pure numerical hexadecimal byte sequences Fuxiang Huang is with the School of Data Science, Lingnan University, to classification labels. In complex enterprise environments, Hong Kong, China. (E-mail: [email protected]) Manuscript received April 19, 2015; revised August 16, 2015. security analysts often encounter the “statistical twin” phe-
N
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015
nomenon, i.e., benign traffic and malicious traffic employing obfuscation techniques exhibit almost identical statistical distributions. Relying solely on unimodal sequence patterns makes these models inadequate to capture the rich, multidimensional semantics required to distinguish such threats. (2) Black-box Property and Limitations of Traditional Explainable Artificial Intelligence (XAI). Purely statistical classifiers cannot provide human-readable, auditable, protocol-level forensic evidence to justify their decisions. While post-hoc interpretation techniques (e.g., SHAP [25], LIME [30] and Grad-CAM [31]) attempted to address this, they can only generate importance scores for features or attention heatmaps. For frontline Security Operations Center (SOC) analysts, knowing that “the byte with offset 42 has high weight” is of no operational value unless the byte can be logically mapped to a specific protocol anomaly, such as a malformed handshake frame or an illegal cipher suite. To overcome the aforementioned semantic limitations and black-box constraints, the deep model is expected to learn to map low-level physical bytes to high-level protocol semantics. However, existing network traffic datasets are primarily collected for the traditional classification task, providing only discrete category labels and inherently lacking the rich, multidimensional semantic annotations required, and thus unable to train generative interpretable models. To bridge this fundamental gap, we innovatively construct a Byte-Grounded Traffic Description (BGTD) dataset. To the best of our knowledge, BGTD is the first benchmark that explicitly pairs raw network traffic bytes with structured, rich expert knowledge. To ensure strong generalization capabilities, the dataset integrates six authoritative public repositories covering a broad ecosystem of applications. Beyond basic classification, BGTD provides fine-grained semantic annotations such as discriminative behavioral features, verifiable chains of evidence, and natural language descriptions. These elements are constructed through an automated expert knowledge generation process powered by Claude Opus. By linking numerical payloads with the highlevel forensic information, BGTD provides the key foundational data required for multimodal reasoning. Building upon this multimodal benchmark, this paper proposes an end-to-end, multi-modal traffic-language representation framework (mmTraffic) to overcome the inherent limitations of semantic void and black-box property in traditional traffic classifiers. Unlike traditional pipelines that strictly freeze the traffic encoder to prevent catastrophic forgetting and often lead to weak semantic alignment, mmTraffic advocates for a joint optimization for perception and cognition modules. By introducing an auxiliary classification head in perception and a semantic-priority guided generation mechanism in cognition, our framework explicitly constrains the continuous feature space and forces the large language model (LLM) to perform accurate classification before reasoning. This intrinsically empowers LLM to understand non-semantic traffic bytes and generate human-readable, evidence-grounded reports. Fig. 1 describes the paradigm difference between mmTraffic and others. The main contributions are summarized as follows: • A Byte-grounded traffic description benchmark (BGTD). We construct the first benchmark to explicitly pair raw network traffic bytes with structured expert
2
knowledge. By providing discriminative behavioral traits and verifiable chains of evidence, BGTD bridges the fundamental data-knowledge gap and enables multimodal reasoning towards interpretable encrypted traffic analysis. • A multi-modal traffic reasoning framework (mmTraffic). We reformulate encrypted traffic analysis as a jointly optimized multimodal alignment pipeline. By unfreezing the traffic encoder and training it synergistically with the LLM, we achieve a deep semantic mapping from physical network bytes to human-readable concepts. • Auxiliary constraint and semantic-priority generation. We introduce a classification head to enforce discriminative constraints on the traffic encoder. Furthermore, we design a semantic-priority generation loss that dynamically assigns higher weights to the categorical tokens, effectively mitigating LLM hallucinations in category prediction and ensuring the quality of generated reports. • Superior performance of traffic interpretation with classification. Extensive evaluations across six diverse traffic benchmarks demonstrate that mmTraffic achieves high-fidelity, auditable report generation, while maintaining exceptional classification accuracy. II. R ELATED W ORK A. Self-supervised Methods for Encrypted Traffic Classification Large-scale self-supervised representation learning for network traffic is one of the most significant breakthroughs in cybersecurity in recent years. Early efforts primarily adapted paradigms from natural language processing and computer vision. For instance, ET-BERT [19] pioneered the application of transformer architectures to traffic sequences via binary segmentation and masked burst flow modeling. Conversely, YaTC structured raw traffic as a multi-level flow representation (MFR) matrix, employing a dual-attention masked autoencoder to explicitly capture hierarchical packet interactions. To address computational bottlenecks and structural limitations, recent research has shifted towards efficiency and behavioral semantics. NetMamba [34] innovatively introduced state-space model (SSM) [13] via a stride-based representation, achieving faster inference suitable for high-speed networks. Meanwhile, FlowletFormer [22] moved beyond fixed-length truncation by encoding explicit multi-layer protocol semantics based on coherent behavioral interaction units. Beyond masked modeling paradigms, contrastive learning has also been explored as a self-supervised pre-training strategy for encrypted traffic analysis. For instance, SmartDetector [32] proposes a Semantic Attribute Matrix (SAM) representation and designs a traffic data augmentation method to improve robustness against obfuscation strategies such as dummy packet injection, pretraining the detection model via contrastive learning to learn deep representations from unlabeled traffic data. Despite the diverse architectures and continuous breakthroughs [10] in accuracy, these models share a fundamental limitation: they are entirely constrained by the nature of unimodal black-box classifiers, as shown in Tables I. While they excel at the classification task, they can only map numerical sequences to discrete labels, but fail to reasoning and generate interpretable reports with chains of evidence.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015
3
TABLE I C OMPARISONS OF DIFFERENT PARADIGMS FOR NETWORK TRAFFIC ANALYSIS . MBM, SBP, MAE, MFM, AND FPT REPRESENT M ASKED B YTE M ODEL , S EGMENT B URST P REDICTION , M ASKED AUTOENCODER , M ASKED F LOW M ODEL , AND F LOW P REDICTION TASK , RESPECTIVELY. Model
Traffic Representation
Core Structure
Pre-training
Limitations
ET-BERT [19]
4-hex Bigram / Burst Segmentation
Transformer Encoder
MBM / SBP
Ignores protocol hierarchy; uses natural language subword tokenization
YaTC [38]
Multi-level Flow Representation (MFR) Matrix
Dual-Attention Transformer
MAE (Matrix Masking)
Fixed matrix dimensions; truncates long-range session features
NetMamba [34]
Stride-based Byte Sequence
Unidirectional Mamba (SSM)
Masked Stride Reconstruction
Purely numerical mapping; lacks interpretability
FlowletFormer [22]
Flowlet Behavioral Unit / Field Tokenization
Transformer Encoder
MFM / FPT
Black-box classifier; unable to output forensic reasoning
diverse vision-language tasks. mmTraffic draws the following Early applications of Large Language Models (LLMs) in insight: rather than relying on disparate training stages with cybersecurity were primarily limited to plain text tasks, such as a frozen perception module, we align an active traffic encoder threat intelligence aggregation [2], log parsing, and vulnerability with a language model through a lightweight MLP connector, description summarization. However, recent study begun to empowering the LLM to perform encrypted traffic interpretation explore domain-specific LLMs capable of directly interpret- with rigorous multimodal reasoning. ing underlying telemetry data. TrafficLLM [7] represents a significant attempt to bridge the modality gap. It employs a D. Explainability in Traffic Analysis traffic-domain tokenizer to compress protocol fields by reducing Despite the strong empirical performance of deep traffic token length to an approximately half. While TrafficLLM [7] classifiers, their black-box nature has motivated a growing has demonstrated the feasibility of feeding continuous/discrete body of work on explainable AI (XAI) [5]. Post-hoc techniques telemetry data into an LLM, this one-tower early fusion archi- such as SHAP [25], LIME [30], and Grad-CAM [31] provide tecture suffers from an inherent structural vulnerability. Forcing feature-level attribution scores, but cannot produce protocolan LLM to simultaneously process discrete natural language level forensic evidence for security analysts. While attentiontokens and high-entropy, non-semantic numerical traffic tokens based mechanisms have been extended to model inter-modal within the same attention layers frequently induces modality interactions [23] and structured multimodal representations [14], interference. Consequently, in high-risk intrusion detection, these approaches remain confined to feature-level enhancethis architecture may neglect the authenticity of underlying ment without producing human-readable explanations. DISphysical bytes in order to maintain the fluency of the language, TILLER [1] proposed a multimodal multitask framework that inevitably generating fictitious security alert logic. In contrast, jointly learns traffic representations and human-readable labels, the proposed mmTraffic explicitly mitigates this limitation by but still lacks free-form natural language generation. mmTraffic reformulating the architecture as an end-to-end multimodal addresses this gap by leveraging large language models to framework, fundamentally bridges the modality gap, prevents produce structured, evidence-grounded forensic reports, moving generative hallucinations, and forces the LLM to ground its beyond importance scores toward auditable reasoning chains. reasoning in authentic physical bytes. B. LLMs for Network Security
III. BGTD B ENCHMARK FOR T RAFFIC R EASONING C. Multimodal Alignment and Cross-Modal Fusion
A. Overview
The problem of bridging heterogeneous modalities is wellA benchmark that explicitly links raw traffic analysis data studied in the vision-language domain. Early approaches to with expert-level semantic reasoning is a prerequisite for cross-modal alignment include graph-based relational mod- training a multimodal traffic reasoning framework, but is eling [15] and semantic-driven hashing for large-scale re- still unexplored. Therefore, we develop a Byte-Grounded trieval [4], which established the importance of preserving Traffic Description (BGTD) benchmark, which, to the best semantic correspondences across modalities. CLIP [27] demon- of our knowledge, bridges the data scarcity and persistent strated that contrastive alignment between image and text data-knowledge gap in encrypted traffic interpretation for encoders produces powerful transferable representations. Subse- the first time. To ensure the diversity of data distribution quent works such as LLaVA [20] and InstructBLIP [8] extended and scenarios, the BGTD dataset integrates six authoritative this paradigm by using lightweight projection connectors to map public traffic repositories, covering different network behaviors, frozen visual encoders into the token space of large language application ecosystems, and encryption protocols. Specifically, models, enabling instruction-following behavior over visual BGTD deeply integrates cross-platform mobile application inputs. Flamingo [3] further showed that cross-modal fusion via traffic (i.e., CrossPlatform-Android [29] and CrossPlatformgated attention layers enables few-shot generalization across iOS [29]), cutting-edge TLS 1.3 encrypted web communication
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015
4
Category Extraction
Class 1 Class 2 Class 3
Class 4
Dataset Generation Pipeline Script
Pcaps Extraction
Split PCAPs
5-Tuple Session Splitting
Raw Traffic Pcaps
Header
Head: First 2 Packets
Mid 6 Packets
Knowledge Base Prompt Template
Tail: Last 2 Packets
Claude Opus-4.6 Knowledge Base
Global Stats Drop: Class < Min Samples
Truncate: Random Sample to Max Cap
Class 1 Class 3
Class 2
Packets List
category "class"： Trafficlabel
"traits"：
Global & Local Features
Payload
Target Byte-level features byte-level
supporting the "evidence"： classification
NPY Fixed Length
Entropy & Payload
64 byte Class 4
Global Statistical features and Local Byte-level attributes
Pattern Match
96 byte
（a）Session Extraction and Class Balancing （b）Fixed-Length Truncation and NPY Array Generation
Natural language
"description"：summary of
traffic type and behavior
"notes"：
Related suggestions
（c）Automated Expert-Knowledge Generation Pipeline
Fig. 2. Pipeline of developing BGTD dataset: (a) session extraction and class balancing from raw PCAP files, (b) fixed-length 10 × 160 NPY array generation via priority-based packet sampling, and (c) LLM-assisted ground-truth synthesis using Claude Opus-4.6 prompted as a senior network security expert.
B. Session Extraction and Class Balancing As shown in Figure 2 (a), the raw PCAP files from various datasets undergo a multi-stage preprocessing. First, each PCAP file is partitioned into the standard five-tuple format (i.e., source IP address, destination IP address, source port, destination port, protocol). To mitigate the impact of the long-tail distribution, the original dataset is filtered by category, with lower and upper sample thresholds applied. Categories below the lower threshold are removed, while categories above the upper threshold are sampled according to the threshold. Specific processing methods for each dataset are provided in Sec. V-A1. The statistics of the BGTD dataset are shown in Figure 3. C. Fixed-Length Truncation and NPY Array Generation