JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
1
FDIFormer:Protocol-Aware Transformer Learning for False Data Injection Attack Detection in Smart Grid Networks
arXiv:2607.06213v1 [cs.CR] 7 Jul 2026
Sandara Sathsarani Wijethunga, Muneeb Ul Hassan, Member, IEEE, and Nasrin Sohrabi, Member, IEEE
Abstract—Smart grids use communication networks and intelligent electronic devices for reliable and automated operation of power systems. As these systems become increasingly interconnected, they are also becoming more exposed to cyberattacks such as message tampering, false command injection, and denialof-service attacks. One particularly concerning threat is False Data Injection (FDI), where attackers manipulate communication messages by deleting, modifying, or adding packets. This is especially crucial in IEC 61850-based substations, where Generic Object-Oriented Substation Event (GOOSE) messages are used for delivering time-critical protection and control information between devices. Detecting FDI attacks in IEC 61850 GOOSE traffic remains challenging because malicious packets can closely resemble legitimate communication, making them difficult to distinguish from normal operational behaviour. Furthermore, many existing detection methods depend heavily on manually engineered protocol features, which require extensive domain knowledge and may not generalise well across different smart grid environments. This paper proposes FDIFormer, a featureengineering-free framework for FDI attack detection using structured textual representations of GOOSE packet sequences and fine-tuned pre-trained Transformer models. The proposed framework converts protocol packets into structured text windows that capture communication behaviour and enables Transformer models to learn attack-related patterns directly from the data. The framework is evaluated using the QUT-ZSS-2023-GOOSE dataset under a scenario-level three-fold cross-validation strategy. Experimental results show that GraphCodeBERT achieves an MCC of 0.595 ± 0.122, achieving performance comparable to the strongest feature-engineered baseline, XGBoost (MCC = 0.604 ± 0.121), while improving MCC by 0.133 compared with the TF-IDF baselines. These findings demonstrate that pre-trained Transformer representations can provide an effective technique for FDI attack detection in IEC 61850 GOOSE communication without relying on manually engineered protocol features. Index Terms—IEC 61850, GOOSE protocol, False Data Injection, Transformer models, pre-trained language models, smart grid cybersecurity, intrusion detection, feature-engineering-free detection.
I. I NTRODUCTION MART Grids have become an important part of modern power systems because they improve efficiency, reliability, and real-time monitoring of electricity networks [19], [20]. With the switch of substations to digital architectures, standardised communication protocols have become essential for coordinating the growing number of interconnected devices
S
S. S. Wijethunga, M. U. Hassan, and N. Sohrabi are with the School of Information Technology, Deakin University, Australia (email: [email protected]; [email protected]; [email protected]).
[36]. The IEC 61850 standard has been widely used in modern substations to support communication between Intelligent Electronic Devices (IEDs), providing a common framework for interoperability and automation [21], [22]. The Generic Object-Oriented Substation Event (GOOSE) protocol is one of the communication services defined by IEC 61850, which is especially important for fast, eventdriven exchange of protection and control information between substation devices [36]. GOOSE messages are developed for speed and reliable communication over shared Ethernet networks and are time-sensitive, not designed for security and thus subject to being hacked [24]. IEC 61850 improves operational performance, but it also introduces cybersecurity challenges. Since GOOSE messages prioritise speed and reliability over security, attackers may exploit these weaknesses to manipulate communication traffic. One of the most critical threats is the False Data Injection (FDI) attack, where malicious data is inserted or modified to influence the operation of power system devices [21], [24]. In such attacks, an adversary may delete, modify, or inject GOOSE messages to mislead protection relays or controllers into making incorrect decisions [24]. Successful FDI attacks can result in incorrect protection actions, equipment damage, or even power system outages. To address these threats, researchers have proposed various intrusion detection systems based on machine learning and deep learning techniques [23]. Many existing approaches achieve strong detection performance using manually engineered protocol features. However, these methods often require significant domain expertise and can be difficult to adapt across different environments. In addition, the effectiveness of such systems is often highly dependent on the quality of the engineered features. Recent advances in pre-trained language models have shown strong performance across a wide range of tasks beyond natural language processing [33]. Models based on the Transformer architecture, including BERT [25], DistilBERT, CodeBERT [29], and GraphCodeBERT [30], have shown a strong ability to learn useful patterns from structured sequential data. This suggests that GOOSE communication traffic can be represented as structured text and analysed using pre-trained Transformer models, reducing the need for manually designed features. Using large language models for cybersecurity problems has attracted growing interest from researchers over recent years [34]. However, there is still limited research on fine-tuned pretrained Transformer models for FDI detection in IEC 61850
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
GOOSE communication. Moreover, there is little evidence that clearly isolates how much of the detection performance is driven by pre-trained Transformer representations compared to baseline machine learning models using the same textual input representation. To address this gap, this paper proposes FDIFormer, an endto-end framework for detecting FDI attacks in IEC 61850 GOOSE traffic. FDIFormer transforms raw GOOSE packet sequences into structured textual representations and finetunes pre-trained Transformer models — including BERT [25], DistilBERT, CodeBERT [29], and GraphCodeBERT [30] — directly on these representations, eliminating the need for manual feature engineering. Unlike approaches that simply apply an existing model off the shelf, FDIFormer defines a complete pipeline that includes packet-to-text conversion, sequence windowing, and model fine-tuning, with GraphCodeBERT serving as the core encoder that captures both semantic and structural patterns in GOOSE traffic. FDIFormer is evaluated against engineered-feature baselines, TF-IDF baselines, and hybrid architectures under a common scenario-level crossvalidation framework, and shows competitive detection performance while removing the dependency on domain-specific feature design [31]. The contributions of this paper are as follows: 1) A structured textual representation is developed for IEC 61850 GOOSE packet sequences so that pre-trained Transformer models can process protocol traffic without any manual feature engineering. 2) FDIFormer is proposed as a complete detection pipeline that integrates packet-to-text conversion, sequence windowing, and fine-tuned Transformer-based classification, with GraphCodeBERT as the core encoder, enabling effective FDI attack detection directly from raw GOOSE traffic. 3) The proposed Transformer-based approach is compared with engineered-feature baselines and TF-IDF baselines using the same experimental framework. 4) The contribution of pre-trained Transformer representations is analysed by comparing Transformer models with classical machine learning models using the same structured text input. 5) Model performance is analysed across different FDI attack behaviours to understand which attack types are more difficult to detect. II. R ELATED W ORKS This section reviews the studies that are most closely related to our work and highlights the key ideas that informed the development of the proposed approach. We start by discussing how large language models have been fine-tuned for specific domains, which forms the basis of our approach to adapt pretrained Transformer models for analysing protocol traffic. Next we review the use of LLMs in cybersecurity and their growing use in smart grid applications. Finally, we study conventional smart grid intrusion detection methods based on manually engineered features, which are used as our baseline models. Together, these areas provide the background for this research and help identify the gap that FDIFormer aims to address.
2
A. LLM Fine-Tuning for Domain-Specific Tasks Recent studies have proven that large language models can be adapted successfully to specialised areas through finetuning. Zhang et al. [1] enhanced a fine-tuned LLaMA-3.2-3B model with BiLSTM and CRF layers for cybersecurity named entity recognition, achieving an F1-score of 98.88%. Their results demonstrate the potential of pre-trained language models when sufficient domain-specific training data is available. Among the studies reviewed, the work of Saber et al. [4] s the most closely aligned with the objectives of this research. They converted relay measurements into structured text prompts and fine-tuned DistilBERT and GPT-2 for cyberattack detection in protective relays. Their results demonstrated that Transformer models can effectively learn from textual representations of power system data. Mehavilla et al. [3] compared fine-tuned LLMs with machine learning baselines using network flow data and reported that XGBoost still provided a strong balance between accuracy and efficiency. Yang et al. [2] proposed LLM-APTDS for advanced persistent threat detection and demonstrated how LLMs can be used for complex cybersecurity reasoning tasks. B. LLMs for Cybersecurity Many researchers conducted research into the application of large language models in cybersecurity. Conceicao and Cruz [5] tested several frontier LLMs on various types of cybersecurity tasks, demonstrating that the performance might greatly differ depending on the application domain. Yang et al. [6] introduced LLM-AE-MP which is based on DistilBERT embeddings, combined with autoencoder, LSTM and MLP for web attack detection. Zhang et al. [7] proposed AttacKG+ for converting threat intelligence reports into structured attack graphs, while Belcastro et al. [8] introduced KLAGE, which combines knowledge graphs with LLM-generated explanations to support explainable threat detection. These studies show the increasing capability of LLMs in the cybersecurity field. However, their focus is mostly on general network security issues rather than packet-level FDI detection in IEC 61850 GOOSE communication. C. LLMs for Smart Grid Applications The use of large language models in smart grid applications remains at an early stage of development. Zaboli et al. [9] evaluated anomaly detection in IEC 61850 communication using ChatGPT, Claude and Bard in a hardware-in-the-loop environment. Their study was interesting however mainly reliant on human interpretation without proper model training or comparison. Other applications of LLMs are non-intrusive load monitoring [10], distributed energy resource monitoring [11], semantic interoperability [12], and others in energy systems [13]. These studies highlight growing interest in applying large language models to power system applications; however, none specifically investigates fine-tuned Transformer models for detecting FDI attacks in GOOSE traffic.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
D. Classical Smart Grid Security Most existing smart grid intrusion detection systems rely on manually engineered protocol features combined with traditional machine learning or deep learning models. Alsirhani et al. [14] proposed an AI-based ensemble framework for smart grid intrusion detection, while Ijeh and Morsi [15] applied fine tree bagging for attack classification. Ciaramella et al. [16] used explainable deep learning for intrusion detection, and Zheng et al. [17] proposed federated learning with unsupervised attack detection methods. Although these methods show good performance, they generally depend on expert-designed protocol features and often require recalibration when applied to new environments. This creates challenges for scalability and deployment across different substations. Overall, the study shows growing interest in both smart grid intrusion detection and Transformer-based learning. However, there is still limited research investigating fine-tuned pre-trained Transformer models for FDI attack detection in IEC 61850 GOOSE communication. Furthermore, the specific contribution of pre-trained Transformer representations has not been systematically evaluated against classical machine learning models using the same textual input representation. Based on our review of the existing literature, we found no prior study that presents a complete end-to-end framework for converting raw IEC 61850 GOOSE packet sequences into structured textual representations and using fine-tuned pretrained Transformer models to detect False Data Injection (FDI) attacks without manual feature engineering, as proposed by FDIFormer. In addition, existing studies have not systematically examined the contribution of pre-trained Transformer representations by comparing them with classical machine learning models using the same textual input. To the best of our knowledge, no previous work has also investigated detection performance across different FDI attack behaviours within a scenario-level cross-validation framework. III. P RELIMINARIES A. IEC 61850 GOOSE Communication IEC 61850 is an international standard developed by the International Electrotechnical Commission (IEC) to support communication networks and systems used in power utility automation. It was introduced to overcome the limitations of specific substation communication protocols and to provide a common framework that allows Intelligent Electronic Devices (IEDs) from different suppliers to communicate and exchange information easily [36]. By defining standardised data models, communication services, and configuration mechanisms, IEC 61850 has become the foundation of modern digital substations, enabling higher levels of interoperability, automation, and operational reliability [21]. The standard consists of multiple parts covering areas such as system engineering, configuration, communication services, and protocol mappings. Among its communication services, Generic Object-Oriented Substation Event (GOOSE) and Sampled Values (SV) are the two most important for real-time operation. GOOSE is used to exchange time-critical protection
3
and control information, while SV is used to transmit digitised measurements from instrument transformers. To meet the strict latency requirements of protection systems, both services operate directly over Ethernet multicast communication rather than relying on higher-layer protocols such as TCP/IP [36]. GOOSE is particularly important because it enables the rapid exchange of protection and control signals between IEDs. It follows a publisher-subscriber communication model, where a publishing device broadcasts messages that can be received by multiple subscribing devices on the same network segment without establishing dedicated connections [23]. The GOOSE messages are retransmitted repeatedly after a state change and continue to be sent periodically during normal operation to improve reliability. This mechanism helps ensure that critical information is delivered even in the absence of acknowledgement messages [21]. Each GOOSE message is encoded using ASN.1 Basic Encoding Rules (BER) and contains a structured set of fields that represent the current state of the device. These fields play an important role in ensuring correct interpretation and synchronisation across communicating devices. The main fields include: • APPID: A 16-bit identifier used to uniquely distinguish each GOOSE publisher on the network. • stNum (State Number): A counter that increases whenever the dataset changes, indicating a new state transition. • sqNum (Sequence Number): A counter that increments with each retransmission within the same state, allowing receivers to track message order and freshness. • TimeAllowedToLive: Defines the maximum time (in milliseconds) a subscriber should wait before considering the message lost, which may trigger a fail-safe action. • Dataset: Contains the actual values being communicated, such as status information, quality indicators, and timestamps. • GOOSElength: Represents the total size (in bytes) of the GOOSE protocol data unit (PDU). The timing behaviour of GOOSE retransmissions can be described mathematically. Let T0 represent the initial retransmission interval after a state change, and let Tmax be the maximum heartbeat interval. The retransmission interval Ti at the i-th retransmission is given by: Ti = min(T0 · 2i , Tmax )
(1)
This exponential back-off mechanism ensures that messages are transmitted frequently immediately after a state change, when timely updates are most important, and less frequently during stable operating conditions to reduce network load. Any deviation from this expected pattern—such as irregular state transitions, unexpected sequence number changes, or abnormal timing behaviour—can be an indicator of potential cyberattacks [21]. Although GOOSE improves communication efficiency and reliability in substations, it was not originally designed with cybersecurity in mind. The messages are broadcast over Ethernet without built-in encryption or authentication, which makes them vulnerable to manipulation if an attacker gains access to
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
4
TABLE I C OMPARATIVE A NALYSIS OF E XISTING S TUDIES R ELATED TO T RANSFORMER M ODELS , S MART G RID C YBERSECURITY, AND FALSE DATA I NJECTION D ETECTION Category
Ref. [1]
Year 2025
Problem Focus Methodology Dataset / Platform Key Findings Cybersecurity named Fine-tuned LLaMA with Cybersecurity article Achieved 98.88% entity recognition BiLSTM and CRF layers corpus score
[2]
2025
Advanced persistent LLM-based APT detection threat detection with graph reasoning
[3]
2026
Flow-based intrusion Fine-tuned LLMs compared detection with ML baselines
[4]
2026
[5]
2025
[6]
2025
[7]
2024
Smart-grid relay cy- Relay measurements berattack detection converted into structured prompts and fine-tuned using Transformer models Cybersecurity task Benchmarking frontier maturity analysis LLMs across cybersecurity tasks Web attack detection DistilBERT embeddings with autoencoder, LSTM, GAN, RL, and MLP Cyber threat intelli- LLM pipeline for attack gence extraction graph construction
[8]
2025
Explainable threat detection
[9]
2024
IEC 61850 anomaly analysis
[10]
2026
Scene-aware energy load monitoring
[11]
2025
[12]
2025
[13]
2026
[14]
2025
[15]
2024
[16]
2025
[17]
2025
Partial tripping detection in distributed energy resources Semantic interoperability in heterogeneous smart grids Review of LLM applications in energy systems Intrusion detection in smart-grid environments Smart-grid cyberattack type classification Explainable smartgrid intrusion detection Privacy-aware cyberattack detection
LLM Fine-Tuning
LLMs for Cybersecurity
LLMs in Smart Grids
Smart Grid Security
Research Gap F1- Focused on entity extraction, not intrusion detection in smart-grid traffic Provenance and APT- Improved attack detec- Not evaluated for IEC related security data tion precision and inter- 61850 or smart-grid pretability communication traffic Zeek network flow text LLMs worked on struc- Did not focus on IEC 61850 data tured network data, but GOOSE protocol or FDI atXGBoost remained com- tacks petitive Protective relay cyberat- Textualised power-system Does not analyse packettack dataset data supported attack de- level GOOSE traffic or FDI tection attack behaviour Malware, honeypot, and Showed that LLM perforCTF tasks mance varies across cybersecurity tasks Web attack datasets Improved web attack detection under imbalance
Not focused on smart-grid protocols or GOOSE traffic
Survey of LLM roles, enhancement methods, and application areas AI-based ensemble intrusion detection framework
Energy systems litera- Identified growing interest ture in LLMs for smart-grid applications Smart-grid intrusion de- Achieved high detection tection datasets accuracy using combined AI models Fine tree bagging ensemble Physical and network Achieved strong attack with feature selection feature dataset classification performance
Shows limited work on protocol-level smart-grid cybersecurity Requires engineered features and labelled datasets
CNN-based intrusion detection with explainable AI methods Autoencoder-based learning with federated detection framework
Depends on manually selected protocol and network features Does not evaluate pretrained Transformer representations Not focused on pre-trained language models or GOOSE FDI detection
Designed for web traffic, not industrial GOOSE communication Cyber threat Converted unstructured Not a real-time packet-level intelligence reports reports into structured intrusion detection system attack graphs Knowledge graph, Graph- Network security data Improved explainability Does not address IEC 61850 BERT, LIME, and LLM exfor threat detection GOOSE FDI detection planations Evaluated ChatGPT, Hardware-in-theShowed early feasibility Exploratory study without Claude, and Bard for smart- loop IEC 61850 test of LLMs for GOOSE- systematic fine-tuning or grid cybersecurity tasks environment related analysis model comparison LLM-based non-intrusive Smart-grid energy con- Improved contextual un- Focused on energy moniload monitoring sumption data derstanding of load be- toring, not cybersecurity or haviour FDI detection LLM-based event interpre- IEEE test network envi- Improved event analysis Does not address intrusion tation and reasoning ronment for distributed energy re- detection in GOOSE comsources munication Fine-tuned LLaMA with Smart-grid Improved semantic inte- Does not focus on cyberatLoRA and retrieval- interoperability data gration between heteroge- tack detection or FDI analaugmented generation neous grid systems ysis
the network. Although IEC 62351 later introduced security enhancements for IEC 61850 systems, its adoption in practice remains limited due to additional processing overhead and the strict timing constraints of protection systems [37]. As a result, operational GOOSE traffic in many substations is still exposed to potential network-level attacks within the local area network. B. False Data Injection Attacks False Data Injection (FDI) attacks are considered as one of the most serious cyber threats for industrial communication systems, including IEC 61850-based substations [21]. FDI attacks are more subtle than denial-of-service attacks, which aim to stop communication entirely. They operate by changing
IEC protocol intrusion Improved interpretability detection data of deep learning-based detection Distributed smart-grid Improved privacydata preserving detection across distributed settings
the content or flow of valid messages thus monitoring systems and protection relays perform faulty operational decisions, while appearing as normal network activity [22], [23]. The attacks on FDI in IEC 61850 GOOSE communication are generally categorised into three major types based on the attacker’s manipulation of the messages: 1) Message Injection (Addition): In this type of attack, the adversary pretends to be a legitimate IED publisher to produce and inject fake GOOSE packets into the network. In these forged messages, invalid values may be inserted for the datasets. For instance, a false breaker trip command, which can cause unintended actions in subscribing devices. Let mt be a valid GOOSE message at time t in formal terms. An injection attack produces a false message m̃t such that:
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
m̃t = mt + δt
5
(2)
where δt denotes the malicious changes introduced by the attacker to the dataset or protocol fields. 2) Message Modification: In a modification attack, the attacker intercepts a valid GOOSE message and changes one or more of its fields before sending the message to its intended recipients. These changes can impact the values within a dataset, sequence data such as stNum or sqNum, or even timing fields. This process can be summarised as: m′t = f (mt )
(3)
where f (·) denotes the malicious transformation of the original message mt . The modified packet would still be valid with respect to the correct GOOSE structure and often hard to detect with traditional signature-based techniques. 3) Message Deletion (Suppression): For a deletion or suppression attack, the adversary selectively drops GOOSE messages before they reach their intended subscribers. This can be especially harmful if messages that correspond to a real state change are missing, as this can cause devices to assume a fault or to initiate fail-safe behaviour. Let M be the set of expected GOOSE messages in time window [t1 , t2 ] A deletion attack leads to a received subset M′ ⊂ M such that: |M′ | < |M|
(4)
which implies that the attacker has deleted some messages. Successful FDI attacks may have an adverse effect on IEC 61850 GOOSE communication. They may lead to unnecessary operations of breakers, malfunctioning of protection devices, damage to equipment and even cascading failures throughout the power system [22], [24]. Furthermore, these attacks are difficult to detect because FDI traffic generally preserves the same structural format as legitimate GOOSE messages, making it difficult to distinguish malicious activity from normal operational communication [21]. C. Pre-Trained Transformer Models Transformer models, first introduced by Vaswani et al. [35], are a family of deep learning architectures based on selfattention mechanisms that enable the learning of relationships between elements in a sequence regardless of their distance from each other. Unlike recurrent models such as LSTMs and GRUs, Transformers do not process tokens in a sequence sequentially, but in parallel. This enables more efficient training on large datasets and improves their ability to capture longrange dependencies [33]. At the core of this architecture is the self-attention mechanism. This mechanism receives a sequence of tokens and returns a weighted representation of each token in the sequence relative to all the other tokens. The attention operation takes as input a query matrix Q, a key matrix K and a value matrix V. It is computed as follows:
QK⊤ Attention(Q, K, V) = softmax √ V dk
(5)
Here, dk is the dimension of the key vectors and the scaling factor √1d is used to prevent very large dot-products, which k make training unstable [35]. In practice, multiple attention heads are used in parallel so that the model can attend to different aspects of the input simultaneously: MultiHead(Q, K, V) = Concat(head1 , . . . , headh )WO , (6) where each headi is defined as Attention(QWiQ , KWiK , VWiV ), and WiQ , WiK , WiV , and WO are learnable parameters. Pre-trained Transformer models build on this architecture by pre-training on large-scale datasets with self-supervised learning objectives and then fine-tuning the learned representations on specific downstream tasks [31]. This two-stage process allows the models to learn general-purpose representations during pre-training and then to adapt efficiently to new tasks with relatively small labelled datasets. The models used in this work are summarised below. 1) BERT: BERT (Bidirectional Encoder Representations from Transformers) [25] proposed bidirectional pre-training for language models. Rather than processing text in one direction, BERT learns context from both sides of a token (left and right). It is trained using Masked Language Modelling (MLM), where some tokens are masked and the model predicts them, and Next Sentence Prediction (NSP), where the model learns whether two sentences follow each other in a text. This bidirectional design allows BERT to produce rich contextual embeddings which are useful in many NLP tasks. 2) DistilBERT: DistilBERT [26] is a lighter and faster BERT version obtained by knowledge distillation. It is 40% smaller and 60% faster than BERT-base, however maintains most of its performance. The model removes some components like token-type embeddings and the pooler layer, and reduces the number of transformer layers from 12 to 6. This makes it more feasible when computational resources are limited. 3) RoBERTa: RoBERTa (Robustly Optimised BERT Pretraining Approach) [27] is an improvement over BERT by optimising the training process. It removes the NSP objective, uses larger batch sizes, is trained on more data, and uses a larger byte-pair encoding vocabulary. These changes enable the model to better use its capacity and consistently improve its performance on benchmarks. 4) ELECTRA: ELECTRA [28] proposes a more efficient pre-training method called Replaced Token Detection. Instead of masking tokens , the small generator replaces some tokens and a discriminator is trained to identify which tokens were replaced . The model is trained on all tokens rather than masked ones, thus achieving strong performance with much improved sample efficiency.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
6
5) CodeBERT: CodeBERT [29] is a bimodal pre-trained model trained on both source code and natural language descriptions. During pre-training, it uses objectives such as MLM and replaced token detection. Its strength in modelling structured and syntax-rich inputs makes it an ideal solution to represent GOOSE packet sequences, which are also expressed in a structured key-value format similar to code. 6) GraphCodeBERT: GraphCodeBERT [30] improves CodeBERT by adding structural information from data flow graphs. It also learns the information flow between variables in a sequence, in addition to token-level relations. This enables the model to learn deeper structural dependencies. It is particularly suitable for analysing GOOSE packet representations because it can model structured relationships. 7) ModernBERT: ModernBERT is a recent encoder-only Transformer architecture with several architectural improvements, including rotary positional embeddings, a combination of local and global attention layers, and an extended context window of up to 8,192 tokens. These improvements make it more suitable for longer structured sequences such as multipacket GOOSE windows. In this work, all seven Transformer models described above are fine-tuned using structured textual representations of GOOSE packet sequences. The implementation uses the HuggingFace Transformers framework [31]. This configuration allows us to evaluate the performance of these models in FDI attack detection, without relying on manual feature engineering in IEC 61850 GOOSE communication.
D. Matthews Correlation Coefficient (MCC) Model performance is evaluated using the Matthews Correlation Coefficient (MCC). Unlike accuracy, which can be misleadingly high when a model predicts only the majority class. MCC incorporates all elements of the confusion matrix and produces values ranging between −1 and +1. Given the number of true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN), MCC is defined as [32]:
MCC = p
TP × TN − FP × FN (T P + F P )(T P + F N )(T N + F P )(T N + F N )
(7)
A value of 0 represents random performance, and +1 represents perfect prediction. Therefore, MCC is more suitable of realistic detection capability in imbalanced binary classification cases.
IV. FDIF ORMER S YSTEM M ODEL This section describes the overall design and architecture of the proposed FDIFormer framework. As illustrated in Fig. 1, the FDIFormer framework consists of three main components: the Smart Grid Substation Environment, the FDI Attack Layer, and the Transformer-based Detection System, supported by an offline training pipeline.
A. Smart Grid Substation Environment The considered environment is a digital substation where multiple Intelligent Electronic Devices (IEDs) communicate continuously over a substation local area network (LAN) using IEC 61850 GOOSE messages [21]. As shown in Zone 1 of Fig. 1, the setup includes three IEDs: two transformer protection units (XFMR1 and XFMR2) and one feeder protection unit (FDR). These devices are connected through an Ethernet switch and exchange protection and control information using the GOOSE publisher-subscriber mechanism [36]. During normal operation, each IED broadcasts GOOSE messages periodically based on a configured heartbeat interval and also transmits rapid updates whenever a state change occurs. All traffic passing through the Ethernet switch is passively captured using a network tap and packet sniffer deployed on the substation LAN. The captured packets are stored as PCAP files and later processed by the FDIFormer pipeline. Since the monitoring is fully passive, the system does not introduce any additional delay into the protection communication path, making it suitable for strict real-time substation requirements [23]. B. Adversary Model The threat model is illustrated as Zone 2 in Fig. 1. We assume that the attacker has gained access to the substation local area network and can monitor, intercept, and manipulate GOOSE traffic as it passes through the Ethernet switch. The attacker does not have physical access to any IED and cannot directly control or reconfigure these devices. As mentioned before, the attacker can perform three main types of FDI attacks: message injection, message modification, and message deletion. The goal of the attacker is to influence the behaviour of protection relays and IEDs, potentially leading to incorrect breaker operations, suppression of valid trip signals, equipment damage, or even cascading failures across the power system [22], [24]. A common characteristic of these attacks is that they are designed to remain stealthy. The manipulated packets still follow the same structural format as legitimate GOOSE messages, which makes them difficult to detect using only rule-based or protocol conformance checks. As a result, detecting such attacks requires learning-based methods that can capture subtle patterns across sequences of packets rather than relying only on individual message inspection [22]. C. Motivation Most existing intrusion detection systems for IEC 61850 GOOSE traffic rely on manually engineered protocol features. These approaches often require deep domain knowledge, are time-consuming to design, and do not generalise well across different substations [22]. In addition, the effectiveness of these approaches depends heavily on the quality of manually engineered features, meaning that previously unseen attack patterns may remain undetected if they are not represented by those features [23]. The main motivation behind FDIFormer is to remove this dependency on manual feature engineering. Instead, raw
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
GOOSE packet sequences are converted into structured textual representations and analysed using pre-trained Transformer models. As discussed earlier, the structured key-value format of GOOSE traffic is similar to the syntax of programming languages, which makes it well-suited for code-aware models such as GraphCodeBERT [30]. This provides a natural opportunity to apply pre-trained representations for FDI detection without relying on domain-specific feature design. D. System Model and Structure As shown in Fig. 1, the FDIFormer framework consists of three main components: (1) the Smart Grid Substation Environment, (2) the FDI Attack Layer, and (3) the Transformerbased Detection System, supported by an offline training pipeline. The data flow between these components is described below. 1) Packet Capture Module: GOOSE traffic is captured passively from the substation network using a network tap and packet sniffer. The captured PCAP files are forwarded to the packet processing engine, where relevant IEC 61850 fields are extracted from each packet. These include APPID, stNum, sqNum, GOOSElength, num of data, and payload information. Additional derived features such as ∆stNum, ∆sqNum, state and sequence reset flags, and inter-packet time delta are also computed. APPID values are mapped to their corresponding IED names (XFMR1, XFMR2, FDR) to improve interpretability and generalisation. 2) Numerical Feature Set: The packet processing engine generates a numerical feature vector Fw for each packet window, as shown in Fig. 1. This includes state and sequence variation features, reset indicators (st reset, sq reset), lengthbased anomalies, payload change indicators, and other aggregated statistics. The feature set Fw is used by traditional machine learning baselines and hybrid models in the experimental evaluation. 3) Structured Text Generator: Each packet window is transformed into a structured text representation by the structured text generator. Five consecutive packets from the same IED are grouped using a sliding window with stride three, forming a window-level representation Tw . Each window is prefixed with the IED identifier and an estimated attack-type label (Normal, Structural, Value, or Sequence), which is derived from observed anomaly patterns within the window. This prefix provides additional contextual information to the Transformer model before processing individual packet fields. 4) Detection Engine: The structured text representation Tw is tokenised and passed into a fine-tuned Transformer model. As shown in Fig. 1, the framework supports several pre-trained models, including BERT, RoBERTa, DistilBERT, CodeBERT, GraphCodeBERT, ELECTRA, and ModernBERT. Each model is fine-tuned as a binary classifier for detecting FDI attacks. The decision module applies a probability threshold θ, selected using validation performance based on MCC, to produce the final prediction ŷ ∈ {0, 1}, where 0 represents benign traffic and 1 represents an attack. 5) Alert and Monitoring Module: If a window is classified as an attack, the alert module generates logs and notifications
7
for operators. The system outputs include predicted labels (benign or FDI), attack category information, event logs, alerts, and performance summaries for monitoring and analysis. E. Offline Training Environment As shown at the bottom of Fig. 1, the FDIFormer model is trained offline using historical GOOSE datasets containing both normal and attack traffic. The dataset is first converted into windowed samples Dw = {Tw , Fw , Yw } and then split into training, validation, and test sets at the scenario level to avoid data leakage. The pre-trained Transformer models and hybrid baselines are trained using this offline dataset and later deployed for inference on unseen or live traffic. In this work, the QUTZSS-2023-GOOSE dataset [18] is used for all training and evaluation experiments. Fig. 1 illustrates the overall FDIFormer:Protocol-Aware Transformer Learning system model used throughout this research. V. M ETHODOLOGY OF FDIF ORMER This research follows a quantitative experimental methodology to evaluate the effectiveness of pre-trained Transformer models for detecting False Data Injection (FDI) attacks in IEC 61850 GOOSE communication. The overall framework consists of data preparation, structured text generation, model training, and performance evaluation. Fig. 2 shows the overall methodology used in this study. A. Methodology Overview The proposed FDIFormer pipeline consists of six main stages, as illustrated in Figure 2. Stage 1 — Data Acquisition: The QUT-ZSS-2023-GOOSE dataset [18] is loaded from CSV files. It includes 11 benign operational scenarios and 9 FDI attack scenarios, identified in the raw dataset using scenario codes (e.g., 821-823, 841-843, and 861-863), as detailed in Table II for the attack scenarios). Each record is labelled in a binary format, where benign traffic is assigned label 0 and FDI attacks are assigned label 1. In total, the dataset contains 46,551 packets collected from three IEDs (XFMR1, XFMR2, and FDR) across 20 scenarios. Stage 2 — Packet Processing: Each GOOSE packet is parsed to extract key protocol fields such as APPID, stNum, sqNum, GOOSElength, num of data, and payload values. Additional derived features are also computed, including changes in state and sequence numbers, reset indicators, and time differences between packets. The APPID is mapped to a meaningful IED name, and each packet is then converted into a structured text format. Stage 3 — Window Construction: Packets are grouped by IED and scenario, and a sliding window approach is applied with a window size of 5 and stride of 3. This produces both structured text windows (Tw ) and numerical feature sets (Fw ). Each window is also prefixed with the corresponding IED name and a high-level indication of the expected behaviour (e.g., normal or attack-related). A window is labelled as an attack if at least one packet inside it contains FDI activity.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
8
Fig. 1. Overview of the proposed FDIFormer framework for detecting False Data Injection attacks in IEC 61850 GOOSE traffic. The framework transforms GOOSE packet sequences into structured text windows and evaluates Transformer, engineered-feature, TF-IDF, and hybrid models for attack classification.
Stage 4 — Data Preparation: After windowing, the dataset is cleaned by removing invalid or overlapping records. The final dataset Dw = {Tw , Fw , Yw } is then split using a scenariolevel three-fold cross-validation strategy. This ensures that no scenario appears in more than one of training, validation, or testing sets, preventing data leakage. Stage 5 — Model Training and Evaluation: Four groups of models are evaluated under the same conditions: engineered-feature models using Fw , TF-IDF models using Tw , Transformer-based models using fine-tuned pre-trained encoders, and hybrid models combining both text and numerical features. For each model, the decision threshold is tuned using the validation set to maximise MCC, and final evaluation is performed on unseen test data. Stage 6 — Results Comparison: Results are reported as the mean and standard deviation across all folds to provide both average performance and variability. The main comparison focuses on Transformer models versus TF-IDF baselines using the same text input, which highlights the benefit of pre-trained representations. Additional comparisons are made against engineered-feature models to evaluate performance differences across feature types. B. Structured Text Representation A key part of this work is converting raw GOOSE packets into structured text, which allows Transformer models to process network traffic without manual feature engineering.
Each packet is transformed into a sequence containing: • The IED name derived from APPID (XFMR1, XFMR2, FDR) • State and sequence numbers (stNum, sqNum) • Changes in state and sequence values • Reset indicators for state and sequence • Protocol length and dataset size information • Payload variation indicators • Binary anomaly flags based on training statistics • Time difference between packets Five consecutive packets are then combined using a sliding window to form Tw . This window-based representation enables the model to capture both short-term behaviours and longer sequential patterns within network traffic. The label for each window is defined as: ( Yw =
1 0
if any packet in the window is FDI otherwise
(8)
C. Cross-Validation Strategy A scenario-level three-fold cross-validation approach was used to ensure a fair evaluation Training, validation, and testing scenarios were separated at the scenario level to prevent data leakage between folds. All anomaly statistics and threshold selection procedures were calculated using training data only. Table II details the fold assignments.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
Fig. 2. Methodology flowchart of the proposed Transformer-based FDI detection framework.
TABLE II S CENARIO - LEVEL DISTRIBUTION OF FDI ATTACK TYPES ACROSS THE THREE CROSS - VALIDATION FOLDS . Fold Training FDI Scenarios 1
2
3
Validation FDI Sce- Testing FDI Scenarios narios 842 (Modification), 843 (Ad- 821 (Deletion), 822 841 (Deletion), 863 dition), 861 (Deletion), 862 (Modification), 823 (Addition) (Modification) (Addition) 821 (Deletion), 841 (Deletion), 823 (Addition), 842 822 (Modification), 863 843 (Addition), 862 (Modifica- (Modification), 861 (Addition) tion) (Deletion) 841 (Deletion), 861 (Deletion), 821 (Deletion), 842 822 (Modification), 823 862 (Modification), 863 (Addi- (Modification), 843 (Addition) tion) (Addition)
D. Model Development To evaluate the effectiveness of different learning approaches for FDI attack detection, three model categories were investigated: Transformer-based models, classical ma-
9
Algorithm 1 Proposed Transformer-Based Detection of FDI Attacks in GOOSE Traffic Require: Dataset D, scenario set S, Transformer model set MT , window size W , stride r, folds K Ensure: Transformer predictions ŶT and evaluation results RT 1: Load GOOSE traffic files from dataset D 2: Select benign and FDI attack scenarios from S 3: Assign binary labels: benign → 0, FDI attack → 1 4: for each GOOSE packet p ∈ D do 5: Extract APPID, stNum, sqNum, GOOSElength, num of data, and payload values 6: Compute ∆stNum, ∆sqNum, st reset, sq reset, length anomaly, payload anomaly, and time delta 7: Map APPID to IED semantic name 8: Construct attack-type hint 9: Convert packet p into structured text representation Tp 10: end for 11: for each scenario s ∈ S do 12: Group packets by scenario and APPID 13: Apply sliding window of size W with stride r 14: Construct window-level text representation Tw 15: Assign label Yw = 1 if any packet in the window is FDI; otherwise Yw = 0 16: end for 17: Split text-window dataset using scenario-level K-fold cross-validation 18: for each fold k = 1 to K do 19: Separate scenarios into train, validation, and test sets 20: Compute class weights using training-fold counts only 21: for each Transformer model m ∈ MT do 22: Tokenise Tw using the tokenizer of model m 23: Fine-tune m for binary classification 24: Tune threshold θ on validation set using MCC 25: Predict on unseen test scenarios 26: Compute MCC, Balanced Accuracy, F1-score, ROC-AUC, and PR-AUC 27: end for 28: end for 29: Aggregate results across all folds return Final Transformer evaluation results RT
chine learning baselines, and hybrid Transformer-numerical architectures. 1) Transformer-Based Models: The first model category consists of fine-tuned pre-trained Transformer models. Structured text windows generated from GOOSE packet sequences were used as inputs to BERT-base, DistilBERT, RoBERTa, ELECTRA-small, CodeBERT, GraphCodeBERT, and ModernBERT. Each model was fine-tuned as a binary classifier to detect between benign and FDI attack traffic. 2) Classical Machine Learning Baselines: The classical machine learning baselines evaluated two baseline approaches. The first used manually engineered protocol features extracted from GOOSE traffic and trained using XGBoost and LightGBM classifiers. The second used TF-IDF representations generated from the same structured text windows and applied
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
Algorithm 2 Classical Machine Learning Baseline Evaluation Require: Windowed dataset Dw , engineered features Fw , text windows Tw , baseline models B, folds K Ensure: Baseline evaluation results RB 1: Split Dw using scenario-level K-fold cross-validation 2: for each fold k = 1 to K do 3: Separate training, validation, and testing scenarios 4: Fit all preprocessing steps using training data only 5: Fit TF-IDF vectoriser using training text windows only 6: for each baseline model b ∈ B do 7: Train b using engineered numerical features Fw 8: Tune threshold θ on validation data using MCC 9: Evaluate b on unseen test scenarios 10: Train b using TF-IDF text features 11: Tune threshold θ on validation data using MCC 12: Evaluate b on unseen test scenarios 13: Record MCC, Balanced Accuracy, F1-score, ROCAUC, and PR-AUC 14: end for 15: end for 16: Aggregate fold-wise results as mean ± standard deviation return Final baseline results RB
the same machine learning algorithms. These baselines were included to evaluate the contribution of pre-trained Transformer representations. 3) Hybrid Transformer-Numerical Models: The hybrid architectures combine Transformer representations with engineered numerical protocol features. Transformer embeddings generated from the text windows were concatenated with numerical features before classification. This approach was evaluated to determine whether combining learned textual representations and protocol-specific numerical information could improve detection performance. E. Model Training Configuration Seven pre-trained Transformer architectures are fine-tuned: BERT-base-uncased, DistilBERT-base-uncased, RoBERTabase, ELECTRA-small-discriminator, GraphCodeBERT-base, CodeBERT-base, and ModernBERT-base. All models share identical hyperparameters: learning rate 2 × 10−5 , cosine annealing scheduler with 10% warmup, maximum 7 training epochs, early stopping patience 3 on validation MCC, label smoothing 0.1, weighted cross-entropy loss proportional to inverse class frequency, and maximum input sequence length 512 tokens. All experiments are implemented in Python using the HuggingFace Transformers library [31], PyTorch, XGBoost, LightGBM, and Scikit-learn, and are executed on Google Colab with an NVIDIA A100 GPU. F. Performance Evaluation Model performance was evaluated using Matthews Correlation Coefficient (MCC), Balanced Accuracy, F1-score, ROC-AUC, and PR-AUC. MCC was selected as the primary evaluation metric because it remains reliable under class
10
Algorithm 3 Hybrid Transformer-Numerical FDI Detection Require: Windowed dataset Dw = {Tw , Fw , Yw }, hybrid model set MH , folds K Ensure: Hybrid evaluation results RH 1: Split Dw using scenario-level K-fold cross-validation 2: for each fold k = 1 to K do 3: Separate training, validation, and testing scenarios 4: Standardise numerical features using training data only 5: Compute class weights using training labels only 6: for each hybrid model h ∈ MH do 7: Tokenise structured text windows Tw 8: Encode Tw using the Transformer branch 9: Process numerical features Fw using numerical branch 10: Concatenate Transformer embedding and numerical representation 11: Train hybrid classifier using weighted loss 12: Predict attack probabilities on validation data 13: Tune decision threshold θ using validation MCC 14: Predict labels on unseen test scenarios 15: Compute MCC, Balanced Accuracy, F1-score, ROC-AUC, and PR-AUC 16: end for 17: end for 18: Aggregate fold-wise results as mean ± standard deviation return Final hybrid results RH
imbalance and provides a balanced assessment of classification performance. The final results were obtained by averaging the performance across all cross-validation folds. The evaluation focuses on comparing Transformer models with engineered-feature baselines, TF-IDF baselines, and hybrid architectures while also analysing behaviour across different FDI attack types. VI. E XPERIMENTAL E VALUATION A. Experimental Environment All experiments were conducted in Google Colab using an NVIDIA A100 40GB GPU. Transformer models are implemented using HuggingFace Transformers, PyTorch, and the datasets library. Classical baselines use Scikit-learn, XGBoost, and LightGBM. All code is version-controlled using Git/GitHub, with fixed random seeds (seed = 42) to ensure full reproducibility. Tokenisation uses each model’s associated HuggingFace tokeniser with padding and truncation to 512 tokens. The QUT-ZSS-2023-GOOSE dataset [18] is processed using pandas and NumPy, with structured text windows generated via a custom Python pipeline. B. Experiment Setup We evaluate 15 models organised into four categories: (i) seven fine-tuned Transformer models; (ii) four hybrid Transformer-numerical models; (iii) two expert-engineered ML baselines (XGBoost and LightGBM with 36 hand-crafted features); and (iv) two same-text TF-IDF baselines (XGBoost and LightGBM on identical text windows). All models are
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
11
evaluated under the same scenario-level three-fold crossvalidation framework described in Section V-C, with the same data splits, class weighting, and threshold optimisation procedure. Performance is reported as mean ± standard deviation across three folds for MCC, Balanced Accuracy, and F1-score, and as mean ROC-AUC. C. Overall Performance Table III reports the complete performance of all 15 models ranked by mean MCC. FDIFormer with GraphCodeBERT achieves MCC = 0.595 ± 0.122, which is statistically equivalent to XGBoost-Engineered (0.604 ± 0.121; difference 0.009, within the standard deviation of both models), while requiring zero protocol-specific feature engineering. TF-IDF baselines are clearly separated below all other model families. Fig. 3 visualises the full MCC ranking with error bars; Fig. 4 presents the performance fingerprint heatmap across all four metrics.
Fig. 5. Comparison between MCC and ROC-AUC for the evaluated models. The figure highlights the performance difference between pre-trained Transformer representations and TF-IDF-based text representations using the same structured GOOSE text input.
D. Pre-Trained Representations vs. Text Format
Fig. 3. Mean MCC with standard deviation error bars across three folds for all evaluated models. Wider error bars indicate higher variance in performance across different attack type compositions.
Fig. 4. Performance heatmap of all evaluated models across MCC, Balanced Accuracy, F1-score, ROC-AUC, and PR-AUC. The heatmap highlights the strong performance of Transformer models compared with TF-IDF baselines and shows that engineered-feature baselines remain highly competitive.
One of the primary objectives of this study was to determine whether performance improvements arise from the structured text representation of the data or from the use of pre-trained Transformer models. To find this, GraphCodeBERT and XGBoost-TF-IDF were evaluated using exactly the same text windows. The results show that GraphCodeBERT achieved an MCC improvement of 0.133 over XGBoost-TFIDF. Because both models were evaluated using identical inputs, the observed improvement can be attributed to the Transformer’s ability to learn contextual relationships and sequential dependencies within packet windows.
Fig. 6. MCC improvement of each fine-tuned Transformer model over the XGBoost (TF-IDF) baseline using identical structured text input Tw . GraphCodeBERT achieves the largest gain, confirming that code-aware pretrained representations provide the strongest contextual advantage over bagof-words features.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
12
TABLE III OVERALL MODEL PERFORMANCE RANKED BY MCC. Rank
Model
Type
MCC
Bal. Acc.
F1
ROC-AUC
PR-AUC
1
XGBoost (Eng.)
Baseline
0.604
0.744
0.639
0.752
0.721
2
LightGBM (Eng.)
Baseline
0.603
0.751
0.646
0.746
0.711
3
GraphCodeBERT
Transformer
0.595
0.754
0.650
0.781
0.715
4
DistilBERT
Transformer
0.573
0.752
0.642
0.810
0.718
5
Hyb. DistilBERT+Num
Hybrid
0.570
0.764
0.662
0.791
0.711
6
ModernBERT
Transformer
0.568
0.740
0.618
0.790
0.704
7
BERT-base
Transformer
0.560
0.742
0.627
0.809
0.729
8
Hyb. ModernBERT+Num
Hybrid
0.539
0.768
0.651
0.797
0.727
9
Hyb. CodeBERT+Num
Hybrid
0.530
0.750
0.634
0.778
0.665
10
Hyb. GraphCB+Num
Hybrid
0.522
0.747
0.630
0.741
0.706
11
ELECTRA-small
Transformer
0.509
0.737
0.608
0.768
0.648
12
CodeBERT
Transformer
0.505
0.725
0.608
0.755
0.697
13
RoBERTa
Transformer
0.493
0.724
0.602
0.770
0.706
14
XGBoost (TF-IDF)
Baseline-TF
0.462
0.733
0.603
0.804
0.664
15
LightGBM (TF-IDF)
Baseline-TF
0.429
0.718
0.580
0.783
0.649
form the expert-engineered baselines. DistilBERT achieves the highest ROC-AUC of 0.810 and BERT-base achieves 0.809, compared to XGBoost-Engineered at 0.752 and LightGBM Engineered at 0.746. This gap of approximately 0.058 in ROCAUC indicates that Transformer models develop a stronger overall ranking capability across detection thresholds, even when their MCC at the optimised threshold is comparable to the engineered baselines.
Fig. 7. Comparison between PR-AUC and ROC-AUC across model families. PR-AUC captures precision-recall performance under class imbalance and reveals differences not visible in ROC-AUC alone.
E. Comparison with Expert-Engineered Baselines A main question of this study is whether FDIFormer can match the detection performance of expert-engineered feature baselines without relying on any manual protocol feature design. As shown in Table III and discussed in Section VIC, GraphCodeBERT achieves statistically equivalent MCC to XGBoost-Engineered. This result is notable because XGBoostEngineered uses 36 hand-crafted protocol features developed with significant domain expertise, whereas GraphCodeBERT learns directly from the raw structured text representation Tw with no manual feature engineering. On ROC-AUC, the Transformer models collectively outper-
Among the Transformer models, code-aware architectures (GraphCodeBERT and CodeBERT) consistently outperform general-purpose language models such as RoBERTa and BERT-base on MCC, with GraphCodeBERT ranking third overall across all 15 evaluated models. This pattern is consistent with the hypothesis that the key-value structured format of GOOSE packet text representations closely resembles the syntax of structured programming languages that GraphCodeBERT and CodeBERT were pre-trained to process [29], [30]. The GraphCodeBERT pre-training objective of data flow graph, which learns the relationship between variables and their usage across sequences, might be very relevant to capture sequential dependencies between stNum, sqNum and anomaly flag fields within GOOSE windows. It is also worth noting that LightGBM-Engineered achieves MCC = 0.603, virtually identical to XGBoost-Engineered, suggesting that the performance of the engineered baseline is primarily determined by the quality of the 36 handcrafted features rather than the choice of gradient boosting algorithm. FDIFormer matches this ceiling without access to any of those features, demonstrating that pre-trained Transformer representations can serve as an effective substitute for domainspecific feature engineering in IEC 61850 GOOSE intrusion detection. Fig. 8 plots MCC against Balanced Accuracy for all 15
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
models. Figure shows that GraphCodeBERT is in the upper right region together with the engineered baselines, while TFIDF models are well separated in the lower left area, further validating the advantages of pre-trained representations over bag-of-words text features.
13
MCC gap between hybrid and standalone models is largest for GraphCodeBERT — the model that can best extract structural information from the prefix encoding. Fig. 9 presents the top5 multi-metric radar profile comparing the strongest models across all five evaluation metrics. The radar plot confirms that hybrid models occupy a different region of the performance space from standalone Transformers — stronger on balanced accuracy and F1-score however not on MCC — and the results indicate that no single model consistently achieves the highest performance across all five evaluation metrics.
Fig. 8. Comparison of MCC and Balanced Accuracy across model families. The figure shows that GraphCodeBERT achieves performance close to expert-engineered XGBoost and LightGBM baselines while avoiding manual protocol feature engineering.
F. Hybrid Model Performance The hybrid Transformer-numerical model family was evaluated to determine whether explicit combination of pre-trained Transformer representations with the 36 engineered numerical features Fw could improve detection performance beyond either modality alone. The picture is more complex as shown in Table III. Hybrid models show statistically significant improvements on secondary metrics. Hybrid DistilBERT+Num outperforms all 15 models (0.662 F1-score) and Hybrid ModernBERT+Num outperforms all models (0.768 Balanced Accuracy), with both models outperforming the standalone Transformer models and the engineered-feature baselines on these metrics. These improvements show that the numerical branch is providing additional discriminative signal for some attack scenarios, especially in cases where the Transformer branch alone might produce borderline classification probabilities. However, on the primary metric MCC, no hybrid model consistently outperforms its corresponding standalone Transformer. Hybrid DistilBERT+Num achieves MCC = 0.570, compared to standalone DistilBERT at 0.573. Similarly, Hybrid GraphCB+Num achieves MCC = 0.522, compared to standalone GraphCodeBERT at 0.595 — a notable decrease. This suggests that the concatenation of Transformer embeddings with numerical features does not provide a consistent MCC benefit and may in some cases introduce noise into the joint representation. A reasonable explanation for such behaviour is that the attack-type prefix encoded in Tw already has some aggregate information based on those same 36 engineered features, including the four binary anomaly flags, length anomaly and payload anomaly indicators. This means the Transformer attention mechanism may already be attending to this information via the prefix token, making the explicit numerical branch somewhat redundant. This is in line with the fact that the
Fig. 9. Top-5 multi-metric radar profile comparing the strongest models across MCC, Balanced Accuracy, F1-score, ROC-AUC, and PR-AUC. The figure shows that hybrid models improve some secondary metrics; however, they do not consistently outperform the strongest single Transformer model on MCC.
G. Attack Type Detectability In Table IV we present the MCC breakdown at the fold level for representative models across the three cross-validation folds. The different composition of FDI attack types in the test set for each fold is: Fold 1 includes deletion and addition attack scenarios, Fold 2 includes modification and addition scenarios and Fold 3 includes only modification scenarios. This design provides a systematic evaluation of the model performance across attack types. The results show a consistent and significant performance degradation from Fold 1 to Fold 3 for all model families. GraphCodeBERT reduces from 0.658 MCC in Fold 1 to 0.455 in Fold 3, a difference of 0.203. Similar drop from 0.690 to 0.465 for XGBoost-Engineered. Even the strongest models fail to reach the MCC = 0.5 threshold in Fold 3, suggesting that modification-only test cases are generally a harder detection task than deletion or addition attacks. As discussed, deletion and addition attacks disrupt multiple protocol fields simultaneously, producing strong multi-feature anomaly signals, whereas modification attacks alter only the payload values while preserving the complete structural format of legitimate GOOSE messages. This explains why modification-only scenarios in Fold 3 produce a much weaker and less reliable detection signal across all model families.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
14
TABLE IV F OLD -L EVEL MCC A NALYSIS BY ATTACK T YPE C OMPOSITION Model GraphCodeBERT DistilBERT ModernBERT BERT-base XGB (Eng.) XGB (TF-IDF)
Fold 1 (Del+Add) 0.658 0.672 0.685 0.664 0.690 0.549
Fold 2 (Mod+Add) 0.673 0.647 0.624 0.609 0.656 0.567
Fold 3 (Mod only) 0.455 0.400 0.396 0.408 0.465 0.271
Mean MCC 0.595 0.573 0.568 0.560 0.604 0.462
FDI attack detection without relying heavily on manually engineered protocol features. These results should be interpreted with some caution, however, given the limited number of attack scenarios and cross-validation folds available in the QUTZSS-2023-GOOSE dataset and given that inference latency on representative substation hardware was not evaluated in this study. Future work will address these limitations by investigating larger and more balanced datasets, explainable AI techniques to support operator trust, real-time edge deployment of compact Transformer models, federated learning across different substations, and extension of the proposed framework to other IEC 61850 communication services and industrial protocols. R EFERENCES
Fig. 10. Fold-level MCC comparison across representative model families. The lower performance in Fold 3 indicates that modification-heavy scenarios are more difficult to detect than deletion and addition attack scenarios.
Fig. 10 visualises the fold-level MCC patterns across model families, clearly showing a performance cliff between Fold 2 and Fold 3 for all evaluated models. VII. C ONCLUSION This paper presented a feature-engineering-free framework for detecting False Data Injection (FDI) attacks in IEC 61850 GOOSE communication using structured textual representations and fine-tuned pre-trained Transformer models. The proposed approach converted GOOSE packet sequences into structured text windows and evaluated multiple Transformer models, classical machine learning baselines, and hybrid architectures under the same experimental framework. The experimental results showed that GraphCodeBERT achieved the strongest performance among the Transformer models and produced results comparable to the best engineered-feature baselines. In contrast, TF-IDF-based models consistently achieved lower performance, demonstrating that the improvement comes mainly from the pre-trained Transformer representations rather than the text format alone. The results also showed that modification attacks remain the most challenging attack type for all model categories due to their ability to preserve normal packet structure while manipulating payload values. Overall, the findings indicate that pre-trained Transformer models can provide an effective and generalisable solution for
[1] H. Zhang, T. Wu, T. Zhu, S. Wen, and Y. Xiang, “CyberLLaMA: A finetuned large language model for cybersecurity named entity recognition,” Knowledge-Based Systems, 2025. [2] L. Yang, A. Ye, Y. Liu, W. Lu, and C. Huang, “LLM-APTDS: A highprecision advanced persistent threat detection system for imbalanced data based on large language models with strong interpretability,” Future Generation Computer Systems, vol. 178, p. 108315, 2025. [3] L. Mehavilla, M. Rodrı́guez, J. Garcı́a, and Á. Alesanco, “Evaluating large language models effectiveness for flow-based intrusion detection: a comparative study with ML and DL baselines,” Artificial Intelligence Review, vol. 59, art. 50, 2026. [4] A. M. Saber, S. Jafari, Z. Ouyang, P. Budnarain, A. Youssef, and D. Kundur, “Large Language Models for Detecting Cyberattacks on Smart Grid Protective Relays,” IEEE Open Access Journal of Power and Energy, vol. 13, pp. 135–144, 2026. [5] T. Conceição and N. Cruz, “Evaluation of the maturity of LLMs in the cybersecurity domain,” International Journal of Information Security, vol. 24, art. 197, 2025. [6] J. Yang et al., “LLM-AE-MP: Web Attack Detection Using a Large Language Model with Autoencoder and Multilayer Perceptron,” Expert Systems with Applications, vol. 274, 2025. [7] Y. Zhang, T. Du, Y. Ma, X. Wang, Y. Xie, G. Yang, Y. Lu, and E.C. Chang, “AttacKG+: Boosting attack graph construction with Large Language Models,” Computers & Security, vol. 150, p. 104220, 2024. [8] L. Belcastro, C. Carlucci, C. Cosentino, P. Liò, and F. Marozzo, “Enhancing network security using knowledge graphs and large language models for explainable threat detection,” Future Generation Computer Systems, vol. 176, p. 108160, 2026. [9] A. Zaboli, S. L. Choi, T.-J. Song, and J. Hong, “ChatGPT and Other Large Language Models for Cybersecurity of Smart Grid Applications,” in Proc. IEEE Power & Energy Society General Meeting (PESGM), 2024, pp. 1–5. [10] H. Chen, J. Chen, Y. Chai, W. Guo, C. Jia, B. Yang, and Z. Xin, “Scene-Aware Non-Intrusive Load Monitoring Using Large Language Models,” IEEE Transactions on Smart Grid, 2025, doi: 10.1109/TSG.2025.3632609. [11] T. Zhao, A. Yogarathnam, and M. Yue, “A Large Language Model for Determining Partial Tripping of Distributed Energy Resources,” IEEE Transactions on Smart Grid, vol. 16, pp. 437–440, 2025. [12] N. Fatemi and J. Fattahi, “Adaptable semantic interoperability in heterogeneous smart grids using Large Language Models,” Energy Reports, vol. 14, pp. 5774–5789, 2025. [13] C. Zhang, J. Zhang, J. Lu, and Y. Zhao, “Large Language Models Meet Energy Systems: Opportunities, Challenges, and Future Perspectives,” Applied Energy, vol. 403, part A, p. 127076, 2026. [14] A. Alsirhani, N. Tariq, M. Humayun, G. N. Alwakid, and H. Sanaullah, “Intrusion detection in smart grids using artificial intelligence-based ensemble modelling,” Cluster Computing, vol. 28, no. 4, 2025. [15] V. O. Ijeh and W. G. Morsi, “Smart grid cyberattack types classification: A fine tree bagging-based ensemble learning approach with feature selection,” Sustainable Energy, Grids and Networks, vol. 38, p. 101291, 2024. [16] G. Ciaramella, F. Martinelli, A. Santone, and F. Mercaldo, “A method for smart grid intrusion detection through explainable deep learning,” Journal of Computer Virology and Hacking Techniques, vol. 21, art. 9, 2025.
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2026
[17] H. Zheng, X. Li, and F. Li, “Unsupervised cyberattack detection in smart grids: A novel approach integrating horizontal federated learning for the control center and substations,” Reliability Engineering & System Safety, 2025. [18] QUT Cybersecurity Research Centre, “QUT-ZSS-2023-GOOSE Dataset,” GitHub: CSCRC-SCREED, 2023. [Online]. Available: https://github.com/CSCRC-SCREED/QUT-ZSS-2023-GOOSE [19] M. U. Hassan, M. H. Rehmani, and J. Chen, “Differential Privacy Techniques for Cyber Physical Systems: A Survey,” IEEE Communications Surveys & Tutorials, vol. 22, no. 1, pp. 746–789, 2020. [20] J. Liu, Y. Xiao, S. Li, W. Liang, and C. L. P. Chen, “Cyber Security and Privacy Issues in Smart Grids,” IEEE Communications Surveys & Tutorials, vol. 14, no. 4, pp. 981–997, 2012. [21] S. E. Quincozes, C. Albuquerque, D. G. Passos, and D. Mossé, “ERENO: A Framework for Generating Realistic IEC-61850 Intrusion Detection Datasets for Smart Grids,” IEEE Transactions on Dependable and Secure Computing, vol. 21, no. 4, pp. 3851–3865, 2024. [22] A. Presekal, A. Ştefanov, V. S. Rajkumar, and P. Palensky, “Attack Graph Model for Cyber-Physical Power Systems Using Hybrid Deep Learning,” IEEE Transactions on Smart Grid, vol. 14, no. 5, pp. 4007–4020, 2023. [23] Y. Li, X. Wei, Y. Li, Z. Dong, and M. Shahidehpour, “Detection of False Data Injection Attacks in Smart Grid: A Secure Federated Deep Learning Approach,” IEEE Transactions on Smart Grid, vol. 13, no. 6, pp. 4862–4872, 2022. [24] V. S. Rajkumar, A. Ştefanov, J. L. R. Torres, and P. Palensky, “Dynamical Analysis of Power System Cascading Failures Caused by Cyber Attacks,” IEEE Transactions on Industrial Informatics, 2024. [25] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186. [26] V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a Distilled Version of BERT: Smaller, Faster, Cheaper and Lighter,” in Proc. NeurIPS Workshop on Energy Efficient Deep Learning, 2019. [27] Y. Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” arXiv preprint arXiv:1907.11692, 2019. [28] K. Clark, M.-T. Luong, Q. V. Le, and C. D. Manning, “ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators,” in Proc. ICLR, 2020. [29] Z. Feng et al., “CodeBERT: A Pre-Trained Model for Programming and Natural Languages,” in Proc. EMNLP (Findings), 2020, pp. 1536–1547. [30] D. Guo et al., “GraphCodeBERT: Pre-training Code Representations with Data Flow,” in Proc. ICLR, 2021. [31] T. Wolf et al., “Transformers: State-of-the-Art Natural Language Processing,” in Proc. EMNLP (System Demonstrations), 2020, pp. 38–45. [32] D. Chicco, N. Tötsch, and G. Jurman, “The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation,” BioData Mining, vol. 14, no. 1, pp. 1–22, 2021. [33] B. Min et al., “Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey,” ACM Computing Surveys, vol. 56, no. 2, pp. 1–40, 2023. [34] M. A. Ferrag, M. Ndhlovu, N. Tihanyi, L. C. Cordeiro, M. Debbah, T. Lestable, and N. S. Thandi, “Revolutionizing Cyber Threat Detection with Large Language Models: A Privacy-Preserving BERTBased Lightweight Model for IoT/IIoT Devices,” IEEE Access, vol. 12, pp. 23733–23750, 2024. [35] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention Is All You Need,” in Proc. Advances in Neural Information Processing Systems (NeurIPS), 2017, pp. 5998–6008. [36] International Electrotechnical Commission, IEC 61850: Communication Networks and Systems for Power Utility Automation, IEC Standard, 2013. [37] International Electrotechnical Commission, IEC 62351: Power Systems Management and Associated Information Exchange – Data and Communications Security, IEC Standard, 2018–2020.
15
Sandara Sathsarani Wijethunga received the B.Sc. (Hons.) degree in Electronics and Telecommunication Engineering from General Sir John Kotelawala Defence University, Sri Lanka, in 2022, graduating with First Class Honours. She is currently pursuing the Master of Applied Artificial Intelligence (Professional) degree at Deakin University, Melbourne, Australia. Her current research focuses on applying pretrained Transformer models for cybersecurity in smart grid communication systems, with particular emphasis on False Data Injection (FDI) attack detection in IEC 61850 GOOSE traffic. Her research interests include smart grid cybersecurity, intrusion detection systems, natural language processing, and Transformer-based learning.
Muneeb Ul Hassan is currently working as a Lecturer (Assistant Professor) in Cybersecurity at Deakin University, Australia. He previously worked as a Postdoctoral Research Associate in Security and Privacy at Swinburne University of Technology, Australia. He received his Ph.D. from Swinburne University of Technology, Australia, in 2021. He completed his Bachelor of Electrical Engineering at COMSATS Institute of Information Technology, Wah Cantt. He was awarded a Gold Medal by the university for being the top student in the Department of Electrical Engineering. He is also a recipient of the IEEE TCSC 2022 Award for Excellence, Early Career Researcher, in Scalable Computing for his research excellence in privacy preservation for blockchain and decentralized energy systems. In addition, he has won several Top Peer Reviewer Awards from Clarivate Web of Science. He has published his research in toptier journals in the field, including IEEE Transactions on Knowledge and Data Engineering, IEEE Transactions on Services Computing, and IEEE Communications Surveys & Tutorials. His publications have an accumulated impact factor of more than 100. His main research interests include privacy preservation, electric vehicles, blockchain technology, differential privacy, cybersecurity, smart grids, cognitive radio ad hoc networks, artificial intelligence, cloud computing, big data security and privacy, wireless networks, cognitive radio sensor networks, and mobile ad hoc networks.
Nasrin Sohrabi is a Lecturer (assistant Professor) in Cybersecurity at Deakin University, Melbourne. She received her Ph.D. in Computer Science from RMIT University. Her research focuses on systems security, anomaly detection, blockchain forensics, and reliability of decentralized systems. She has published in leading venues, including IEEE International Conference on Data Engineering (ICDE), IEEE Transactions on Dependable and Secure Computing (TDSC), IEEE Transactions on Services Computing (TSC), IEEE Transactions on Information Forensics and Security (TIFS), ACM Computing Surveys, and Journal of Parallel and Distributed Computing (JPDC). She serves on the program committees of conferences such as ACM International Conference on Information and Knowledge Management (CIKM), IEEE/IFIP International Conference on Dependable Systems and Networks (DSN), FAB, and Australasian Information Security Conference (AISC). In addition, she regularly serves as a reviewer for prestigious journals, including IEEE Transactions on Parallel and Distributed Systems (TPDS), TDSC, TSC, and IEEE Transactions on Knowledge and Data Engineering (TKDE). She is also the founder and leader of the Resilient and Scalable Computing Group (RSCG).