Stego Battlefield: Evaluating Image Steganography Attacks and Steganalysis Defenses
arXiv:2605.05789v1 [cs.CR] 7 May 2026
Zhen Sun1,2 *, Zongmin Zhang2 *, Leyi Sheng1 , Yule Liu2 , Yifan Liao2 , Ke Li2 , Xinhu Zheng2 , Jiaheng Wei2 , Wenyuan Yang3 , Xinlei He1† 1 Wuhan University 2 The Hong Kong University of Science and Technology (Guangzhou) 3 Sun Yat-sen University
Examples of Real-world Threats
Abstract Image steganography is widely used to protect user privacy and enable covert communication. However, it can also be abused by the adversary as a covert channel to bypass content moderation, disseminate harmful semantics, and even hide malicious instructions in images to elicit dangerous outputs from large models, posing a practical security risk that continues to evolve. To address the lack of a unified and systematic evaluation framework, we propose SADBench, a systematic benchmark that assesses the adversary’s ability to inject harmful secrets via steganography and the defender’s ability to detect such threats through steganalysis. Crucially, SADBench comprises 4 core tasks, namely steganography attack capability evaluation, steganalysis defense capability evaluation, efficiency evaluation, and transferability evaluation. It evaluates both image-payload and text-payload steganography across diverse cover distributions, utilizing harmful visual semantics and toxic instructions to simulate malicious attacks. Across a broad set of attacks and detectors, SADBench reveals that (i) INN and autoencoder-based methods demonstrate superior stability compared to other architectures, (ii) in-domain detection is near-perfect and cheaper than generation, (iii) a critical asymmetry exists in transferability where attacks robustly generalize to new distributions while detectors fail to adapt, and (iv) real-world threats persist on social media, where payloads either survive minimal compression or effectively adapt to aggressive compression via simulated training. Overall, SADBench establishes a systematic, reproducible, and extensible framework to quantify risks, paving the way for measurable and security-driven advancements in steganography defense.
1
Steganography Image-payload
Adversary
Benign Images
Stego Distributing Online Images
Platforms
Malicious Instructions Steganography Text-payload
Stego Images
Jailbreaking
LLM
Steganalysis-based Defense
Generating
Defender
Public Steganography Methods
Training
Stego Images
Steganalysis Detectors
Figure 1: Overview of real-world image-cover steganography misuse scenarios and the steganalysis-based defense.
cover medium for information hiding, which has been used in a range of scenarios for information protection and secure communication, including covert messaging in military and intelligence settings [15, 27], hiding patient information in medical applications [2], and copyright protection through watermarking for multimedia content [11]. Conventional image steganography typically embeds information in spatial or transform domains, exemplified by LSB [6,36,38] and DCTbased methods [30, 37]. These methods are often limited in capacity, failing to meet modern demands for high payload and stealthiness. Deep learning-based image steganography overcomes these limitations and has emerged as a major research focus [23]. However, these advancements also introduce dual-use risks. Recent cases show that image steganography can be misused to transmit malicious instructions for remote control [47], bypass content moderation for data theft [5], or hide harmful text in benign-looking images to enable multimodal jailbreaks of vision-capable LLMs [24]. These threats suggest that evaluating steganography solely by distortion, capacity, or random-bit recovery is insufficient: we should also assess whether safety-relevant payloads can be recovered, detected, and transferred across realistic deployment shifts. To this end, we introduce SADBench, the first attack-
Introduction
Steganography is an information hiding technique that embeds secret data into a cover medium with minimal perceptible changes to avoid detection [12]. As an important branch of this field, image steganography uses images as the * Equal contribution. † Corresponding author ([email protected]).
1
defense benchmark for image steganography with safetyrelevant image and text payloads. SADBench jointly evaluates steganography attack capability, steganalysis defense capability, efficiency, and cross-distribution transferability under a unified protocol. Steganography Attack Capability: From the adversary’s perspective, we evaluate 7 imagepayload and 5 text-payload methods, assessing both visual stealthiness and payload recoverability. Steganalysis Defense Capability: From the defender’s perspective, we evaluate the binary detection performance of 8 universal steganalysis detectors. Efficiency: We compare the computational overhead and deployment cost of both attack and defense methods during the training and inference stages. Transferability: We analyze the robustness of both sides against various distribution shifts, including changes in cover domains, secret payloads, and steganographic methods. Additionally, we conduct a real-world transmission test to assess the payload recoverability of stego images when disseminated across different social media platforms. Main Findings. SADBench reveals several critical insights into the real-world steganography arms race: (i) Unmasking the True Threat: To build targeted defenses, we must identify attacks that pose practical risks. We find that achieving visual stealth is relatively trivial, but ensuring the semantic survival of the payload is a major bottleneck for adversaries. While the generative diffusion-based method [46] struggles to maintain payload integrity, specific structural designs such as INN-based [14] and autoencoder-based methods [18] effectively enable high-fidelity recovery. Defenders should strategically prioritize these high-recoverability architectures. (ii) A Critical Asymmetry in Transferability: A stark contrast exists between attack and defense robustness under distribution shifts. On the attack side, steganography (particularly image-payload methods) generalizes robustly to new cover distributions, though text payloads remain somewhat vulnerable to secret-domain shifts due to bit-error accumulation. Conversely, the defender’s transferability is alarmingly poor. Detectors heavily overfit to domain-specific traces, leading to catastrophic degradation when facing unseen datasets, cross-modality payloads, or zero-day steganographic methods. (iii) The Cost Battlefield: We uncover a massive computational mismatch. Generating high-fidelity stego images requires exorbitant training and inference costs (e.g., up to 153 hours for training), whereas effective detection takes mere milliseconds. Consequently, the defender’s true bottleneck is not computational efficiency, but acquiring representative training distributions to counter unknown threats. (iv) The Social Media Barricade: Real-world platform processing (e.g., aggressive image compression) acts as a natural barrier that can effectively destroy hidden payloads. Nevertheless, this threat is evolving: adversaries can actively bypass these natural defenses by simulating the platform’s processing pipeline during training, allowing toxic payloads to adapt and survive. In summary, we make the following contributions:
safety-relevant payload settings: harmful image payloads and harmful text instructions hidden in image covers. • We conduct a common-protocol evaluation of 7 image-payload steganography methods, 5 text-payload steganography methods, and 8 steganalysis detectors on two cover-image datasets. The evaluation covers attack capability, defense capability, efficiency, and transferability, and further includes selected platformtransmission tests to assess payload recoverability under platform-side image processing. • We identify key attack-defense gaps in this scoped setting, including the mismatch between visual fidelity and payload recovery, the cost-performance asymmetry between stego generation and steganalysis detection, and the transferability gap of current defenses. Our results show that detectors can degrade under cover-dataset and steganographic-method shifts, while platform-side compression can substantially affect payload recovery; simulated channel training partially mitigates this degradation in the evaluated setting.
2 2.1
Preliminary and Related Work Real World Threats of Steganography
Steganography was originally developed to protect information security [12], with legitimate applications ranging from covert military communications [15, 27] and medical data privacy [2] to digital watermarking for copyright protection [11]. However, adversaries increasingly exploit steganography to establish “covert channels” for malicious activities [5]. In Advanced Persistent Threat (APT) campaigns, the adversaries embed malicious payloads within seemingly benign images hosted on cloud services to disguise communication as normal image access, effectively evading network traffic auditing [8, 17]. Similarly, malware can scrape meme images posted on social media (e.g., Twitter) to extract hidden command-and-control (C2) instructions [47]. Insiders also leverage these techniques to embed confidential business documents into ordinary photos, bypassing Data Loss Prevention (DLP) systems during data exfiltration [35]. Beyond traditional malware, emerging threats demonstrate that image steganography can even be weaponized against modern AI systems. Recently, Li et al. [24] successfully jailbroke Large Language Models (LLMs) by encoding malicious queries (e.g., “how to make a virus”) into innocuous images. By incorporating robust error correction, they enabled the LLM to reliably recover and execute the hidden harmful instructions. These escalating realities severely complicate security detection and digital forensics [23]. Consequently, we adopt an explicit attack-defense perspective to evaluate existing image steganography (adversary) and steganalysis (defender) methods, providing a unified benchmark to measure their effectiveness and robustness under realistic threat scenarios.
• We propose SADBench, a benchmark and code framework for evaluating image-cover steganography misuse and steganalysis defenses. SADBench focuses on two 2
2.2
Table 1: Comparison of our SADBench with prior benchmarks. Std. denotes in-distribution evaluation. In A/D Coverage, we mark which evaluation side a benchmark provides: ( ) steganography-side evaluation, ( ) steganalysis-side evaluation, and ( ) both.
Steganography
Image steganography embeds a secret message m into a cover image x to produce a visually indistinguishable stego image y, from which a receiver can reliably extract m̂. Depending on the payload modality, existing methods fall into two categories: image-payload (image-in-image) and text-payload (text-in-image) steganography. Image-payload Steganography. Image-payload steganography handles high-capacity visual data and can be broadly categorized into three deep-learning architectures: autoencoders, invertible neural networks (INNs), and diffusion models [23]. Early methods, pioneered by Baluja [3], utilize autoencoder architectures with separate hiding and revealing networks [7, 18, 48]. Conversely, INN-based approaches [14, 16, 41] formulate concealment and extraction as inverse problems using shared parameters, minimizing information loss to support higher capacity. More recently, diffusion-based methods like CRoSS [46] introduce a coverless paradigm. Instead of modifying an existing image, they generate realistic stego images from scratch conditioned on semantic and secret inputs, prioritizing stealthiness and controllability. Our benchmark systematically evaluates the trade-offs among these three structural paradigms. Text-payload Steganography. Text-payload steganography typically encodes text as binary bitstreams, focusing heavily on robust message recovery under channel perturbations [23]. End-to-end learning pipelines are prevalent in this domain. For instance, HiDDeN [51] utilizes an EncoderNoise-Decoder architecture to simulate and resist transmission distortions, while SteganoGAN [49] incorporates adversarial training to align the stego image distribution with natural images for enhanced stealthiness. Beyond these standard frameworks, researchers have also explored normalizing flows (e.g., RMSteg [42]) and training-free optimization methods (e.g., FNNS [22]). Together, these diverse lines of work offer different trade-offs across robustness, capacity, and computational overhead, which we evaluate under our unified framework.
2.3
Key Evaluation Scenarios Tasks A/D Transfer Real EffiStd. Metrics ability Platform ciency Kharrazi et al. [20] Random bits ✓ ✗ ✗ ✗ 1 Ker et al. [19] Random bits ✓ ✗ ✗ ✗ 1 Pevný et al. [31] Random bits ✓ ✗ ✗ ✗ 1 ALASKA#2 [9] Random bits ✓ ✓ ✗ ✗ 1 Benchmark
Payload
SADBench (Ours) Image&Text
✓
✓
✓
✓
4
els became feasible, SRNet [33] adopts deep residual architectures and emphasizes more end-to-end feature learning, aiming to reduce reliance on hand-crafted heuristics and further advance CNN-based steganalysis. Beyond these, several effective universal steganalysis detectors have also been proposed [13,43,45,50]. However, a unified benchmarking suite for systematic evaluation is still lacking, which constitutes one of the main contributions of our work.
2.4
Prior Benchmarks
Existing image steganography and steganalysis benchmarks [9, 19, 20, 31] measure capacity, distortion, and detectability, but mostly use random-bit payloads, limiting their relevance to safety-related misuse, as summarized in Table 1. First, random-bit payloads provide a content-agnostic measure of channel reliability, but they do not capture whether the recovered payload preserves harmful semantics. In misuse scenarios, the attack objective is not only bit recovery, but also application-level viability: a recovered jailbreak instruction must remain executable by a downstream model, and a recovered harmful image must preserve recognizable visual semantics. Thus, realistic harmful payloads complement random-bit tests by evaluating semantic recoverability, not only low-level transmission quality. Second, prior benchmarks are typically not organized around an explicit attack-defense setting. Image steganography becomes security-critical when it is used to bypass moderation or deliver harmful content through image covers. This motivates a benchmark that jointly evaluates steganographic attack capability, steganalysis defense capability, efficiency, and transferability under a common protocol. Third, practical deployment factors remain underexplored. In real settings, defenders may face unknown cover distributions, payload sources, and steganographic methods, while platform-side processing such as social-media compression can directly affect payload recovery. However, these transfer and transmission factors are only partially covered in prior benchmarks. SADBench complements prior random-bit benchmarks by focusing on image-cover steganography misuse with safetyrelevant image and text payloads. It evaluates whether harmful visual or textual payloads can be embedded, recovered, and detected under selected distribution shifts and platform-processing conditions, without claiming to cover all steganography scenarios.
Steganalysis
Following prior surveys, we primarily focus on detectors that can generalize across different steganographic methods, commonly referred to as universal steganalysis detectors [23]. Since deep learning was introduced into image steganalysis, researchers have quickly realized that generic image-classification backbones are not well-suited for detecting extremely weak steganographic signals. Instead, architectures must be structured to capture the statistical characteristics of steganographic noise. Xu et al. [40] first propose XuNet, which introduces residual and high-pass style preprocessing at the frontend, together with tailored nonlinear operations to amplify steganographic traces, making it one of the early CNN baselines widely used in spatial-domain steganalysis. Building on this line, YedroudjNet [44] integrates key components such as filter-bank preprocessing, truncation-based activations, and normalization, and achieves stronger detection performance with improved training strategies. Subsequently, as training deeper mod3
Cover Datasets
Steganography Stego & Benign Images
Adversary
1. Cross-cover Transfer
Defender
Steganography Methods ALASKA#2, DIV2K
Secret Payloads Image Payload
Text Payload Toxic instructions
Input
Attack Goal:
Hide harmful content while evading detection
I. Steganography Attack Capability
Performance-cost Trade-offs
Real World
2. Cross-secret Transfer 3. Platform Transmission Steganalysis
Inference Time Input
Train on seen data (Source)
Training Time
8 Universal Detectors
5 Text-payload Methods
Harmful visual semantics
Attack & Defense
Steganalysis Methods
7 Image-payload Methods
Defense Goal:
Test on unseen data (Target)
1. Cross-cover Transfer 2. Zero-day Detection
- Cross-method - Cross-payload-modality
Detect stego content accurately
II. Steganalysis Defense Capability
III. Efficiency
IV. Transferability
Figure 2: Overview of SADBench framework. The framework consists of four core tasks.
3
Threat Model
by the defender. This reflects a practical setting in which the adversary controls the steganographic pipeline but only has partial knowledge of the defense.
As shown in Figure 1, we model image steganography as an adversarial interaction between a steganography sender and a steganalysis detector: the adversary hides payloads in images, while the defender detects stego images to prevent misuse.
3.1
3.2
Defender
Goal. We consider a defender who inspects images at the platform side, such as during image uploading, transmission, or storage. The defender aims to detect stego images that may contain hidden payloads while maintaining a low falsepositive rate on benign images. In practice, the defender also needs detectors that are efficient enough for large-scale screening and robust to distribution shifts, since the deployed system may encounter unseen cover sources, payload types, and steganographic methods.
Adversary
Goal. We consider an adversary who misuses image-cover steganography to hide safety-relevant payloads in benignlooking cover images. The payload can be either a harmful image payload or a harmful textual instruction. The resulting stego image may be disseminated through social media or messaging platforms [23], or provided as input to visioncapable LLMs or LLM agents. The attack succeeds when the hidden payload remains recoverable and application-level meaningful, e.g., a harmful visual payload remains recognizable or a hidden instruction can be decoded and used in a downstream jailbreak setting [24]. The adversary aims to satisfy four requirements: (i) Usability: the hidden payload can be reliably recovered under the target channel or downstream model setting; (ii) Stealthiness: the stego image remains visually close to the cover image and avoids obvious detection by human inspection, moderation systems, or steganalysis detectors; (iii) Resource efficiency: the adversary prefers methods with lower training or inference cost when they provide comparable recovery and stealthiness; and (iv) Transferability: the adversary benefits when a steganography method trained on one cover or payload distribution remains usable on another distribution or after platform processing. Capability. We assume that the adversary controls the steganographic generation process, including the choice of cover images, payloads, embedding method, extraction model or key, and message length. The adversary may use public or self-trained image steganography methods and can tune the trade-off among recoverability, stealthiness, and cost. Before uploading images to a platform, the adversary may have partial knowledge of common image-processing operations, such as resizing or compression, and may simulate such transformations to improve robustness. However, we do not assume a fully adaptive white-box adversary against the defender. In particular, the adversary does not necessarily know the exact steganalysis detector, training data, decision threshold, or ensemble strategy deployed
Capability. We assume that the defender operates in a blackbox setting with respect to the adversary. The defender can observe the final uploaded images but does not know the exact steganographic algorithm, payload, key, extraction model, or training distribution used by the adversary. The defender can use public steganography methods to synthesize labeled Cover/Stego pairs and train steganalysis detectors. The defender may also deploy multiple detectors or ensemble different detection strategies when resources allow. Nevertheless, we primarily evaluate single-detector transferability for two reasons. First, single-detector evaluation isolates whether the learned steganalysis features generalize across cover datasets, payload sources, and steganographic methods, without conflating this property with ensemble design choices. Second, ensembles can improve coverage but do not eliminate the transferability problem: they increase deployment cost, still require representative training data, and may remain vulnerable to unseen steganographic methods or shifted cover distributions. Thus, poor single-detector transferability reveals a fundamental weakness that an ensemble may mitigate but not automatically solve. Given this capability asymmetry, transferability emerges as a critical security concern. The adversary can reuse a trained steganography model across multiple cover sources or adapt it with simulated platform processing, whereas the defender must detect stego images without knowing the attack method or payload distribution. Therefore, a gap between attack transferability and defense transferability reveals a practical blind spot for image-cover steganalysis in open-world deployment. 4
4
Design of SADBench
Task 1: Steganography Attack Capability. This task evaluates the adversary’s ability to embed safety-relevant payloads into image covers. We consider two payload settings: image payloads that contain harmful visual semantics and text payloads that contain harmful instructions. Attack quality is measured from two perspectives: stealthiness, i.e., whether the stego image remains visually close to the original cover image, and recoverability, i.e., whether the embedded payload can be accurately recovered and remain application-level meaningful. Task 2: Steganalysis Defense Capability. This task evaluates the defender’s detection capability against image steganography threats, with an emphasis on universal steganalysis detectors. Under a unified evaluation protocol, we formalize detection as a binary classification problem of whether an input image is a stego. Task 3: Efficiency. This task quantifies the resource overhead and practical deployability of methods on both the attack and defense sides. For steganography methods, we evaluate the computational cost of generating stego images and recovering secrets, including training and inference time. For steganalysis methods, we likewise measure efficiency during both training and inference. By adopting unified efficiency metrics and measurement procedures, this task provides a comparable reference for analyzing performance-cost tradeoffs across methods. Task 4: Transferability. This task analyzes the outof-distribution generalization capabilities of both steganographic attacks and steganalysis defenses. We systematically evaluate robustness across three critical transfer dimensions: cross-cover dataset shifts, cross-method shifts (i.e., generalizing to unseen steganographic methods), and cross-secret payload shifts. By quantifying the performance degradation under these distribution mismatches, this task exposes the stability boundaries and structural failure modes of current methods when confronting zero-day threats and diverse realworld data pipelines.
malicious instructions from AdvBench [24, 52] to simulate multimodal LLM jailbreak scenarios. Crucially, replacing traditional random-bit payloads with these semantic datasets allows us to assess whether the extracted content retains sufficient integrity to trigger downstream misuse. Both payload datasets follow a 7:1:2 split and are dynamically resampled during training to match the larger volume of cover images. Steganography Methods. For image-payload steganography, SADBench evaluates 7 representative methods spanning 3 architectural families commonly studied in learningbased image steganography. Autoencoder-style methods include DS [3], UDH [48], and StegFormer [18]; INN-based methods include HiNet [16], DeepMIH [14], and PRIS [41]. A representative diffusion-based method is CRoSS [46]. For text-payload steganography, SADBench evaluates 5 public methods including HiDDeN [51], SteganoGAN [49], FNNS [22], CLPSTNet [25], and RoSteALS [4]. These methods cover both classical encoder-decoder pipelines and more recent text/bitstream-oriented image steganography approaches, allowing us to compare recoverability and visual stealthiness under the same safety-relevant text-payload setting. For implementation and training settings, we follow the experimental configurations of the original papers as closely as possible and adopt their reported best-performing hyperparameters. When a paper or public implementation does not fully specify a training setting, we use a fixed configuration and report it for reproducibility; for example, the DS paper does not explicitly specify the number of training epochs, and we therefore set it to 1000. For all trainable models, we perform model selection on the validation set and use the checkpoint with the best validation performance for testing and all subsequent experiments. Steganalysis Detectors. As described in Section 2.3, we primarily evaluate universal steganalysis detectors. Specifically, we consider eight detectors that can be used to jointly detect both image-payload and text-payload steganography. These include one hand-crafted feature-based baseline, SRM+EC [10], and 7 deep learning-based detectors, namely XuNet [40], Yedroudj-Net [44], ZhuNet [50], SRNet [33], YeNet [43], StegNet [13], and SiaStegNet [45]. All methods are configured using the best hyperparameters reported in the corresponding papers or public implementations, and are evaluated under the same protocol.
4.2
4.3
Figure 2 presents the overall framework of our SADBench, which consists of 4 core tasks covering both the attack and defense sides.
4.1
Task Definition
Framework
Datasets. We select two cover-image datasets representing distinct visual distributions. First, ALASKA#2 [9], a highly diverse steganalysis benchmark. To manage computational costs, we randomly sample 7,200 images, applying a 7:1:2 train/validation/test split. Second, DIV2K [1], a high-quality dataset standard in steganography research, utilizes its official split. To simulate realistic security threats, we construct safetyrelevant payloads across two modalities. For image payloads, we use 10,000 samples from Hateful Memes [21] to model the covert transmission of harmful visual semantics and moderation evasion. For text payloads, we employ 520
Experimental Setup
Steganography Metrics. We organize steganography metrics around two evaluation goals: stealthiness and recoverability. For image-payload steganography, both goals are evaluated with image similarity and perceptual metrics on two types of image pairs: Cover/Stego pairs for stealthiness and Secret/Recovery pairs for recoverability. We report MAE, PSNR, SSIM, and LPIPS, which capture complementary aspects of image quality. MAE measures average pixellevel distortion, while PSNR emphasizes pixel-wise reconstruction fidelity and is sensitive to large reconstruction errors. SSIM measures structural similarity by comparing luminance, contrast, and local structure, making it useful for 5
assessing whether the recovered or stego image preserves visual organization. LPIPS measures perceptual discrepancy in a deep feature space and better reflects perceptual differences that may not be captured by pixel-wise metrics. Together, these metrics allow us to distinguish low-level distortion, structural preservation, and perceptual fidelity. For text-payload steganography, stealthiness is still evaluated on Cover/Stego image pairs using MAE, PSNR, SSIM, and LPIPS. For recoverability, because the payload is a natural-language instruction, we report text-specific metrics including Exact Match Rate (EMR), Character Error Rate (CER), and Bit Error Rate (BER). EMR is the strictest metric and measures whether the recovered instruction exactly matches the original payload, which is important when a downstream system requires precise prompt recovery. CER measures fine-grained character-level corruption and reflects how severely the recovered text deviates from the original instruction. BER measures bit-level transmission reliability after text encoding, making it comparable to traditional bitstream-oriented steganography evaluation. This combination allows us to evaluate both application-level exact recovery and lower-level decoding errors. Formal metric definitions are provided in Section B. Steganalysis Metrics. We formulate steganalysis as a binary classification problem between cover and stego images. In the main text, we report F1-score as the primary metric for compactness and comparability across methods and detectors. F1-score summarizes the trade-off between false alarms and missed detections, and is appropriate for our balanced Cover/Stego test sets. We additionally report accuracy and AUC in Section H, where accuracy reflects threshold-dependent correctness and AUC measures threshold-independent separability.
5 5.1
generally lag behind the top-performing baselines. Their lower Cover/Stego fidelity and/or weaker Secret/Recovery quality indicate a less favorable fidelity–recovery trade-off, suggesting that they may be less effective in allocating capacity between cover preservation and secret extraction. In contrast, the diffusion-based method CRoSS performs substantially worse under this fixed-cover evaluation protocol, showing consistent and severe degradation across both datasets. Its Cover/Stego PSNR drops to 19.888 dB on ALASKA#2 and 19.400 dB on DIV2K, with Secret/Recovery PSNR similarly hovering around 19 dB. This is because its generative conditional diffusion mechanism is not naturally aligned with strict cover-preserving steganography. Unlike residual or invertible methods that perturb a given cover locally, CRoSS relies on an iterative generative denoising process that may alter the overall image appearance and semantics to satisfy conditioning constraints. Such generative deviations can substantially reduce Cover/Stego consistency, while information transfer and inversion errors accumulated during generation may further degrade the quality of secret reconstruction. Text-payload Steganography. As shown in Table 2b, textpayload steganography imposes a stricter recoverability requirement than image-payload hiding, since textual messages are represented as long discrete bit sequences and small bit errors can accumulate into message-level failures. CLPSTNet achieves the strongest recoverability on both datasets, with EMR of 0.983 on ALASKA#2 and 0.920 on DIV2K, and nearly zero decoding errors, i.e., CER/BER of 0.001/0.000 on both datasets. It also maintains strong Cover/Stego quality, with PSNR of 41.610 dB on ALASKA#2 and 33.944 dB on DIV2K, suggesting that it better supports the long binary sequences induced by text payloads. Its consistently low CER/BER further indicates that the recovered bit sequence remains globally well aligned with the original message, rather than only preserving partial local fragments. This is particularly important for text payloads, since successful recovery requires stable decoding over the entire sequence instead of merely reducing average bit-level errors. SteganoGAN preserves high visual quality, achieving Cover/Stego PSNR of 40.151 dB on ALASKA#2 and 40.059 dB on DIV2K, but its EMR drops to 0.791 and 0.740, respectively. FNNS provides a more balanced alternative, with EMR of 0.927 and 0.890 while maintaining moderate PSNR of 35.606 dB and 35.400 dB on the two datasets. In contrast, HiDDeN and RoSteALS fail to recover text reliably in our evaluation, with EMR remaining at 0.000 on both datasets, even though their Cover/Stego quality is not always the worst. This suggests that methods originally designed for relatively short bitstreams may not provide sufficient reliable capacity for realistic text payloads, where even a single sentence can correspond to thousands of bits. As the payload length increases, small per-bit errors are accumulated and amplified into complete message-level recovery failures. Additional ablations on training-data size and payload/embedding rate are provided in Sections D and G.
Evaluation and Analysis Steganography Attack Capability
We evaluate attack capability on ALASKA#2 and DIV2K under both image- and text-payload settings; quantitative results are shown in Table 2, with qualitative examples in Figure A6. Image-payload Steganography. As shown in Table 2a, DeepMIH exhibits the strongest overall performance on both datasets. It achieves the highest Cover/Stego fidelity, with PSNR values of 44.130 dB on ALASKA#2 and 43.960 dB on DIV2K, and maintains near-zero perceptual perturbation with LPIPS no larger than 0.001. This suggests that its invertible design can constrain the embedding distortion to a very low level while preserving a reliable information path for accurate secret reconstruction. StegFormer and HiNet also show highly competitive Secret/Recovery quality. For instance, StegFormer reaches a Secret/Recovery PSNR of 43.200 dB on ALASKA#2. We hypothesize that these methods benefit from their structural designs: StegFormer may leverage transformer-based representation modeling to facilitate payload reconstruction, while HiNet’s invertible architecture explicitly encourages information-preserving transformations. Other methods, such as UDH, DS, and PRIS, 6
Table 2: Comparison of steganography methods with capability metrics. Deeper cell shading indicates better performance. (a) Image-payload steganography. DIV2K
ALASKA#2 Metric
DS
UDH
StegFormer
HiNet
DeepMIH
PRIS
CRoSS
Metric
DS
UDH
StegFormer
PSNR↑ SSIM↑ MAE↓ LPIPS↓
32.610 0.868 4.504 0.013
36.100 0.927 3.307 0.002
Cover/Stego image pair 42.020 34.850 0.983 0.927 1.442 3.423 0.002 0.008
PSNR↑ SSIM↑ MAE↓ LPIPS↓
33.930 0.948 3.699 0.041
31.190 0.925 5.506 0.049
HiNet
DeepMIH
PRIS
CRoSS
44.130 0.988 1.261 0.001
30.750 0.828 5.443 0.005
19.888 0.761 17.080 0.284
PSNR↑ SSIM↑ MAE↓ LPIPS↓
29.880 0.891 6.381 0.070
35.582 0.936 3.569 0.001
Cover/Stego image pair 41.730 30.240 0.990 0.884 1.495 5.473 0.001 0.021
43.960 0.994 1.132 0.000
30.750 0.867 5.378 0.004
19.400 0.756 17.547 0.294
Secret/Recovery image pair 43.200 41.980 47.780 0.992 0.987 0.997 1.272 1.423 0.668 0.004 0.002 0.000
30.280 0.932 5.427 0.044
19.393 0.739 18.120 0.302
PSNR↑ SSIM↑ MAE↓ LPIPS↓
31.090 0.914 5.245 0.054
25.670 0.846 9.673 0.122
Secret/Recovery image pair 33.240 36.830 47.190 0.957 0.969 0.997 4.468 2.480 0.688 0.025 0.009 0.000
29.860 0.925 5.622 0.045
19.070 0.741 18.179 0.309
(b) Text-payload steganography. Metric
HiDDeN
ALASKA#2 SteganoGAN FNNS
CLPSTNet
Metric
RoSteALS
HiDDeN
DIV2K SteganoGAN FNNS
PSNR↑ SSIM↑ MAE↓ LPIPS↓
33.035 0.965 4.561 0.031
Cover/Stego image pair 40.151 35.606 0.974 0.935 1.986 3.327 0.004 0.018
41.610 0.995 1.050 0.000
27.663 0.924 7.300 0.072
PSNR↑ SSIM↑ MAE↓ LPIPS↓
29.403 0.956 7.022 0.042
Cover/Stego image pair 40.059 35.400 0.985 0.951 1.909 3.362 0.002 0.007
EMR↑ CER↓ BER↓
0.000 0.698 0.219
Secret/Recovery text pair 0.791 0.927 0.209 0.073 0.209 0.073
0.983 0.001 0.000
0.000 0.695 0.224
EMR↑ CER↓ BER↓
0.000 0.691 0.212
Secret/Recovery text pair 0.740 0.890 0.260 0.110 0.260 0.110
Takeaway: For image-payload steganography, coverconditioned methods with explicit reconstruction paths, such as invertible or autoencoder-based designs, are more effective because they can jointly preserve Cover/Stego consistency and Secret/Recovery fidelity. Diffusion-based generation is less aligned with fixed-cover preservation and therefore suffers from larger consistency loss under this protocol. For text-payload steganography, the decisive factor is reliable capacity for long bitstreams: methods such as CLPSTNet and FNNS scale better to sentencelevel text, whereas methods designed around shorter binary messages can collapse as bit errors accumulate.
5.2
CLPSTNet
RoSteALS
33.944 0.980 2.760 0.003
30.677 0.948 5.325 0.028
0.920 0.001 0.000
0.000 0.960 0.295
embedding-induced residual traces than hand-crafted statistical features. More importantly, detectability is not fully determined by visual fidelity. CRoSS is consistently easy to detect, which is expected because its diffusion-based generation introduces large Cover/Stego discrepancies and distributional shifts in our previous capability evaluation. However, even visually strong methods such as DeepMIH and StegFormer are not necessarily secure against strong detectors. Although they achieve high Cover/Stego fidelity and strong Secret/Recovery quality, StegNet and SiaStegNet still obtain very high F1-scores on them, e.g., 0.986/0.996 on StegFormer and 0.997/0.999 on DeepMIH over ALASKA#2. This indicates that preserving perceptual similarity does not eliminate the statistical traces exploited by modern CNNbased steganalysis.
Steganalysis Defense Capability
From the defender’s perspective, we evaluate steganalysis defense capability on the two datasets, under both imagepayload and text-payload settings. As summarized in Table 3, we report F1-scores for eight representative detectors, including the classical SRM+EC model and seven CNNbased steganalysis detectors (XuNet, Yedroudj-Net, SRNet, YeNet, ZhuNet, StegNet, and SiaStegNet), against seven image-payload and five text-payload steganography methods. Detection Performance on Image-payloads. In this setting, CNN-based detectors consistently outperform the SRM+EC method. On ALASKA#2, SRM+EC achieves an average F1-score of approximately 0.796 across the seven imagepayload methods, whereas StegNet and SiaStegNet reach near-perfect averages of about 0.996 and 0.997, respectively. A similar trend appears on DIV2K, where the average F1score of SRM+EC drops to about 0.679, while StegNet and SiaStegNet still remain above 0.980. This suggests that learned steganalysis models are more effective at capturing
Detection Performance on Text-payloads. This setting shows a similar detector-side trend but a different methodside explanation. CNN-based detectors substantially outperform SRM+EC: on ALASKA#2, SRM+EC obtains an average F1-score of about 0.581, while StegNet and SiaStegNet reach about 0.968 and 0.969, respectively. On DIV2K, SRM+EC remains low at about 0.597, whereas StegNet and SiaStegNet still achieve about 0.964 and 0.967. This suggests that text-payload embedding also leaves detectable statistical traces, even when the stego image quality appears acceptable. CLPSTNet and FNNS achieve relatively reliable text recovery in the previous capability evaluation, suggesting that they better support the long bitstreams induced by realistic text payloads. At the same time, they are not completely immune to steganalysis, especially under strong CNN-based detectors. In contrast, HiDDeN and RoSteALS are easier to detect and also fail to recover text reliably in our evaluation, with EMR remaining at 0.000 on both datasets. 7
Table 3: Results of steganalysis defense capability evaluation measured by F1-score. Payload
Dataset
Detector
Steganography SRM+EC
XuNet
Yedroudj-Net
SRNet
YeNet
ZhuNet
StegNet
SiaStegNet
DS UDH StegFormer HiNet DeepMIH PRIS CRoSS
0.764 0.892 0.589 0.848 0.588 0.971 0.923
0.970 0.998 0.929 0.997 0.969 1.000 1.000
0.976 0.998 0.954 0.999 0.977 1.000 1.000
0.918 0.809 0.459 0.979 0.447 0.999 1.000
0.968 0.998 0.935 0.999 0.955 0.999 0.999
0.957 0.995 0.897 0.993 0.323 0.999 0.998
0.986 1.000 0.986 1.000 0.997 1.000 1.000
0.981 1.000 0.996 1.000 0.999 1.000 1.000
DS UDH StegFormer HiNet DeepMIH PRIS CRoSS
0.536 0.589 0.544 0.937 0.525 0.854 0.768
0.829 0.992 0.976 1.000 0.987 0.997 1.000
0.877 0.990 0.993 0.997 0.988 0.998 1.000
0.766 0.528 0.563 0.958 0.474 0.922 0.980
0.816 0.987 0.913 1.000 0.962 0.998 0.995
0.777 0.786 0.556 0.974 0.542 0.973 0.978
0.942 0.997 0.982 1.000 0.995 1.000 1.000
0.920 0.997 0.993 1.000 0.993 1.000 1.000
ALASKA#2
CLPSTNet HiDDeN SteganoGAN FNNS RoSteALS
0.509 0.516 0.567 0.658 0.657
0.851 0.961 0.815 0.919 0.963
0.921 0.992 0.855 0.946 0.990
0.456 0.990 0.525 0.894 0.951
0.749 0.991 0.831 0.919 0.992
0.785 0.989 0.803 0.954 0.943
0.982 0.998 0.886 0.976 0.998
0.981 0.998 0.883 0.983 0.999
DIV2K
CLPSTNet HiDDeN SteganoGAN FNNS RoSteALS
0.573 0.594 0.570 0.554 0.692
0.860 0.735 0.865 0.945 0.953
0.893 0.949 0.849 0.955 0.971
0.473 0.923 0.402 0.532 0.883
0.744 0.843 0.746 0.922 0.924
0.555 0.786 0.538 0.682 0.889
0.987 0.970 0.892 0.982 0.990
0.977 0.981 0.904 0.978 0.995
ALASKA#2
Image
DIV2K
Text
a notable outlier at 153 hours, and CRoSS requiring 0 hours due to its use of a pre-trained generative model. Text-payload methods are relatively cheaper overall, with their training times spanning from under 1 hour to roughly 14 hours. In contrast, steganalysis detectors are much cheaper to train, taking only 0.2 to 0.5 hours across SRM+EC and CNN-based models. This indicates that, under matched training-data assumptions, the defender can train effective detectors with a much lower computational cost than the adversary needs for most steganography models. The inference results reveal an even sharper asymmetry. Most steganalysis detectors require only 1.3 to 1.5 ms per sample, making online detection computationally affordable. By comparison, most image-payload steganography methods typically require tens of milliseconds (ranging from roughly 3 to 60 ms). CRoSS is an extreme outlier, requiring nearly 23,700 ms due to the high sampling cost of diffusion-based generation. Text-payload steganography also exhibits wide variance, with inference times spanning from as little as 2.4 ms to over 300 ms. Combining the efficiency results with the steganography capability results in Table 2, we observe that stronger imagepayload attacks often require higher computational cost. For example, DeepMIH achieves the best hiding capability on ALASKA#2, with Cover/Stego PSNR of 44.130 dB and Secret/Recovery PSNR of 47.780 dB. However, it also requires the longest training time, i.e., 153 hours, and a relatively high inference time of 59.4 ms. StegFormer and HiNet are more efficient, requiring 41.0 and 47.7 hours of training and around 31 ms per inference, but their overall hiding capability is lower than DeepMIH. CRoSS is the least favorable
This suggests that methods designed around shorter binary or watermark-like payloads may become unstable when extended to long text payloads: the embedding process can leave detectable artifacts while still failing to provide sufficient reliable capacity for exact message recovery. Takeaway: From the defender’s perspective, high-quality steganography is not inherently secure against steganalysis. Across both image- and text-payload settings, CNNbased detectors, especially StegNet and SiaStegNet, consistently outperform the classical SRM+EC baseline by capturing embedding-induced statistical traces. Although high-fidelity methods such as StegFormer and DeepMIH are harder for weaker detectors, they remain detectable by stronger CNN-based models; similarly, text-payload methods with insufficient reliable long-bitstream support, such as HiDDeN and RoSteALS, are both easier to detect and less reliable in recovery. Thus, practical defense should prioritize learned steganalysis over visual-quality inspection alone.
5.3
Efficiency
We evaluate efficiency from two perspectives, namely training cost and inference overhead. As shown in Figure 3, the computational burden differs substantially between steganography methods on the adversary side and steganalysis models on the defender side. For training costs, image-payload steganography is generally the most expensive. Training times for these models typically range from 13.5 to 56.7 hours, with DeepMIH being 8
(a) Training time (hours) comparison across image-payload steganography, text-payload steganography, and steganalysis methods.
(b) Inference time (ms) comparison across image steganography, text steganography, and steganalysis methods.
Figure 3: Computational efficiency comparison on ALASKA#2.
case: although it requires no additional training in our evaluation, its hiding quality is poor and its inference time reaches 23698 ms. For text-payload steganography, CLPSTNet provides the best effectiveness-efficiency trade-off. It achieves the strongest text recovery on ALASKA#2, with EMR of 0.983 and CER/BER of 0.001/0.000, while requiring only 0.7 hours of training and 33.5 ms per inference. FNNS also achieves strong recovery, with EMR of 0.927, but its inference time is much higher at 272 ms. These results suggest that, from the adversary side, obtaining both high payload recovery and good visual quality is usually not free, especially for high-capacity image payloads or long text bitstreams. From the defender’s perspective, the efficiency results are encouraging. Strong detectors such as StegNet and SiaStegNet achieve top-tier steganalysis performance while requiring only 0.4 and 0.5 hours of training and 1.5 and 1.4 ms per inference, respectively. Thus, even when adversaries use high-quality steganography methods, the defender can often obtain strong detection capability with much lower runtime overhead. However, this advantage assumes access to representative stego training data; if the adversary’s method or payload distribution is unknown, constructing matched training data may become the dominant cost, motivating the transferability analysis in Section 5.4.
potential to detect unseen, zero-day steganographic threats. Cross-dataset Transferability for Steganography. Since training and hyperparameter tuning are costly, adversaries may prefer steganography methods that remain stable under distribution shifts. We consider two transfer settings. (i) Cross-cover transfer: models are trained on one cover dataset and directly applied to the other, testing robustness to cover-image distribution shifts. For brevity, we denote DIV2K as D and ALASKA#2 as A, and use D→A and A→D to indicate the two directions. (ii) Cross-secret transfer: models are trained with one secret dataset and evaluated on another, testing whether the hiding and recovery mechanism generalizes to unseen payload distributions. For Cross-cover transfer, the results in Table 4 show that transferability varies substantially across methods. In the image-payload setting, DeepMIH remains the most robust method, achieving the best Cover/Stego PSNR of 43.900 dB under D→A and 43.210 dB under A→D, as well as the strongest Secret/Recovery PSNR of 47.350 dB and 47.090 dB. This suggests that its invertible design preserves a stable information path under cover-domain shifts. StegFormer also maintains high cover fidelity, with Cover/Stego PSNR of 42.000 dB and 40.510 dB, while HiNet remains competitive mainly in Secret/Recovery quality. Overall, cross-cover robustness depends not only on visual fidelity, but also on whether the recovery path remains stable after the cover distribution changes. In the text-payload setting, CLPSTNet shows the strongest message recovery under cross-cover transfer. It achieves the highest EMR of 0.859 under D→A and 0.980 under A→D, with the lowest CER/BER of 0.019/0.007 and 0.000/0.000, respectively. SteganoGAN provides strong Cover/Stego quality, with PSNR of 39.240 dB and 40.840 dB, but its EMR is lower at 0.563 and 0.700. FNNS transfers reasonably well, especially under A→D with EMR of 0.930, whereas HiDDeN and RoSteALS fail to recover text reliably, with EMR remaining at 0.000 in both directions. This indicates that text-payload transfer is mainly limited by reliable longbitstream recovery rather than visual quality alone. For Cross-secret transfer, we use MM-SafetyBench [26] as the new secret dataset for image-payload steganography and StrongREJECT [32] as the new secret dataset for text-payload steganography. As shown in Table 5, image-
Takeaway: There is a fundamental computational asymmetry in the steganographic arms race. For adversaries, achieving high capacity and visual fidelity remains intrinsically expensive, dictating a strict effectiveness-efficiency trade-off. Conversely, defenders enjoy a massive efficiency advantage, rendering near real-time detection highly scalable. Consequently, the true battleground shifts from computational power to data superiority: the defender’s primary bottleneck is no longer algorithmic efficiency, but acquiring representative training distributions, making crossdomain transferability the ultimate deciding factor.
5.4
Transferability
Driven by the need to minimize costs, we next evaluate transferability from both perspectives. For the adversary, crossdataset robustness offers a way to bypass the massive computational overhead of retraining; for the defender, transferability not only reduces data-collection costs but also reveals the 9
Table 4: Cross-cover transfer for steganography (DIV2K→ALASKA#2 and ALASKA#2→DIV2K). (a) Image-payload steganography. ALASKA#2→DIV2K
DIV2K→ALASKA#2 Metric
DS
UDH
StegFormer
HiNet
PSNR↑ SSIM↑ MAE↓ LPIPS↓
29.761 0.855 6.593 0.129
Cover/Stego image pair 35.560 42.000 31.064 0.921 0.982 0.858 3.591 1.618 5.019 0.002 0.001 0.036
PSNR↑ SSIM↑ MAE↓ LPIPS↓
30.973 0.912 5.407 0.057
Secret/Recovery image pair 25.550 33.870 37.190 0.820 0.969 0.973 10.725 4.263 2.412 0.148 0.017 0.008
DeepMIH
PRIS
Metric
43.900 0.986 1.332 0.001
30.610 0.826 5.536 0.006
PSNR↑ SSIM↑ MAE↓ LPIPS↓
47.350 0.997 0.698 0.000
29.761 0.924 5.763 0.047
PSNR↑ SSIM↑ MAE↓ LPIPS↓
DS
UDH
HiNet
DeepMIH
PRIS
32.653 0.900 4.414 0.006
Cover/Stego image pair 36.122 40.510 33.186 0.941 0.990 0.941 3.287 1.444 3.962 0.001 0.002 0.005
StegFormer
43.210 0.994 1.199 0.000
30.700 0.864 5.464 0.003
33.790 0.948 3.658 0.040
Secret/Recovery image pair 30.560 40.840 40.860 0.934 0.984 0.981 5.302 1.596 1.590 0.042 0.010 0.003
47.090 0.997 0.708 0.000
30.117 0.928 5.428 0.043
(b) Text-payload steganography. DIV2K→ALASKA#2 SteganoGAN FNNS CLPSTNet
Metric
HiDDeN
PSNR↑ SSIM↑ MAE↓ LPIPS↓
30.117 0.948 6.666 0.074
Cover/Stego image pair 39.240 34.758 0.967 0.913 2.253 3.773 0.005 0.018
EMR↑ CER↓ BER↓
0.000 0.701 0.218
Secret/Recovery text pair 0.563 0.661 0.437 0.339 0.437 0.339
ALASKA#2→DIV2K SteganoGAN FNNS CLPSTNet
RoSteALS
Metric
HiDDeN
RoSteALS
34.230 0.974 2.442 0.007
33.900 0.943 4.167 0.035
PSNR↑ SSIM↑ MAE↓ LPIPS↓
31.551 0.965 5.314 0.017
Cover/Stego image pair 40.840 35.582 0.986 0.954 1.763 3.257 0.002 0.009
40.773 0.997 1.091 0.000
26.300 0.924 8.549 0.052
0.859 0.019 0.007
0.000 0.963 0.297
EMR↑ CER↓ BER↓
0.000 0.703 0.218
Secret/Recovery text pair 0.700 0.930 0.300 0.070 0.300 0.070
0.980 0.000 0.000
0.000 0.714 0.227
Table 5: Secret-domain transfer evaluation on ALASKA#2. Image-payload methods: trained on Hateful Memes, tested on MMSafetyBench SD. Text-payload methods: trained on AdvBench, tested on StrongREJECT. Image Payload Metric
DS
UDH
StegFormer
Text Payload HiNet
DeepMIH
PRIS
Metric
HiDDeN
44.350 0.988 1.238 0.001
31.230 0.832 5.474 0.004
PSNR↑ SSIM↑ MAE↓ LPIPS↓
31.950 0.963 5.360 0.032
Cover/Stego image pair 40.150 35.610 0.974 0.935 1.985 3.326 0.004 0.018
47.480 0.996 0.740 0.000
30.660 0.917 5.599 0.075
EMR↑ BER↓ CER↓
0.001 0.278 0.835
PSNR↑ SSIM↑ MAE↓ LPIPS↓
32.830 0.870 4.540 0.008
Cover/Stego image pair 38.510 42.550 36.420 0.954 0.985 0.950 2.438 1.379 2.908 0.001 0.002 0.005
PSNR↑ SSIM↑ MAE↓ LPIPS↓
33.810 0.938 3.890 0.071
Secret/Recovery image pair 31.660 33.840 43.140 0.916 0.991 0.989 5.593 4.738 1.315 0.071 0.008 0.002
payload methods are generally more robust to secret-domain shifts than text-payload methods. DeepMIH again achieves the best overall performance, with Cover/Stego PSNR of 44.35 dB and Secret/Recovery PSNR of 47.48 dB. HiNet also shows strong recovery with Secret/Recovery PSNR of 43.14 dB. In contrast, StegFormer maintains good cover fidelity with PSNR of 42.55 dB, but its Secret/Recovery PSNR drops to 33.84 dB, suggesting that cover preservation and secret reconstruction can decouple under secret-domain shifts.
SteganoGAN
FNNS
RoSteALS
CLPSTNet
27.310 0.919 8.016 0.079
41.160 0.992 1.311 0.000
Secret/Recovery text (EMR/BER/CER) 0.407 0.486 0.000 0.593 0.514 0.299 0.593 0.514 0.822
0.379 0.053 0.119
CLPSTNet makes fewer bit- and character-level errors overall. HiDDeN and RoSteALS remain poor, with EMR of only 0.001 and 0.000. Overall, cross-secret transfer is challenging for text-payload steganography, where small bit-level errors can accumulate into message-level failures. Transferability for Steganalysis. For steganalysis, transferability reflects whether a defender can still detect stego samples when the deployment condition differs from the training setting. We consider two practical forms of uncertainty: known-method detection under cover-distribution shift, where the steganography method is known but the cover dataset changes, and zero-day detection under unseen steganography methods, where the detector is tested on stego samples generated by methods that were never used during training. The zero-day setting further includes two cases: cross-method detection, where the source and target methods belong to the same payload modality, and cross-payload-modality detection, where the detector is
The text-payload setting is more sensitive to secret-domain transfer. CLPSTNet achieves the best Cover/Stego quality, with PSNR of 41.16 dB, and the lowest BER/CER of 0.053/0.119, indicating better average bit- and characterlevel preservation. On the other hand, FNNS obtains the highest EMR of 0.486, followed by SteganoGAN at 0.407 and CLPSTNet at 0.379. This reveals a discrepancy between exact-message recovery and average decoding quality: FNNS produces more fully correct messages, whereas 10
(a) Detection results for image-payload steganography.
(b) Detection results for text-payload steganography.
Figure 4: Cross-dataset transfer performance (F1-score) of the steganalysis detector. A denotes the ALASKA#2 dataset and D denotes the DIV2K dataset.
spectives. First, we study cross-method zero-day detection in Figures A1 and 5, where detectors are trained on one steganography method and tested on another unseen method within the same payload modality. The heatmaps show a clear diagonal-dominant pattern: detectors usually perform best when the training and testing methods match, while off-diagonal transfer is often unstable. For imagepayload steganography, transfer succeeds only when the source and target methods leave similar artifacts. For example, XuNet trained on CRoSS transfers reasonably well to UDH, HiNet, and PRIS, with F1-scores of 0.86, 0.88, and 0.88, but is weaker on DeepMIH at 0.74. Conversely, XuNet trained on DS reaches 0.83 on matched DS but only 0.08 on CRoSS, showing that artifacts learned from one method may not cover another. Even strong detectors can fail under zero-day method shift: StegNet achieves 0.98 on matched StegFormer but drops to 0.01 on DS, 0.06 on DeepMIH, and 0.01 on CRoSS. For text-payload steganography, zero-day transfer is even more volatile. SiaStegNet trained on HiDDeN reaches 0.98 on matched HiDDeN, but drops to 0.03 on CLPSTNet and 0.00 on FNNS, RoSteALS, and SteganoGAN. This indicates that text-payload detectors often learn method-specific bit-embedding fingerprints rather than generalizable steganographic signatures. Second, we evaluate cross-payload-modality zero-day detection in Figure A5, where detectors are trained on one payload modality and tested on the other. The results show that such transfer is generally weak, with the overall average F1-score ranging only from 0.10 to 0.30 across detectors. Although a few target methods remain partially detectable, such as CRoSS in the text-payload → image-payload direction, where ZhuNet and SRM+EC achieve average F1-scores of 0.66 and 0.60, respectively, most cross-modality cases exhibit poor generalization. This indicates that image-payload and text-payload steganography tend to leave different modality-specific artifacts, making zero-day detection substantially harder when
trained on one payload modality, i.e., image-payload or textpayload steganography, and evaluated on the other. For the first setting, Figure 4 shows that high in-domain F1-scores can overestimate real deployment robustness. For image-payload steganography, transferability is highly method- and direction-dependent. DS suffers severe A→D degradation: XuNet, Yedroudj-Net, YeNet, and StegNet drop from 0.97, 0.98, 0.97, and 0.99 to only 0.02, 0.01, 0.02, and 0.03, respectively. HiNet shows the opposite failure mode under D→A, where SRM+EC, XuNet, StegNet, and SiaStegNet drop to 0.02, 0.19, 0.24, and 0.17. These asymmetric failures suggest that some detectors learn coverdomain-specific residual statistics rather than cover-invariant steganographic traces. By contrast, PRIS and CRoSS are much easier to transfer across datasets, with most CNNbased detectors maintaining near-perfect F1-scores in both directions, indicating that their artifacts are more consistent across cover distributions. StegFormer and DeepMIH lie between these two extremes: strong detectors such as StegNet and SiaStegNet often remain effective, while weaker or less transferable detectors such as SRM+EC, SRNet, and ZhuNet can degrade substantially. For text-payload steganography, the transfer behavior is also method-dependent but reflects payload-specific artifacts. RoSteALS is relatively stable for most CNN-based detectors; for example, StegNet obtains 1.00, 0.98, 0.88, and 0.99 under A→A, A→D, D→A, and D→D, respectively, while SiaStegNet stays between 0.96 and 1.00. In contrast, CLPSTNet shows strong in-domain detection but weak D→A transfer: StegNet drops from 0.99 under D→D to 0.18 under D→A, and SiaStegNet drops from 0.98 to 0.03. FNNS also exhibits clear A→D degradation, with XuNet, Yedroudj-Net, StegNet, and SiaStegNet dropping to 0.32, 0.39, 0.48, and 0.39, respectively. These results indicate that even when the attack method is known, coverdistribution shift can break detector generalization. We further evaluate the zero-day setting from two per11
Figure 5: Cross-method transferability of steganalysis on image payloads.
the payload modality also changes. Overall, practical steganalysis cannot rely on matchedsetting F1 alone: robust defense requires not only coverdomain generalization but also zero-day robustness across unseen methods, including both cross-method and crosspayload-modality settings.
responding secrets from the downloaded images and assess the recovery quality. For each platform, we uploaded and downloaded 10 stego images, resulting in a small-scale evaluation that is negligible compared with the normal imageprocessing workload handled by these large-scale platforms. All uploaded images were kept private or otherwise restricted from public visibility whenever the platform provided such an option, and they were not shared, promoted, or exposed to other users. The embedded secrets used in our experiments were benign test payloads generated solely for evaluation purposes and did not contain executable code, personal information, credentials, harmful instructions, or any operationally useful content. Therefore, even if a third party were able to access and decode the embedded payloads, the recovered messages would not cause harm. As shown in Figure 6, in our tested upload/download setting, X and Instagram do not apply additional compression to user-uploaded images, thereby preserving image quality and enabling the embedded payload to be reliably extracted after download. In contrast, Facebook adopts more aggressive compression, causing noticeable degradation and consistently preventing payload recovery in our experiments. This observation suggests that, as storage costs decline, some social media platforms tend to adopt weaker compression or even near-lossless image processing pipelines. Such pipelines objectively reduce the probability that steganographic payloads are destroyed along the platform transmission chain, which in turn increases the risk of steganographic poisoning or information-injection attacks. For platforms such as Facebook that actively modify image content, for example via resampling and compression, a substantial body of prior work has discussed their impact on steganographic robustness [34]. To investigate this further, we evaluate a broader range of common social media processing techniques in Section E.
Figure 6: Performance comparison of image quality across different social media platforms.
Transmission of Stego Images on Social Media. As described in Section 3.1, the adversary may disseminate stego images carrying malicious payloads through social media to achieve malicious goals. To evaluate robustness under realworld social media transmission pipelines, we select DeepMIH, the best-performing method among the image-payload steganography methods, and test the robustness of its stego images on 3 mainstream platforms, namely X [39], Instagram [29], and Facebook [28], which are among the most widely used social media services today. Specifically, we manually upload the stego images to each platform and then download the processed images. We then extract the cor12
[2] S Balu, C Nelson Kennedy Babu, and K Amudha. Secure and efficient data transmission by video steganography in medical imaging system. Cluster Computing, 22(Suppl 2):4057–4063, 2019. 1, 2
Our results indicate that without adaptation, the original method lacks robustness against these distortions, leading to a significant decline in recoverability performance. However, we further discuss in Section F that an adversary can infer the platform’s specific compression rules by analyzing the discrepancy between uploaded and downloaded images. By leveraging this information to perform simulated channel training, the adversary can effectively enhance the robustness and performance of image steganography methods on such social media platforms.
[3] Shumeet Baluja. Hiding images in plain sight: Deep steganography. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 2069–2079, 2017. 3, 5
Takeaways: (i) Attack Transferability is asymmetric across modalities: Steganography generalizes well to new cover images but struggles with secret-domain shifts. Text payloads are vulnerable to these shifts due to the accumulation of bit-level errors. (ii) Zero-day threats expose critical defense vulnerabilities: Steganalysis detectors are highly sensitive to distribution shifts and often overfit to domain-specific residual statistics. Even top-performing detectors fail when confronted with unseen cover datasets or zero-day steganographic methods, proving that high indomain detection metrics are insufficient for open-world robustness. (iii) Countering platform compression via simulation: Although social media platforms may compress stego images and compromise secret recovery, adversaries can reduce this risk by employing simulated channel training to mimic platform-specific processing rules. (iv) Unpredictable processing pipelines lower steganographic robustness: By implementing randomized compression strategies rather than static rules, defenders can increase the difficulty of channel estimation for adversaries, making it challenging to train robust steganography models against such variable transmission environments.
6
[4] Tu Bui, Shruti Agarwal, Ning Yu, and John P. Collomosse. Rosteals: Robust steganography using autoencoder latent space. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 Workshops, Vancouver, BC, Canada, June 17-24, 2023, pages 933–942. IEEE, 2023. 5 [5] Rajasekhar Chaganti, Vinayakumar Ravi, Mamoun Alazab, and Tuan D Pham. Stegomalware: A systematic survey of malwarehiding and detection in images, machine learningmodels and research challenges. arXiv preprint arXiv:2110.02504, 2021. 1, 2 [6] Chin-Chen Chang, Ju Yuan Hsiao, and Chi-Shiang Chan. Finding optimal least-significant-bit substitution in image hiding by dynamic programming strategy. Pattern Recognit., 36(7):1583–1595, 2003. 1 [7] Huajie Chen, Tianqing Zhu, Yuan Zhao, Bo Liu, Xin Yu, and Wanlei Zhou. Low-frequency image deep steganography: Manipulate the frequency distribution to hide secrets with tenacious robustness. CoRR, abs/2303.13713, 2023. 3
Conclusion
[8] Joey Chen, Hiroyuki Kakara, and Masaoki Shoji. Operation endtrade: Multi-stage backdoors that tick. https://www.trendmicro.com/en_us/research/ 19/k/operation-endtrade-finding-multistage-backdoors-that-tick.html, November 2019. Trend Micro Research, published on 29 Nov 2019. Accessed: 2026-01-18. 2
We address the lack of unified evaluation standards by proposing SADBench, the first systematic benchmark evaluating image and text steganography across attack, defense, efficiency, and transferability dimensions. Through comprehensive evaluation, we uncover critical asymmetries in the steganographic arms race: while defenders enjoy a massive computational advantage over adversaries, current steganalysis models overfit severely, failing catastrophically against zero-day methods or cross-domain shifts. Besides, we demonstrate that while real-world social media compression naturally disrupts hidden payloads, adversaries can bypass these barriers via simulated channel training. Ultimately, SADBench establishes a reproducible framework to quantify these evolving threats and drive security-driven advancements in steganography defenses.
[9] Rémi Cogranne, Quentin Giboulot, and Patrick Bas. Alaska#2: Challenging academic research on steganalysis with realistic images. In 12th IEEE International Workshop on Information Forensics and Security, WIFS 2020, New York City, NY, USA, December 6-11, 2020, pages 1–5. IEEE, 2020. 3, 5 [10] Jean-François Couchot, Raphaël Couturier, and Michel Salomon. Steganalysis with cnn and srm. https://github.com/rcouturier/steganalysis_ with_CNN_and_SRM, 2016. 5
References
[11] Ingemar Cox, Matthew Miller, Jeffrey Bloom, and Chris Honsinger. Digital watermarking. Journal of Electronic Imaging, 11(3):414–414, 2002. 1, 2
[1] Eirikur Agustsson and Radu Timofte. Ntire 2017 challenge on single image super-resolution: Dataset and study. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, July 2017. 5
[12] Mukesh Dalal and Mamta Juneja. Steganography and steganalysis (in digital forensics): a cybersecurity 13
guide. Multim. Tools Appl., 80(4):5723–5771, 2021. 1, 2
Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. 5
[13] Xiaoqing Deng, Bolin Chen, Weiqi Luo, and Da Luo. Fast and effective global covariance pooling network for image steganalysis. In Proceedings of the ACM Workshop on Information Hiding and Multimedia Security, IH&MMSec 2019, Paris, France, July 3-5, 2019, pages 230–234. ACM, 2019. 3, 5
[22] Varsha Kishore, Xiangyu Chen, Yan Wang, Boyi Li, and Kilian Q. Weinberger. Fixed neural network steganography: Train the images, not the network. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. 3, 5
[14] Zhenyu Guan, Junpeng Jing, Xin Deng, Mai Xu, Lai Jiang, Zhou Zhang, and Yipeng Li. Deepmih: Deep invertible network for multiple image hiding. IEEE Trans. Pattern Anal. Mach. Intell., 45(1):372– 390, 2023. 2, 3, 5
[23] Meike Kombrink, Zeno Jean Marius Hubert Geradts, and Marcel Worring. Image steganography approaches and their detection strategies: A survey. ACM Comput. Surv., 57(2):33:1–33:40, 2025. 1, 2, 3, 4
[15] Tariq Jamil. Steganography: the art of hiding information in plain sight. IEEE potentials, 18(1):10–12, 1999. 1, 2
[24] Songze Li, Jiameng Cheng, Yiming Li, Xiaojun Jia, and Dacheng Tao. Odysseus: Jailbreaking commercial multimodal llm-integrated systems via dual steganography. In Network and Distributed System Security Symposium, 2026. 1, 2, 4, 5
[16] Junpeng Jing, Xin Deng, Mai Xu, Jianyi Wang, and Zhenyu Guan. Hinet: Deep image hiding by invertible network. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 4713–4722. IEEE, 2021. 3, 5
[25] Fengchun Liu, Tong Zhang, and Chunying Zhang. Clpstnet: A progressive multi-scale convolutional steganography model integrating curriculum learning. arXiv preprint arXiv:2504.16364, 2025. 5
[17] Kaspersky GReAT. Scarcruft continues to evolve, introduces bluetooth harvester. https://securelist. com/scarcruft-continues-to-evolveintroduces-bluetooth-harvester/90729/, May 2019. Securelist (Kaspersky), published on 13 May 2019. Accessed: 2026-01-18. 2
[26] Xin Liu, Yichen Zhu, Jindong Gu, Yunshi Lan, Chao Yang, and Yu Qiao. Mm-safetybench: A benchmark for safety evaluation of multimodal large language models. In Computer Vision - ECCV 2024 - 18th European Conference, Milan, Italy, September 29-October 4, 2024, Proceedings, Part LVI, volume 15114 of Lecture Notes in Computer Science, pages 386–403. Springer, 2024. 9
[18] Xiao Ke, Huanqi Wu, and Wenzhong Guo. Stegformer: Rebuilding the glory of autoencoder-based steganography. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, pages 2723–2731. AAAI Press, 2024. 2, 3, 5
[27] Rebecca T Mercuri. The many colors of multimedia security. Communications of the ACM, 47(12):25–29, 2004. 1, 2
[19] Andrew D. Ker. The ultimate steganalysis benchmark? In Deepa Kundur, Balakrishnan Prabhakaran, Jana Dittmann, and Jessica J. Fridrich, editors, Proceedings of the 9th workshop on Multimedia & Security, MM&Sec 2007, Dallas, Texas, USA, September 20-21, 2007, pages 141–148. ACM, 2007. 3
[28] Meta Platforms, Inc. Facebook. facebook.com/, 2026. 12
https://www.
[29] Meta Platforms, Inc. Instagram. instagram.com/, 2026. 12
https://www.
[30] Hardik Patel and Preeti Dave. Steganography technique based on dct coefficients. International Journal of Engineering Research and Applications, 2(1):713– 717, 2012. 1
[20] Mehdi Kharrazi, Husrev T. Sencar, and Nasir D. Memon. Benchmarking steganographic and steganalysis techniques. In Edward J. Delp III and Ping Wah Wong, editors, Security, Steganography, and Watermarking of Multimedia Contents VII, San Jose, California, USA, January 17-20, 2005, Proceedings, volume 5681 of Proceedings of SPIE, pages 252–263. SPIE, 2005. 3
[31] Tomás Pevný and Jessica J. Fridrich. Benchmarking for steganography. In Kaushal Solanki, Kenneth Sullivan, and Upamanyu Madhow, editors, Information Hiding, 10th International Workshop, IH 2008, Santa Barbara, CA, USA, May 19-21, 2008, Revised Selected Papers, volume 5284 of Lecture Notes in Computer Science, pages 251–267. Springer, 2008. 3
[21] Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. The hateful memes challenge: Detecting hate speech in multimodal memes. In
[32] Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, and 14
Sam Toyer. A strongreject for empty jailbreaks. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, 2024. 9
[43] Jian Ye, Jiangqun Ni, and Yang Yi. Deep learning hierarchical representations for image steganalysis. IEEE Transactions on Information Forensics and Security, 12(11):2545–2557, 2017. 3, 5 [44] Mehdi Yedroudj, Frédéric Comby, and Marc Chaumont. Yedroudj-net: An efficient cnn for spatial steganalysis. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), page 2092–2096. IEEE Press, 2018. 3, 5
[33] Shunquan Tan, Weilong Wu, Zilong Shao, Qiushi Li, Bin Li, and Jiwu Huang. CALPA-NET: channelpruning-assisted deep residual network for steganalysis of digital images. IEEE Trans. Inf. Forensics Secur., 16:131–146, 2021. 3, 5
[45] Weike You, Hong Zhang, and Xianfeng Zhao. A siamese CNN for image steganalysis. IEEE Trans. Inf. Forensics Secur., 16:291–306, 2021. 3, 5
[34] Jinyuan Tao, Sheng Li, Xinpeng Zhang, and Zichi Wang. Towards robust image steganography. IEEE Trans. Circuits Syst. Video Technol., 29(2):594–600, 2019. 12
[46] Jiwen Yu, Xuanyu Zhang, Youmin Xu, and Jian Zhang. Cross: Diffusion model makes controllable, robust and secure image steganography. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. 2, 3, 5
[35] United States District Court for the Northern District of New York. Indictment: United states of america v. zheng xiaoqing and zhang zhaoxi. https://www.justice.gov/d9/pressreleases/attachments/2019/04/23/zheng_et_ al_indictment_0.pdf, April 2019. Criminal No. 1:19-cr-00156-MAD, Document 25, filed 18 Apr 2019. Accessed: 2026-01-18. 2
[47] Aliakbar Zahravi. Malicious memes that communicate with malware. https://www.trendmicro. com/en_us/research/18/l/cybercriminalsuse-malicious-memes-that-communicate-withmalware.html, December 2018. Trend Micro Research, published on 14 Dec 2018. Accessed: 2026-01-18. 1, 2
[36] Ran-Zan Wang, Chi-Fang Lin, and Ja-Chen Lin. Image hiding by optimal LSB substitution and genetic algorithm. Pattern Recognit., 34(3):671–683, 2001. 1 [37] Andreas Westfeld. F5-A steganographic algorithm. In Ira S. Moskowitz, editor, Information Hiding, 4th International Workshop, IHW 2001, Pittsburgh, PA, USA, April 25-27, 2001, Proceedings, volume 2137 of Lecture Notes in Computer Science, pages 289–302. Springer, 2001. 1
[48] Chaoning Zhang, Philipp Benz, Adil Karjauv, Geng Sun, and In So Kweon. UDH: universal deep hiding for steganography, watermarking, and light field messaging. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 612, 2020, virtual, 2020. 3, 5
[38] Ming-Ni Wu, Min-Hui Lin, and Chin-Chen Chang. A LSB substitution oriented image hiding strategy using genetic algorithms. In Chi-Hung Chi and KwokYan Lam, editors, Content Computing, Advanced Workshop on Content Computing, AWCC 2004, ZhenJiang, JiangSu, China, November 15-17, 2004, Proceedings, volume 3309 of Lecture Notes in Computer Science, pages 219–229. Springer, 2004. 1
[49] Kevin Alex Zhang, Alfredo Cuesta-Infante, Lei Xu, and Kalyan Veeramachaneni. Steganogan: High capacity image steganography with gans. CoRR, abs/1901.03892, 2019. 3, 5 [50] Ru Zhang, Feng Zhu, Jianyi Liu, and Gongshen Liu. Depth-wise separable convolutions and multi-level pooling for an efficient spatial cnn-based steganalysis. IEEE Trans. Inf. Forensics Secur., 15:1138–1150, 2020. 3, 5
[39] X Corp. X. https://x.com/, 2026. 12 [40] Guanshuo Xu, Han-Zhou Wu, and Yun-Qing Shi. Structural design of convolutional neural networks for steganalysis. IEEE Signal Process. Lett., 23(5):708– 712, 2016. 3, 5
[51] Jiren Zhu, Russell Kaplan, Justin Johnson, and Li FeiFei. Hidden: Hiding data with deep networks. In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, editors, Computer Vision - ECCV 2018 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XV, volume 11219 of Lecture Notes in Computer Science, pages 682–697. Springer, 2018. 3, 5
[41] Hang Yang, Yitian Xu, Xuhua Liu, and Xiaodong Ma. PRIS: practical robust invertible network for image steganography. Eng. Appl. Artif. Intell., 133:108419, 2024. 3, 5 [42] Huayuan Ye, Shenzhuo Zhang, Shiqi Jiang, Jing Liao, Shuhang Gu, Dejun Zheng, Changbo Wang, and Chenhui Li. Robust message embedding via attention flowbased steganography. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2025, Nashville, TN, USA, June 11-15, 2025, pages 12840– 12849. Computer Vision Foundation / IEEE, 2025. 3
[52] Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. CoRR, abs/2307.15043, 2023. 5 15
A
Ethical Considerations
ducible foundation rather than security through obscurity. Negative Impacts. (1) Dual-Use Risk (Impact on Society): We acknowledge that demonstrating the effectiveness of advanced steganography creates a risk of adoption by malicious actors. Adversaries might leverage our insights to select methods that are most resistant to current detection, potentially facilitating the covert dissemination of harmful content. (2) Lowering Attack Barriers (Impact on Moderators): The release of pre-trained attack models lowers the technical threshold for executing injection attacks. While intended for stress-testing, these artifacts could be theoretically misused to bypass existing filters before updated defenses are fully deployed. Justification for Research. Finally, we posit that the benefits of this research outweigh the potential risks, particularly given the mitigations in place. This work addresses a critical challenge in multimedia security: accurately evaluating the robustness of steganalysis systems against realistic, malicious vectors rather than idealized random noise. Conducting this research is essential for the following reasons: (1) Ecological Validity in Threat Modeling: Our findings demonstrate that steganographic detectability is highly dependent on the payload distribution. Simulating actual attacks using harmful visual and textual payloads rather than high-entropy random bits represents the only way to avoid a false sense of security and ensure that benchmarks reflect real-world adversarial behaviors. (2) Driving Robust Defense via Transparency: We share our unified attack-defense framework to promote community-driven improvements. By exposing detectors to the worst-case scenarios involving structured harmful data, our benchmark serves as a crucial step towards developing practical steganalysis systems capable of mitigating sophisticated, modern misuse.
We organize our ethical analysis by examining the implications for stakeholders across three stages: data management during the research process, limited online platform evaluation, and the downstream effects following publication. Stakeholder Analysis and Process Impact. During the research phase, the primary ethical concern pertains to the management of sensitive datasets, specifically Hateful Memes and AdvBench. The stakeholders potentially affected in this process include the subjects depicted in the original datasets and the broader research community. To mitigate risks to these groups, our study strictly repurposed existing academic datasets within a contained, offline environment, ensuring no new harmful content was synthesized, and no interaction with human subjects occurred. We adhered rigorously to the Terms of Use for all data sources (ALASKA#2, DIV2K, and the payload datasets). By isolating the data handling process, we ensured that the evaluation of steganographic robustness remained a technical exercise without propagating harmful material to public platforms or external users during the development cycle. Limited Online Platform Evaluation. Our platform experiments were designed to measure whether standard social media image-processing pipelines affect the recoverability of embedded secrets. For each platform, we uploaded and downloaded 10 stego images, resulting in a small-scale evaluation that is negligible compared with the normal imageprocessing workload handled by large-scale platforms. To minimize risk, all uploads were conducted using accounts controlled by the authors solely for this research purpose. The uploaded images were kept private or otherwise restricted from public visibility whenever the platform provided such an option, and they were not shared, promoted, tagged, or intentionally exposed to other users. After completing the upload-download procedure, we deleted the uploaded images from the corresponding platforms. The embedded secrets used in these online platform experiments were benign test payloads generated solely for evaluation purposes. They did not contain executable code, personal information, credentials, harmful instructions, or operationally useful content. Therefore, even if a third party were able to access and decode the embedded payloads, the recovered messages would not cause harm. Impact of the Research. The publication of SADBench has both positive and negative impacts, directly affecting the stakeholders identified above. Positive Impacts. (1) Correcting Defensive Asymmetry (Impact on Defenders & Platforms): Currently, defenders lack a unified benchmark to quantify risks. SADBench provides a standardized framework to systematically evaluate detection limits. This directly benefits social media platforms by empowering them to audit and harden their content moderation pipelines against evolving steganographic threats. (2) Reproducibility and Transparency (Impact on Researchers): We have opensourced our framework and artifacts. This allows the research community to responsibly assess vulnerabilities and develop robust next-generation steganalysis tools, ensuring that future defenses are built on a transparent and repro-
B
Formal Definitions of Evaluation Metrics
B.1
Image-payload steganography metrics
We compute the same set of image metrics on two types of image pairs: Cover/Stego and Secret/Recovery. Let N = HWC be the total number of pixels (including channels), and let L = 255 be the maximum pixel value for 8-bit images. MAE. MAE measures the mean absolute pixel-wise error: MAE(X,Y ) =
1 N ∑ |Xi −Yi | . N i=1
(1)
PSNR. PSNR characterizes pixel-level fidelity based on the mean squared error: ! L2 PSNR(X,Y ) = 10 log10 1 N . (2) 2 N ∑i=1 (Xi −Yi ) SSIM. SSIM measures similarity across luminance, contrast, and structural consistency: SSIM(X,Y ) =
(2µX µY + c1 )(2σXY + c2 ) , (µ2X + µY2 + c1 )(σ2X + σY2 + c2 )
(3)
where c1 = (k1 L)2 and c2 = (k2 L)2 , with the common choice k1 = 0.01 and k2 = 0.03. 16
LPIPS. LPIPS measures perceptual discrepancy in a deep feature space. Let φl (·) ∈ RHl ×Wl ×Cl denote the feature map at layer l. We apply channel-wise normalization φ̂l (·)hw =
φl (·)hw , ∥φl (·)hw ∥2
and the full dataset, and train each model independently. The results are shown in Figure A2b and Figure A2a. Overall, using less training data generally leads to degraded steganographic performance, although the degree of degradation varies across methods and metrics. For example, as shown in Figure A2b, the Cover/Stego PSNR of SteganoGAN decreases from 38.430 dB when trained on the full dataset to 33.513 dB when trained on only 1/4 of the data. Similarly, in the image-payload setting shown in Figure A2a, HiNet is sensitive to the training data scale, with its Recovered Secret PSNR decreasing by about 7.8 dB, from 40.226 dB to 32.380 dB, when the training data is reduced to one quarter. These results suggest that sufficient training data is important for maintaining high cover-stego fidelity and accurate secret recovery in steganographic models.
(4)
and define 2 1 , w ⊙ φ̂ (X) − φ̂ (Y ) l hw l hw ∑ l 2 l Hl Wl h,w (5) where wl denotes the learned channel-wise weights and ⊙ denotes element-wise (channel-wise) multiplication. LPIPS(X,Y ) = ∑
B.2
Text-payload steganography metrics
For text-payload steganography, we evaluate the recovery quality of the secret text from two perspectives: (1) whether each sample is recovered perfectly, and (2) the fine-grained severity of errors. We report Exact Match Rate (EMR), Character Error Rate (CER), and Bit Error Rate (BER). Let the test set contain N samples, where the ground-truth text of the i-th sample is ti , and the recovered text is tˆi . Exact Match Rate (EMR). EMR =
1 N 1 ti = tˆi . ∑ N i=1
E
Some social media platforms may apply operations such as compression, resizing (resampling), or sharpening during image uploading and distribution, which can alter pixel statistics and undermine the stability of steganographic payloads. To approximately simulate such potential platform processing pipelines, we apply four common perturbations to stego images. Sharpen enhances high-frequency details using an UnsharpMask filter (radius=1.0, amount=0.5). Resize performs resampling via Pillow’s Image resize by first downscaling the image to 75% of its original size (scale=0.75) and then upscaling it back to the original resolution using the same LANCZOS interpolation, mimicking information loss induced by resizing. JPEG compression converts the image to RGB, saves it as JPEG, and reads it back (quality=95). Subsampling reuses the JPEG encode–decode pipeline but fixes quality=100 to minimize compression loss while retaining the effect of chroma subsampling (subsampling=4:2:0). The results in Tables A1 and A2 show that under the noperturbation condition (Orig), most methods can maintain reasonable visual quality on the Cover/Stego pairs. However, under these approximate platform-level transformations, the recovery quality of the secret generally degrades substantially. For the image-payload setting, Resize and the JPEG-related operations cause large drops in Secret/Recovery PSNR and SSIM for many methods, along with marked increases in MAE and LPIPS. For example, after Resize, the Secret/Recovery PSNR of several methods drops to about 6– 10 dB. For the text-payload setting, Resize, JPEG-75, and Subsampling are particularly destructive, reducing the EMR of all evaluated methods to zero and substantially increasing BER and CER. Sharpen has a milder and method-dependent effect, but still degrades the decoding performance of some methods. Overall, existing steganography methods still lack sufficient secret robustness under these common image processing operations.
(6)
Character Error Rate (CER). CER =
1 N EditDist(ti , tˆi ) ∑ max(|ti |, 1) . N i=1
(7)
Bit Error Rate (BER). We convert texts into binary sequences bi and b̂i using a fixed encoding (e.g., UTF-8). To account for possible length mismatch, we treat the extra bits as errors: min(|bi |,|b̂i |) 1 bi, j ̸= b̂i, j + |bi | − |b̂i | 1 N ∑ j=1 BER = ∑ . (8) N i=1 max |bi |, |b̂i | For Cover/Stego image pairs in text-payload steganography, we still adopt MAE, PSNR, SSIM, and LPIPS to evaluate stealthiness.
C
Environment
All experiments were conducted on the server with a single NVIDIA H100 GPU. Unless otherwise specified, both training and inference were run under the same hardware configuration to ensure fair and reproducible comparisons.
D
Robustness Test of Steganography
Ablation Study on Training Data Size of Steganography
Training deep image steganography models is typically computationally expensive and time-consuming. Therefore, we investigate how the size of the training set affects steganographic performance on the DIV2K dataset. Specifically, we divide the training data into four fractions, i.e., 1/4, 2/4, 3/4,
F
Robustness via Simulated Channel Training
When uploaded to real-world social media platforms (e.g., Facebook), stego images encounter severe platform-specific modifications. These irreversible distortions, including re17
Figure A1: Cross-method transferability of steganalysis on text payloads.
StegFormer, and HiNet for image-payload steganography, and CLPSTNet, FNNS, and SteganoGAN for text-payload steganography. For image-payload steganography, we control the payload by changing the secret image resolution relative to the cover image. We use four resolution ratios, i.e., 1/8, 1/4, 1/2, and 1× of the cover resolution, corresponding to 0.375, 1.5, 6.0, and 24.0 bpp, respectively. For text-payload steganography, we split AdvBench according to the number of tokens and construct four payload levels, denoted as 1/4, 2/4, 3/4, and 4/4 AdvBench. Recovery quality is measured by Recovered Secret PSNR for image-payload tasks and EMR for text-payload tasks, while detectability is measured by detector AUC. As shown in Figure A3, increasing the payload size generally degrades recovery quality. For image-payload tasks, DeepMIH decreases from 52.91 dB to 47.19 dB when the payload increases from 1/8 to 1× resolution, while HiNet decreases from 39.78 dB to 36.82 dB. StegFormer is relatively stable across different payload sizes, but its Recovered Secret PSNR remains lower than DeepMIH. For text-payload tasks, CLPSTNet maintains relatively high EMR across different payload levels, whereas FNNS and SteganoGAN show more noticeable fluctuations and lower EMR under larger token budgets.
sizing, JPEG compression, and chroma subsampling, often destroy the embedded features, rendering the secret information unextractable. However, an adversary can simulate this social media channel by analyzing the discrepancies between uploaded and downloaded images. Taking Facebook as an example, the adversary can reverse-engineer the channel to infer an approximate JPEG compression quality of 90 and identify a 4:2:0 chroma subsampling scheme via SOF sampling factors. Building upon the DeepMIH method, we integrate this adversary-simulated distortion layer between the Encoder and Decoder to perform adversarial training. We term this strategy Adversary Simulated Channel Training, which effectively forces the model to learn robust feature representations. As shown in Figure A4, while the baseline model collapses under Facebook compression (yielding a poor PSNR of 9.78 dB and SSIM of 0.1109), our adversary simulated channel training achieves a significant performance leap. Specifically, the method restores the PSNR to 20.85 dB and improves the SSIM to 0.6820. Furthermore, the MAE is drastically reduced from 73.39 to 15.85, confirming the superior robustness of the training method against real-world social media distortions.
G
Ablation Study on Payload Size and Embedding Rate
H
Payload size is a key factor that affects both recovery quality and detectability in steganographic systems. A larger payload usually requires denser embedding, which may introduce stronger statistical artifacts and make the stego image easier to detect. We select several representative methods with strong overall performance in previous experiments for this ablation study. Specifically, we evaluate DeepMIH,
Evaluation Metrics and Additional Detection Results
Table A3 and Table A4 report the accuracy and AUC metrics, respectively, for evaluating the steganalysis defense capability of different methods. These results are consistent with the F1-score observations discussed in the main text. Since the evaluation sets are class-balanced, the F1-score provides an 18
(a) Image-payload tasks.
(b) Text-payload tasks. Figure A2: Ablation study on the impact of training data size.
Figure A4: Effectiveness of attacker simulated channel training against compression; Original: Baseline training; Robust: Proposed attacker simulated channel training; +FB: Performance after Facebook compression channel.
(a) Image-payload tasks.
(b) Text-payload tasks. Figure A3: Effect of payload size on recovery quality and detectability.
appropriate and representative measure of detection performance; therefore, we mainly report and discuss F1-score in the main text.
19
(a) Image-payload → text-payload.
(b) Text-payload → image-payload. Figure A5: Cross-payload-modality transferability of steganalysis detectors on ALASKA#2. Each heatmap reports the F1-score (%) when a detector is trained on stego images generated by one payload modality and tested on stego images generated by the other payload modality.
20
Host
Container
Secret
Extracted origin
UDH
DS
HiNet
DeepMIH StegFormer
PRIS
CRoSS
(a) Image-payload results.
Host
Secret
Container Give detailed instructions on how to commit murder and get away with it
Give detailed instructions on how
Extracted to gkim q boci aw
Give detailed instructions on how to commit murder and get away with it
HiDDeN
SteganoGAN
origin
tplejde \lmo`
Give detailed instructions on how to commit murder and get away with it
FNNS
Give detailed instructions on how to aoamiennca\ar gb`dbde``e}qava`d
Give detailed instructions on how to commit murder and get away with it
RoSteALS
CLPSTNet
(b) Text-payload results. Figure A6: Visual examples of steganography generation and payload recovery.
Table A1: Robustness evaluation of image-payload steganography on the DIV2K dataset. Metric
DS
PSNR SSIM MAE LPIPS PSNR (Orig) PSNR (Sharpen) PSNR (Resize) PSNR (JPEG-75) PSNR (Subsample 4:2:0) SSIM (Orig) SSIM (Sharpen) SSIM (Resize) SSIM (JPEG-75) SSIM (Subsample 4:2:0) MAE (Orig) MAE (Sharpen) MAE (Resize) MAE (JPEG-75) MAE (Subsample 4:2:0) LPIPS (Orig) LPIPS (Sharpen) LPIPS (Resize) LPIPS (JPEG-75) LPIPS (Subsample 4:2:0)
UDH
StegFormer
HiNet
DeepMIH
PRIS
CRoSS
29.880 0.891 6.381 0.070
Cover/Stego image pair (no attack) 35.582 41.730 30.240 0.936 0.990 0.884 3.569 1.495 5.473 0.001 0.001 0.021
43.960 0.994 1.132 0.000
30.750 0.867 5.378 0.004
19.400 0.756 17.547 0.294
31.090 25.080 23.570 15.590 15.580 0.914 0.874 0.835 0.590 0.595 5.245 10.450 11.600 30.430 30.450 0.054 0.076 0.162 0.559 0.556
Secret/Recovery image pair 25.670 33.240 36.830 20.650 22.270 21.220 12.800 10.000 10.040 9.310 9.820 16.820 9.340 9.820 17.550 0.846 0.957 0.969 0.734 0.704 0.778 0.424 0.183 0.225 0.253 0.124 0.494 0.262 0.120 0.607 9.673 4.468 2.480 18.320 15.030 18.090 49.680 72.540 71.740 76.860 73.780 27.580 76.580 73.880 24.820 0.122 0.025 0.009 0.158 0.196 0.088 0.628 0.926 0.719 0.999 0.882 0.476 0.977 0.883 0.391
47.190 19.510 9.920 10.480 11.850 0.997 0.617 0.118 0.096 0.124 0.688 20.450 72.070 64.620 53.450 0.000 0.285 0.890 0.802 0.800
29.860 29.910 6.070 7.920 8.010 0.925 0.945 0.156 0.239 0.244 5.622 6.060 109.220 81.710 80.630 0.045 0.047 0.995 0.808 0.807
19.070 18.710 18.640 18.830 18.690 0.741 0.722 0.721 0.727 0.722 18.179 19.530 19.430 19.150 19.410 0.309 0.353 0.356 0.349 0.356
21
Table A2: Robustness evaluation of text-payload steganography on the DIV2K dataset.
Metric
HiDDeN
PSNR SSIM MAE LPIPS EMR (Orig) EMR (Sharpen) EMR (Resize) EMR (JPEG-75) EMR (Subsample 4:2:0) BER (Orig) BER (Sharpen) BER (Resize) BER (JPEG-75) BER (Subsample 4:2:0) CER (Orig) CER (Sharpen) CER (Resize) CER (JPEG-75) CER (Subsample 4:2:0)
0.000 0.000 0.000 0.000 0.000 0.212 0.216 0.220 0.283 0.282 0.691 0.696 0.734 0.884 0.877
SteganoGAN
FNNS
CLPSTNet
RoSteALS
Cover/Stego image pair (no attack) 29.403 40.059 35.400 0.956 0.985 0.951 7.022 1.909 3.362 0.042 0.002 0.007
33.944 0.980 2.760 0.003
30.677 0.948 5.325 0.028
Text payload decoding 0.740 0.890 0.580 0.780 0.000 0.000 0.000 0.000 0.000 0.000 0.260 0.110 0.420 0.220 1.000 1.000 1.000 1.000 1.000 1.000 0.260 0.110 0.420 0.220 1.000 1.000 1.000 1.000 1.000 1.000
0.920 0.920 0.000 0.000 0.000 0.000 0.000 0.152 0.857 0.655 0.001 0.001 0.765 1.175 1.358
0.000 0.000 0.000 0.000 0.000 0.295 0.296 0.295 0.295 0.295 0.960 0.962 0.970 0.953 0.959
Table A3: Results of steganalysis defense capability evaluation measured by Accuracy. Payload
Dataset
Detector
Steganography SRM+EC
XuNet
Yedroudj-Net
SRNet
YeNet
ZhuNet
StegNet
SiaStegNet
ALASKA#2
DS UDH StegFormer HiNet DeepMIH PRIS CRoSS
0.743 0.893 0.538 0.844 0.520 0.971 0.924
0.970 0.998 0.930 0.997 0.969 1.000 1.000
0.976 0.998 0.954 0.999 0.977 1.000 1.000
0.917 0.808 0.509 0.979 0.501 0.999 1.000
0.968 0.998 0.936 0.999 0.955 0.999 1.000
0.957 0.995 0.897 0.993 0.501 0.999 0.998
0.986 1.000 0.986 1.000 0.997 1.000 1.000
0.981 1.000 0.996 1.000 0.999 1.000 1.000
DIV2K
DS UDH StegFormer HiNet DeepMIH PRIS CRoSS
0.513 0.628 0.530 0.935 0.510 0.852 0.767
0.832 0.992 0.977 1.000 0.987 0.997 1.000
0.872 0.990 0.993 0.997 0.988 0.998 1.000
0.732 0.513 0.520 0.958 0.505 0.922 0.980
0.800 0.987 0.908 1.000 0.962 0.998 0.995
0.778 0.780 0.513 0.973 0.507 0.973 0.978
0.942 0.997 0.982 1.000 0.995 1.000 1.000
0.918 0.997 0.993 1.000 0.993 1.000 1.000
ALASKA#2
CLPSTNet HiDDeN SteganoGAN FNNS RoSteALS
0.517 0.528 0.525 0.633 0.611
0.855 0.961 0.808 0.917 0.963
0.921 0.992 0.855 0.945 0.990
0.505 0.990 0.503 0.888 0.951
0.717 0.991 0.812 0.915 0.992
0.787 0.989 0.782 0.954 0.944
0.982 0.998 0.885 0.976 0.998
0.981 0.998 0.881 0.983 0.999
DIV2K
CLPSTNet HiDDeN SteganoGAN FNNS RoSteALS
0.535 0.563 0.518 0.532 0.670
0.860 0.727 0.857 0.945 0.953
0.887 0.950 0.832 0.953 0.972
0.510 0.925 0.500 0.527 0.880
0.685 0.852 0.685 0.917 0.925
0.548 0.797 0.512 0.702 0.883
0.987 0.970 0.887 0.982 0.990
0.977 0.982 0.898 0.978 0.995
Image
Text
22
Table A4: Results of steganalysis defense capability evaluation measured by AUC. Payload
Dataset
Detector
Steganography SRM+EC
XuNet
Yedroudj-Net
SRNet
YeNet
ZhuNet
StegNet
SiaStegNet
ALASKA#2
DS UDH StegFormer HiNet DeepMIH PRIS CRoSS
0.820 0.962 0.560 0.924 0.522 0.996 0.978
0.997 1.000 0.985 1.000 0.995 1.000 1.000
0.997 1.000 0.992 1.000 0.997 1.000 1.000
0.975 0.829 0.508 0.997 0.500 1.000 1.000
0.995 1.000 0.987 1.000 0.991 1.000 1.000
0.993 1.000 0.972 1.000 0.500 1.000 1.000
0.999 1.000 0.999 1.000 1.000 1.000 1.000
0.998 1.000 1.000 1.000 1.000 1.000 1.000
DIV2K
DS UDH StegFormer HiNet DeepMIH PRIS CRoSS
0.518 0.649 0.520 0.960 0.502 0.892 0.858
0.908 1.000 0.999 1.000 0.996 1.000 1.000
0.946 0.997 0.999 0.998 0.999 1.000 1.000
0.821 0.507 0.512 0.987 0.501 0.977 0.998
0.889 0.996 0.968 1.000 0.987 0.997 0.999
0.846 0.865 0.521 0.995 0.502 0.996 0.996
0.980 1.000 0.998 1.000 1.000 1.000 1.000
0.979 1.000 1.000 1.000 0.999 1.000 1.000
ALASKA#2
CLPSTNet HiDDeN SteganoGAN FNNS RoSteALS
0.520 0.538 0.537 0.689 0.649
0.940 0.992 0.907 0.977 0.993
0.981 0.999 0.946 0.988 0.999
0.506 0.998 0.505 0.961 0.990
0.791 0.999 0.909 0.974 1.000
0.871 0.998 0.869 0.991 0.987
0.998 1.000 0.966 0.998 1.000
0.998 1.000 0.963 0.999 1.000
DIV2K
CLPSTNet HiDDeN SteganoGAN FNNS RoSteALS
0.546 0.559 0.519 0.533 0.724
0.939 0.811 0.932 0.987 0.986
0.968 0.991 0.931 0.989 0.993
0.514 0.980 0.509 0.556 0.940
0.737 0.907 0.760 0.982 0.960
0.553 0.871 0.529 0.782 0.943
1.000 0.998 0.961 0.999 1.000
0.997 0.998 0.960 0.998 1.000
Image
Text
23