Rethinking Forgery Attacks on Semantic Watermarks in Black-Box Settings: A Geometric Distortion Perspective
Cheng-Yi Lee 1 Yichi Zhang 2 Yuchen Yang 2 Chun-Shien Lu 1 Jun-Cheng Chen 1
arXiv:2606.29807v1 [cs.CR] 29 Jun 2026
Abstract Recent studies have shown that semantic watermarks, which embed information into the initial noise of latent diffusion models (LDMs), are vulnerable to black-box forgery attacks. However, existing methods primarily rely on empirical evidence and lack a rigorous theoretical understanding of the conditions under which such attacks succeed or fail. To bridge this gap, we rethink the nature of such attacks through the lens of ratedistortion in the latent space. Our analysis identifies an irreducible distortion floor due to structural mismatches between proxy and target models, which fundamentally limits the fidelity of forged watermarks. We further characterize this distortion as structured geometric deviations on the latent manifold, in the form of global drift and local deformation rather than stochastic noise. Leveraging these insights, we propose a scheme-agnostic detection method that distinguishes forged samples before watermark verification. Extensive experiments demonstrate the effectiveness of our method across diverse black-box scenarios, while preserving robustness to common distortions.
Figure 1. Illustration of the black-box forgery attack. An adversary uses a proxy model to invert a watermarked image into a latent representation and generates a forged image via different strategies, which preserves the service provider’s watermark while falsifying content provenance.
media (Goodman, 2024) and the dissemination of misinformation (Jaidka et al., 2025). For instance, deepfakes (Westerlund, 2019), highly realistic AI-generated media, have been used to perpetrate fraud, damage personal reputations, and spread disinformation. In response, governments (Biden, 2023; Legislature, 2024; Union, 2024) have begun mandating that companies ensure the detectability and traceability of generated images or their underlying models, as well as the identification of responsible users. Generative image watermarking has emerged as a promising paradigm to address these challenges. By injecting watermarks into generated images, the service provider (SP) can enable reliable detection and source attribution through watermark extraction. However, post-hoc watermarking schemes (Cox et al., 2008; Bui et al., 2023; Sander et al., 2025) alter the data distribution and degrade visual fidelity, regardless of whether operating in the spatial (Li et al., 2009) or frequency domain (Al-Haj, 2007). To balance utility and robustness, the semantic watermarks (Wen et al., 2023; Yang et al., 2024b; Gunn et al., 2025) modify the initial latent noise to embed a predefined pattern, which can later be recovered through inversion of the denoising process. This design enables straightforward deployment into existing diffusion models, achieving greater robustness against diverse image transformations and adversarial attacks.
1. Introduction The increasing proliferation of AI-generated content (AIGC) (Cao et al., 2025) has attracted widespread interest across various fields and contributed to substantial commercial value (Betker et al., 2023; Midjourney, Inc., 2025). In visual content generation, diffusion models (Ho et al., 2020; Song et al., 2021; Rombach et al., 2022) allow individuals from diverse backgrounds to produce high-quality images with minimal effort. However, this advancement has raised concerns regarding the erosion of trust in digital 1
Academia Sinica, Taipei, Taiwan, ROC 2 The Pennsylvania State University, PA, US. Correspondence to: Jun-Cheng Chen <[email protected]>.
Despite these advantages, recent studies (Müller et al., 2025) reveal that an adversary can perform a watermark forgery attack using only black-box access to the SP’s model. Fig. 1 illustrates this attack. The adversary takes a watermarked
rd
Proceedings of the 43 International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026. Copyright 2026 by the author(s).
1
Geometric Distortion in Black-Box Watermark Forgery
image generated by an SP, inverts it into the latent space using a proxy diffusion model, and regenerates a new image such that the SP’s watermark is preserved while the image content is no longer produced by the SP. Such an attack undermines trust in the watermarking system by falsely attributing image provenance to an SP and wrongly accusing regular users of originating harmful content. However, prior work primarily demonstrates such attacks empirically, with limited insight into the conditions under which black-box forgery succeeds or fails. As a result, it remains unclear how much security semantic watermarking systems can provide in practice, or how to design defenses that generalize beyond specific attack implementations.
Figure 2. Latent distortion under model mismatch. Discrepancies between the target model (Θ) and the proxy model (ΘA ) exacerbate reconstruction distortion in the latent space, arising from both intrinsic and external errors. (See Sec 4.2 for more details.)
In this paper, we revisit the feasibility of black-box forgery attacks on semantic watermarks.We posit that the success of such attacks is constrained by distortions introduced in the latent space when the adversary relies on a mismatched proxy model. To formalize this intuition, we model blackbox forgery as a rate–distortion problem, where the adversary must trade off successful watermark information transfer (rate) against the preservation of latent quality (distortion). Under this framework, we show that proxy–target model mismatch induces an irreducible distortion floor (see Fig. 2), which fundamentally limits the adversary’s ability to achieve high-fidelity forgery. Crucially, we find that this distortion is not stochastic noise, but manifests as structured geometric deviations from the intrinsic latent manifold, characterized by global drift and local deformation. By leveraging these geometric differences, forged samples can be detected as a pre-verification step, without requiring any modification to existing watermarking schemes. Experimental results validate the effectiveness of our approach across a wide range of black-box forgery settings. Our contributions can be summarized as follows:
process is applied to the watermarked image to recover the corresponding watermarked latent. This approach is highly effective because it is easy to implement without additional training. For example, Tree-Ring (TR) (Wen et al., 2023) embeds circular patterns into the frequency domain of the (w) latent zT . For detection, it verifies the pattern by checking (w) if the frequency representation of ẑT is sufficiently close to the original pattern. Gaussian Shading (GS) (Yang et al., 2024b) combines stream cipher encryption with distributionpreserving sampling to ensure that watermarked images follow the same distribution as non-watermarked ones. During verification, this process is inverted to recover a bit string, which is compared against registered keys. However, recent works (Müller et al., 2025) show that an adversary can readily forge a watermarked pattern using any arbitrary proxy model through reprompting and imprint attacks. Further details on semantic watermarks are provided in Sec. A.1. Rate-Distortion Theory. Shannon first explored the essential balance between the minimum amount of information (rate) required to represent a source and the distortion that arises when the data is reconstructed (Shannon, 1948; Shannon et al., 1959). By establishing theoretical limits on compression performance, rate–distortion (RD) theory guides the design of practical source coding schemes (Ballé et al., 2020) and enables evaluation of their capabilities. Recent studies (Blau & Michaeli, 2018; 2019; Zhang et al., 2021) have extended this theory to include perceptual quality, revealing a three-way trade-off among rate, distortion, and perception. In this work, we leverage RD theory to characterize latent distortions arising from model mismatch. Our analysis reveals that forged latents exhibit measurable geometric drift, which aids in the separation of forged samples.
• To the best of our knowledge, we are the first to formalize black-box forgery attacks within a rate–distortion framework, identifying an irreducible distortion floor that fundamentally constrains the adversary’s capability. • We characterize forgery-induced distortion as structured geometric deviations on the intrinsic latent manifold, manifesting as global drift and local deformation. • We propose a scheme-agnostic detection based on these geometric findings. Extensive evaluations demonstrate its effectiveness under diverse black-box forgery scenarios.
2. Related Works
3. Preliminary
Semantic Watermarking. Semantic watermarks (Wen et al., 2023; Yang et al., 2024b; Lee & Cho, 2025) embed or map a specific, recoverable structure into the starting latent, from which the diffusion process generates watermarked images. During extraction, the denoising inversion
3.1. Diffusion Models and DDIM Inversion Denoising Diffusion Probabilistic Models (DDPM) (Ho et al., 2020) formulate the process of adding and removing noise as a Markov chain. Denoising Diffusion Implicit 2
Geometric Distortion in Black-Box Watermark Forgery
Definition 3.1 (RD Function of a Gaussian Source). Let X ∼ N (0, σ 2 ) be a Gaussian source. The RD function under mean-squared error distortion D is defined as: 2 σ 1 log 0 < D < σ2 , (4) R(D) = 2 D 0 D > σ2 .
Models (DDIM) (Song et al., 2021) extend DDPM to generate high-quality images with fewer sampling steps. Unlike DDPM, DDIM follows a deterministic and non-Markovian process, which enables reversible noising and denoising. To reduce memory usage and computational cost, Latent Diffusion Models (LDM) (Rombach et al., 2022) perform the diffusion process in a latent space. Given an image x ∈ RH×W ×3 , LDM employs an encoder E(·) maps x to its latent representation z0 = E(x), and a decoder D(·) reconstructs the image as x′ = D(z0 ). Let βt denote the variance schedule at t, Q where t ∈ {0, 1, . . . , T −1}, Qtimestep t t and define ᾱt = i=1 αi = i=1 (1 − βi ). At each denoising step, a learned noise predictor ϵθ (zt , t, C) estimates the noise added to z0 . The corresponding estimate of z0 at timestep t is given by: √ zt − 1 − ᾱt ϵθ (zt , t, C) √ ẑ0t = , (1) ᾱt
Since the latent variable zT in diffusion models follows an isotropic Gaussian distribution N (0, I), we adopt this form to characterize the RD trade-off under model mismatch (see Sec. A.3 for details).
4. Methodology and Theoretical Analysis 4.1. Threat Model (w)
(f )
Adversary’s capability: Let ẑT , ẑT denote the recovered latents of watermarked and forged samples, respectively. The adversary’s objective is to generate forged images that successfully deceive both the watermark detector (w) and extractor by minimizing the difference between zT (w) and ẑT . To achieve this, a proxy model ΘA invert a water(w) marked image x(w) and obtain ẑT , which is then decoded by ΘA to produce the forged image x̂. Meanwhile, the adversary aims to preserve the visual fidelity of x̂, ensuring the forgery remains indistinguishable from genuine.
where C denotes the text condition. Using ẑ0t , the latent at the previous timestep can be computed as: p √ zt−1 = ᾱt−1 ẑ0t + 1 − ᾱt−1 ϵθ (zt , t, C). (2) Diffusion inversion reverses the generative process by recovering the latent representation from a given image. DDIM inversion accomplishes this by reversing the time steps and applying the same update rule used in DDIM generation. Starting from the latent representation z0 , noise is incrementally added, with the t-th step defined as: p √ zt+1 = ᾱt+1 ẑ0t + 1 − ᾱt+1 ϵθ (zt , t, C). (3)
Adversary’s knowledge: We assume a black-box setting in which the adversary has limited knowledge of the semantic watermarking methods and the SP’s model. Specifically, in practice, the adversary does not know the model architecture or parameters, nor does it have access to the prompts used by legitimate users. However, the adversary can acquire watermarked images that are publicly shared or uploaded by users and are known to originate from a generative model.
3.2. Black-Box Forgery Attacks In the black-box forgery setting (Müller et al., 2025), an adversary is given a watermarked image x(w) generated by the SP’s model Θ and has only black-box query access to this model. The adversary relies on a proxy model ΘA (w) to invert x(w) and obtain an estimated latent z̃T . From this latent, two forgery strategies emerge: Guidance- and Optimization-based methods. While guidance-based meth(w) ods initialize the reverse diffusion process from z̃T using auxiliary textual or structural conditions, optimization-based methods seek a perturbation δ to align a perturbed cover (c) (w) latent z̃T + δ with z̃T under the proxy inversion IA , such (c) (w) that I0→T (z̃T + δ; uA ) ≈ z̃T (See Sec. A.2 for details).
Provider’s capability: We assume that the SP has full control over the generative pipeline (e.g., watermark embedding or detection) and complete knowledge of the model architecture and its parameters. However, storing the original generative features for every synthesized image is unfeasible in large-scale deployments. Therefore, in our threat model, the provider must rely on the latent noise ẑT obtained through an inversion process to detect forged samples. 4.2. Theoretical Framework Rate–Distortion Perspective. We study the theoretical limits of semantic watermark forgery from an informationtheoretic perspective inspired by RD theory. Under this view, the forward and reverse processes of generative models are abstracted as the encoding and decoding stages of a lossy compression, in which the adversary’s reconstruction acts as a lossy decoder approximating a target watermarked latent.
3.3. Rate-Distortion Theory The RD function (Koga et al., 2013) aims to study the relation between the input X and the output X̂ of an encoderdecoder pair, and is a mapping defined by a conditional distribution pX̂|X . For a Gaussian source X ∼ N (0, σ 2 ) under common distortion measures (e.g., mean-square error), the RD function admits a closed form:
Let X denote the target latent source, X̂ its lossy recon3
Geometric Distortion in Black-Box Watermark Forgery
struction, and Yw the corresponding decoded watermarked image in the data space (see Fig. 2). This formulation serves as an abstraction for analyzing fundamental limits, rather than an exact equivalence to the diffusion inversion process. Assumptions. Our analysis relies on two standard assumptions regarding latent statistics and the distortion measure. First, we assume that latent representations follow a Gaussian distribution with shared covariance (Assumption B.1). Second, we adopt the MSE as the distortion measure for the information-theoretic analysis (Assumption B.2). These assumptions facilitate a tractable characterization of fundamental distortion limits. We emphasize that the geometric metrics introduced in Sec 4.3 are not used as distortion measures in the rate–distortion sense, but rather to characterize how inevitable distortion manifests in the latent space. Further theoretical analysis is provided in Sec. B, with empirical validations in Sec. D.
(a) Hyperspherical Space
Figure 3. Geometric interpretation of latent discrepancy. (a) illustrates angular separation θ on a hypersphere; (b) depicts the contrast between the geodesic distance (red solid line) and the Euclidean distance (blue dashed line) on the SPD manifold, with (w) the dark blue dot denoting the watermarked latent zT .
provides a standard control of their total-variation discrepancy in terms of the KL mismatch. Therefore, the effective penalty Ipen in Def. 4.1 prevents the expected distortion from vanishing.
4.2.1. R ATE -D ISTORTION L OWER B OUND UNDER M ODEL M ISMATCH
From Thm. 4.2, we identify two key consequences of model mismatch. First, the irreducible information loss Dirr introduces a rate-independent distortion floor that cannot be eliminated by increasing the information rate R. Second, the penalty Ipen reduces the adversary’s usable information rate, leading to a slower decay of distortion as R increases. Together, these effects indicate that high-fidelity semantic forgery is intrinsically constrained when the adversary relies on an imperfect proxy model.
As illustrated in Fig. 2, an adversary must rely on a proxy model ΘA that generally differs from the target model Θ. This mismatch imposes intrinsic limits on the achievable inversion accuracy. We characterize these constraints in terms of an irreducible information loss and a corresponding effective rate penalty, which together yield a lower bound on the achievable distortion. Definition 4.1 (Irreducible Information Loss and Effective Rate Penalty). Let PΘ (X̂ | Yw ) and PΘA (X̂ | Yw ) denote the target and proxy posterior distributions, respectively. The irreducible information loss induced by model mismatch is defined as
The following corollary formalizes the existence of an irreducible distortion implied by Thm. 4.2. Corollary 4.4 (Existence of Distortion Error). Under the conditions of Theorem 4.2, there exists a constant ϵ > 0 such that Dmin (R) ≥ ϵ for any rate R. In particular, even with unbounded rate or computational resources, a blackbox adversary cannot achieve arbitrarily small inversion distortion under model mismatch.
Dirr := EYw [DKL (PΘ (· | Yw )∥PΘA (· | Yw ))] . Under the Gaussian latent approximation with variance σ 2 , we instantiate the corresponding effective rate penalty as 1 Dirr Ipen := log2 1 + 2 . 2 σ
These distortions can be attributed to both intrinsic inaccuracies of the target model and external errors induced by proxy models; a detailed description is deferred to Sec. C.1.
Here, Dirr measures the posterior discrepancy caused by model mismatch, while Ipen serves as a Gaussian surrogate for the reduction in the adversary’s usable information rate.
4.3. Geometric Interpretation of Latent Discrepancy
Theorem 4.2 (Rate-Distortion Lower Bound under Model Mismatch). Under Assumptions B.1 and B.2, the minimal achievable distortion for an adversary operating at information rate R, denoted Dmin (R), is lower-bounded by Dmin (R) ≥ Dirr + σ 2 · 2−2(R−Ipen ) .
(b) SPD Riemannian Manifold
4.3.1. F ROM D ISTORTION TO L ATENT G EOMETRY Under a Gaussian approximation, the latent space features a well-defined geometric structure, shared by both the recovered latent ẑT and the reference latent zT . In contrast, for forged samples, the distortion induced by a proxy model disrupts this intrinsic structure, resulting in systematic, measurable geometric deviations (see Fig. 3). Such a discrepancy enables the SP to distinguish forged samples by comparing a given recovered latent ẑT with the reference watermarked
(5)
Remark 4.3. The positivity of the distortion floor follows from the nonzero posterior mismatch. When Dirr > 0, the target and proxy posteriors are distinct. Pinsker’s inequality 4
Geometric Distortion in Black-Box Watermark Forgery (w)
Scos . Thus, we employ cosine similarity as a practical metric to evaluate global directional drift on the hypersphere.
latent zT from the target model (see Fig. 8 in Appendix). In the following, we examine how such geometric distortion manifests along specific dimensions of the latent space.
4.3.3. D EFORMATION ON THE SPD M ANIFOLD 4.3.2. D IRECTIONAL D RIFT ON THE H YPERSPHERE
Beyond point-wise drift, forgery perturbations compromise the local structural integrity of the latent manifold. As shown in Fig. 3b, we adopt a complementary perspective by mapping local statistics onto the manifold of Symmetric Positive Definite (SPD) matrices (Bhatia, 2009), PN . When endowed with the Affine-Invariant Riemannian Metric (AIRM) (Pennec et al., 2006), PN forms a smooth manifold where the geodesic distance between P1 , P2 ∈ PN (the red solid path in Fig. 3b) is:
With an isotropic Gaussian prior N (0, IN ), where N denotes the dimension of the latent space, any latent z̃ admits a polar decomposition r · u, where the magnitude r = ∥z̃∥2 satisfies r2 ∼ χ2 (N ) and the direction u is uniformly distributed on the unit hypersphere SN −1 . Since r and u are statistically independent, this decomposition effectively decouples the radial energy from the latent’s geometric orientation. In high dimensions (N ≫ 1), the radial component r concentrates sharply around its mean, making magnitude variations negligible. Thus, magnitude variations become uninformative, and the discriminative geometric variation is almost entirely preserved in the angular component u.
−1/2
dAIRM (P1 , P2 ) = log(P1
−1/2
P2 P 1
) F,
where ∥ · ∥F denotes the Frobenius norm. This metric is congruence-invariant, ensuring that the intrinsic geometry of covariance representations remains preserved across various distortion attacks. By contrast, the Euclidean distance (the blue dashed line in Fig. 3b) ignores the intrinsic curvature of the SPD manifold and lacks the invariance properties for detection. Definition 4.8 (Local SPD Geometric Inconsistency (LGI)). Let B(zT ) = {ti }M i=1 denote a local neighborhood of tokens around a latent z. We define the local covariance mapping C : z 7→ P ∈ PN as
This rotational invariance further implies that semantic perturbations from forgeries cannot be absorbed by radial variation and instead manifest as angular distortions, as shown in Fig. 3a. We treat such perturbations as geometric noise that drives the recovered latent ẑT to deviate from the original zT on the hypersphere. To characterize this discrepancy, we define the Spherical Angular Distortion (SAD): Definition 4.5 (Spherical Angular Distortion (SAD)). Let zT , ẑT ∈ RN denote the original and recovered latent, respectively. The Spherical Angular Distortion (SAD) between zT and ẑT is defined as the geodesic distance between their normalized projections on the unit hypersphere SN −1 , SAD(zT , ẑT ) = arccos Scos (zT , ẑT ) ,
M
C(z) :=
1 X (ti − t̄)(ti − t̄)⊤ + εIN , M i=1
where t̄ is the sample mean and ε > 0. The Local SPD Geometric Inconsistency is then defined using AIRM:
T ,ẑT ⟩ where Scos (zT , ẑT ) = ∥z⟨z denotes the cosine simiT ∥2 ∥ẑT ∥2 larity between zT and ẑT .
LGI(zT , ẑT ) = dAIRM (C(zT ), C(ẑT )). Lemma 4.9 (Inevitable Structural Deformation under (w) Forgery Perturbations). Let zT ∈ RN be the initial water(w) marked latent with its associated mapping C(zT ) ∈ PN . (w) For a recovered watermarked latent ẑT , the local struc(w) (w) tural coherence is preserved such that LGI(zT , ẑT ) ≈ 0. (f ) Conversely, consider a forged recovered latent ẑT = (w) ẑT + δ that induces a non-congruent deformation of the local token neighborhood B. Under such structural perturba(f ) tions, the resulting covariance C(ẑT ) is no longer related (w) to C(zT ) by a congruent transformation, and thus h i the in-
Lemma 4.6 (Unavoidable Angular Distortion under Forgery (w) Perturbations). Let zT ∈ RN ∼ N (0, IN ). For a re(w) covered watermarked latent ẑT , the orientation remains (w) (w) well aligned, such that SAD(zT , ẑT ) ≈ 0. Conversely, (f ) (w) for a forged recovered latent ẑT = zT + δ stemming from model ΘA , the distortion satisfies: h a mismatched proxy i (w) (f ) E SAD(zT , ẑT ) > 0. In high-dimensional regimes (N ≫ 1), this angular displacement remains bounded away from zero, acting as a manifest signature of forgery.
(w)
Lem. 4.6 establishes that the orientation of the watermarked latent is inherently sensitive to proxy-induced noise. From a geometric perspective, this angular drift can be quantified through the alignment of latent vectors on SN −1 . Remark 4.7 (Implementation via Cosine Similarity). Since the geodesic (angular) distance θ = arccos(Scos ) is a monotonically decreasing in Scos on [0, π], the angular distortion characterized in Lem. 4.6 can be equivalently captured by
(f )
duced inconsistency satisfies E LGI(zT , ẑT ) = Ω(1).
Instead of uniform scaling, forgery attacks induce nonuniform deformations that disrupt the relative positioning between neighboring tokens. By measuring distances on PN , we can distinguish between origin (target) and forgeryinduced (proxy) recovered latent, as the latter exhibit anomalous geometric behavior. 5
Geometric Distortion in Black-Box Watermark Forgery Table 2. Our detection performance (AUC) under guidance-based attacks. “G-Cos” and Table 1. Watermark TPR under guidance-based “L-SPD” denote the global and local geometric deviation metric in Sec. 4, respectively. forgery attacks (TPR@X-FPR). Frequency-domain Frequency-domain
Bitstream-level
Proxy
Target
TR
RingID
HSTR
HSQR
GS
TAG
SD2.1
SD2.1 SDXL PixArt-Σ FLUX SD3
0.991 0.976 0.855 0.161 0.657
1.000 1.000 1.000 0.892 0.187
0.996 0.991 0.656 0.002 0.000
0.993 0.989 0.614 0.095 0.478
0.999 0.999 0.943 0.588 0.819
1.000 0.999 0.946 0.550 0.868
SD3
SD2.1 SDXL PixArt-Σ FLUX SD3
0.972 0.957 0.887 0.612 0.849
0.966 0.994 0.994 0.582 0.836
0.224 0.612 0.507 0.022 0.343
0.869 0.974 0.968 0.889 0.973
0.991 0.992 0.943 0.995 0.996
0.974 0.992 0.958 0.981 0.997
TR
RingID
Bitstream-level
HSTR
HSQR
GS
TAG
Proxy
Target
G-Cos
L-SPD
G-Cos
L-SPD
G-Cos
L-SPD
G-Cos
L-SPD
G-Cos
L-SPD
G-Cos
L-SPD
SD2.1
SD2.1 SDXL PixArt-Σ FLUX SD3
0.955 1.000 1.000 1.000 1.000
0.929 0.997 0.992 0.993 1.000
0.982 1.000 1.000 1.000 0.923
0.968 0.998 0.981 0.996 0.832
0.983 1.000 1.000 1.000 1.000
0.966 0.997 0.953 0.988 0.996
0.958 1.000 1.000 1.000 1.000
0.933 0.998 0.998 0.999 0.988
0.975 1.000 1.000 1.000 1.000
0.948 0.995 0.995 1.000 0.999
0.974 1.000 1.000 1.000 1.000
0.964 0.988 0.875 0.994 0.922
SD3
SD2.1 SDXL PixArt-Σ FLUX SD3
1.000 1.000 1.000 0.997 0.943
0.989 0.991 0.990 0.995 1.000
1.000 1.000 1.000 0.999 0.919
0.997 0.994 0.984 0.994 0.832
1.000 1.000 1.000 0.911 0.981
0.999 0.992 0.927 0.880 0.982
1.000 1.000 1.000 0.997 0.939
0.985 0.997 0.994 0.998 0.919
1.000 1.000 1.000 0.998 0.933
0.999 0.985 0.988 1.000 0.993
1.000 1.000 1.000 0.999 0.944
0.993 0.977 0.847 0.993 0.804
4.3.4. G EOMETRIC I MPLICATIONS OF D ISTORTION
following (Müller et al., 2025), using Stable Diffusion 2.1 (SD2.1) (Rombach et al., 2022) and Stable Diffusion 3 (SD3) (Esser et al., 2024) as proxy models and five commonly used target models: SD2.1, SDXL (Podell et al., 2024), PixArt-Σ (Chen et al., 2024), FLUX.1 (Labs, 2024), SD3. This evaluation is conducted on 1,000 samples. Regarding the optimization-based scenario, we randomly select 100 cover images from the MS-COCO dataset (Lin et al., 2014) and use SD2.1 as the proxy model with the same target set. All experiments generate images at a size of 512 × 512 using prompts from Stable-Diffusion-Prompt1 . Further details are provided in Sec. D.
The established lemmas cast forgery detection as a manifold separability problem. By projecting recovered latents into the joint bi-metric space, the inherent geometric displacement implies that forged samples are statistically distinguishable from watermarked ones, as formalized below: Theorem 4.10 (Probabilistic Geometric Separability under (w) Black-Box Forgery). Let zT ∈ RN be the initial water(w) (f ) marked latent, and let ẑT and ẑT denote the recovered latents corresponding to the watermarked and forged samples, respectively. Given the conditions in Lemma 4.6 and Lemma 4.9, there exists a rejection region R ⊂ R2+ such that the joint geometric distortion satisfies: h i (w) (f ) (w) (f ) Pr (SAD(zT , ẑT ), LGI(zT , ẑT )) ∈ R
Watermarking Methods. We consider six state-of-the-art semantic watermarks: TR (Wen et al., 2023), RingID (Ci et al., 2024), HSTR (Lee & Cho, 2025), HSQR (Lee & Cho, 2025), GS (Yang et al., 2024b) and TAG (Chen et al., 2025). The first four methods embed watermarks in the Fourier frequency domain, while the latter two encode the bitstreams as Gaussian perturbations within the initial latent noise. We refer the reader to Sec. A for more details.
≥ 1 − e−Ω(N ) . (w)
Conversely, for any watermarked recovered latent ẑT , the pair (SAD, LGI) remain concentrated outside R with high (w) probability, as (SAD, LGI) → (0, 0) for ẑT . Thus, as N → ∞, forged and watermarked latents become asymptotically separable in the joint geometric space.
Evaluation Metrics. For watermark detection, we report the true positive rate (TPR) at fixed false positive rates (FPRs), using 10−3 for frequency-domain schemes and a stricter 10−6 for bitstream-level schemes. The corresponding detection thresholds are determined by p-values (for TR) and ℓ1 distance in frequency-domain methods; bitstreamlevel schemes are based on bit accuracy. Furthermore, we adopt the Area Under the Curve (AUC) as a thresholdindependent metric to evaluate our detection performance.
The proof of Thm. 4.10 can be found in Sec. B. Thm. 4.10 formalizes the complementary effects of global drift (Lem. 4.6) and local deformation (Lem. 4.9), ensuring robust detection even under partial perturbations. In practice, however, finite-dimensional overlap may persist if: (i) High Proxy Fidelity minimizes trajectory drift; or (ii) Robustness Margins allow heavily distorted watermarked samples to exhibit geometric features akin to those of forgeries. Therefore, perfect separability cannot be guaranteed in finite dimensions, despite high-probability separability.
Our Detection Details. To quantify latent geometric discrepancies, we use global cosine similarity (G-Cos) as an empirical realization of SAD to capture directional drift, and localized SPD-based distance (L-SPD) as a measure of LGI to characterize structural deformation. For L-SPD, we partition the latent space into a regular 16 × 16 spatial grid and compute SPD covariance matrices for each patch. We then aggregate these localized discrepancies using a top-k mean strategy (with k = 5) to identify the most significant regional distortions.
5. Experiments 5.1. Experimental Settings Models and Datasets. We consider two adversarial scenarios: guidance-based and optimization-based. For the guidance-based scenario, we focus on reprompting attacks
1
6
Stable-Diffusion-Prompts
Geometric Distortion in Black-Box Watermark Forgery Table 3. Watermark TPR under optimization- Table 4. Our detection performance (AUC) under optimization-based attacks. “G-Cos” and “L-SPD” are used here as in the guidance-based attack scenarios (see Tab. 2). based forgery attacks (TPR@X-FPR). Frequency-domain
Bitstream-level
Target
Step
TR
RID
HSTR
HSQR
GS
TAG
SD2.1
20 50 100
0.95 0.99 1.00
0.98 1.00 1.00
0.67 0.99 1.00
0.99 1.00 1.00
1.00 1.00 1.00
1.00 1.00 1.00
20 50 100
0.26 0.67 0.83
0.35 0.85 1.00
0.04 0.20 0.55
0.30 0.76 0.94
0.97 0.99 1.00
0.96 0.99 0.99
PixArt-Σ
20 50 100
0.18 0.42 0.73
0.21 0.59 0.85
0.01 0.03 0.14
0.02 0.20 0.42
0.43 0.94 0.97
0.33 0.92 0.99
FLUX.1
20 50 100
0.04 0.13 0.14
0.01 0.02 0.01
0.01 0.01 0.00
0.01 0.01 0.01
0.00 0.43 0.88
0.11 0.60 0.79
SD3
20 50 100
0.18 0.45 0.63
0.00 0.00 0.00
0.00 0.00 0.00
0.00 0.01 0.04
0.20 0.74 0.94
0.28 0.74 0.89
SDXL
(a) TR
(b) RID
Frequency-domain TR
RID
Bitstream-level
HSTR
HSQR
GS
TAG
Target
Step
G-Cos
L-SPD
G-Cos
L-SPD
G-Cos
L-SPD
G-Cos
L-SPD
G-Cos
L-SPD
G-Cos
L-SPD
SD2.1
20 50 100
1.000 0.997 0.978
0.978 0.955 0.905
1.000 1.000 0.999
0.999 0.991 0.958
1.000 1.000 0.999
0.999 0.990 0.971
1.000 0.996 0.968
0.979 0.953 0.911
1.000 0.998 0.990
0.998 0.992 0.970
1.000 1.000 0.994
0.997 0.989 0.961
SDXL
20 50 100
1.000 1.000 1.000
0.997 0.997 0.990
1.000 1.000 1.000
0.999 0.997 0.996
1.000 1.000 1.000
0.996 0.993 0.989
1.000 1.000 1.000
1.000 0.999 0.998
1.000 1.000 1.000
0.997 0.995 0.992
1.000 1.000 1.000
0.994 0.986 0.978
PixArt-Σ
20 50 100
1.000 1.000 1.000
0.956 0.952 0.954
1.000 1.000 1.000
0.999 0.998 0.995
1.000 1.000 1.000
0.995 0.995 0.992
1.000 1.000 1.000
1.000 1.000 1.000
1.000 1.000 1.000
0.957 0.954 0.950
1.000 1.000 1.000
0.985 0.983 0.976
FLUX.1
20 50 100
1.000 1.000 1.000
0.994 0.994 0.995
1.000 1.000 1.000
0.993 0.992 0.991
1.000 1.000 1.000
0.991 0.992 0.992
1.000 1.000 1.000
1.000 1.000 1.000
1.000 1.000 1.000
1.000 1.000 1.000
1.000 1.000 1.000
1.000 0.999 0.999
SD3
20 50 100
1.000 1.000 1.000
0.994 0.994 0.993
1.000 1.000 1.000
0.929 0.931 0.929
1.000 1.000 1.000
0.995 0.992 0.992
1.000 1.000 1.000
0.995 0.995 0.994
1.000 1.000 1.000
0.993 0.993 0.993
1.000 1.000 1.000
0.987 0.987 0.987
(c) HSTR (a) TR
(b) GS
Figure 5. Distributions of cosine similarity for watermarked and forged samples, under the same attack setting as in Fig. 4.
(d) HSQR
(e) GS
semantic shift computed by cosine similarity, respectively.
(f) TAG
Optimization-based Forgery Attacks. Tab. 3 reports the forgery success rates of watermarking schemes under optimization-based attacks, where higher values indicate greater attack effectiveness. When the target model matches the proxy (SD2.1), the adversary achieves near-perfect forgery success (i.e., 1.00 TPR) even under limited steps. Conversely, for DiT-based target models (e.g., FLUX.1 and SD3), the attack performance degrades significantly across all watermarking schemes.
Figure 4. Log-scale ROC curves for our local structural metric across watermarking schemes, evaluated on SDXL as the target model and SD2.1 as the proxy under guidance-based attacks.
5.2. Experimental Results Guidance-based Forgery Attacks. Tab. 1 illustrates the robustness of various watermarking schemes under guidancebased attacks. We find that forgery attacks are highly successful when the target and proxy models are identical (e.g., both are SD2.1); however, the attack efficacy is notably constrained in cross-model scenarios (e.g., an SD3 target with an SD2.1 proxy), consistent with the analysis in Sec. 4.2.
Tab. 4 presents our detection performance under diverse conditions, following the same metrics as in Tab. 2. While G-Cos and L-SPD show a marginal decrease with more optimization steps, both retain strong detection capability, e.g., the L-SPD metric decreases slightly from 0.997 to 0.990 for TR on SDXL. A similar trend is also evident in Fig. 6, which shows a clear separation between cosine similarity distributions at different optimization steps. Notably, a more significant performance drop is observed when the target model matches the proxy (i.e., SD2.1), as discussed in Sec. 4.2. However, we argue that such a matched-model scenario is less representative of real-world threats.
Tab. 2 reports our detection performance measured by global and local geometric deviations. Both detection metrics, GCos and L-SPD, achieve near-perfect AUC values, exceeding 0.990 in many cases, reflecting substantial semantic deviations induced by architectural discrepancies between target and proxy models. By contrast, detection performance degrades slightly in the most challenging scenarios where target and proxy models are identical (e.g., both are SD3) and approximation errors are minimized. Despite this, our method retains meaningful detection performance under these conditions (e.g., above 0.990 in SD2.1). Figs. 4 and 5 present the detection performance using ROC curves and the
Comparison with Alternative Metrics. Although our framework is not tied to a specific metric, G-Cos and LSPD are chosen as empirical realizations of the geometric deviations characterized in Lemmas 4.6 and 4.9. We fur7
Geometric Distortion in Black-Box Watermark Forgery Table 5. Distortion between watermarked and reconstructed latents. Baseline error (w/o) represents the intrinsic reconstruction error without proxy guidance, while values in parentheses denote the empirical Dirr estimate. TR
(a) Step T = 20
(b) Step T = 50
(c) Step T = 100
Figure 6. Cosine similarity distributions under optimization-based attacks, with increasing numbers of optimization iterations on the TR. The target and proxy models are consistent with Fig. 4.
GS
Target
w/o
SD2.1A (Dirr )
SD3A (Dirr )
w/o
SD2.1A (Dirr )
SD3A (Dirr )
SD2.1 SDXL PixArt-Σ FLUX SD3
0.253 0.405 0.473 0.422 0.573
0.621 (0.368) 1.222 (0.817) 1.244 (0.771) 1.528 (1.106) 1.496 (0.923)
1.285 (1.032) 1.229 (0.824) 1.257 (0.784) 1.322 (0.900) 1.005 (0.432)
0.217 0.430 0.480 0.399 0.570
0.534 (0.317) 1.239 (0.809) 1.252 (0.772) 1.552 (1.153) 1.512 (0.942)
1.262 (1.045) 1.251 (0.821) 1.271 (0.791) 1.333 (0.934) 1.004 (0.434)
5.4. Reconstruction Distortion Analysis To empirically quantify the additional reconstruction error associated with Dirr , we compute the MSE between zT and ẑT , as reported in Tab. 5. We compare MSEtarget , obtained from the target model’s standard inversion for watermark verification, with MSEproxy , obtained after proxy-based inversion. We estimate the proxy-induced distortion as D̂irr = MSEproxy − MSEtarget . For simplicity, the values in parentheses in Tab. 5 are reported as Dirr , which denotes this empirical estimate. When the target and proxy models share the same architecture (e.g., SD2.1), both the reconstruction error and the estimated D̂irr remain small. For example, under TR with SD2.1 as both target and proxy, the MSE is 0.621, with D̂irr = 0.368. In contrast, cross-model settings produce larger distortion, reaching an MSE of 1.496 for TR when targeting SD3 with an SD2.1 proxy. This shows that proxy-induced distortion is measurable and increases with target-proxy discrepancy, consistent with our formulation, although MSE alone is not reliable for detection under common image distortions (see Sec. D).
Figure 7. Similarity distributions under 14 types of image distortions for TR and GS, evaluated on SDXL as the target model.
ther compare against a local cosine baseline (i.e., grid size 16, mean aggregation), which also achieves competitive performance in both cross-model and identical-model settings (e.g., AUC = 1.000 for both TR and GS under SDXL → SD2.1, and 0.954/0.976 in the identical-model setting). These results suggest that capturing geometric discrepancies is central to detection, while G-Cos and L-SPD provide principled geometric metrics.
6. Discussions 6.1. Pseudo-randomness Property PRC watermark (PRCW) (Gunn et al., 2025) incorporates pseudo-random error-correcting codes (PRC) (Christ & Gunn, 2024) to ensure that an adversary cannot distinguish between watermarked and unwatermarked images, even under adaptive query access. As shown in Tab. 9, this pseudo-randomness significantly enhances robustness against forgery attacks. However, successful forgery can still occur in the special case where the proxy and target models are identical. In this setting, our approach serves as an orthogonal defense that remains effective when model mismatch no longer provides protection. We provide further descriptions in Sec. D.
5.3. Robustness Against Image Distortion Attacks Fig. 7 evaluates the robustness of our method against various post-processing distortions. For each distortion and intensity level, we report the average performance over 100 randomly selected samples. The results indicate that our method maintains high geometric consistency under most distortions across both TR and GS. Nevertheless, in a few cases involving stochastic noise and nonlinear signal degradations, the similarity distributions exhibit increased dispersion. Such distortions partially corrupt the latent structural coherence, degrading the intrinsic watermarked features and verification performance (see Sec. D for details).
6.2. Impact of Proxy Model Architecture on Forgery In Tab. 1, we observe that the proxy model with higher latent channel dimension (i.e., SD3 with 16 channels) exhibits stronger forgery capability compared to those with lower di8
Geometric Distortion in Black-Box Watermark Forgery
mensions (i.e., SD2.1). For example, when SD2.1 serves as the target model, SD3 achieves a remarkably high detection rate (e.g., 0.972 for TR and 0.991 for GS). This indicates that increased channel capacity enables the proxy to capture and manipulate finer-grained latent structures, facilitating more accurate alignment with the watermarked latent during forgery. Nevertheless, the unavoidable proxy–target mismatch still induces a structured geometric deviation in latent space, which our method consistently exploits to distinguish forged samples from watermarked ones (see Tab. 2).
the effectiveness of our approach across diverse black-box forgery scenarios. Our findings offer a new perspective for the design of more robust semantic watermarking schemes.
This work was supported by the National Science and Technology Council (NSTC) with Grants NSTC 114-2221-E001-010-MY2, 114-2634-F-001-001-MBK, 114-2634-F002-004, and AS-IAIA-114-M10 for Academia Sinica.
6.3. Practical Limitations and Trade-offs
Impact Statement
While capturing robust local descriptors (e.g., SIFT (Lowe, 2004) or ORB (Rublee et al., 2011)) or deep feature correspondences (e.g., DIFT (Tang et al., 2023)) at intermediate denoising or sampling steps could enhance forgery detection, such approaches incur prohibitive storage overhead for SPs. This requirement contradicts the zero-storage principles inherent in watermarking. Accordingly, our framework operates exclusively on the latent noise zT and ẑT . This design ensures that our approach remains lightweight and maintains seamless compatibility with existing schemes.
This paper enhances the security of generative models by identifying forged watermarks through the lens of geometric distortion. Our approach helps mitigate the threats posed by black-box forgery attacks, thereby preventing unauthorized use and malicious dissemination. Overall, this work contributes to ongoing efforts toward the responsible and secure deployment of generative AI systems.
Acknowledgements
6.4. Security Boundaries under Stronger Attackers The identical target-proxy setting represents an important security boundary in which the proxy-induced architectural mismatch (ϵmis ) is minimized. However, the practical forgery boundary is constrained by an unavoidable internal accumulated error (ϵint ), as discussed in Sec. C.1. This error arises from the asymmetry between the forward generation and inversion processes and accumulates throughout the diffusion trajectory, even under identical architectures. Consequently, exact trajectory reversal remains difficult in practice, leading to measurable latent discrepancies. Empirically, even under this challenging attacker setting, our method achieves robust detection performance, with AUCs above 0.955 for G-Cos and 0.928 for L-SPD. In addition, the proposed pre-verification stage serves as an early warning mechanism by flagging samples with anomalous geometric drift as potential forgeries, even when watermark verification succeeds.
7. Conclusion In this work, we demonstrate that the intrinsic limitations of black-box forgery attacks stem from proxy-target model mismatch, as formalized through the lens of rate–distortion theory. We reveal that this mismatch manifests as structured geometric deviations on the intrinsic latent manifold, characterized by global drift and local deformation. These geometric deviations provide a principled basis for identifying forged samples. Extensive experimental results validate 9
Geometric Distortion in Black-Box Watermark Forgery
References
Christ, M. and Gunn, S. Pseudorandom error-correcting codes. In CRYPTO, pp. 325–347. Springer, 2024.
Al-Haj, A. Combined dwt-dct digital image watermarking. Journal of Computer Science, 3(9):740–746, 2007.
Ci, H., Yang, P., Song, Y., and Shou, M. Z. Ringid: Rethinking tree-ring watermarking for enhanced multi-key identification. In ECCV, pp. 338–354. Springer, 2024.
Ballé, J., Chou, P. A., Minnen, D., Singh, S., Johnston, N., Agustsson, E., Hwang, S. J., and Toderici, G. Nonlinear transform coding. IEEE Journal of Selected Topics in Signal Processing, 15(2):339–353, 2020.
Cover, T. and Thomas, J. Elements of Information Theory. Wiley, 2012. ISBN 9781118585771.
Bernstein, D. J. Chacha, a variant of salsa20. In Workshop Record of SASC, pp. 3–5, 2008.
Cox, I. J., Miller, M. L., Bloom, J. A., Fridrich, J., and Kalker, T. Digital watermarking. Morgan Kaufmann Publishers, 2008.
Betker, J., Goh, G., Jing, L., Brooks, T., Wang, J., Li, L., Ouyang, L., Zhuang, J., Lee, J., Guo, Y., et al. Improving image generation with better captions. Computer Science. https://cdn.openai.com/papers/dall-e-3.pdf, 2(3):8, 2023.
Dong, Z., Shuai, C., Ba, Z., Cheng, P., Qin, Z., Wang, Q., and Ren, K. WMCopier: Forging invisible watermarks on arbitrary images. In NeurIPS, 2025.
Bhatia, R. Positive definite matrices. In Positive Definite Matrices. Princeton University Press, 2009.
Efron, B. Tweedie’s formula and selection bias. Journal of the American Statistical Association, 106(496):1602– 1614, 2011.
Biden, J. R. Executive order on the safe, secure, and trustworthy development and use of artificial intelligence. https://www.whitehouse.gov/briefing-r oom/presidential-actions/2023/10/30/ executive-order-on-the-safe-secure-a nd-trustworthy-development-and-use-o f-artificial-intelligence/, 2023. Accessed: 2024-09-24.
Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., et al. Scaling rectified flow transformers for high-resolution image synthesis. In ICML, 2024. Goodman, E. P. Synthetic content: Default to distrust. Case W. Rsrv. L. Rev., 75:961, 2024. Gunn, S., Zhao, X., and Song, D. An undetectable watermark for generative image models. In ICLR, 2025.
Blau, Y. and Michaeli, T. The perception-distortion tradeoff. In CVPR, pp. 6228–6237, 2018.
Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. NeurIPS, 33:6840–6851, 2020.
Blau, Y. and Michaeli, T. Rethinking lossy compression: The rate-distortion-perception tradeoff. In ICML, pp. 675– 685. PMLR, 2019.
Hu, R., Zhang, J., Li, Y., Li, J., Guo, Q., Qiu, H., and Zhang, T. Videoshield: Regulating diffusion-based video generation models via watermarking. In ICLR, 2025.
Brooks, D., Schwander, O., Barbaresco, F., Schneider, J.-Y., and Cord, M. Riemannian batch normalization for spd neural networks. NeurIPS, 32:15463–15474, 2019.
Huang, Z. and Van Gool, L. A riemannian network for spd matrix learning. AAAI, 31(1), 2017.
Bui, T., Agarwal, S., Yu, N., and Collomosse, J. Rosteals: Robust steganography using autoencoder latent space. In CVPR, pp. 933–942, 2023.
Jaidka, K., Chen, T., Chesterman, S., Hsu, W., Kan, M.-Y., Kankanhalli, M., Lee, M. L., Seres, G., Sim, T., Taeihagh, A., et al. Misinformation, disinformation, and generative ai: Implications for perception and policy. Digital Government: Research and Practice, 6(1):1–15, 2025.
Cao, Y., Li, S., Liu, Y., Yan, Z., Dai, Y., Yu, P., and Sun, L. A survey of ai-generated content (aigc). ACM Computing Surveys, 57(5):1–38, 2025.
Koga, H. et al. Information-spectrum methods in information theory, volume 50. Springer Science & Business Media, 2013.
Chen, J., Jincheng, Y., Chongjian, G., Yao, L., Xie, E., Wang, Z., Kwok, J., Luo, P., Lu, H., and Li, Z. Pixart-α: Fast training of diffusion transformer for photorealistic text-to-image synthesis. In ICLR, 2024.
Labs, B. F. Flux. https://github.com/black-f orest-labs/flux, 2024.
Chen, Y., Ma, Z., Fang, H., Zhang, W., and Yu, N. Tag-wm: Tamper-aware generative image watermarking via diffusion inversion sensitivity. In ICCV, pp. 16723–16732, 2025.
Lee, D. Z., Fang, H., Wang, H., and Chang, E.-C. Removal attack and defense on ai-generated content latent-based watermarking. In ACM CCS, pp. 2174–2188, 2025. 10
Geometric Distortion in Black-Box Watermark Forgery
Lee, S. J. and Cho, N. I. Semantic watermarking reinvented: Enhancing robustness and generation quality with fourier integrity. In ICCV, pp. 18759–18769, 2025.
Shannon, C. E. A mathematical theory of communication. The Bell system technical journal, 27(3):379–423, 1948. Shannon, C. E. et al. Coding theorems for a discrete source with a fidelity criterion. IRE Nat. Conv. Rec, 4(142-163): 1, 1959.
Legislature, C. S. California assembly bill ab-3211 california digital content provenance standards. https://le giscan.com/CA/text/AB3211/id/2984195, February 2024. Accessed: 2026-01-05.
Song, J., Meng, C., and Ermon, S. Denoising diffusion implicit models. In ICLR, 2021.
Li, K., Huang, Z., Hou, X., and Hong, C. Gaussmarker: Robust dual-domain watermark for diffusion models. In ICML, 2025a.
Tang, L., Jia, M., Wang, Q., Phoo, C. P., and Hariharan, B. Emergent correspondence from image diffusion. NeurIPS, 36:1363–1389, 2023.
Li, W., Zhang, H., and Qu, Q. Shallow diffuse: Robust and invisible watermarking through low-dim subspaces in diffusion models. In NeurIPS, 2025b.
Union, E. Artificial intelligence act: Regulation (eu) 2024/1689 of the european parliament and of the council. https://eur-lex.europa.eu/legal-con tent/EN/TXT/?uri=CELEX:32024R1689, June 2024. Accessed: 2026-01-05.
Li, X., Yang, B., Cheng, D., and Zeng, T. A generalization of lsb matching. IEEE SPL, 16(2):69–72, 2009.
Wang, T., Wang, A., Cong, T., Ran, D., Liu, J., and Wang, X. Cryptanalysis of pseudorandom error-correcting codes. arXiv preprint arXiv:2512.17310, 2025.
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In ECCV, pp. 740–755. Springer, 2014.
Wen, Y., Kirchenbauer, J., Geiping, J., and Goldstein, T. Tree-rings watermarks: Invisible fingerprints for diffusion images. NeurIPS, 36:58047–58063, 2023.
Lowe, D. G. Distinctive image features from scale-invariant keypoints. IJCV, 60(2):91–110, 2004.
Westerlund, M. The emergence of deepfake technology: A review. Technology innovation management review, 9 (11), 2019.
Luo, C. Understanding diffusion models: A unified perspective. arXiv preprint arXiv:2208.11970, 2022.
Yang, P., Ci, H., Song, Y., and Shou, M. Z. Can simple averaging defeat modern watermarks? In NeurIPS, 2024a.
Midjourney, Inc. Midjourney (version 7.0), 2025. URL https://www.midjourney.com/. Accessed: 2026-01-05.
Yang, Z., Zeng, K., Chen, K., Fang, H., Zhang, W., and Yu, N. Gaussian shading: Provable performance-lossless image watermarking for diffusion models. In CVPR, pp. 12162–12171, 2024b.
Müller, A., Lukovnikov, D., Thietke, J., Fischer, A., and Quiring, E. Black-box forgery attacks on semantic watermarks for diffusion models. In CVPR, pp. 20937–20946, 2025.
You, K. and Park, H.-J. Re-visiting riemannian geometry of symmetric positive definite matrices for the analysis of functional connectivity. NeuroImage, 225:117464, 2021.
Pennec, X., Fillard, P., and Ayache, N. A riemannian framework for tensor computing. IJCV, 66(1):41–66, 2006.
Zhang, G., Qian, J., Chen, J., and Khisti, A. Universal rate-distortion-perception representations for lossy compression. NeurIPS, 34:11517–11529, 2021.
Podell, D., English, Z., Lacey, K., Blattmann, A., Dockhorn, T., Müller, J., Penna, J., and Rombach, R. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In ICLR, 2024.
Zhang, L., Rao, A., and Agrawala, M. Adding conditional control to text-to-image diffusion models. In ICCV, pp. 3836–3847, 2023.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In CVPR, pp. 10684–10695, 2022.
Zhao, X., Zhang, K., Su, Z., Vasan, S., Grishchenko, I., Kruegel, C., Vigna, G., Wang, Y.-X., and Li, L. Invisible image watermarks are provably removable using generative ai. NeurIPS, 37:8643–8672, 2024.
Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. Orb: An efficient alternative to sift or surf. In ICCV, pp. 2564– 2571, 2011.
Zhu, C., Li, Z., Yang, R., Birke, R., Chen, P.-Y., Ho, T.Y., and Chen, L. Y. Optimization-free universal watermark forgery with regenerative diffusion models. arXiv preprint arXiv:2506.06018, 2025.
Sander, T., Fernandez, P., Durmus, A. O., Furon, T., and Douze, M. Watermark anything with localized messages. In ICLR, 2025. 11
Geometric Distortion in Black-Box Watermark Forgery
The content of Supplementary Material is summarized as follows: 1) In Sec. A, we provide the background information to facilitate a better understanding of our methodology; 2) In Sec. B, we state the underlying assumptions and provide formal proofs for Theorem 4.2 and Theorem 4.10; 3) In Sec. C, we offer an in-depth analytical extension of our main findings, focusing on the theoretical underpinnings of detection errors and the practical scope of our evaluation; 4) In Sec. D, we elaborate on our implementation, including the datasets, the model architectures, and an extended set of experimental evaluations; 5) Finally, Sec. E showcases the visual examples of forged images generated by guidance- and optimization-based black-box forgery attacks.
A. Background A.1. Semantic Watermarking Methods Tree-Ring (TR). Tree-Ring (Wen et al., 2023) introduces the concept of inversion-based watermarking, which can be divided into two phases: (i) Generation and (ii) Detection. • Generation. During image generation, TR injects a concentric circular pattern into the frequency representation of a (w) clean initial latent zT ∼ N (0, IN ), generating a watermarked noise zT . This latent then serves as the starting point for the diffusion process, which involves denoising and decoding to generate the final watermarked image x(w) . (w)
• Detection. We first reconstruct the initial noise ẑT = I0→T (E(x(w) ); U) and analyze its frequency spectrum. A statistical test aggregates the squared absolute differences between observed and expected frequency values across all rings; a resulting p-value below the threshold τ indicates the presence of the watermarked pattern. In this paper, we use a ring pattern with a radius of 10 and apply zero-bit watermarking. Following the prior work (Müller et al., 2025), we adopt the same detection thresholds derived from statistics on 5,000 watermarked and 5,000 clean images to achieve the target false positive rate (FPR). (w)
RingID. As a subsequent advancement of TR, RingID (Ci et al., 2024), ensures that the distribution of zT more closely aligns with N (0, IN ). Furthermore, it introduces support for multi-bit watermarking, enabling the embedding of multi-bit messages for user identification. Note that we follow the default parameters of RingID for implementation. Symmetric Fourier-based Watermarks. Compared to existing frequency-domain methods, Symmetric Fourier-based watermarks (Lee & Cho, 2025) ensure frequency integrity by enforcing Hermitian symmetry (HS). Based on this principle, the authors introduce two semantic variants, HSTR and HSQR, both of which exhibit superior performance and robustness over established baselines such as TR and RingID. To ensure a fair comparison, our experimental settings align with the original (Lee & Cho, 2025) (i.e., central-aware embedding with a 44 × 44 frequency mask, and HSQR configured with QR version 1 and a box size of 2). Gaussian Shading (GS). Gaussian Shading (Yang et al., 2024b) is the first bit-stream level semantic watermarking by incorporating the cryptographic primitive to achieve preserving generation. We describe the two phases of GS as follows: • Generation. Given a message s of length k, we first replicate it ρ times to obtain sd , which is then encrypted using the symmetric stream cipher ChaCha20 (Bernstein, 2008). The cipher takes a secret key and a unique random seed as input for each image, generating an encrypted bitstream m. The encrypted message m is then used to modulate the sampling (w) process of the initial latent zT . Specifically, GS partitions the Gaussian distribution into 2ℓ bins with equal probability. In this work, we adopt the standard setting with ℓ = 1, which effectively bisects the Gaussian distribution into two regions (w) corresponding to negative and positive values. Each corresponding latent bit of m[i] ∈ {0, 1} determines whether zT [i] is sampled from the negative or the positive region of the Gaussian distribution. In addition, the encryption ensures that the (w) bits in m are uniformly distributed, thereby preserving the Gaussian distribution of zT . The watermarked image is then (w) generated by continuing the standard sampling process: x(w) = GT →0 (zT ; U). (w)
• Verification. To verify the watermark, GS first applies the inversion process I0→T to obtain the estimated x̂T . The (w) (w) inverted latent x̂T is then quantized to retrieve the encrypted message bits m′ . Specifically, we set m′ [i] = 0 if ẑT [i] < 0 ′ ′ ′d and m [i] = 1 otherwise. The recovered bitstream m is then decrypted to reconstruct the repeated message s . Finally, the original message of s′ is determined via majority voting over ρ replicated bits, which corrects bit errors and improves robustness to noise.
1
Geometric Distortion in Black-Box Watermark Forgery
Following the settings in (Müller et al., 2025), we use an encoding window of ℓ = 1, with a unique random key and message per image. The message length k is 256, resulting in 1024 bits. The repetition factor ρ is 64 for SD2.1, and 256 for FLUX.1 and SD3, which uses 16-channel latents compared to four channels in the other models. In the detection scenario, we count a true positive if r(s, s′ ) exceeds a threshold calibrated to achieve a specified FPR (i.e., 10−6 ). The threshold is 0.70703. In addition, since our objective is centered on detection, we exclude attribution scenarios from this study. In addition, Videoshield (Hu et al., 2025) adapts the GS scheme to both text-to-video and image-to-video diffusion models by incorporating intrinsic tamper localization. This training-free approach preserves spatio-temporal integrity while ensuring the latent trajectory remains consistent throughout the temporal domain. Crucially, our detection framework can be extended to assess the authenticity of each recovered latent frame, effectively distinguishing between watermarked recovered latents and forged recovered latents before watermark extraction. The extension of our method to such video-based scenarios is deferred to future work. TAG Watermarks. Building on GS, TAG watermark (Chen et al., 2025) employs a dual-mark joint sampling strategy to co-embed copyright and localization watermarks without quality loss. By leveraging the sensitivity of the inversion process, it can localize manipulated regions to guide tamper-aware decoding, which excludes compromised bits and robustly restores messages under tampering. For our experiments, we embed the 256-bit watermark into the initial latent noise zT using the TAG scheme. To maximize error resilience, we implement an adaptive repetition strategy, where the redundancy factor Nrep T) is dynamically determined by Nrep = ⌊ Dim(z 256 ⌋, thereby ensuring the watermark payload fully populates the available latent dimensions Dim(zT ). The threshold follows the same setting as GS. PRC Watermarks. The PRC Watermark (PRCW) (Gunn et al., 2025) embeds information into the latent representation of diffusion models by using pseudorandom error-correcting codes (PRC) (Christ & Gunn, 2024). Instead of stream cipher techniques, PRCW focuses on achieving statistical undetectability alongside robustness, ensuring that an adversary cannot distinguish between watermarked and unwatermarked images, even after making many adaptive queries. However, recent studies (Lee et al., 2025; Wang et al., 2025) have exposed emerging threats in such schemes, ranging from direct signal removal to cryptanalytic exploits against the PRC mechanism. In our implementation, following the original settings in (Gunn et al., 2025), we set the undetectable parameter t = 3 and maintain a FPR of 10−5 . Other Related Works. Beyond the schemes discussed above, several studies (Li et al., 2025a;b) have proposed alternative approaches to enhance the consistency of semantic watermarks. GaussMarker (Li et al., 2025a) embeds watermark signals in both spatial and frequency domains, forming a dual-domain representation. To handle geometric transformations, it employs a learnable Gaussian Noise Restorer (GNR) that maps distorted latents back to a Gaussian distribution, mitigating the effects of rotation and cropping. Similarly, Shallow Diffuse (Li et al., 2025b) adopts a decoupling strategy that exploits the low-dimensional subspace of the diffusion process. By embedding the watermark primarily in the null space of this subspace, the method separates the watermark signal from the sampling trajectory. In practice, GaussMarker can be viewed as a combination of TR and GS, while Shallow Diffuse is derived from the TR scheme. Accordingly, in this work, we focus our comparisons on representative frequency-domain and bitstream-level watermarking schemes. A.2. Black-box forgery Attacks Guidance-based Methods. An adversary performs guidance-based forgery (Müller et al., 2025; Zhu et al., 2025) by first (w) applying DDIM inversion with a proxy model ΘA to estimate a latent noise vector ẑT from the public watermarked image (w) x . This estimated latent serves as the initialization for a guided reverse diffusion process. Guidance can be introduced either through a textual prompt t or via a controllable model such as ControlNet (Zhang et al., 2023), which conditions generation on a cover image x(c) . In the latter case, a trainable control module FA extracts structural information from x(c) (i.e., edge or depth representations) and an encoder EA maps this information into a visual condition embedding (Zhu et al., 2025). This embedding guides the generation process jointly with a textual embedding TA (t(c) ), which is derived from a descriptive prompt t(c) associated with the cover image and conditioned on a pretrained, frozen U-Net (UA ). We denote the resulting guided reverse diffusion process by the operator G, which is parameterized by the U-Net and the control module, (w)
ẑ0′ = GA,T →0 (ẑT
| EA (x(c) ), TA (t(c) ); UA , FA ),
where EA (x(c) ) and TA (t(c) ) provide the visual and textual conditions, respectively. Finally, the decoder D maps the refined 2
Geometric Distortion in Black-Box Watermark Forgery
latent vector ẑ0′ back to the pixel space, producing the forged image x̂(w) . Optimization-based Methods. In contrast to guidance-based methods that manipulate the reverse diffusion process, optimization-based forgery operates by directly solving for an optimal latent variable. The core objective is to find a minimal perturbation to a clean image’s latent state which, upon forward diffusion to timestep T , aligns with the known latent representation of a target watermarked image. As demonstrated in (Müller et al., 2025), performing this optimization in the near-noiseless latent space at t = 0 is an effective strategy. (c)
The process begins by encoding the cover image x(c) to obtain its latent representation ẑ0 = EA (x(c) ). The optimization objective is then formulated as minimizing the squared L2 distance between the diffused perturbed latent and the target, as defined by the loss function: (c)
(w)
Lforgery (δ) = I0→T (ẑ0 + δ; uA ) − ẑT
, 2
where I0→T denotes the deterministic forward diffusion process that applies noise according to the predefined schedule up to timestep T , and u represents the denoising backbone model. The adversary applies gradient descent for up to N steps to (c) minimize this loss w.r.t. δ. Once the optimal perturbation δ ∗ is found, the forged latent ẑ0 + δ ∗ is decoded using the proxy (c) model to generate the final forged image x̂ . A.3. Rate-distortion Theory Rate-distortion theory (Shannon et al., 1959; Koga et al., 2013) characterizes the fundamental trade-off between the rate used to represent samples from a data source X ∼ pX and the expected distortion incurred in decoding those samples from their compressed representations. Formally, the relation between the input X and output X̂ of an encoder-decoder pair is a (possibly stochastic) mapping defined by some conditional distribution pX̂|X . The expected distortion is given by E[∆(X, X̂)],
(6)
where the expectation is over the joint distribution pX,X̂ = pX̂|X pX , and ∆ : X × X̂ → R+ is any full-reference distortion measure such that ∆(X, X̂) = 0 if and only if X = X̂. A fundamental result states that for an i.i.d. source X, the minimum achievable rate under a distortion constraint D is given by the rate-distortion function: R(D) = min I(X, X̂) s.t. E[∆(X, X̂)] ≤ D, (7) pX̂|X
where I denotes mutual information (Cover & Thomas, 2012). Closed-form expressions for the rate-distortion function R(D) exist only for a few source distributions and simple distortion measures (e.g., mean-squared error or Hamming distance), but in general, R(D) is known to be non-increasing, convex, and continuous. A.4. Symmetric Positive-Definite (SPD) Manifold and AIRM The space of SPD matrices forms a differentiable manifold known as the SPD manifold, which has been successfully applied in various fields (Huang & Van Gool, 2017; Brooks et al., 2019; You & Park, 2021). Since SPD matrices lie on a curved, non-Euclidean space, standard similarity measures (e.g., Euclidean distance or Pearson correlation) fail to capture their intrinsic geometry. To account for the manifold structure, we adopt the Affine-Invariant Riemannian Metric (AIRM) (Pennec et al., 2006) to measure the distance between two SPD matrices P1 and P2 . The AIRM distance is defined as: v uN uX d(P1 , P2 ) = t ln2 λi , (8) i=1
where λi are the eigenvalues of P1−1 P2 . The AIRM is characterized by its invariance under affine transformations and inversion. This metric provides a principled framework for analyzing manifold-valued data and effectively mitigates the “swelling effect,” a common artifact where the determinant of an intermediate matrix exceeds that of its endpoints in Euclidean space. 3
Geometric Distortion in Black-Box Watermark Forgery
B. Proofs of Theorems and Lemmas in Sec. 4 B.1. Assumptions Assumption B.1 (Gaussian Approximation). We assume the adversary’s inversion task can be modeled by conditional probability distributions that are approximately multivariate Gaussian. Specifically, the true posterior distribution (from the target model, Θ) and the adversary’s assumed posterior (from the proxy model, ΘA ) are given by: PΘ (X0 |Yw ) ≈ N (µΘ (Yw ), σ 2 Id ),
PΘA (X0 |Yw ) ≈ N (µA (Yw ), σ 2 Id ),
where Yw is the observed watermarked output, and σ 2 is a shared noise variance. The means µΘ and µA differ due to the model mismatch. The shared-variance assumption in Assumption B.1 reflects a simplified setting in which the adversary has access to a generative model with similar uncertainty behavior. While variance may differ across architectures in practice, adopting a shared variance allows us to isolate the effect of posterior mean deviation, which constitutes the dominant source of mismatch in latent representations. Moreover, this assumption establishes a conservative analytical baseline: any additional variance mismatch would further increase the reconstruction distortion, reinforcing the theoretical limits derived in Sec. 4. To further justify the shared-variance premise, we consider the underlying score-matching objective. According to Tweedie’s Formula (Efron, 2011; Luo, 2022), the posterior mean of clean data x0 is determined by the score function: 1 E[x0 | xt ] = √ xt + (1 − αt )∇ log pt (xt ) . αt
(9)
Crucially, the posterior covariance is related to the local curvature of the log-density, i.e., ∇2 log pt (xt ). Because these quantities are defined by the data distribution and noise schedule rather than the specific model architecture, modern backbones (e.g., U-Net, DiT) trained to approximate the same target score ∇ log pt (xt ) exhibit statistically consistent estimation errors and uncertainty structures. We note that modern diffusion/flow-matching frameworks (e.g., EDM, FLUX) increasingly adopt straightened transport paths and deterministic sampling (i.e., solving ODEs via Euler or Heun methods) for stable generation. Under such deterministic ODE trajectories, inversion and reconstruction are primarily governed by the learned vector field (drift) rather than stochastic noise injection. Consequently, the sensitivity to exact covariance structures is significantly reduced, mitigating residual variance discrepancies across different models. Thus, Assumption B.1 is not merely a mathematically convenient simplification, but a realistic reflection of practical model standardization. Second, we define a distortion metric to quantify representation quality. In rate-distortion theory, the choice of distortion measure is critical for determining the theoretical limits of compression. Under the Gaussian assumption, mean squared error (MSE) is both standard and analytically tractable. We thus adopt MSE to assess the fidelity of recovered latents. Assumption B.2 (Mean Squared Error Distortion). The latent recovery quality is measured by the normalized mean squared error (MSE) between the original latent variable X0 and the adversary’s estimated latent X̂0 : 1 D(X̂0 , X0 ) = E ||X0 − X̂0 ||22 , d where the expectation is over the joint distribution of (X0 , X̂0 ). B.2. Proof of Theorem 4.2 Proof. We derive the lower bound under the Gaussian latent approximation and MSE distortion. Step 1: Proxy-target Posterior Mismatch. Let P = PΘ (· | Yw ), Q = PΘA (· | Yw ) denote the target and proxy posteriors. By Def. 4.1, the mismatch-induced irreducible information loss is Dirr = EYw [DKL (P ∥Q)]. When Dirr > 0, the proxy posterior differs from the target posterior. This posterior mismatch captures an irreducible source of reconstruction error that cannot be removed by increasing the information rate. Under our Gaussian rate-distortion surrogate, this mismatch contributes to the additive term Dirr in the distortion lower bound. 4
Geometric Distortion in Black-Box Watermark Forgery
Step 2: Effective Information Rate. Since the adversary reconstructs the latent through the proxy posterior rather than the target posterior, only part of the nominal information rate R is usable for target-consistent reconstruction. Under the Gaussian latent approximation, we model this loss by the effective rate penalty defined in Def. 4.1, Dirr 1 Ipen = log2 1 + 2 . 2 σ Thus, the adversary operates at an effective rate Reff ≤ R − Ipen . Step 3: Gaussian Rate-distortion Lower Bound. For a Gaussian source with variance σ 2 under MSE distortion, the inverse rate-distortion function is D(R) = σ 2 2−2R . Since D(R) is monotonically decreasing in R, substituting Reff ≤ R − Ipen gives D(Reff ) ≥ σ 2 · 2−2(R−Ipen ) . Since Dirr > 0, the additive mismatch term prevents the lower bound from vanishing, while Ipen > 0 further reduces the effective rate. This establishes the theorem. Relation to Total Variation. The KL mismatch above also implies distributional separation between the target and proxy posteriors. When Dirr > 0, the target and proxy posteriors q are distinct on a set of nonzero measure. Pinsker’s inequality 1 provides the standard upper control ∥P − Q∥TV ≤ 2 DKL (P ∥Q). This relation is used only to interpret posterior mismatch as a distributional discrepancy. The rate penalty in Thm. 4.2 is the Gaussian surrogate defined in Def. 4.1.
B.3. Proof of Theorem 4.10 The proof of Theorem 4.10 leverages two complementary geometric perspectives. Lemma 4.6 establishes the existence of a persistent global angular shift on the latent hypersphere, whereas Lemma 4.9 accounts for the local structural inconsistency on the SPD manifold. Proof of Lemma 4.6. We analyze the geometric effect of a proxy-induced perturbation on the hyperspherical representation (w) of the latent. Let z ≡ zT ∼ N (0, IN ) and consider a forged recovered latent ẑ = z + δ. We begin by decomposing the perturbation δ into components parallel and orthogonal to z: δ = δ ∥ + δ⊥ ,
δ∥ = αz,
δ⊥ ⊥ z.
By construction, the parallel component affects only the radial magnitude, while the orthogonal component governs orientation changes. The angular distortion between z and ẑ is given by cos θ =
⟨z, z + δ⟩ . ∥z∥2 ∥z + δ∥2
Using the above decomposition and noting that ⟨z, δ⊥ ⟩ = 0, we obtain ⟨z, z + δ⟩ = ∥z∥22 + ⟨z, δ∥ ⟩, and ∥z + δ∥22 = ∥z∥22 + 2⟨z, δ∥ ⟩ + ∥δ∥ ∥22 + ∥δ⊥ ∥22 . Assuming ∥δ∥2 ≪ ∥z∥2 , a first-order Taylor expansion of the denominator yields cos θ ≈ 1 −
∥δ⊥ ∥22 . 2∥z∥22
This approximation shows that the induced angular deviation depends solely on the orthogonal component of the perturbation. Finally, since z ∼ N (0, IN ), its norm concentrates sharply around ∥z∥22 ≈ N as N → ∞. Under the proxy-target model mismatch, the perturbation δ cannot be consistently aligned with z in expectation, implying that the orthogonal energy 5
Geometric Distortion in Black-Box Watermark Forgery
∥δ⊥ ∥22 remains strictly positive. Consequently, the angular distortion is bounded away from zero, acting as a manifest signature of forgery. Substituting into the above expression yields E[SAD(z, ẑ)] = E[θ] = Ω(1), which completes the proof of Lemma 4.6. Proof of Lemma 4.9. In this proof, we exploit the congruence invariance of the AIRM, which states that for any P, Q ∈ PN and any invertible matrix G ∈ RN ×N , the distance satisfies dAIRM (P, Q) = dAIRM (GP G⊤ , GQG⊤ ). (w)
(w)
Let P := C(zT ) ∈ PN denote the local covariance associated with the reference latent zT . To derive the probability bound, we analyze the geometric deviation in two distinct cases: (w)
Case 1: Watermarked recovered latents. For a watermarked recovered latent ẑT produced by the target model, the relative configuration of tokens within the local neighborhood B is preserved up to negligible estimation error. Thus, the (w) induced covariance Q(w) := C(ẑT ) remains close to P in the SPD manifold. Since the regularization εI ensures strict positive definiteness, the AIRM metric is continuous on PN . Therefore, such structural preservation under an affine (congruent) transformation yields: (w)
(w)
LGI(zT , ẑT ) = dAIRM (P, Q(w) ) ≈ 0. (f )
(w)
Case 2: Forged recovered latents. Consider a forged recovered latent ẑT = ẑT + δ that induces a non-congruent deformation of the local token neighborhood B. As a result, the induced covariance Q(f ) cannot be expressed as GP G⊤ for any invertible matrix G sufficiently close to the identity, and thus lies outside the local congruence orbit of P on the SPD manifold. Mathematically, let S := P −1/2 Q(f ) P −1/2 . The structural mismatch implies that S cannot be close to the identity. Specifically, there exists a constant η0 > 0 such that, with non-vanishing probability p0 > 0, the spectrum of S satisfies: λmax (S) ≥ eη0
or
λmin (S) ≤ e−η0 .
Recall that the AIRM geodesic distance is characterized by the log-eigenvalues of S. Thus, by the property of the Frobenius norm: sX (w) (f ) LGI(zT , ẑT ) = ∥ log S∥F = ln2 λi (S) ≥ max | ln λi (S)| ≥ η0 . i
i
Finally, combining the probability bound p0 with the spectral gap η0 , and taking the expectation over the dataset, we obtain: h i (w) (f ) E LGI(zT , ẑT ) ≥ η0 p0 = Ω(1). This completes the proof of Lemma 4.9. Proof of Theorem 4.10. The proof relies on a concentration-of-measure argument in high-dimensional spaces, combining the global geometric drift established in Lemma 4.6 and the local structural deformation from Lemma 4.9. First, we define a rejection region R ⊂ R2+ in the joint metric space defined by SAD and LGI: R := {(s, ℓ) ∈ R2+ | s > τs and ℓ > τℓ }, where τs and τℓ are positive constants. • Forged recovered latents: By Lemma 4.6, the expected angular distortion induced by the orthogonal perturbation component ∥δ⊥ ∥ is bounded away from zero (i.e., E[SAD] = µs = Ω(1)). Similarly, Lemma 4.9 demonstrates that non-congruent structural deformation leads to a non-vanishing local geometric inconsistency (i.e., E[LGI] = µℓ = Ω(1)). However, divergence in expectation alone is insufficient to imply separability. To bridge this gap, we invoke concentrationof-measure results in high-dimensional spaces (N ≫ 1). Under the assumption that the considered geometric metrics are 6
Geometric Distortion in Black-Box Watermark Forgery
Lipschitz continuous w.r.t the latent variables, it is a well-established result that for isotropic Gaussian distributions, and their normalized projections onto the hypersphere, such quantities concentrate sharply around their expectations, with tails that decay exponentially in the dimension. Consequently, the probability that the observed distortions fall significantly below their expectations decays exponentially with the dimension N . For thresholds chosen such that 0 < τs < µs and 0 < τℓ < µℓ , we obtain: Pr(SAD > τs ) ≥ 1 − e−Ω(N ) ,
and
Pr(LGI > τℓ ) ≥ 1 − e−Ω(N ) .
Applying a union bound, the forged sample falls within the rejection region R with high probability: Pr((SAD, LGI) ∈ R) ≥ 1 − 2e−Ω(N ) = 1 − e−Ω(N ) . (w)
• Watermarked recovered latents: For a watermarked recovered latent ẑT generated by the target model, the recovery error is minimal and structurally coherent. As N → ∞, both SAD and LGI converge to zero in probability. Therefore, the probability that a watermarked sample falls within the rejection region R vanishes asymptotically. Combining these results, we find that as N → ∞, the overlap probability between the distributions of watermarked and forged latents in the joint geometric space vanishes. Thus, there exists a decision boundary that separates the two classes with probability approaching one.
C. Additional Analysis C.1. Intrinsic and External Errors in Diffusion Models Internal Accumulated Error. Internal accumulated error εint arises from systematic mismatches between the forward generation and backward inversion processes. In practice, forward diffusion is conditioned on prompts and amplified by classifier-free guidance, while inversion is typically performed under unconditional settings with approximate solvers. Although each stepwise discrepancy is small, such biases accumulate over the diffusion horizon, resulting in a non-negligible estimation error, internal latent drift: (w) (w) εint := dist IΘ (z0 ), zT , where dist(·, ·) denotes a distance metric (e.g., L2 norm) in the latent space, measuring the divergence of the reconstructed (w) (w) latent IΘ (z0 ) from the ground-truth zT . This intrinsic error is unavoidable and establishes a baseline distortion level (lower bound) for evaluating watermark robustness within the latent domain. External Inevitable Perturbations. Beyond internal drift, we consider two primary external sources of degradation: (i) common image-domain distortions εimg , and (ii) black-box forgery-induced model mismatch εmis . The latter occurs when a watermarked image is manipulated by an unknown proxy model ΘA . We define this mismatch error εmis as the latent deviation arising from the internal processing of ΘA : (w) (w) εmis := IΘA ◦ GΘA (zT ) − zT , which characterizes the effective latent deviation induced by ΘA . The magnitude of εmis is correlated with the architectural discrepancy between ΘA and Θ: as this discrepancy increases, the induced inversion–sampling mapping departs further from identity, resulting in a larger latent mismatch. Therefore, εmis constitutes an intrinsic, architecture-induced deviation that is unavoidable without access to the exact target parameters Θ. C.2. Comparison of Forgery Paradigms and Threat Models Our study focuses on black-box forgery scenarios, in which an adversary exploits proxy models to regenerate watermarked content. We distinguish this setting from alternative forgery paradigms, such as statistical forgery (Yang et al., 2024a) or watermark copying (Dong et al., 2025), which are based on different assumptions and threat models. Specifically, the statistical averaging method (Yang et al., 2024a) requires access to a collection of watermarked samples in order to estimate the underlying watermark distribution. Such requirements are often impractical in realistic settings, where an adversary typically has limited access to watermarked data. Moreover, existing evaluations primarily focus on frequency-based semantic watermarks (e.g., TR), and their effectiveness against bitstream-level schemes (e.g., GS) remains unclear. In 7
Geometric Distortion in Black-Box Watermark Forgery
contrast, WMCopier (Dong et al., 2025) transplants invisible signatures onto arbitrary images through unconditional diffusion and iterative optimization. However, its evaluation mainly targets post-processing watermarks (Cox et al., 2008), rather than the semantic watermarking schemes (e.g., TR and GS) considered in our work. To the best of our knowledge, black-box forgery (Müller et al., 2025) remains the only approach demonstrated to produce forged outputs against semantic watermarking schemes successfully. C.3. Discussion on Adaptive Attacks The adversary may be aware of the deployment of anti-forgery detection and could develop adaptive attacks to circumvent it. We consider one possible such adaptive attack, Reforge, described below. Reforge. Reforge adopts a two-stage adaptive strategy: cleansing and re-imprinting. The adversary first removes the original watermark through purification and then imprints an estimated watermark pattern using the proxy model. This two-stage process aims to deceive the watermark detector while preserving the semantic content of the original image. In practice, such operations inevitably lead to perceptual degradation (e.g., reduced PSNR and SSIM), as removal attacks typically require additive noise to drive the latent representation away from the reference watermarked latent zT . The subsequent re-imprinting further shifts the latent toward the watermarked latent, but this manipulation compounds the structural damage, severely compromising image fidelity with noticeable visual artifacts. We exclude Reforge from our comparison due to its prohibitive computational overhead and poor image fidelity. Furthermore, this two-stage adaptive strategy, which relies on a proxy model, significantly distorts the underlying geometric structure of the original latent manifold. Thus, our metric remains effective against such adaptive attempts. C.4. Our Detection Pipeline As shown in Fig 8, the detection metrics are computed between the reference watermarked latent zT and the recovered latent ẑT before watermark verification. Importantly, our detection framework is agnostic to the specific watermarking scheme and can be generalized to any latent-based (i.e., semantic) watermarking that utilizes deterministic reverse diffusion trajectories.
Figure 8. Illustration of the proposed detection framework.
C.5. Connection between Removal Guarantees and Distortion Bounds Recent work (Zhao et al., 2024) theoretically analyzes the removability of invisible watermarks under stochastic regeneration perturbations in diffusion models. Although that work focuses on post-hoc watermarking, whereas our work studies in-generation watermark schemes, both are related through the distortion introduced during watermark manipulation. Specifically, (Zhao et al., 2024) shows that successful watermark removal requires sufficient stochastic perturbation, which progressively degrades reconstruction fidelity. In contrast, our analysis reveals that black-box forgery remains subject to Dirr caused by proxy-target mismatch and inversion-generation asymmetry, even without explicit stochastic perturbation. We further find that this distortion manifests geometrically as directional drift on the hypersphere and localized deformation on the SPD manifold. These observations indicate a fundamental trade-off among watermark suppression, forgery fidelity, and geometric consistency in generative models. 8
Geometric Distortion in Black-Box Watermark Forgery
D. Experimental Details In this section, we provide a comprehensive overview of experiments. We elaborate on diffusion model architectures, the datasets, and the computational runtime. Besides, we present additional experiments to provide further insights. D.1. Target and Proxy Models We evaluate our method across diverse diffusion model architectures, with target models, including SD2.1, SDXL, PixArt-Σ, SD3, and FLUX.1. Tab. 6 summarizes the sampling configurations and key hyperparameters for all models used in our experiments. Notably, FLUX.1 and SD3, which are based on rectified-flow matching, employ fewer inference steps (i.e., 20 for FLUX.1) and a lower guidance scale (i.e., 7.0 for SD3). Table 6. Overview of model settings used in the experiments. All images are generated at a 512 × 512 resolution. L. Ch. denotes the number of latent channels; Scheduler indicates the algorithm used for generation and inversion; Steps refers to the number of inference steps; G. Scale represents the guidance scale during generation. Model
Hugging Face ID
Type
L. Ch.
SD2.1 SDXL Pixel-Σ FLUX.1 SD3
stabilityai/stable-diffusion-2-1-base stabilityai/stable-diffusion-xl-base-1.0 PixArt-alpha/PixArt-Sigma-XL-2-512-MS black-forest-labs/FLUX.1-dev stabilityai/stable-diffusion-3-medium
UNet UNet DiT DiT DiT
4 4 4 16 16
Scheduler
Steps
G. Scale
DDIM DDIM DPM FlowMatchEuler FlowMatchEuler
50 50 20 20 28
7.5 7.5 4.5 3.5 7.0
D.2. Prompting and Cover Image Datasets We employ the Stable-Diffusion-Prompt (SDP)2 dataset to generate the watermarked images from all target models. For guidance-based forgery attacks, we draw prompts from the same dataset but select a disjoint set to re-guide image generation, ensuring that the forged images are visually distinct from the corresponding watermarked ones. Importantly, we intentionally maintain a consistent prompt distribution between the service provider and the adversary, as this setting provides the adversary with a stronger and more realistic forgery capability. Accordingly, all experiments are conducted within the SDP dataset, rather than using an external prompt source (i.e., the Inappropriate Image Prompts (I2P)3 dataset adopted in (Müller et al., 2025)). For optimization-based attacks, we adopt the MS-COCO-2017 Dataset (Lin et al., 2014) as cover images, consistent with (Müller et al., 2025). D.3. Runtime of Attack Algorithms The experiments described in Sec. 5 are conducted on a single A6000 GPU, and all methods are evaluated under identical system conditions. To evaluate computational overhead, we report the approximate per-sample execution time for each attack, conducted on a single GPU in a single-batch configuration: • The optimization-based forgery attacks (Sec. A.2) require approximately between 15 and 20 minutes to complete 100 steps. The most time-consuming part of these algorithms is the gradient-based optimization done by the adversary’s model (SD2.1 by default). In contrast, verification by the target model, conducted after 20, 50, or 100 optimization steps, is comparatively fast and incurs negligible overhead. • The guidance-based attack (Sec. A.2) requires between 30 seconds for smaller models (i.e., SD2.1, SDXL, PixArt-Σ), and 4 minutes for larger models such as FLUX.1 and SD3. This time accounts for all stages of the attack, including generating a watermarked image with the target model, inverting and regenerating it using the adversary’s model, and verifying the presence of the watermark with the target model. D.4. Distortion Methods and Parameters To evaluate the robustness of our detection framework, we apply 14 commonly used post-processing distortions to the watermarked images before the inversion process. These distortions are designed to emulate realistic image modifications that may interfere with watermark extraction. Visual examples of each attack type are provided in Fig. 9. 2 3
Stable-Diffusion-Prompts Inappropriate Image Prompts (I2P)
9
Geometric Distortion in Black-Box Watermark Forgery
(a) Signal Processing Attacks
(b) Geometric Attacks
Figure 9. Visual examples of the post-processing distortions used in our robustness evaluation. (a) illustrates signal processing distortions such as JPEG compression levels and noise intensities, while (b) shows geometric and erasure-based attacks.
• Signal Processing Attacks: We consider a broad spectrum of signal distortions with varying intensities to simulate realistic transmission and editing artifacts. These include brightness and contrast adjustments, JPEG compression, Gaussian and Median blurring, and additive noise (i.e., both Gaussian and salt-and-pepper types). We examine these attacks across the parameter ranges specified in Fig. 7 of the main text. • Geometric Attacks: We further consider a range of geometric transformations that alter the spatial structure of images while largely preserving semantic content. These include scaling, cropping, horizontal/vertical translation, flipping, rotation, and random region dropping (occlusion). Parameter ranges for each geometric attack are specified in Fig. 7 of the main text. D.5. Empirical Validation of Theoretical Assumptions (f )
(w)
We empirically examine Assumption B.1 by analyzing the inversion residuals (ẑT − zT ) under the reprompting forgery attack with the TR scheme. To assess the Gaussian latent approximation, we apply a random-projection normality test to the residuals. As shown in Tab. 7, across different target models with SD2.1 as the proxy, the average skewness and kurtosis remain close to 0 and 3, respectively. This indicates that the residual statistics are broadly consistent with a Gaussian approximation. Table 7. Normality test of inversion residuals (Proxy: SD2.1). Target
Skewness
Kurtosis
SD2.1 SDXL PixArt-Σ FLUX SD3
0.0092 0.0015 -0.0083 -0.0040 -0.0106
3.1694 3.0272 3.0050 3.0612 3.0539
We further examine the shared-variance premise by comparing latent posterior variances across model architectures. The measured variance ratios are 1.2265 for SDXL/SD2.1 and 0.9508 for PixArt-Σ/SD2.1, both close to the ideal ratio of 1.0. We also observe nontrivial per-dimension correlations of 0.4421 and 0.3496, respectively. These results suggest that, although the architectures differ, their latent uncertainty structures remain sufficiently aligned to support the tractable shared-variance approximation used in our analysis. D.6. Additional Experiments Distributional Analysis under Guidance-based Attacks. We analyze the distributions of unwatermarked, watermarked, and forged samples under guidance-based attacks to understand the geometric separability induced by proxy-based generation. • SDXL (Target) → SD2.1 (Proxy): As shown in Fig. 10, the distributions of unwatermarked (blue) and watermarked samples (green) are nearly identical, indicating that the semantic watermark preserves the original latent distribution, i.e., a Gaussian distribution, and maintains high perceptual fidelity. However, forged images generated by the proxy model inevitably introduce structural perturbations in the latent space, enabling forged samples to be clearly distinguished from watermarked ones, even under a similar target–proxy model setting. • FLUX (Target) → SD3 (Proxy): We further report the distribution between watermarked and forged samples under the 10
Geometric Distortion in Black-Box Watermark Forgery
flow matching model. As shown in Fig. 11, the two distributions are clearly separable under the local SPD distance. When FLUX is used as the target model, the distribution is broader, whereas the SD3-based distribution is more concentrated. A similar separation is observed in Fig. 12 using cosine similarity. Notably, FLUX-generated samples exhibit a wider cosine similarity range (i.e., 0.4 to 0.8), forming a longer tail compared to the range (i.e., 0.6 to 0.8) reported in Tab. 5.
(a) TR
(b) HSTR
(c) GS
(d) TAG
Figure 10. Distributions of local SPD (AIRM) distances for unwatermarked, watermarked, and forged samples. The results are shown for SDXL as the target model and SD2.1 as the proxy.
(a) TR
(b) GS
(a) TR
Figure 11. Distributions of local SPD distances for watermarked and forged samples, under the FLUX (target) and SD3 (proxy).
(b) GS
Figure 12. Distributions of cosine similarity for watermarked and forged samples. The target-proxy setup is consistent with Fig. 11.
Hyperparameter Sensitivity of L-SPD. To evaluate the robustness of L-SPD to hyperparameter choices, we conduct a sensitivity analysis under the SDXL (target) and SD2.1 (proxy) setting. As shown in Table 8, L-SPD exhibits high stability across different grid sizes (G ∈ {4, 8, 16}), top-k selections (k ∈ {5, 8, 16}), and aggregation methods (Mean, Top-k Mean, and Std). The AUC consistently remains above 0.932 and reaches up to 1.000, indicating that the proposed metric captures intrinsic geometric deformation rather than relying on specific hyperparameter configurations. Table 8. Hyperparameter sensitivity analysis of L-SPD. Group
Parameter Variation
TR (AUC)
GS (AUC)
Default Grid Size Patch Top-k Aggr.
G = 16, k = 5, Top-k Mean G ∈ {4, 8, 16} k ∈ {5, 8, 16} {Mean, Top-k Mean, Std}
0.997 0.932/0.983/0.997 0.997/0.999/1.000 1.000/0.997/0.997
0.995 0.963/0.995/0.995 0.995/0.998/0.999 1.000/0.995/0.994
Detection Performance via MSE Metric. As discussed in Sec. 5.4, the MSE metric can be regarded as a quantitative indicator of feature-level distortion, but remains insufficient for robust detection across a wide range of image distortions. As shown in Fig. 13, the latent MSE distributions exhibit substantial overlap under standard distortions (e.g., Cropping, Gaussian Blurring). Moreover, increasing discrepancies between proxy and target models further shift the MSE distributions toward higher values (red arrow), complicating the selection of reliable detection thresholds. Robustness Analysis on PRCW. As discussed in Sec. 6.1, the PRCW scheme exhibits inherent robustness against guidance-based forgeries in cross-model settings, stemming from its undetectable design, which prevents the adversary from accurately estimating the watermarked latent through a proxy model. However, this robustness degrades substantially when the adversary employs a proxy model identical to the target (i.e., to 0.974 for SD2.1 and 0.527 for SD3); in such cases, PRCW becomes highly susceptible to successful forgery, as reported in Tab. 9. In contrast, our proposed method remains effective (i.e., G-Cos = 0.977 for SD2.1 and 0.931 for SD3) even under this stronger threat model, as shown in Tab. 10. This implies that our latent-drift analysis successfully captures the fundamental discrepancy between forged and watermarked 11
Geometric Distortion in Black-Box Watermark Forgery
Figure 13. MSE distributions under 14 types of image distortions for TR and GS, evaluated on SDXL as the target model.
samples, enabling robust detection across a broader range of attack scenarios. We omit the analysis of optimization-based attacks here, as they exhibit marginal attack efficacy compared to guidance-based methods. Table 9. PRCW TPR under guidance-based attacks (TPR@XFPR). The metric is consistent with those described in Tab. 1. SD2.1 (Proxy)
Table 10. Our detection performance (AUC) under optimizationbased attacks. “G-Cos” and “L-SPD” are used here as in the guidance-based attack scenarios (see Tab. 2).
SD3 (Proxy)
Target
Det.
Det.
SD2.1 SDXL PixArt-Σ FLUX SD3
0.974 0.003 0.024 0.000 0.000
0.117 0.012 0.027 0.008 0.952
Proxy
Target
G-Cos
L-SPD
SD2.1
SD2.1
0.977
0.951
SD3
SD3
0.931
0.700
E. Example of Forgery Images In this section, we provide additional example images generated by guidance- and optimization-based forgery methods. All images are generated at a resolution of 512 × 512, and the adversary employs SD2.1 as the proxy model. E.1. Guidance-based Methods Fig. 14 presents a visual comparison between watermarked images produced by different target models and forged images generated via the proxy model using GS and TR schemes. E.2. Optimization-based Methods Fig. 15 illustrates the progression of the imprinting forgery attack as the number of optimization steps increases. As discussed in (Müller et al., 2025), successful forgeries typically require at least 20 optimization steps during which the accumulation of adversarial noise severely degrades image fidelity (e.g., with PSNR values dropping below 24 dB).
12
Geometric Distortion in Black-Box Watermark Forgery
Figure 14. Examples of guidance-based attacks on different target models and watermarking schemes. The top row shows watermarked images, generated using prompts from the SDP dataset with the target model indicated at the top of each panel. The subsequent rows present the corresponding forged outputs, where the adversary employs SD2.1 and SD3 as proxy models to perform forgery.
Figure 15. Progression of optimization-based forgery attack as the number of optimization steps increases. The target model is SDXL, and the forged watermark is TR and GS. Results are conducted on SDXL with TR and GS watermarks and four cover images x(c) . For each cover image, the top row shows the initial cover and the corresponding forged images at different optimization steps, while the bottom row illustrates the absolute pixel-wise differences from the initial cover.
13