Conceptio › Archive › arXiv CS

arXiv CSopen access

What Breaks Local Watermarks? A Robustness Benchmark for Local Invisible Image Watermarking

· arxiv_cs

arXiv CS · Papers · License: Open Access

Open Source ↗Direct PDF ↓

cryptography cybersecurity privacy security

cryptography, security, privacy, cybersecurity

arXiv:2609.16832v1 [cs.CV] 15 Sep 2026

What Breaks Local Watermarks? A Robustness Benchmark for Local Invisible Image Watermarking Kai Yao∗

Bence Szilágyi∗

Sebestyén Kamp

University of Edinburgh Edinburgh, UK [email protected]

Garandor Edinburgh, UK [email protected]

Garandor; University of Edinburgh Edinburgh, UK [email protected]

Máté Poór

Máté Szilveszter

Matyas K. Zsoldos

Garandor Edinburgh, UK [email protected]

Marc Juarez† University of Edinburgh Edinburgh, UK [email protected] This work has been accepted for publication in the proceedings of the 19th ACM Workshop on Artificial Intelligence and Security (AISec 2026), co-located with ACM CCS 2026. The final version will be published in the ACM Digital Library.

Abstract

Keywords

Local image watermarking embeds an invisible signal into selected image regions rather than spreading it across the entire image, enabling payload recovery from specific objects or regions without perceptibly altering the image. Existing studies evaluate the robustness of payload recovery and localization under image transformations, but they often focus on their own proposed method, resulting in narrow evaluations with inconsistent choices of transformations, datasets, and metrics. These inconsistencies across studies limit direct comparisons across methods and muddle the overall picture of local watermark robustness. To address this gap, we present the first systematic robustness benchmark for local watermarks across 55 image transformations, including (i) signal distortions, (ii) changes in image coordinate alignment, (iii) indirect local edits, and (iv) direct watermark edits. The benchmark evaluates MaskWM, WAM, OmniGuard, TrustMark, and PixelSeal, all methods that either provide native localization or require minimal adaptation to support it. Our results show that all evaluated methods are vulnerable to some transformation, with MaskWM standing out as offering the strongest payload recovery and localization, although it has the lowest image quality in the clean setting. Synchronization further improves MaskWM’s payload recovery under several geometric transformations, albeit at an additional cost to image quality. A key finding is that local watermark robustness depends strongly on the nature of the transformation: signal distortions are often tolerated by the strongest methods, while geometric misalignment and generative local edits, such as inpainting and outpainting, can completely impair payload recovery. We observe that payload recovery and localization are related but not interchangeable, and both strongly depend on the transformation’s impact on the watermark region.

image watermarking, content provenance, robustness benchmarking, watermark localization, generative image editing

Introduction

Generative image models have enabled new forms of image manipulation: objects can be easily inserted, erased, or restyled, and images increasingly combine content from multiple human and AI sources [19, 24]. In response, local image watermarking has emerged as a leading approach for protecting intellectual property under such complex image manipulations [13, 20]. As opposed to global watermarking, which spreads a detectable signal across the entire image, local watermarking embeds the signal in specific regions, so that the watermark can still be detected even when the image has been cropped, outpainted, or spliced with other content. Several methods to address this problem have been proposed in the academic literature. However, the robustness of local watermarks remains difficult to compare, because existing studies tend to focus on the methods they propose and differ in their scheme configurations, choices of datasets, considered image transformations, localization targets, and evaluation metrics [13, 20]. This lack of a systematic evaluation makes it difficult to directly compare schemes and to quantify their relative robustness to manipulation. To address this gap, we present a unified evaluation framework that measures robustness of five state-of-the-art localization-related watermarking methods to 55 image transformations across four datasets. The suite includes four main categories of transformations, including conventional image distortions such as (i) signal distortions, which alter pixel values or frequency content while preserving image coordinates, and (ii) changes in alignment, which modify the image coordinate frame and canvas extent. It also covers two types of local edits: (iii) indirect local edits and (iv) direct watermark edits, including splicing and AI-based inpainting and

∗ Both authors contributed equally to this research. † Corresponding author.

Yao et al.

outpainting—transformations that are underrepresented in existing evaluations despite their relevance for local watermarking. Besides robustness, these schemes aim to keep the watermark imperceptible and, in some cases, to segment the region where it was embedded. Our evaluation thus covers three axes: payload recovery, i.e., whether the embedded message remains decodable; imperceptibility, approximated by image fidelity in the clean setting; and localization accuracy, measured as the Intersection over Union (IoU) between the predicted and ground-truth watermarked regions. Each method is also evaluated with and without SyncSeal [10], an auxiliary synchronization technique that embeds an additional signal to correct geometric image transformations and restore the coordinate frame expected by the watermark decoder. Our results show that no evaluated method is robust across all transformations. In the clean setting, MaskWM achieves the strongest payload recovery and localization accuracy, but its watermarked images have the lowest fidelity. Performance also varies with watermark-region area: smaller regions are associated with weaker payload recovery and, for MaskWM and WAM, worse localization accuracy. Under transformations, for (i) signal distortion, only MaskWM remains robust across all five signal-distortion subcategories; for (ii) changes in alignment, we observe that geometric misalignment causes the sharpest reduction in payload recovery. SyncSeal partly restores the recovery lost to geometric misalignment, although its benefit varies widely across methods and transformations; for (iii) indirect local edits, the payload recovery can fail for generative edits even when there’s no overlap between the edit and the watermarked region; and for (iv) direct watermark edits, replacing or regenerating the region where the watermark was embedded pushes recovery to random decoding for nearly all methods. Taken together, our results indicate that current local watermarking methods remain too brittle across realistic image manipulations for reliable production deployment. Because the transformations we evaluate are non-adaptive, i.e., their parameters are not tuned against any individual scheme, our results have direct security consequences: a scheme’s lack of robustness to a transformation in our benchmark implies a practical black-box attack that removes the watermark. This means that the schemes are vulnerable to simple attacks, agnostic to the watermarking scheme and underlying generative models. Given the limited robustness we observe, these schemes are unlikely to provide provenance in realistic copyright scenarios, where there are adversaries with vested interests in removing the watermarks.

helps or harms recovery, and highlight implications for watermark detection evasion. The benchmark code will be available at https://github.com/ garandorinc/markbench.

Background and Related Work

In this section, we first introduce local watermarking, a subcategory of image watermarking that embeds the payload message within a selected region to enable provenance of individual image objects. We then distinguish local watermarking from the closely related task of watermark-based tamper localization. Finally, we review the literature on robustness evaluations of local watermarking.

2.1

Local watermarking

Image watermarking embeds a hidden payload through changes intended to remain visually imperceptible [6, 7, 12]. While early methods were based on signal-processing techniques, state-of-theart approaches jointly train a neural network encoder–decoder pipeline that learns to embed and recover the payload from the image, subject to an imperceptibility constraint. Traditional image watermarking methods have been global, meaning that the watermark signal was spread over the full image [4, 11, 21]. However, as generative models enable new forms of image manipulation and images increasingly combine content from multiple sources, there is a growing need to provide provenance of individual objects within an image, rather than of the entire image. Local watermarking has emerged as the leading technical approach to address these challenges. Local watermarking embeds the watermark payload in specific regions and, during detection, not only decodes the payload but also identifies where these regions are. Representative methods include the Watermark Anything Model (WAM) [20] and MaskWM [13]. WAM reformulates watermark extraction as a segmentation problem: its extractor predicts a watermark-presence score and payload bits at each pixel, enabling pixel-level localization and payload recovery [20]. MaskWM introduces mask-guided training into the standard encoder–decoder watermarking pipeline. By applying random image masks on regions of the images, it restricts the encoder to specific regions and teaches the decoder where the watermark signal is confined. Through a dedicated localization module, MaskWM can also identify watermarked regions before payload extraction [13]. A related but distinct direction of work is designed specifically for tamper localization–estimating where an image has been edited or manipulated. Tamper localization methods also embed payloads into the image, but they aim to estimate a manipulation mask indicating where the protected image has been altered after payload embedding. For example, OmniGuard embeds two complementary signals: a payload for copyright protection and another payload for localization. Local edits can selectively corrupt the recovered localization payload, while the copyright payload is designed to remain recoverable [26, 27].

Our contributions. • We introduce the first systematic local invisible image watermarking benchmark that evaluates payload recovery, localization accuracy, and imperceptibility, and separately assesses the effect of optional synchronization. • Our benchmark evaluates five methods on four datasets under 55 non-adaptive transformations spanning four categories: (i) signal distortions, (ii) changes in alignment, (iii) indirect local edits, and (iv) direct watermark edits. • We distinguish and analyze failures caused by signal distortion, misalignment, pixel replacement or regeneration, and diffusionbased reconstruction. We show that payload recovery and localization can fail independently, identify when synchronization

2.2

Robustness evaluations

A substantial body of work has evaluated the robustness of conventional full-image watermarking against common signal-processing 2

What Breaks Local Watermarks?

distortions and watermark removal attacks [3, 8, 22]. More recent studies have further examined robustness against generative transformations and attacks enabled by modern image generation and editing models [18, 28]. By contrast, robustness evaluation for local watermarking remains underexplored, with existing evidence derived primarily from method-specific evaluations introduced in WAM and MaskWM. Sander et al. evaluate full-image watermark detection and payload recovery under geometric transformations, valuemetric distortions, splicing, and inpainting [20]. For local watermarking, they construct images containing centered watermarked rectangles of varying sizes and evaluate watermark localization and payload recovery on these controlled composites, but do not apply additional distortions to the locally watermarked images for this evaluation. Thus, although WAM introduces local watermarking capabilities, it does not specifically evaluate their robustness under transformations or attacks. Hu et al. go further by directly evaluating the robustness of local payload recovery and watermark-region localization [13]. They construct a controlled local watermarking test set spanning twelve watermark-region size intervals from 1–5% to 95–99% of the full image. Across these region sizes, the authors evaluate and compare payload bit accuracy and localization performance under ten valuemetric distortions and three geometric transformations. However, this evaluation is built around a single mask compositing procedure and a limited set of conventional transformations. Therefore, it provides an important initial evaluation of local watermark robustness, but its results do not generalize to a diverse range of realistic transformations, including generative and local edits.

watermark decoder then performs payload recovery and, when supported, tries to identify the watermarked region. Finally, we assess the scheme’s performance on an evaluation run along three dimensions: payload recovery, localization accuracy, and imperceptibility (Section 3.5).

3.2

Localization methods. We evaluate five state-of-the-art local watermarking methods with different designs, each with a public repository and pretrained weights. MaskWM and WAM are the two natively local watermarking methods evaluated in this work. The other three methods (OmniGuard, TrustMark, and PixelSeal) were originally designed for global (i.e., full-image) watermarking, and we have adapted them for local watermarking by confining the embedding of the watermarks to a specific region. We include OmniGuard because it is often compared with MaskWM and WAM for localization capability, though it was originally proposed for tamper localization, as described in Section 2.1. TrustMark is relevant for the evaluation because its official implementation provides an optional watermark-region detector that returns a rectangular region for watermark localization capability [1, 4]. Finally, PixelSeal is a recent high-capacity global image and video watermarking method [17, 21]. We adapt it to a local watermarking setting to benchmark how state-of-the-art global watermarking techniques perform when only part of the embedded signal is utilized, compared to watermarking methods specifically designed for localization. We summarize these embedding setups and decoder outputs in Table 1. Appendix A.5 details the embedding and decoding procedures used to adapt OmniGuard, TrustMark, and PixelSeal to local watermarking.

Evaluation Framework

This section describes our evaluation framework in detail, including the watermarking methods, datasets, transformations and metrics that we used.

3.1

Localization and synchronization

We distinguish between two types of methods used in the watermarking pipeline: localization and synchronization methods.

Synchronization methods. Synchronization and watermark localization address different problems. Watermark localization refers to a decoder capability within the watermarking pipeline: identifying the image region that contains a local watermark. Synchronization, by contrast, is performed by a separate, optional module that estimates and reverses geometric transformations that may have been applied to the image before decoding. We evaluate SyncSeal as a representative standalone synchronization module that can be integrated with different watermarking methods [10]. It embeds a learned synchronization signal across the image and uses the recovered signal to estimate the applied geometric transformation, allowing the image to be rectified before watermark decoding.

Overview

Figure 1 depicts one run of our evaluation pipeline, which is divided into three stages: embedding, transformation, and decoding. • In the embedding stage, we draw an input image from our datasets and generate a watermark mask (Section 3.3). For each image and method, we sample a binary payload of the length supported by that method, drawing each bit uniformly from {0, 1}. The watermarking method then embeds the payload within the region defined by the watermark mask. The same payload is used across all image transformations, both with and without SyncSeal. When SyncSeal is enabled, the SyncSeal embedder adds a synchronization signal to the image after the payload embedding. • In the transformation stage, we apply an image transformation from our comprehensive taxonomy comprising 55 image manipulation functions (Section 3.4). • The decoding stage is blind: the decoder receives only a possibly transformed image, without access to the unwatermarked cover image. Then, if SyncSeal was used during embedding, it estimates the geometric transformation and attempts to map the image back to the coordinate frame expected by the decoder [10]. The

3.3

Image datasets and watermark masks

For the evaluation, we selected four datasets chosen to cover different image distributions and mask sources. • COCO 2017 (COCO) [16] provides everyday scenes with instance segmentation masks, enabling object-aligned watermark regions in cluttered natural images. • SA-1B [15] provides large-scale segmentation masks over diverse image content. 3

Yao et al.

Table 1: Watermarking methods used in the evaluation. All methods are evaluated for payload recovery. For MaskWM and WAM, which return pixel-level maps of watermark presence, we report watermark-region localization using intersection-over-union (IoU). MaskWM and WAM are native local methods; OmniGuard, TrustMark, and PixelSeal are adapted reference methods originally designed for global (i.e., full-image) watermarking. Method

Embedding setup

Payload bits

MaskWM [13] Native embedding only within local regions WAM [20] Native embedding only within local regions OmniGuard [27] Global embedding confined to watermark region TrustMark [4] Global embedding confined to watermark region PixelSeal [21] Global embedding confined to watermark region

64 32 100 100 256

Decoder output Payload + pixel-level map of watermark presence Payload + pixel-level map of watermark presence Payload + tamper map† Payload + rectangular region∗ Payload only

∗ For TrustMark, the predicted watermark region is the rectangular output of its optional watermark-region detector. † OmniGuard’s tamper map estimates edited content rather than the watermark region.

• DIV2K [2] includes high-quality, high-resolution images originally curated for super-resolution, with fine texture and large spatial dimensions. • MIRFLICKR-25K (MIRFLICKR) [14] is a large in-the-wild photography dataset with diverse capture conditions and content.

evaluate each transformation with and without SyncSeal. This category also includes outpainting (expanded canvas), which resizes and places the watermarked image within a larger canvas before generating the surrounding content, thereby changing the scale and position of the original image content. • Indirect local edits include transformations that do not deliberately target the watermark region itself. Blackout fills a randomly sampled region with black pixels, splicing replaces pixels in a sampled region with content from another image in the same dataset, and inpainting uses a Stable Diffusion inpainting pipeline to regenerate the selected region from the surrounding image context [19]. For these three transformations, the edit mask is sampled without reference to the watermark mask, so the two masks overlap only by chance. Because the diffusion pipeline may also alter pixels outside the edit mask, the inpainting (independent region, outside restored) control run copies the watermarked input back outside that mask after generation. Outpainting (fixed canvas) keeps the image size unchanged and uses the complement of the watermark region as the generation mask to outpaint. Appendix Figure 8 illustrates the effect of inpainting and outpainting protocols for generative edits. • Direct watermark edits use the watermark region itself as the edit mask and replacement target, applying the same inpainting or splicing operations directly to this region. These conditions test whether any recoverable watermark signal remains after the pixels intended to carry the payload are replaced or regenerated.

Watermark masks. For COCO and SA-1B, we use a segmentation mask provided with each image as the watermark region. DIV2K and MIRFLICKR do not provide segmentation masks, so we generate either a rectangular or an irregular region under the procedure described in Appendix A.1. Although MaskWM and WAM support multiple watermark regions, our evaluation uses a single region per image to focus on robustness comparisons across schemes while limiting the additional complexity introduced by multi-region configurations.

3.4

Transformation taxonomy

To comprehensively evaluate robustness of local watermarks, we include one clean setting as a baseline and 55 transformations. These transformations are organized into a taxonomy of four categories depending on the nature of the underlying transformation mechanism as described below. The signal distortions and changes in alignment categories cover conventional image distortions that treat the image as a whole, without targeting any local regions. In contrast, we also evaluate two categories of local edits: indirect local edits and direct watermark edits. These four categories are mutually exclusive, with no transformation belonging to more than one category in our taxonomy. Table 3 in Appendix A lists every transformation by category. Appendix Table 4 specifies the transformation operations and parameter settings; Appendix A.6 explains their sampling.

Together, these transformations capture realistic image manipulation workflows that alter or regenerate image content.

3.5

Evaluation metrics

We evaluate three main dimensions of local watermarking performance: (i) payload recovery, (ii) localization accuracy, and (iii) imperceptibility. Each dimension is characterized by specific metrics, as described below in detail.

• Signal distortions are transformations that preserve image coordinates but alter pixel values or frequency content, such as brightness adjustment, blurring and noising, JPEG compression, and perturbations to coefficients in the frequency domain. • Changes in alignment cover transformations that change the image coordinate frame, such as cropping, rotation, flipping, and perspective warping, so the embedded watermark may no longer align with the coordinates expected by the decoder. To evaluate whether synchronization techniques can restore alignment, we

Payload recovery. Payload recovery is quantified with two complementary metrics: bit accuracy and evidence score. Bit accuracy is the fraction of payload bits decoded correctly, commonly used in the literature for assessing watermark recovery. However, the payload lengths supported by the schemes range from 32 bits for WAM to 256 bits for PixelSeal, so the same bit accuracy does not provide the same evidence of payload recovery against random 4

What Breaks Local Watermarks?

Figure 1: Evaluation pipeline. (i) Embedding. Given a cover image, a watermark mask, and a binary payload, the watermarking method embeds the payload within the masked region. When SyncSeal is enabled, its embedder then adds a synchronization signal across the image. (ii) Transformation. The resulting image undergoes one of the 55 benchmark transformations. (iii) Decoding. When SyncSeal is enabled, its rectifier first attempts to restore the coordinate frame expected by the watermark decoder; otherwise, both SyncSeal operations are skipped. The watermark decoder then recovers the payload and, when supported, localizes the watermarked region. For runs with and without SyncSeal, we report payload recovery for all methods, pixel-level localization for MaskWM and WAM, and also image fidelity in the clean setting. decoding for every method. Hence, we report bit accuracy for consistency with previous evaluations but also report the evidence score, a measure that takes into account the payload length, enabling a fairer comparison across methods with different payload lengths. This evidence score is defined as the observed recovery against random decoding, under which each decoded bit matches the corresponding embedded bit independently with probability 0.5 [9]. For 𝑘 correctly decoded bits in an 𝑛-bit payload, the probability that random decoding would produce at least 𝑘 correct bits is 𝑝 random = Pr[Binomial(𝑛, 0.5) ≥ 𝑘]. We report − log10 (𝑝 random ), so larger values indicate stronger evidence of successful payload recovery. A score of 3 corresponds to 𝑝 random = 10−3 . Equivalently, if recovery were declared whenever the score is at least 3, the theoretical false positive rate for an individual decoding attempt would be at most 0.1%, an operating point used in prior watermarking studies [3]. We therefore use 3 as a qualitative reference level.

isolate the visual distortion introduced by watermark embedding itself. After a transformation, fidelity metrics would conflate this distortion with changes caused by the transformation.

Results

In this section, we compare the watermarking methods along the three evaluation axes: payload recovery (Figure 2), watermarkregion localization (Figure 3), and image fidelity in the clean setting (Table 2). We then further examine in detail how local watermark performance varies with watermark-region area (Figure 4), and is affected by generative local edits (Figure 5) and SyncSeal synchronization (Figure 6). For each method, we evaluate a fixed set of 200 images per dataset (800 in total). Unless otherwise stated, we report results without SyncSeal, and average values first within each dataset and then take the statistics across the four datasets. SyncSeal results and comparisons are labeled explicitly. More details on empirical evaluation are deferred to Appendix A.1.

Localization accuracy. Watermark localization accuracy is measured by thresholding the predicted map at 0.5 to obtain a predicted watermark-region mask, consistent with MaskWM and WAM’s reference implementations. then computing intersection-over-union (IoU) with the ground-truth watermark-region mask after applying the transformation’s coordinate mapping. The localization comparison is restricted to MaskWM and WAM because only these two methods return pixel-level maps of watermark presence, making them comparable to each other. In contrast, OmniGuard produces a tamper map indicating the region of manipulation instead of the region of watermark; TrustMark’s detector returns a rectangular region and not a pixel-level map, creating a shape mismatch; and PixelSeal does not provide spatial output. Hence, the localization accuracy of these three methods cannot be directly compared to MaskWM and WAM.

4.1

Payload recovery

Figure 2 presents payload recovery across the transformations in terms of both bit accuracy and evidence score (see Table 3 in the appendix for details). These results represent the broadest benchmark of our evaluation as it covers payload recovery for all five methods under every evaluation condition. Payload recovery in the clean setting. The clean setting establishes how well each method recovers its payload before any transformation is applied. Our results show that payload recovery in the clean setting varies substantially across methods (first row of Figure 2). MaskWM, one of the native local methods, gives the strongest recovery, with bit accuracy 0.973 ± 0.018 and an evidence score of 17.08 ± 1.37, though it has the lowest image fidelity in the clean setting among the evaluated methods (Section 4.3). WAM is also a native local method, but its clean-setting recovery is much weaker, with bit accuracy of 0.702 ± 0.033 and an evidence score of 3.39 ± 0.35.

Imperceptibility. We quantify the imperceptibility of the scheme by image fidelity in the clean setting, measured between the original cover image and the watermarked image, using metrics including PSNR, SSIM [23], and LPIPS [25], which respectively capture pixel-level distortion, structural similarity, and perceptual similarity based on deep features. We evaluate only in the clean setting to 5

Yao et al. Category Clean

Subcategory

Evidence score (−log10 prandom)

Bit accuracy Reference

Color and tone transformations Filtering and enhancement

Signal

Noise Compression and quantization Frequency-domain transformations

Geometric transformations Diagonal flip Horizontal flip

Alignment

Vertical flip Row/column removal Chained transformation Outpainting (expanded canvas)

Blackout (independent region) Inpainting (independent region)

Indirect local edits

Inpainting (independent region, outside restored) Splicing (independent region) Outpainting (fixed canvas)

Direct watermark edits

Inpainting (watermark region) Splicing (watermark region)

0.5

0.6

0.7

0.8

MaskWM SyncSeal off

0.9 WAM SyncSeal on

1.0

OmniGuard

5 TrustMark

50% chance baseline

15 PixelSeal

prandom = 10−3

Figure 2: Payload recovery over the transformation taxonomy. Rows follow the transformation taxonomy in Appendix Table 3, with categories and subcategories shown at left. The left panel reports bit accuracy; the right panel reports the payload-lengthaware evidence score, − log10 (𝑝 random ), where larger values indicate stronger evidence against random decoding after accounting for payload length. Open markers denote evaluation without SyncSeal and filled markers with SyncSeal; horizontal bars show the sample standard deviation across the four dataset means. The gray dashed line appears only in the bit-accuracy panel and marks the 50% chance baseline; the brown dash-dot line in the evidence-score panel marks the threshold 𝑝 random = 10−3 . One design difference between the two native local methods is how they use masks during training and decoding. WAM uses masks mainly for mask-based augmentation and pixel-level supervision, while MaskWM instead uses masks to guide local extraction and

embedding directly, which may be one of the reasons why MaskWM offers better localization. Notably, strong clean-setting recovery is not limited to the native local methods: PixelSeal reaches an evidence score of 6.73. Although 6

What Breaks Local Watermarks?

(IV) Direct watermark edits. For direct watermark edits, the distinction between localized replacement and broader reconstruction matters less because they both directly replace pixels in the watermarked region. Our results confirm this: direct inpainting and splicing both leave every method with a mean evidence score below 1 and drive bit accuracy toward random decoding.

PixelSeal is a global watermarking scheme adapted for localization, its evidence score even exceeds that of WAM in the clean setting. This result shows that a method need not be designed specifically for local embedding to provide strong evidence of local payload recovery. Payload recovery under transformations. The clean-setting performance on payload recovery provides a baseline against which we assess the impact of image transformations on local watermarking methods. Overall, we find that payload recovery varies substantially across transformation categories and methods, and we provide a breakdown of the results by category below.

4.2

Localization accuracy

Besides payload recovery, another important indicator of local watermarking performance is the method’s ability to localize the watermark region. Figure 3 shows localization accuracy, measured by localization IoU, for MaskWM and WAM, the two methods that return directly comparable pixel-level maps of watermark presence. In the clean setting, MaskWM achieves a localization IoU of 0.902, substantially higher than WAM’s 0.293, indicating stronger localization performance. Across transformations, localization degrades most sharply under changes in alignment, which disrupt the spatial correspondence between the predicted watermark map and the original watermark region. By restoring this correspondence, SyncSeal raises MaskWM’s IoU from 0.741 to 0.848 and WAM’s from 0.085 to 0.192, showing that synchronization can partially recover localization performance under misalignment. Some transformations separate localization from payload recovery, where the watermark region can be localized but payload recovery fails. For example, under vertical flip, MaskWM’s localization IoU remains high, at 0.901 without SyncSeal and 0.864 with SyncSeal, while the corresponding bit accuracies remain at random decoding (0.502 and 0.505). Direct watermark inpainting and splicing show a similar distinction between localization and payload recovery. Under inpainting, MaskWM and WAM retain localization IoUs of 0.779 and 0.906, respectively; under splicing, the corresponding IoUs are 0.673 and 0.701, demonstrating the capability to localize the watermark region even though both methods’ bit accuracies remain at the random-decoding level for both inpainting and splicing. These results show that localization and payload decoding can respond differently to the transformed signal under certain image transformations: a pixel-level map of watermark presence can remain informative even when bit accuracy is near random decoding. Payload recovery and localization therefore capture distinct failure modes and should be treated separately for local watermarking methods.

(I) Signal distortions. Signal distortions alter pixel values or frequency content without changing the image coordinate frame or replacing any local region. They therefore provide the most direct evaluation of the watermark signal itself: the payload region remains aligned with the coordinates expected by the decoder and is not directly replaced or regenerated. Even in this setting, robustness remains strongly method-dependent. MaskWM’s mean evidence score remains well above the threshold 𝑝 random = 10−3 in all five signal subcategories, with scores of 11.51–15.25. PixelSeal’s mean score exceeds the threshold in four of five subcategories, ranging from 3.82 to 5.54, but falls below it under noise (2.21). The mean scores for WAM, OmniGuard, and TrustMark remain below the threshold in all five subcategories. Thus, only MaskWM shows consistently strong evidence of payload recovery across all five signal subcategories. (II) Changes in alignment. Unlike signal distortions, misalignment disrupts the geometric correspondence between the pixels that carry the payload and those observed by the decoder. Its effect varies sharply across individual transformations. For MaskWM without SyncSeal, the mean evidence score remains high under horizontal flip (12.35) and row/column removal (11.31), but falls below 0.5 under diagonal flip, vertical flip, chained transformation, and outpainting (expanded canvas). SyncSeal improves recovery in some cases: MaskWM’s mean score rises from 5.14 to 10.91 for geometric transformations and from 0.48 to 12.41 for diagonal flip, but remains below 0.5 for vertical flip and outpainting (expanded canvas). Section 4.6 examines this effect in detail. (III) Indirect local edits. Indirect local edits can be further distinguished by whether they modify only pixels within the edit region or also affect pixels outside it. This distinction splits indirect local edits into localized pixel replacement (e.g., blackout, splicing) and generative editing (e.g., inpainting, outpainting) which may induce whole-image reconstruction. MaskWM and PixelSeal retain mean evidence scores above the threshold under blackout (independent region) and splicing (independent region). Under the same setting, inpainting (independent region) and outpainting (fixed canvas) instead leave every watermarking method below the threshold. However, under inpainting (independent region, outside restored), where the outside of the edit region is restored with the exact pixels before transformation, the mean score returns above the threshold for MaskWM, PixelSeal, and WAM. This control indicates that changes outside the edit mask induced by whole-image reconstruction contribute substantially to the loss of recovery. We analyze this further in detail in Section 4.5.

4.3

Imperceptibility

Payload recovery and localization characterize a local watermarking scheme’s ability to recover the watermark and identify the region in which it is embedded. However, imperceptibility must be also considered, as it is a practical constraint in most local watermarking use cases. Here we report image fidelity in the clean setting (Table 2) using multiple metrics as defined in Section 3.5. In the clean setting, MaskWM offers the strongest payload recovery and localization accuracy, but it also introduces the largest embedding distortion. Its mean clean-setting PSNR is 49.48 dB, significantly lower than the corresponding values for WAM (62.55), PixelSeal (56.80), TrustMark (54.09), and even OmniGuard (51.36). MaskWM also has the highest mean LPIPS (0.006, compared with 7

Yao et al. Category Clean

Subcategory

Localization IoU

Table 2: Image fidelity in the clean condition, used here as a proxy for imperceptibility. Rows report results with SyncSeal off and on; superscripts give the sample standard deviation across the four dataset means. Higher PSNR and SSIM and lower LPIPS indicate better image fidelity.

Reference Color and tone transformations Filtering and enhancement

Signal

Noise Compression and quantization Frequency-domain transformations

Method

PSNR (dB)

Geometric transformations Diagonal flip Horizontal flip

Alignment

MaskWM WAM PixelSeal TrustMark OmniGuard

Vertical flip Row/column removal Chained transformation Outpainting (expanded canvas)

SSIM

LPIPS

SyncSeal off

SyncSeal on

SyncSeal off

SyncSeal on

SyncSeal off

SyncSeal on

49.481±1.082 62.549±1.185 56.800±1.243 54.092±2.075 51.361±0.837

42.309±0.617 43.562±0.824 43.353±0.811 42.978±0.857 42.730±0.841

0.998±0.001 1.000±0.000 0.999±0.000 0.999±0.001 0.998±0.001

0.990±0.004 0.992±0.003 0.992±0.003 0.991±0.003 0.991±0.003

0.006±0.001 0.000±0.000 0.000±0.000 0.000±0.000 0.000±0.000

0.009±0.002 0.003±0.001 0.003±0.001 0.003±0.000 0.003±0.001

Blackout (independent region)

Indirect local edits

Inpainting (independent region) Inpainting (independent region, outside restored) Splicing (independent region)

whereas DIV2K and MIRFLICKR use procedurally randomly sampled masks; all four distributions are shown in Figure 7 in the appendix. The average watermark-region area is small across datasets: 15.22% for COCO, 12.50% for SA-1B, and 8.80% for DIV2K and MIRFLICKR. Much of the evaluation therefore focuses on small regions, which both stress-tests the evaluated watermarking methods and reflects practical settings in which the provenance of small objects within an image must be tracked. Across COCO and SA-1B, larger watermark regions are associated with higher payload-recovery evidence scores (Figure 4). On COCO, MaskWM’s post-transformation evidence score rises from 3.84 in the 0–3% bin to 9.71 in the 40–85% bin. PixelSeal shows an even larger observed increase, from 0.36 to 27.42. SA-1B shows the same trend. One reasonable explanation is that a larger region provides the embedder with more pixels over which to distribute the payload and the decoder with more potential redundancy. Localization accuracy follows a similar pattern to payload recovery across varying region areas. MaskWM’s post-transformation IoU increases from 0.634 to 0.825 on COCO with increased watermark-region area. WAM increases from 0.052 to 0.359 over the same range, but remains below MaskWM. WAM also degrades more from the clean setting, indicating that its localization output is less stable under transformations even for larger regions.

Outpainting (fixed canvas)

Direct watermark edits

Inpainting (watermark region) Splicing (watermark region)

0.00

0.25

0.50

MaskWM SyncSeal off

0.75

1.00

WAM SyncSeal on

Figure 3: Localization accuracy over the transformation taxonomy. Rows follow the transformation taxonomy in Appendix Table 3. The plot reports localization IoU for MaskWM and WAM, the methods that return directly comparable pixellevel maps of watermark presence. Open markers denote SyncSeal off and filled markers denote SyncSeal on; horizontal bars show the sample standard deviation across the four dataset means.

values that are approximately 0.000 for the others), indicating lower image fidelity under this metric. Mean SSIM remains uniformly high across methods, ranging from 0.998 to 1.000. Importantly, the fidelity cost goes up when SyncSeal is enabled because SyncSeal adds a full-image synchronization signal. With SyncSeal enabled, mean PSNR remains above 42 dB for all evaluated methods. Relative to the corresponding values without SyncSeal, mean PSNR drops by 7.17 dB for MaskWM, 18.99 dB for WAM, 8.63 dB for OmniGuard, 11.11 dB for TrustMark, and 13.45 dB for PixelSeal. These results indicate that synchronization techniques, such as SyncSeal, should be applied carefully, weighing their gains in robustness against the associated image fidelity loss.

4.4

4.5

A deeper analysis of generative edits

One important aspect of our benchmark is its evaluation of the effect of generative edits, a type of manipulation that is underrepresented in prior work. We therefore provide a deeper analysis here, complementing the results presented in Section 4.1, to explore which factors contribute to the degradation of payload recovery under inpainting transformations. We first analyze whether this degradation is explained by the spatial relationship between the edit and watermark regions. We then examine whether restoring the changes outside the edit region recovers payload. Finally, we investigate whether characteristics of the watermark region, particularly its edge density, affect payload recovery under such edits.

Performance by watermark-region area

Watermarking method design is not the only source of variation in these performance outcomes. Because local embedding confines the watermark signal to a selected region, performance may also vary with the amount of image area available for embedding. We report how this variation unfolds below. We use COCO and SA-1B for this analysis because their watermark regions are derived from native object-segmentation masks,

Recovery does not improve with less overlap or greater distance. In Figure 5, we first test whether overlap or distance between the edit region and the watermark region explains the poor recovery under inpainting in an independently sampled edit region. If 8

What Breaks Local Watermarks?

Evidence score (−log10 prandom)

A. Payload recovery (COCO)

3 0

0-3

3-5

5-8 8-12 12-20 20-40 40-85

Localization IoU

C. Localization IoU (COCO)

0-3

3-5

Restoring pixels outside the edit mask restores payload recovery. Since overlap and distance do not explain the recovery loss, we next compare splicing, inpainting, and the outside-restored control to isolate the changes that the generative pipeline induces outside the edit mask. Splicing an independently sampled edit region replaces only the edit region with external content and leaves surrounding pixels unchanged. As a result, MaskWM’s splicing evidence score remains high across the overlap bins (15.95, 15.95, 10.42, 10.42, 8.75). Inpainting, however, passes the entire image through the reconstruction pipeline, so pixels outside the edit mask change even when the image appears visually preserved. This drives inpainting’s evidence score to near zero. The outside-restored control tests whether changes outside the edit mask contribute to the loss of payload recovery: it restores all pixels outside the edit mask to their original values in the watermarked image. The results show that restoring these outside pixels recovers performance: MaskWM’s evidence score remains high across all overlap bins (14.62–17.45), and WAM shows the same pattern. This result indicates that the whole-image reconstruction induced by the Stable Diffusion inpainting pipeline accounts for the loss in payload-bearing signal. When the pixels outside the edit mask are restored, payload recovery improves accordingly.

5-8 8-12 12-20 20-40 40-85

D. Localization IoU (SA-1B)

1.0

0.8

0.6

0.4

0.2

0.0

B. Payload recovery (SA-1B)

Higher edge density is associated with weaker recovery. We next ask which characteristics of the watermark region outside the edit mask are associated with the magnitude of the effect. One hypothesis is that regions containing high-frequency content undergo larger pixel-level value changes from the generative pipeline than smooth, low-frequency regions, such as sky and meadows. We test this hypothesis by measuring Canny edge density [5] over watermark-region pixels outside the edit mask, investigating whether it affects payload recovery (Panel C of Figure 5). Interestingly, for MaskWM, the evidence score decreases monotonically from 1.53 in the smoothest quintile to 0.36 in the highest-edge quintile. Bit accuracy falls from 0.560 to 0.499 over the same range. The same trend also applies to WAM, though less significantly. This result suggests that payload recovery is inversely correlated with the edge density of the preserved watermark region, indicating that watermarks embedded in high-frequency content may be more susceptible to changes introduced by the evaluated generative pipeline. This can be concerning for watermark applications because high-frequency regions are often considered favorable for watermark embedding because they allow larger perturbations while preserving imperceptibility [7, 12].

0.0 0-3

3-5

5-8 8-12 12-20 20-40 40-85

Watermark-region area (% of image)

MaskWM

WAM

OmniGuard

Clean condition After transformations

0-3

3-5

5-8 8-12 12-20 20-40 40-85

Watermark-region area (% of image)

TrustMark

PixelSeal

prandom = 10−3

Figure 4: Larger watermark regions are associated with stronger payload recovery and watermark-region localization. The four panels group COCO and SA-1B examples by the fraction of image area covered by the watermark region, evaluated without SyncSeal. Panels A and B report the evidence score. Panels C and D report localization IoU for MaskWM and WAM. Dashed curves show clean-setting values. For the solid curves, bit accuracy and localization IoU are first averaged across all transformations for each image, and the evidence score is computed from the resulting mean bit accuracy. The brown dash-dot line in the payload-recovery panels marks 𝑝 random = 10−3 . Vertical bars show the sample standard deviation across image-level values within each area bin.

4.6

Synchronization is selective

We also examine synchronization and its effect on payload recovery under transformations, particularly those that change image alignment, since synchronization is designed to correct for exactly this kind of misalignment. We examine where it helps, where it reduces payload recovery, and what fidelity cost it adds. We adopt SyncSeal as a representative synchronization technique, with extension to other approaches left for future investigation.

degradation were mainly caused by overlap with the watermark region, recovery should improve when the edit mask barely overlaps the watermark. However, the results do not show improvement with less overlap: for MaskWM, the evidence score remains close to zero across the five overlap bins from < 10% to ≥ 70% (Panel A, left, blue line); and WAM follows the same pattern (Panel A, right, blue line). Increasing the distance between the edit region and the watermark region also does not restore recovery. For example, MaskWM’s evidence score is 0.451 when the edit touches the watermark region and 0.346 across the non-touching distance bins (Panel B).

SyncSeal helps when transformations match its scope. Our results show that SyncSeal’s gains concentrate on global geometric mappings within its reported scope in SyncSeal’s original paper, consistent with its intended role of estimating and inverting such 9

Evidence score (−log10 prandom )

Yao et al.

MaskWM

15 10 5 0

WAM

Inpainting (independent region) Splicing (independent region) Inpainting (independent region, outside restored)

prandom = 10−3

<10 10-30 30-50 50-70 >=70

B. Inpainting (independent region)

<10 10-30 30-50 50-70 >=70

Edit-mask overlap / watermark-region area (%)

touch

0-5

5-20

C. Inpainting (independent region) MaskWM WAM prandom = 10−3

0 0.0

>=20

0.1

Edit distance (% image diagonal)

0.2

0.3

0.4

Canny edge density

Figure 5: Payload recovery under local edits. The panels examine three factors in local edits: edit-mask overlap, distance from the watermark region, and edge density within preserved watermark-region pixels. Panel A compares inpainting (independent region), splicing (independent region), and the inpainting (independent region, outside restored) control for MaskWM (left) and WAM (right). Panel B shows that increasing the edit-to-watermark distance for inpainting (independent region) does not restore recovery. Panel C divides inpainting (independent region) examples with less than 5% edit-mask overlap into quintiles by Canny edge density over watermark-region pixels outside the edit mask. Markers show the mean edge density and evidence score in each quintile, and vertical bars show standard errors of the evidence-score means across examples within that quintile. MaskWM has higher evidence scores when those preserved pixels have lower Canny edge density, while WAM’s evidence scores remain close to zero. Brown dash-dot lines mark the evidence-score threshold 𝑝 random = 10−3 .

Random affine transform

Affine transformations

Crop

Synchronization can conflict with payload decoding. For PixelSeal, the added synchronization signal reduces rather than improves recovery in several cases. As shown in Fig. 2, SyncSeal lowers the evidence score for PixelSeal in both the clean setting and the alignment category. The evidence score also drops under signal distortions and indirect local edits when evaluated with SyncSeal. One possible explanation is that the additional full-image signal shifts the residual seen by the PixelSeal decoder away from its expected input distribution. The same trend also applies to TrustMark for certain transformations such as cropping, though relatively mildly.

SyncSeal change

Transformation

transformations. Figure 6 shows how enabling SyncSeal changes the evidence score for the evaluated transformations that specifically alter alignment. For MaskWM, SyncSeal increases the evidence score by +12.36 on shearing, +11.93 on diagonal flip, +9.59 on translation, +8.67 on perspective transform, +8.47 on rotation, and +8.24 on random affine transform. However, the gains do not extend to every transformation in this category. Under vertical flip, for example, the evidence score remains close to zero after SyncSeal.

Resize and crop Rotation Shearing Translation Horizontal flip Vertical flip Diagonal flip

Non-affine transformations

Chained transformation

Discussion and Conclusion

Interpolation resize Perspective transform Small geometric distortions Grid warping Row/column removal Outpainting (expanded canvas)

−5

Local watermarking has been proposed to track provenance at the level of individual objects within an image, yet current evaluation of local watermarking methods remains fragmented. We address this gap by developing a comprehensive benchmark spanning five watermarking methods, four datasets, and 55 non-adaptive image transformations, and by analyzing the robustness vulnerabilities of state-of-the-art local watermarking schemes. We now conclude with several key takeaways from our results.

Change in evidence score MaskWM WAM

OmniGuard TrustMark

PixelSeal No change

Figure 6: SyncSeal effect on payload recovery under changes in alignment. Rows include only transformations in the alignment category, grouped into affine and non-affine blocks. Points report the change in payload-length-aware evidence score, − log10 (𝑝 random ), from enabling SyncSeal; positive values indicate stronger payload recovery with SyncSeal. Horizontal bars show the sample standard deviation across the four dataset-level changes.

Current methods fail on smaller regions. Tracing a small object within an image is one of the main motivations for local watermarking in composited imagery, with applications such as e-commerce product image compositing. However, we find that even in the clean 10

What Breaks Local Watermarks?

setting with no transformations applied, current local watermarking methods perform poorly on smaller region sizes. Future designs of local watermarking methods should improve this capability for smaller regions, for example, by dynamically scaling the number of embedded payload bits according to region area, or by embedding watermark signals across multiple spatial resolutions so that decoding remains robust even when the target region is small.

[2] Eirikur Agustsson and Radu Timofte. 2017. NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 1122–1131. doi:10.1109/CVPRW.2017.150 [3] Bang An, Mucong Ding, Tahseen Rabbani, Aakriti Agrawal, Yuancheng Xu, Chenghao Deng, Sicheng Zhu, Abdirisak Mohamed, Yuxin Wen, Tom Goldstein, and Furong Huang. 2024. WAVES: Benchmarking the Robustness of Image Watermarks. In Proceedings of the 41st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 235). PMLR, 1456–1492. https://proceedings.mlr.press/v235/an24a.html [4] Tu Bui, Shruti Agarwal, and John Collomosse. 2025. TrustMark: Robust Watermarking and Watermark Removal for Arbitrary Resolution Images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 18629–18639. doi:10.1109/ICCV51701.2025.01731 [5] John Canny. 1986. A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-8, 6 (1986), 679–698. doi:10.1109/TPAMI.1986.4767851 [6] François Cayre, Caroline Fontaine, and Teddy Furon. 2005. Watermarking Security: Theory and Practice. IEEE Transactions on Signal Processing 53, 10 (2005), 3976–3987. doi:10.1109/TSP.2005.855418 [7] Ingemar J. Cox, Joe Kilian, F. Thomson Leighton, and Talal Shamoon. 1997. Secure Spread Spectrum Watermarking for Multimedia. IEEE Transactions on Image Processing 6, 12 (1997), 1673–1687. doi:10.1109/83.650120 [8] Mucong Ding, Bang An, Tahseen Rabbani, Chenghao Deng, Anirudh Satheesh, Souradip Chakraborty, Mehrdad Saberi, Yuxin Wen, Kyle Rui Sang, Aakriti Agrawal, Xuandong Zhao, Mo Zhou, Mary-Anne Hartley, Lei Li, Yu-Xiang Wang, Vishal M. Patel, Soheil Feizi, Tom Goldstein, and Furong Huang. 2025. A Technical Report on “Erasing the Invisible”: The 2024 NeurIPS Competition on Stress Testing Image Watermarks. In Advances in Neural Information Processing Systems, Vol. 38. https://proceedings.neurips.cc/ paper_files/paper/2025/hash/15389039ccef67b6bed9b2df8414400f-AbstractDatasets_and_Benchmarks_Track.html [9] Pierre Fernandez, Hady Elsahar, I. Zeki Yalniz, and Alexandre Mourachko. 2024. Video Seal: Open and Efficient Video Watermarking. arXiv:2412.09492 [cs.MM] https://arxiv.org/abs/2412.09492 [10] Pierre Fernandez, Tomáš Souček, Nikola Jovanović, Hady Elsahar, SylvestreAlvise Rebuffi, Valeriu Lacatusu, Tuan Tran, and Alexandre Mourachko. 2026. Geometric Image Synchronization With Deep Watermarking. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 9522–9526. doi:10.1109/ICASSP55912.2026.11464560 [11] Sven Gowal, Rudy Bunel, Florian Stimberg, David Stutz, Guillermo Ortiz-Jimenez, Christina Kouridi, Mel Vecerik, Jamie Hayes, Sylvestre-Alvise Rebuffi, Paul Bernard, Chris Gamble, Miklos Z. Horvath, Fabian Kaczmarczyck, Alex Kaskasoli, Aleksandar Petrov, Ilia Shumailov, Meghana Thotakuri, Olivia Wiles, Jessica Yung, Zahra Ahmed, Victor Martin, Simon Rosen, Christopher Savcak, Armin Senoner, Nidhi Vyas, and Pushmeet Kohli. 2025. SynthID-Image: Image Watermarking at Internet Scale. arXiv:2510.09263 [cs.CR] https://arxiv.org/abs/2510. 09263 [12] Frank Hartung and Martin Kutter. 1999. Multimedia Watermarking Techniques. Proc. IEEE 87, 7 (1999), 1079–1107. doi:10.1109/5.771066 [13] Runyi Hu, Jie Zhang, Shiqian Zhao, Nils Lukas, Jiwei Li, Qing Guo, Han Qiu, and Tianwei Zhang. 2025. Mask Image Watermarking. In Advances in Neural Information Processing Systems, Vol. 38. https://openreview.net/forum?id=da1u6Cy7Mq [14] Mark J. Huiskes and Michael S. Lew. 2008. The MIR Flickr Retrieval Evaluation. In Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval. 39–43. doi:10.1145/1460096.1460104 [15] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. 2023. Segment Anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 4015–4026. doi:10.1109/ ICCV51070.2023.00371 [16] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. In Proceedings of the European Conference on Computer Vision (ECCV). 740–755. doi:10.1007/978-3-319-10602-1_48 [17] Meta FAIR. 2025. VideoSeal and PixelSeal Official Implementation. https://github. com/facebookresearch/videoseal. Accessed: 2026-06-06. [18] Yunyi Ni, Finn Carter, Ze Niu, Emily Davis, and Bo Zhang. 2025. DiffusionBased Image Editing for Breaking Robust Watermarks. arXiv:2510.05978 [cs.CV] https://arxiv.org/abs/2510.05978 [19] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-Resolution Image Synthesis With Latent Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10684–10695. doi:10.1109/CVPR52688.2022.01042 [20] Tom Sander, Pierre Fernandez, Alain Oliviero Durmus, Teddy Furon, and Matthijs Douze. 2025. Watermark Anything With Localized Messages. In The Thirteenth International Conference on Learning Representations. https://openreview.net/ forum?id=IkZVDzdC8M

Payload recovery trades off with image fidelity. We also find a significant trade-off between payload recovery and image fidelity. For example, MaskWM achieves the strongest clean payload recovery but also produces the lowest clean image fidelity. SyncSeal improves recovery for several transformations but also adds a further fidelity cost across every evaluated method. This trade-off creates a deployment problem. There may be use cases where image fidelity is less critical, such as social media previews, where the watermark’s provenance signal matters more than visual quality, but for quality-critical applications such as professional photography or medical imaging, it is unacceptable for image quality to degrade noticeably. Future work should vary the strength of the added signal within each method to identify the point at which recovery gains stop justifying further image distortion. Generative editing can break a local watermark without even targeting it. Inpainting edits affect pixels outside the edit region, which can reduce payload recovery without visible image quality loss in the watermarked region itself. The two outpainting transformations also bring payload recovery down to random guessing. This makes generative edits a serious threat to local watermarking robustness, because modern image processing workflows increasingly apply those edits. We therefore hope future benchmarks will include these types of transformations, and encourage future local watermarking methods to be robust to them. Robustness vulnerabilities expose a security gap. Several transformations in our benchmark reduce payload recovery substantially, sometimes to the level of random guessing. When applied deliberately, these transformations become non-adaptive black-box removal attacks. Therefore, our results generalize to a broader security concern: if a non-adaptive black-box attacker can already achieve this level of degradation, an adaptive, strategic attacker could in principle perform even better. Future work should evaluate adaptive attacks that may use knowledge of the method design or access to decoder outputs. Conclusion. Local watermarking becomes increasingly important as AI-assisted editing makes it easier to blend human-created and AI generated content within a single image. However, our benchmark shows that, within the evaluated scope, none of the current local watermarking methods is ready for reliable production deployment. Payload recovery performs unsatisfactorily on small target regions, trades off against image fidelity, and remains vulnerable to various transformations, including generative edits. Future work will need to address these challenges before local watermarking can be deployed reliably.

References [1] Adobe Research. 2025. TrustMark Official Implementation. https://github.com/ adobe/trustmark. Accessed: 2026-06-06. 11

Yao et al.

[21] Tomáš Souček, Pierre Fernandez, Hady Elsahar, Sylvestre-Alvise Rebuffi, Valeriu Lacatusu, Tuan Tran, Tom Sander, and Alexandre Mourachko. 2025. Pixel Seal: Adversarial-Only Training for Invisible Image and Video Watermarking. arXiv:2512.16874 [cs.CV] https://arxiv.org/abs/2512.16874 [22] Sviatoslav Voloshynovskiy, Shelby Pereira, Victor Iquise, and Thierry Pun. 2001. Attack Modelling: Towards a Second Generation Watermarking Benchmark. Signal Processing 81, 6 (2001), 1177–1214. doi:10.1016/S0165-1684(01)00039-1 [23] Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600–612. doi:10.1109/TIP.2003. 819861 [24] Tao Yu, Runseng Feng, Ruoyu Feng, Jinming Liu, Xin Jin, Wenjun Zeng, and Zhibo Chen. 2023. Inpaint Anything: Segment Anything Meets Image Inpainting. arXiv preprint arXiv:2304.06790 (2023). [25] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 586–595. doi:10.1109/CVPR.2018.00068 [26] Xuanyu Zhang, Runyi Li, Jiwen Yu, Youmin Xu, Weiqi Li, and Jian Zhang. 2024. EditGuard: Versatile Image Watermarking for Tamper Localization and Copyright Protection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 11964–11974. doi:10.1109/CVPR52733.2024.01137 [27] Xuanyu Zhang, Zecheng Tang, Zhipei Xu, Runyi Li, Youmin Xu, Bin Chen, Feng Gao, and Jian Zhang. 2025. OmniGuard: Hybrid Manipulation Localization via Augmented Versatile Deep Image Watermarking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 3008–3018. doi:10. 1109/CVPR52734.2025.00286 [28] Xuandong Zhao, Kexun Zhang, Zihao Su, Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu-Xiang Wang, and Lei Li. 2024. Invisible Image Watermarks Are Provably Removable Using Generative AI. In Advances in Neural Information Processing Systems, Vol. 37. doi:10.52202/079017-0276

What Breaks Local Watermarks?

A A.1

Additional Evaluation Details Methodology details

A.3

Transformation Protocol Examples

Figure 8 shows the local-edit controls.

Images. For each method, we evaluate a fixed set of 200 images from each dataset, giving 800 images per method. Before embedding, input images are resized to 512 × 512 pixels, and watermark masks are generated in the same coordinate frame. Masks. For COCO and SA-1B, we sample uniformly among segmentation masks covering at least 2% of the image. DIV2K and MIRFLICKR lack segmentation masks, so we generate content-independent masks. For methods that accept arbitrary shapes, rectangles and irregular masks are equally likely. We sample a target area of 4%–14%, a log-uniform bounding-box aspect ratio of 0.6–1.8, and a random valid location; irregular masks fill 55%–80% of the box. Because TrustMark requires a rectangle, we use rectangular masks for DIV2K and MIRFLICKR and the smallest rectangle enclosing the selected segmentation mask for COCO and SA-1B. Transformations. For each stochastic transformation, such as random cropping, Gaussian noise, or blackout applied to a randomly sampled region, we sample a single realization per image using a fixed random seed.

Figure 8: Inpainting and outpainting protocols. Examples show edit masks, transformed images, and absolute differences. The outside-restored control restores pixels outside the edit mask; fixed-canvas outpainting regenerates surrounding content, whereas expanded-canvas outpainting extends and rescales the canvas.

Reproducibility. We use the released pretrained weights for all watermarking methods and SyncSeal without further training, and fix random seeds for transformation parameters, watermark mask sampling, and payload generation. The benchmark code will be available at https://github.com/garandorinc/ markbench. Reporting convention. Unless otherwise stated, the text reports results without SyncSeal; SyncSeal results and direct comparisons are labeled explicitly. We first average within each dataset and then report the unweighted mean across the four dataset means.

A.2

A.4

Watermark-Region Area Distributions Number of images

Figure 7 reports the area distributions used in Figure 4. COCO