Conceptio › Archive › arXiv CS

arXiv CSopen access

RLSpoofer: A Lightweight Evaluator for LLM Watermark Spoofing Resilience

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

cryptography, security, privacy, cybersecurity

RLSpoofer: A Lightweight Evaluator for LLM Watermark Spoofing Resilience.

Hanbo Huang1

Xuan Gong1 Yiran Zhang1 Hao Zheng1 1 Shanghai Jiao Tong University

Shiyu Liang1 †

arXiv:2604.11546v1 [cs.CR] 13 Apr 2026

{hhuang417, lsy18602808513}@sjtu.edu.cn

Abstract Large language model (LLM) watermarking has emerged as a promising approach for detecting and attributing AI-generated text, yet its robustness to black-box spoofing remains insufficiently evaluated. Existing evaluation methods often demand extensive datasets and white-box access to algorithmic internals, limiting their practical applicability. In this paper, we study watermark resilience against spoofing fundamentally from a distributional perspective. We first establish a local capacity bottleneck, which theoretically characterizes the probability mass that can be reallocated under KL-bounded local updates while preserving semantic fidelity. Building on this, we propose RLSpoofer, a reinforcement learning-based black-box spoofing attack that requires only 100 human-watermarked paraphrase training pairs and zero access to the watermarking internals or detectors. Despite weak supervision, it empowers a 4B model to achieve a 62.0% spoof success rate with minimal semantic shift on PF-marked texts, dwarfing the 6% of baseline models trained on up to 10,000 samples. Our findings expose the fragile spoofing resistance of current LLM watermarking paradigms, providing a lightweight evaluation framework and stressing the urgent need for more robust schemes.

Introduction

With the rapid advancement and increasing availability of large language models (LLMs), they are being widely applied across diverse applications to generate fluent and human-like content [1, 2]. However, this widespread use raises major concerns about model misuse, including the generation of misinformation [3], copyright violations [4], data contamination [5], and academic dishonesty [6]. As a safeguard, text watermarking has become an important defense. By subtly embedding statistical signals into model outputs, watermarking enables reliable AI text detection and source tracking while maintaining the quality of the text [7, 8]. Most existing watermarking schemes predominantly operate on a generate-and-detect paradigm, utilizing detection algorithms to identify hidden patterns that differentiate AI-generated text from human authorship [9, 10]. However, this paradigm inadvertently creates an avenue for watermark spoofing. Malicious actors can query a watermarked LLM to gather text samples, using this data to construct a surrogate model capable of approximating the underlying watermarking rules [11]. By successfully forging the watermark, such attacks pose a fundamental threat to the reliability of existing watermarking systems. To rigorously assess the spoofing resistance of LLM watermarking schemes, researchers have investigated various attack paradigms. However, existing methods typically suffer from at least one of three major limitations: (1) reliance on partial algorithmic knowledge or detector API access [7, 12]; (2) the need for large training corpora [13, 14] (up to 10k); or (3) an inability to generate semantically † Corresponding author.

Preprint.

Unwatermarked Reference Model

Human Written Text

Sequence-level Semantic Reward

Group Normalization

Capacity-aware Token-Level Reward = Watermarked Model

Policy Model

KL reward Capacity True reward

Policy Model

... Tokens

Tokens

Stabilize‌distribution shifting Watermarked Paraphrase

Cross-entropy Anchor Watermarked

Figure 1: Overview of RLSpoofer. Given human–watermarked rewrite pairs, RLSpoofer jointly optimizes sequence-level semantic rewards, capacity-aware token-level rewards, and a cross-entropy anchor, shifting the policy distribution Pθ from the human-like distribution Ph toward the watermarked distribution Pwm , thereby effectively spoofing the watermark. coherent text that seamlessly matches the original context [15, 16]. These limitations reduce their practicality as evaluation tools for watermark spoofing resilience. As LLMs continue to advance, there is an urgent need for a practical, generalizable, and data-efficient methodology for benchmarking the robustness of watermarking schemes. In this paper, we investigate the resilience of LLM watermarks to spoofing attacks. In the blackbox setting, direct instance-level spoofing is intractable because the adversary has no access to the watermark details or the detector. Motivated by recent evidence that watermarked text exhibits systematic distributional discrepancy from human-written text [17], even for distortion-free watermarking schemes [18], we take a distributional view of watermark spoofing. Specifically, the attacker seeks to shift the paraphraser’s output distribution away from the human-like distribution and toward the watermarked distribution. Leveraging the autoregressive structure of LLM generation, we establish a local capacity bottleneck that characterizes how much token-level probability mass can be reallocated under KL-bounded local updates while preserving semantic fidelity. Building on this insight, we propose RLSpoofer, a reinforcement-learning-based black-box attack that requires no access to the watermarking internals or the detector. As illustrated in Figure 1, RLSpoofer jointly optimizes semantic fidelity, capacityaware token-level reward, and a cross-entropy anchor, thereby shifting the model outputs toward the target watermarked distribution. Using only 100 training pairs, RLSpoofer enables a compact 4B model to achieve a 62.0% spoof success rate against PF-Watermark, substantially surpassing the 6.0% of baseline trained on 10,000 examples. Extensive experiments across five attacker models and six watermarking schemes show that RLSpoofer provides a practical, sample-efficient stress test for spoofing resilience, exposing critical vulnerabilities in current watermarking defenses. Our contributions are as follows: • We establish a local capacity bottleneck that characterizes the token-level bottleneck on probability mass reallocation under KL-bounded local updates while preserving semantic integrity. (Sec. 3.1.) • We propose RLSpoofer, an effective and sample-efficient black-box attack. It successfully spoofs watermarks with high semantic fidelity, requiring only limited human-watermarked rewrite pairs and zero access to the watermarking scheme or detector. (Sec. 3.2.) • We conduct extensive experiments across five architectures and six watermarking schemes from both logit-based and sampling-based families. The results demonstrate the broad effectiveness of RLSpoofer, exposing critical vulnerabilities in current watermark defense. (Sec. 4.)

Preliminaries

LLM Paraphraser and Logit-based Watermark. Let V be a finite vocabulary and V ∗ denote the space of all finite token sequences. For a source input X ∈ V ∗ , an autoregressive paraphraser πθ induces a probability distribution over generated output sequences X′ = (x′1 , . . . , x′|X′ | ) ∈ V ∗ : Q|X′ | Pθ (X′ | X) := t=1 πθ (x′t | x′<t , X). Independent of the paraphrasing model, a watermark is embedded into generated text using a secret key s and a target green-token rate q ∈ (0, 1). At each generation step, key s defines a prefix-dependent pseudorandom subset of the vocabulary with expected size q|V|, whose tokens are softly upweighted during sampling. Given a sequence X′ , the 2

detector f computes a score f (X′ , s) under the key s and flags the sequence watermarked whenever f (X′ , s) > δ, where δ is a fixed detection threshold. Threat Model: Semantics-Preserving Paraphrasing. We consider a black-box adversary aiming to spoof the watermark into an unwatermarked sequence X via paraphrasing. The adversary is defined by: (1) Objective: Generate an output X′ that preserves the semantic meaning of X while triggering the watermark detector. (2) Knowledge: The adversary lacks access to the secret key s, the detector function f , and the threshold δ, but can observe watermarked text by querying the generator. (3) Capability: The adversary can construct a training set by querying the watermarked model for paraphrases of given texts, and can subsequently fine-tune a paraphrasing model πθ without requiring access to the detector. Adversarial Objective. Let d : V ∗ × V ∗ → [0, ∞) be a distance metric where smaller values indicate higher semantic similarity. For a tolerance ε > 0, we define the instance-level spoof success rate as: SSRX (θ) := Pθ ({X′ ∈ V ∗ : d(X′ , X) ≤ ε, f (X′ , s) > δ} | X). The adversary seeks to maximize this rate. However, without access to f and s, direct optimization is intractable. Distributional Surrogate. Prior work suggests a distributional discrepancy between watermarked and human-written text [17, 18]. Since the detector function f and the secret key s are unavailable in the black-box setting, directly optimizing the instance-level spoof success rate is intractable. We therefore adopt a distributional surrogate objective that favors outputs relatively more likely under a watermarked distribution than under a human-like distribution, while enforcing semantic fidelity. Let Ph (· | X) and Pwm (· | X) denote the human-like and watermarked distributions, respectively. We consider the following surrogate objective based on the Kullback–Leibler (KL) divergences DKL : max DKL Pθ (· | X) ∥ Ph (· | X) − DKL Pθ (· | X) ∥ Pwm (· | X) . θ

Under the mild regularity conditions in Appendix A.1, the KL-difference objective admits an expected log-likelihood-ratio form. Imposing the semantic-fidelity constraint Pθ (d(X ′ , X) < ε | X) ≥ 1 − ρ for ρ ∈ [0, 1) yields the following constrained surrogate objective: Pwm (X′ | X) max EX′ ∼Pθ (·|X) log s.t. Pθ d(X′ , X) < ε | X ≥ 1 − ρ. (1) θ Ph (X′ | X) This formulation yields a tractable surrogate objective that prefers semantically faithful outputs with higher relative likelihood under the watermarked reference than under the human-like reference.

Methods

3.1

Local Capacity Bottleneck for Semantics-Preserving KL-Bounded Redistribution

To optimize the sequence-level objective in Eq. 1, we decompose it into token-level steps. Let the generation history at step t be ht := (x′<t , X). Since both reference distributions are autoregressive, P|X′ | P (x′t |ht ) (X′ |X) log PPwm = t=1 log Pwm . Accordingly, the adversarial objective becomes ′ ′ h (X |X) h (x |ht ) t

|X′ |

max EX′ ∼Pθ (·|X) θ

X t=1

log

Pwm (x′t | ht ) Ph (x′t | ht )

s.t.

Pθ d(X′ , X) < ε | X ≥ 1 − ρ.

(2)

This decomposition identifies a token-level surrogate signal. To localize the sequence-level semantic constraint, we consider KL-bounded local updates around the human-like reference Ph (· | ht ), using such neighborhoods as a tractable proxy for compatible modifications. This leads to the following local capacity question: under a fixed KL budget, how much probability mass can be reassigned? Formally, for a fixed history ht , we consider a local update from Ph (· | ht ) to πθ (· | ht ) under a KL constraint, and bound the resulting increase of probability mass on an arbitrary subset of tokens. Let dkl (p∥q) := p log pq + (1 − p) log 1−p 1−q , p, q ∈ [0, 1], with the standard conventions 0 log 0 := 0 and b log(b/0) := +∞ for b > 0. We then establish the following theorem. Theorem 1 (Local capacity characterization). Fix a history ht . For any next-token distribution πθ (· | ht ) and any subset A ⊆ V, we have n o πθ (A | ht ) − Ph (A | ht ) ≤ sup λ ∈ [0, 1] : dkl λ 1 − maxx∈V Ph (x | ht ) ≤ DKL πθ (· | ht ) ∥ Ph (· | ht ) .

(3)

Moreover, for any fixed C ≥ 0, lim

sup πθ (A | ht ) − Ph (A | ht ) = 0.

sup

1−maxx∈V Ph (x|ht )→0 πθ : DKL (πθ (·|ht ) ∥ Ph (·|ht ))≤C A⊆V

(4)

Remark 1. The proof is provided in Appendix A.2. Theorem 1 characterizes a local bottleneck for KL-bounded redistribution around the human-like reference: when the next-token distribution is highly concentrated, only limited probability mass can be reassigned under a fixed KL budget. While purely distributional, this provides a tractable local proxy for compatible modification. In particular, taking A = {x} recovers the token-level case. Let x∗t ∈ arg maxx∈V Ph (x | ht ), and define ct := 1 − Ph (x∗t | ht ). We refer to ct as the local capacity mass. By Theorem 1, ct quantifies the local bottleneck for semantics-consistent redistribution: a more concentrated human-like distribution leaves less probability mass available for reassignment under a fixed KL budget. Moreover, if a local update preserves the dominant human-like continuation, i.e., πθ (x∗t | ht ) = Ph (x∗t | ht ), then the local surrogate gain factorizes as P P (x|ht ) πθ (x|ht )−Ph (x|ht ) Pwm (x|ht ) log PPwm . (5) x∈V πθ (x | ht ) − Ph (x | ht ) log Ph (x|ht ) = ct x̸=x∗ ct h (x|ht ) t Thus, ct serves as a local indicator of both the feasible amount of redistribution and the scale of the token-level surrogate gain. This suggests that token positions with larger ct may admit more flexible redistribution, thereby motivating a token-dependent weighting scheme. 3.2

RLSpoofer: Capacity-Aware RL Attack for Watermark Spoofing

From surrogate objective to policy optimization. Equation 1 defines an expected sequence-level objective under the paraphraser-induced distribution over discrete generations, which naturally admits an episodic Markov decision process formulation with πθ serve as the policy. Motivated by this view, we propose RLSpoofer, a GRPO-based [19] watermark spoofing attack that requires only limited sample pairs (X, X′wm ), where X is a human-written text and X′wm is its semanticpreserving paraphrase generated by a watermarked model. Using such pairs, the attack steers the paraphrasing distribution Pθ (· | X) toward the watermarked distribution Pwm (· | X) and away from the human-written distribution Ph (· | X), thereby encouraging watermark-spoofed generations. Approximating the target distributions. Since the ground-truth watermarked and human-like distributions are not directly accessible, we approximate both using a lightweight reference model πref . As LLMs are trained to model human text [2], we approximate Ph by querying the πref conditioned on the original input X, namely Ph (x′t | x′<t , X) ≈ πref (x′t | x′<t , X). Similarly, motivated by the observation that weak rewriting often preserves both semantics and watermark patterns [7], we approximate the watermarked distribution by conditioning the same reference model on the watermarked instance X′wm , i.e., Pwm (x′t | x′<t , X) ≈ πref (x′t |x′<t , X′wm ). Intuitively, ′ conditioning on the semantics-preserving rewrite Xwm provides a tractable proxy for the watermarked distribution’s lexical and semantic preferences, yielding a tractable local approximation of Pwm . P

(x′ |h )

t t Reward design. Based on Eq. 2, the token-level spoofing signal is naturally given by log Pwm . ′ h (xt |ht ) Moreover, Theorem 1 together with Eq. 5 suggest that, under semantics-consistent local updates around Ph (· | ht ), the attainable improvement of this signal is modulated by the local capacity mass ct . Motivated by this, we define the reward for a sampled token x′t ∼ πθ (· | ht ) as rt := P (x′t |ht ) ct log Pwm , where ct = 1−maxx∈V Ph (x | ht ) ≈ 1−maxx∈V πref (x | x′<t , X). The quantity ′ h (xt |ht ) ct represents the human-plausible mass outside the dominant continuation, thereby capturing the local room for lexical redistribution under semantic fidelity. As a result, rt rewards tokens preferred by the surrogate watermarked distribution while attenuating updates at positions where the human-like distribution is already sharply concentrated.

However, token-level redistribution alone is insufficient, since a successful attack must also preserve semantics at the sequence level. We therefore introduce a sequence-level reward A based on the P-SP score [20], followed by a sigmoid transformation to improve gradient sensitivity. Specifically, we compute the semantic similarity between the generated paraphrase and each of X and X′wm , and take the minimum as A. This conservative design preserves the meaning of the original text while keeping the policy close to the watermarked rewrite, thereby improving the quality of the surrogate 4

target for watermark-favored generation. We further normalize A within each rollout group to obtain Âi = Ai −mean(A) , which stabilizes training and improves semantic consistency across samples. std(A) Training objective. Recent work has shown that distillation-based objectives can be effective for watermark spoofing [14, 13]. Accordingly, in addition to the reward design above, RLSpoofer also incorporates a cross-entropy term to anchor optimization toward the target watermarked distribution. Concretely, the attack policy πθ is iteratively updated by sampling a set of outputs {x′1 , . . . , x′G } from the previous policy πθold , maximizing the following training objective: |x′i | " G πθ (x′i,t | X, x′i,<t ) 1 X 1 X w1 Âi + w2 ri,t J (θ) ≈ E{x′i }∼πθold ′ ′ ′ G i=1 |xi | t=1 πθold (xi,t | X, xi,<t ) # − βDKL πθ ∥ πref − w3 LCE πθ , {(X, X′wm )} ,

(6)

where πref denotes the reference model, and w1 , w2 , w3 control the contributions of the respective components. The KL term regularizes the policy against the reference model at the token level, while LCE denotes the teacher-forced cross-entropy loss for predicting X′wm conditioned on X.

Experiments

4.1

Experimental Setup

We provide detailed experimental setups and configurations in Appendix B. Victim Models and Attackers. We use Llama3.1-8B-Instruct [2] as the victim model to generate watermarked paraphrases. For attackers, we consider five lightweight models of varying sizes from three well-known model families: Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B [1], Qwen2.5-3BInstruct [21], and Llama3.2-3B-Instruct. Watermarking Schemes. We evaluate six watermarking schemes drawn from both logit-based and sampling-based families. Specifically, the logit-based methods include EWD [9], SWEET [10], KGW [7], and Unigram [22]. To capture recent advances, we additionally consider two samplingbased distortion-free methods: semantic watermark PMark [23] and the cryptographic PFWatermark [8]. We implement PMark based on its official codebase, whereas the remaining schemes are implemented and detected using the widely adopted MarkLLM toolkit [24]. Datasets. We construct the training set from C4-RealNewslike subset [25] and the test set from four datasets: Reddit WritingPrompts [26], LFQA [27], and the BookReport and FakeNews subsets of MMW [28], following [11]. Specifically, we prompt Qwen3-8B to generate human-like unwatermarked texts and use watermarked Llama3.1-8B-Instruct to rewrite them into semantically preserved watermarked responses. For each watermarking scheme, we use 100 unwatermarked–watermarked response pairs for training and 400 unwatermarked test samples drawn evenly from the four datasets. We standardize the length of all training and test samples to 500 tokens. Spoof Methods. Under the black-box threat model, in addition to RLSpoofer, we consider three spoofing baselines to further reveal vulnerabilities in watermarking schemes. Distill [14] distills the distribution of the watermarked model using 10,000 huamn–watermarked rewritten pairs. DITTO [13] further exploits the distilled model by analyzing its output distribution and reproducing the statistical preferences induced by the target watermark. Additionally, DPO [29] casts watermark spoofing as a preference alignment and optimizes the attacker on 7,000 samples preference pairs. Implementation Details of RLSpoofer. RLSpoofer utilizes a reparameterized sigmoid scaling function for the semantic reward, with a threshold of 0.85 to separate positive and negative rewards, thereby encouraging higher-quality rephrasings. Training on 100 samples with a batch size of 48 and group size G = 12 converges efficiently, completing in approximately 1.5 hours for Qwen3-4B and 45 minutes for Qwen3-0.6B on four NVIDIA Pro 6000 GPUs. In the experiments, we use the attack model itself as the reference model to generate the surrogate watermark distribution. Metric. We primarily evaluate spoofing effectiveness using Spoof Success Rate (SSR), defined as the proportion of rephrased texts detected as watermarked under a semantic similarity constraint (P-SP > 0.7, following prior work [11]). We also report Spoof Rate (SR), which omits the semantic constraint. Rephrasing quality is further assessed using perplexity and GPT-as-a-Judge (GPTS) [30]. 5

Table 1: Spoof Success Rate (SSR, %) and P-SP scores across models and watermarking schemes. Bold indicates the best SSR for each model. Higher SSR indicates stronger spoofing performance. EWD Models

Methods

SWEET

KGW

Unigram

PMark

SSR P-SP SSR P-SP SSR P-SP SSR P-SP SSR P-SP SSR P-SP

Distill DITTO Qwen3-0.6B DPO RLSpoofer

42.3 7.50 0.25 54.3

0.75 0.43 0.57 0.73

20.0 6.75 0.00 50.5

0.76 0.41 0.76 0.79

35.8 1.00 1.00 52.0

0.68 0.33 0.66 0.72

13.8 0.25 0.25 49.5

0.84 0.32 0.32 0.70

6.50 5.50 2.50 33.3

0.97 0.79 0.57 0.66

20.0 11.8 6.25 29.5

0.90 0.63 0.63 0.92

Distill DITTO Qwen3-1.7B DPO RLSpoofer

43.8 21.5 0.25 53.5

0.80 0.53 0.94 0.76

26.8 29.0 0.00 52.0

0.79 0.56 0.84 0.71

44.5 13.8 1.00 52.0

0.75 0.52 0.96 0.71

19.5 1.50 0.50 54.8

0.81 0.36 0.87 0.73

7.00 7.50 4.25 29.0

0.96 0.86 0.58 0.73

20.3 16.5 22.5 29.5

0.90 0.68 0.93 0.90

Distill DITTO DPO RLSpoofer

51.3 56.0 0.25 56.5

0.81 0.68 0.78 0.73

37.3 43.3 0.00 52.3

0.82 0.66 0.78 0.75

57.0 36.5 1.00 58.0

0.79 0.61 0.93 0.75

28.0 2.25 0.75 54.8

0.80 0.39 0.59 0.74

6.00 3.50 5.25 62.0

0.96 0.88 0.88 0.77

21.5 16.5 17.5 36.3

0.92 0.71 0.68 0.91

Distill Qwen2.5-3B DITTO DPO -Instruct RLSpoofer

55.5 14.0 0.00 53.5

0.76 0.50 0.88 0.70

49.8 22.3 0.00 54.5

0.77 0.54 0.87 0.75

60.3 9.50 1.25 57.3

0.74 0.48 0.87 0.72

25.8 0.25 2.50 54.5

0.80 0.34 0.78 0.77

6.50 5.25 6.25 50.3

0.93 0.72 0.83 0.68

22.3 11.3 24.3 30.3

0.91 0.56 0.95 0.89

Distill Llama3.2-3B DITTO DPO -Instruct RLSpoofer

53.8 19.3 2.50 54.5

0.77 0.54 0.49 0.70

45.5 24.3 0.50 54.5

0.76 0.56 0.53 0.74

56.3 14.0 0.75 55.3

0.75 0.51 0.36 0.76

26.0 1.00 7.75 52.0

0.77 0.35 0.60 0.72

8.75 6.50 6.25 49.8

0.93 0.79 0.67 0.85

23.3 18.0 25.0 33.3

0.89 0.65 0.87 0.92

Qwen3-4B

4.2

Main Results

In this subsection, we evaluate the spoofing resilience of different watermarking schemes across four attacks, with results shown in Table 1. Details of rephrase quality are provided in Appendix C.1. Logit-based watermarks are vulnerable to baseline attacks. Table 2: Spoofing performance on We observe that across all attack settings, Distill consistently EWD using Qwen3-4B. achieves strong spoofing performance against logit-based watermarks. As shown in Table 1, Distill enables Qwen3-0.6B to Method SSR SR P-SP GPTS achieve 42.3% SSR and a P-SP score of 0.75 on EWD, signifDistill 51.3 62.5 0.81 7.66 icantly outperforming DITTO (7.5%) and DPO (0.25%), with DITTO 56.0 94.3 0.68 6.66 similar trends observed across models. Furthermore, Table 2 DPO 0.25 0.50 0.78 7.55 shows that Distill and DITTO achieve spoof rates (SR) of 62.5% RLSpoofer 56.5 87.0 0.73 6.68 and 94.3%, respectively, on Qwen3-4B with EWD. This effectiveness arises because logit-based watermarking induces pronounced shifts in the output distribution [18], which can be captured by distribution-matching approaches. In contrast, DPO yields consistently low SSR, spoof rates, and P-SP scores across most watermarks, suggesting that preference-based optimization is less suitable for watermark spoofing. Sampling-based distortion-free watermarks exhibit stronger Table 3: Spoofing performance on resilience. We observe that all three baselines struggle to spoof PF using Qwen3-4B. sampling-based distortion-free watermarks effectively. Distill achieves only ∼7% SSR on PF-Watermark and at most 23.3% Method SSR SR P-SP GPTS on PMark, with the other baselines showing similarly limited Distill 6.00 6.50 0.96 7.67 performance. Furthermore, as shown in Table 3, all three meth- DITTO 3.50 6.25 0.87 6.01 ods obtain spoof rates below 9% on PF-Watermark, indicating DPO 5.25 8.75 0.88 9.30 that they fail to replicate the watermark distribution. We hy- RLSpoofer 62.0 82.8 0.77 6.60 pothesize this resilience stems from the fact that PF and PMark watermarks are distortion-free in expectation [8], rendering their latent signals exceptionally difficult to model from large training corpora. RLSpoofer consistently compromises the resilience of watermarking schemes. We observe that, with only 100 training samples, RLSpoofer matches and surpasses baselines requiring 10,000 samples across all evaluated watermarks. Specifically, on SWEET, it enables Qwen3-0.6B to achieve a 50.5% spoofing success rate (SSR), significantly exceeding Distill (20.0%) and DITTO (6.75%). 6

EWD: Watermarked Text EWD: Unwatermarked Text EWD: Paraphrased Text

SWEET: Watermarked Text SWEET: Unwatermarked Text SWEET: Paraphrased Text

0.0

2.0

4.0

6.0

Detection Score

8.0

10.0

12.0

2.0

(b) POS-wise Reward Magnitude Raw reward True reward 𝑟 𝑡 Capacity Mass

0.8

1.5

0.6

1.0

0.4

0.5

0.2

0.0

Capacity Mass

(a) Detection Score Distribution

Reward Magnitude

Frequency Frequency

25.0 20.0 15.0 10.0 5.0 0.0 30.0 25.0 20.0 15.0 10.0 5.0 0.0 -2.0

noun adj verb adv num adp pron det

0.0

POS

Figure 2: (a) illustrates the detection score distributions for EWD and SWEET watermarks across unwatermarked outputs, watermarked outputs, and paraphrased texts generated by Qwen3-4B trained with RLSpoofer; (b) shows the POS-wise reward magnitudes for SWEET watermark on Qwen3-4B. Furthermore, RLSpoofer empowers Qwen3-4B to effectively spoof the PF watermark with a 62.0% SSR, far surpassing the best baseline of 8.75%. This efficacy stems from its ability to align with the distribution of watermarked text, successfully exploiting the sample-level distribution shifts induced even by distortion-free watermarks under a fixed key [18]. Our analysis of EWD and SWEET detection scores (400 samples, 500 tokens per input) confirms this alignment: as shown in Figure 2(a), RLSpoofer shifts the z-score distribution of its generated rephrasings toward that of watermarked text, maintaining clear separation from the unwatermarked distribution. Details are in Appendix C.2. 4.3

Empirical Analysis of RLSpoofer

In this subsection, we analyze the effectiveness of RLSpoofer on three complementary perspectives: feasible token-level watermark injection, sequence-level semantic preservation, and optimization stability. Additional details can be found in Appendix C.3 - C.5. Local capacity mass identifies feasible room for watermark injection. Theorem 1 suggests that the local capacity mass ct captures the local bottleneck of semantics-consistent redistribution, and thus favors positions with greater semantics-preserving substitutability. Since ct directly modulates token-level rewards, its effect should be reflected in the POS-wise distribution of rt . We therefore plot POS-wise token-level rewards of the Qwen3-4B RLSpoofer on the SWEET training set. Figure 2 (b) shows that the induced reward is small on numerals, which are typically critical for semantic fidelity, but larger on adjectives, verbs, and adverbs, which allow greater lexical flexibility. Thus, local capacity mass suppresses overly sharp rewards on semantically rigid tokens while amplifying rewards on more substitutable ones. This suggests that RLSpoofer exploits genuinely flexible positions rather than uniformly favoring all watermark-preferred tokens. We next verify that this weighting is not only intuitively aligned Table 4: SSR on reward weights. with semantic flexibility but also critical to spoofing performance. Specifically, we replace 1 - pmax with either a uniform weight of 1 Model Weight EWD SWEET or the reverse weight pmax . Table 4 shows that both replacements 1 - pmax 54.3 50.5 consistently degrade performance. On EWD, uniform weighting Qwen3 uniform 42.8 39.0 (0.6B) and pmax reduce the SSR of Qwen3-4B from 56.5% to 35.5% pmax 40.5 29.8 and 28.0%, respectively; similar trends hold on SWEET and for 52.3 Qwen3-0.6B. These results show that spoofing does not benefit Qwen3 1 - pmax 56.5 uniform 35.5 28.8 from uniformly enlarging token-level rewards. Instead, it requires (4B) pmax 28.0 25.0 identifying contexts with sufficient semantically permissible redistribution room, where probability mass can be shifted toward watermark-preferred continuations without compromising semantic fidelity. A conservative semantic reward improves surrogate quality. Beyond identifying feasible token positions, successful spoofing must preserve the meaning of the original text while staying close to the watermarked rewrite, which serves as a surrogate for the watermark-favored distribution. We therefore use a conservative semantic reward (Min.), defined by the weaker of the two semantic matches. Compared with averaging the two matches (Avg.) or using only one reference (Hum./W.M.), our design avoids one-sided alignment and produces a better surrogate target for optimization. As shown in Table 5, we observe Min. consistently achieves the best SSR across models and 7

Table 5: SSR across semantic rewards.

Table 6: CE-anchor ablation (SSR vs. RLS.).

Scheme

Min.

Avg.

Hum.

W.M.

Model

Scheme

W.O.

Dis.(100)

Qwen3 (0.6B)

EWD SWEET PF

54.3 50.5 33.3

44.8 48.8 26.3

42.3 43.0 24.5

48.5 46.8 25.5

Qwen3 (0.6B)

EWD SWEET PF

29.3 (-25.0) 27.3 (-23.2) 12.3 (-19.0)

0.25 (-54.0) 0.25 (-50.3) 0.00 (-31.3)

Qwen3 (4B)

EWD SWEET PF

56.5 52.3 62.0

47.3 34.3 50.3

47.5 47.0 48.5

45.5 48.5 51.0

Qwen3 (4B)

EWD SWEET PF

33.5 (-23.0) 26.8 (-25.5) 25.5 (-36.5)

0.00 (-56.5) 0.25 (-52.0) 0.00 (-62.0)

Table 7: SSR across Training set. Samples Models Scheme Qwen3 (0.6B) Qwen3 (4B)

100

200

EWD 44.3 54.3 56.8 SWEET 42.5 50.5 51.3 PF 12.3 31.3 20.5 EWD 46.5 56.5 58.5 SWEET 43.0 52.3 54.3 PF 20.8 60.8 25.3

Spoof Success Rate (SSR)

Model

(a) Sensitivity on Qwen3-4B 62.0 56.5

56.5 52.3

55 53.3

(b) Sensitivity on Qwen3-8B 60.5 58.5 54.3 45.3

45 EWD SWEET PF

35 25

0.6B

EWD SWEET PF

Self

Surrogate Model Choice

0.6B

35.0 29.8

Self

Surrogate Model Choice

Figure 3: (a) and (b) compare SSR of RLSpoofer under two surrogate choices, Qwen3-0.6B (0.6B) and attack model (Self).

watermarking schemes, suggesting that successful spoofing benefits from preserving the original meaning while maintaining a better surrogate target for optimization. Cross-entropy (CE) anchoring stabilizes optimization. We observe that even with carefully designed token-level and semantic rewards, unconstrained policy optimization can still drift away from the base model and exploit brittle reward shortcuts. Cross-entropy anchoring mitigates this by keeping optimization aligned with the base-model distribution while still allowing watermark-favored redistribution. As shown in Table 6, removing the anchor (W.O.) causes large SSR drops of 23.0 36.5 points, while replacing RL with supervised distillation on the same 100 training pairs (Dis.(100)) leads to near-complete failure. These results indicate that the gain comes from stable distributional alignment rather than limited-pair imitation alone. 4.4

Ablation Study

In this subsection, we study the sensitivity of RLSpoofer to the training data, the choice of surrogate distribution generated model, and its ability to generalize to OOD test data. Small training sets suffice for strong RLSpoofer performance. We examine the impact of training set size on RLSpoofer under three watermarking schemes, EWD, SWEET, and PF, using Qwen3-0.6B and Qwen3-4B. Table 7 shows that RLSpoofer can achieve strong spoofing performance with only 50 training samples. For instance, on EWD, Qwen3-4B attains an SSR of 46.5%, which increases to 58.5% with 200 samples. A similar pattern holds for SWEET. However, for the PF watermark, enlarging the training set does not yield consistent gains in SSR. This is likely because PF’s distortionfree design creates a distribution gap that becomes harder to capture with more data. Overall, these findings confirm that relatively small training sets are sufficient for RLSpoofer to obtain strong spoofing performance. We provide details in Appendix C.6. RLSpoofer exhibits increased sensitivity to surrogate selection as attacker capacity grows. We evaluate how the surrogate model used to approximate the target watermarked distribution impacts RLSpoofer’s efficacy. Specifically, we evaluate two surrogates, Qwen3-0.6B (0.6B) and the base attacker (Self), against EWD, SWEET, and PF watermarks across the 4B and 8B Qwen3 attackers. Fig. 3 demonstrates that surrogate selection minimally affects the weaker Qwen3-4B, which yields comparable SSR across all settings. Conversely, the stronger Qwen3-8B is highly sensitive: the 0.6B surrogate consistently outperforms the Self surrogate. For instance, on the PF watermark, utilizing 0.6B instead of Self elevates the SSR from 29.8% to 58.5%, with substantial gains similarly observed for EWD and SWEET. We attribute this disparity to Qwen3-8B’s superior watermark8

removal capabilities [17]; its rewrites heavily weaken the embedded signal, rendering the model itself an ineffective proxy for the target watermark distribution. Details are in Appendix C.7. EWD SWEET

Train watermark

Spoof Success Rate (SSR)

Cross-watermark Transferability RLSpoofer demonstrates strongly asymmetric and directional 60 cross-watermark transfer. We evaluate zero-shot transferability 56.5 50.0 6.00 with Qwen3-4B by training RLSpoofer on one watermarking scheme 50 and testing it on the others. As shown in Fig. 4, transferability is 40 not strictly constrained by watermark family. Although the KGW47.8 52.3 47.0 30 style, logit-based watermarks EWD and SWEET [9, 10] exhibit 20 substantial bidirectional transfer (SSR ∼ 47.8% - 50%), interactions involving the sampling-based PF watermark are notably directional. 52.5 0.25 62.0 10 Specifically, RLSpoofer transfers effectively from SWEET to PF 0 EWD SWEET PF (SSR 47%) and from PF to EWD (52.5%), whereas the reverse Test watermark directions are far weaker: PF to SWEET achieves only 0.25% SSR, and EWD to PF only 6%. This pronounced asymmetry suggests Figure 4: Cross watermark that transferability is not determined solely by mechanism-level transferability on Qwen3-4B. similarity, but also by more intricate and directional overlaps in the vulnerabilities induced by different watermarking schemes. We provide details in Appendix C.8.

Related Works

LLM watermarks. As LLMs become increasingly widespread, the human-written and machinegenerated text grows harder to distinguish. By embedding imperceptible yet algorithmically detectable signals into generated text, watermarking has become important tool for content attribution, copyright protection, and the mitigation of malicious misuse [31–33]. Existing methods mainly follow two paradigms. Logit-based approaches [7, 9] use a secret pseudo-random hash key to partition the vocabulary into green and red token lists at each generation step. They then bias the logits of green-list tokens to increase their sampling probability, and detection is performed through statistical tests for green-token overrepresentation [11]. Conversely, sampling-based methods [8, 23, 34] embed watermark signals directly into the sampling process, rather than modifying logits explicitly, offering finer-grained control while reducing quality degradation. Across both paradigms, recent advances prioritize minimizing statistical distortion, ensuring watermarked outputs remain indistinguishable from natural text [35]. LLM watermark spoofing attack. LLM watermark spoofing attacks seek to forge a target watermark signal and thereby falsely attribute arbitrary text to a specific model provider. Existing spoofing methods can generally be grouped into three categories. The first, piggyback spoofing [15], makes subtle edits to authentically watermarked text in order to inject toxic content while preserving detector confidence. However, this strategy substantially limits the attacker’s semantic flexibility. The second, feedback-guided spoofing attacks [15, 36], rely on repeated interaction with the watermark detector during generation to search for text that remains watermarked. Such attacks are typically query-intensive and depend on detector access, limiting their practical applicability. The third, learning-based spoofing [11, 13, 14], queries the target model to construct a training set for a surrogate model that internalizes the watermark signal. However, these approaches typically require either the knowledge of the watermarking mechanism [11] or a large amount of training data (e.g., up to 10K samples [13]). Consequently, the restrictive assumptions and inefficiencies of these approaches make them unsuitable for practically evaluating the spoofing resilience of watermarking schemes.

Conclusion

In this paper, we study LLM watermark spoofing attack from a distributional perspective. We establish a local capacity bottleneck to characterize how token-level probability mass can be redistributed under KL-bounded, semantics-consistent local updates. Building on this insight, we propose RLSpoofer, a black-box RL-based spoofing attack that requires only 100 human–watermarked paraphrase pairs and no access to watermark internals or detectors. Across the evaluated setting of five attacker models and six watermarking schemes, RLSpoofer achieves strong spoofing performance on both logit-based and sampling-based watermarks. These results suggest that current LLM watermarking designs may remain vulnerable to spoofing attacks. We hope RLSpoofer can serve as a useful stress test for future watermark evaluation and for the development of more spoofing-resistant schemes. 9

References [1] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. [2] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. [3] Canyu Chen and Kai Shu. Can llm-generated misinformation be detected? arXiv preprint arXiv:2309.13788, 2023. [4] Zhenhua Xu, Xubin Yue, Zhebo Wang, Qichen Liu, Xixiang Zhao, Jingxuan Zhang, Wenjun Zeng, Wengpeng Xing, Dezhang Kong, Changting Lin, et al. Copyright protection for large language models: A survey of methods, challenges, and trends. arXiv preprint arXiv:2508.11548, 2025. [5] Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. The curse of recursion: Training on generated data makes models forget. arXiv preprint arXiv:2305.17493, 2023. [6] Debby RE Cotton, Peter A Cotton, and J Reuben Shipway. Chatting and cheating: Ensuring academic integrity in the era of chatgpt. Innovations in education and teaching international, 61(2):228–239, 2024. [7] John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In International Conference on Machine Learning, pages 17061–17084. PMLR, 2023. [8] Xuandong Zhao, Lei Li, and Yu-Xiang Wang. Permute-and-flip: An optimally stable and watermarkable decoder for llms. arXiv preprint arXiv:2402.05864, 2024. [9] Yijian Lu, Aiwei Liu, Dianzhi Yu, Jingjing Li, and Irwin King. An entropy-based text watermarking detection method. arXiv preprint arXiv:2403.13485, 2024. [10] Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim. Who wrote this code? watermarking for code generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4890–4911, 2024. [11] Nikola Jovanović, Robin Staab, and Martin Vechev. Watermark stealing in large language models. arXiv preprint arXiv:2402.19361, 2024. [12] Ruibo Chen, Yihan Wu, Junfeng Guo, and Heng Huang. De-mark: Watermark removal in large language models. arXiv preprint arXiv:2410.13808, 2024. [13] Hyeseon An, Shinwoo Park, Suyeon Woo, and Yo-Sub Han. Ditto: A spoofing attack framework on watermarked llms via knowledge distillation. arXiv preprint arXiv:2510.10987, 2025. [14] Chenchen Gu, Xiang Lisa Li, Percy Liang, and Tatsunori Hashimoto. On the learnability of watermarks for language models. arXiv preprint arXiv:2312.04469, 2023. [15] Qi Pang, Shengyuan Hu, Wenting Zheng, and Virginia Smith. Attacking llm watermarks by exploiting their strengths. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models, 2024. [16] Thibaud Gloaguen, Nikola Jovanović, Robin Staab, and Martin Vechev. Discovering spoofing attempts on language model watermarks. arXiv preprint arXiv:2410.02693, 2024. [17] Hanbo Huang, Yiran Zhang, Hao Zheng, Xuan Gong, Yihan Li, Lin Liu, and Shiyu Liang. Rlcracker: Exposing the vulnerability of llm watermarks with adaptive rl attacks. arXiv preprint arXiv:2509.20924, 2025. 10

[18] Aiwei Liu, Sheng Guan, Yiming Liu, Leyi Pan, Yifei Zhang, Liancheng Fang, Lijie Wen, Philip S Yu, and Xuming Hu. Can watermarked llms be identified by users via crafted prompts? arXiv preprint arXiv:2410.03168, 2024. [19] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. [20] John Wieting, Kevin Gimpel, Graham Neubig, and Taylor Berg-Kirkpatrick. Paraphrastic representations at scale. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 379–388, 2022. [21] Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025. URL https://arxiv.org/abs/2412.15115. [22] Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu-Xiang Wang. Provable robust watermarking for ai-generated text. arXiv preprint arXiv:2306.17439, 2023. [23] Jiahao Huo, Shuliang Liu, Bin Wang, Junyan Zhang, Yibo Yan, Aiwei Liu, Xuming Hu, and Mingxun Zhou. Pmark: Towards robust and distortion-free semantic-level watermarking with channel constraints. arXiv preprint arXiv:2509.21057, 2025. [24] Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, et al. Markllm: An open-source toolkit for llm watermarking. arXiv preprint arXiv:2405.10051, 2024. [25] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1–67, 2020. [26] Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. Ghostbuster: Detecting text ghostwritten by large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 1702–1717, 2024. [27] Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense. Advances in Neural Information Processing Systems, 36:27469–27500, 2023. [28] Julien Piet, Chawin Sitawarin, Vivian Fang, Norman Mu, and David Wagner. Markmywords: Analyzing and evaluating language model watermarks. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 68–91. IEEE, 2025. [29] Abdulrahman Diaa, Toluwani Aremu, and Nils Lukas. Optimizing adaptive attacks against watermarks for language models. arXiv preprint arXiv:2410.02440, 2024. [30] Yixin Cheng, Hongcheng Guo, Yangming Li, and Leonid Sigal. Revealing weaknesses in text watermarking through self-information rewrite attacks. arXiv preprint arXiv:2505.05190, 2025. [31] Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Lidia Sam Chao, and Derek Fai Wong. A survey on llm-generated text detection: Necessity, methods, and future directions. Computational Linguistics, 51(1):275–338, 2025. [32] Aiwei Liu, Qiang Sheng, and Xuming Hu. Preventing and detecting misinformation generated by large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 3001–3004, 2024. 11

[33] Ruisi Zhang, Shehzeen Samarah Hussain, Paarth Neekhara, and Farinaz Koushanfar. {REMARK-LLM}: A robust and efficient watermarking framework for generative large language models. In 33rd USENIX Security Symposium (USENIX Security 24), pages 1813–1830, 2024. [34] Abe Hou, Jingyu Zhang, Yichen Wang, Daniel Khashabi, and Tianxing He. k-semstamp: A clustering-based semantic watermark for detection of machine-generated text. In Findings of the Association for Computational Linguistics: ACL 2024, pages 1706–1715, 2024. [35] Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. Robust distortionfree watermarks for language models. arXiv preprint arXiv:2307.15593, 2023. [36] Tong Zhou, Xuandong Zhao, Xiaolin Xu, and Shaolei Ren. Bileve: Securing text provenance in large language models against spoofing with bi-level signature. Advances in Neural Information Processing Systems, 37:56054–56075, 2024. [37] Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thailand, 2024. Association for Computational Linguistics. URL http://arxiv.org/abs/2403.13372. [38] Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, and Nathan Lambert. Trl: Transformer reinforcement learning. https://github.com/lvwerra/ trl, 2020. [39] Hugging Face. Open r1: A fully open reproduction of deepseek-r1, January 2025. URL https://github.com/huggingface/open-r1.

Details for the distributional surrogate

A.1

Absolute continuity and finite KL divergences

In this subsection, we state the regularity conditions under which the distributional surrogate in Section 2 is well defined and can be expanded into the expected log-likelihood-ratio objective in Eq. 1. These conditions ensure that the relevant KL divergences are finite and that the log-likelihood ratio is integrable under the attack distribution. Assumption 1 (Absolute continuity and finite KL divergences). For a fixed input X, Pθ (· | X) is absolutely continuous with respect to both reference distributions (Pθ ≪ Ph and Pθ ≪ Pwm ). Furthermore, the log-likelihood ratio is integrable: EX′ ∼Pθ [| log(Pwm (X′ | X)/Ph (X′ | X))|] < ∞. Moreover, DKL Pθ (· | X) ∥ Ph (· | X) < ∞ and DKL Pθ (· | X) ∥ Pwm (· | X) < ∞. Under Assumption 1, both KL terms in the surrogate objective are well defined, and their difference can be expanded as Pwm (X′ | X) DKL Pθ ∥ Ph − DKL Pθ ∥ Pwm = EX′ ∼Pθ (·|X) log , Ph (X′ | X) where we suppress the conditioning on X for readability. A.2

Proof of Theorem 1

In this subsection, we use the binary KL divergence dkl (p∥q) := p log

1−p p + (1 − p) log , q 1−q

p, q ∈ [0, 1],

with the standard conventions 0 log(0/q) := 0 for q ∈ [0, 1], 0 log(0/0) := 0, and b log(b/0) := +∞ for b > 0. For convenience, we restate Theorem 1. Theorem 2 (Local capacity characterization). Fix a history ht . For any next-token distribution πθ (· | ht ) and any subset A ⊆ V, we have n o πθ (A | ht ) − Ph (A | ht ) ≤ sup λ ∈ [0, 1] : dkl λ 1 − maxx∈V Ph (x | ht ) ≤ DKL πθ (· | ht ) ∥ Ph (· | ht ) .

(7)

Moreover, for any fixed C ≥ 0, lim

sup πθ (A | ht ) − Ph (A | ht ) = 0.

sup

1−maxx∈V Ph (x|ht )→0 πθ : DKL (πθ (·|ht ) ∥ Ph (·|ht ))≤C A⊆V

(8)

Proof. Fix a history ht , and let x∗t ∈ arg maxx∈V Ph (x | ht ) and q := 1 − Ph (x∗t | ht ) = 1 − maxx∈V Ph (x | ht ). Thus, q is the total mass assigned by Ph (· | ht ) to all tokens other than the dominant human-like continuation x∗t . We first prove Eq. (7). Consider the measurable map T : V → {0, 1} defined by T (x) = 1{x ̸= x∗t }, which induces the binary partition {x∗t } and V \ {x∗t }. Under Ph (· | ht ), the pushforward distribution of T is (1 − q, q), whereas under πθ (· | ht ) it is πθ (x∗t | ht ), 1 − πθ (x∗t | ht ) . By the data processing inequality for KL divergence under the map T , dkl 1 − πθ (x∗t | ht ) ∥ q ≤ DKL πθ (· | ht ) ∥ Ph (· | ht ) . (9) Hence, n o 1 − πθ (x∗t | ht ) ≤ sup λ ∈ [0, 1] : dkl (λ∥q) ≤ DKL πθ (· | ht ) ∥ Ph (· | ht ) . Moreover, since dkl (q∥q) = 0, we also have n o q ≤ sup λ ∈ [0, 1] : dkl (λ∥q) ≤ DKL πθ (· | ht ) ∥ Ph (· | ht ) . Now fix any subset A ⊆ V. We distinguish two cases. 13

(10)

(11)

Case 1: x∗t ∈ / A.

Since A ⊆ V \ {x∗t }, we have πθ (A | ht ) ≤ 1 − πθ (x∗t | ht ). Therefore, πθ (A | ht ) − Ph (A | ht ) ≤ πθ (A | ht ) ≤ 1 − πθ (x∗t | ht ).

Case 2: x∗t ∈ A. Then Ac ⊆ V \ {x∗t }, so Ph (Ac | ht ) ≤ q. Using πθ (A | ht ) − Ph (A | ht ) = Ph (Ac | ht ) − πθ (Ac | ht ), we obtain πθ (A | ht ) − Ph (A | ht ) ≤ Ph (Ac | ht ) ≤ q. Combining the two cases yields n o πθ (A | ht ) − Ph (A | ht ) ≤ max 1 − πθ (x∗t | ht ), q .

(12)

Finally, Eqs. (10), (11), and (12) imply n o πθ (A | ht ) − Ph (A | ht ) ≤ sup λ ∈ [0, 1] : dkl (λ∥q) ≤ DKL πθ (· | ht ) ∥ Ph (· | ht ) , which is exactly Eq. (7). We next prove Eq. (8). Fix C ≥ 0, and define n o ρ(q; C) := sup λ ∈ [0, 1] : dkl (λ∥q) ≤ C . By Eq. (7), it suffices to show that ρ(q; C) → 0

as q → 0.

(13)

Suppose otherwise. Then there exist ε > 0, a sequence qn → 0, and λn ∈ [ε, 1] such that dkl (λn ∥qn ) ≤ C for all n. Since dkl (λn ∥qn ) = λn log

λn 1 − λn + (1 − λn ) log , qn 1 − qn

and the second term is bounded below by (1 − λn ) log(1 − λn ), we obtain dkl (λn ∥qn ) ≥ λn log

1 + λn log λn + (1 − λn ) log(1 − λn ). qn

Using the elementary bound x log x ≥ −1/e on [0, 1] and λn ≥ ε, it follows that dkl (λn ∥qn ) ≥ ε log

1 2 − . qn e

The right-hand side tends to +∞ as qn → 0, contradicting dkl (λn ∥qn ) ≤ C. This proves Eq. (13). Finally, for every πθ satisfying DKL πθ (· | ht ) ∥ Ph (· | ht ) ≤ C, Eq. (7) yields sup πθ (A | ht ) − Ph (A | ht ) ≤ ρ(q; C). A⊆V

Taking the supremum over all such πθ , and then letting q = 1 − maxx∈V Ph (x | ht ) → 0, Eq. (13) gives Eq. (8).

Experimental Setup and Configuration

B.1

Watermark algorithm setting

In this subsection, we report the hyperparameter settings for the watermarking algorithms evaluated in Section 4. For consistency and reproducibility, we use the official PMark [23] codebase† with its default settings for both watermark generation and evaluation. For all other watermarking methods, we use the MarkLLM toolkit [24]† to generate watermarked text. Specifically, for KGW, we follow [13] and set δ = 3, while for EWD, SWEET, PF, and Unigram, we use the default configurations provided † https://github.com/PMark-repo/PMark † https://github.com/THU-BPM/MarkLLM

by MarkLLM. MarkLLM is widely adopted in the watermarking literature due to its robustness and ease of integration.

Hyperparameters for the PMark watermark "algorithm_name": "PMark", "num_samples": 64, "pivot": rand, "median_method": prior, "msig": 2

Hyperparameters for the KGW watermark "algorithm_name": "KGW", "gamma": 0.5, "delta": 3.0, "hash_key": 15485863, "prefix_length": 1, "z_threshold": 4.0, "f_scheme": "time", "window_scheme": "left"

Hyperparameters for the EWD watermark "algorithm_name": "EWD", "gamma": 0.5, "delta": 2.0, "hash_key": 15485863, "prefix_length": 1, "z_threshold": 4.0

Hyperparameters for the SWEET watermark "algorithm_name": "SWEET", "gamma": 0.5, "delta": 2.0, "hash_key": 15485863, "z_threshold": 4.0, "prefix_length": 1, "entropy_threshold": 0.9

Hyperparameters for the PF watermark "algorithm_name": "PF", "ngram": 8, "seed": 0, "seeding": "hash", "salt_key": 35317, "payload": 0, "max_seq_len": 8192

Hyperparameters for the Unigram watermark "algorithm_name": "Unigram", "gamma": 0.5, "delta": 2.0, "hash_key": 15485863, "z_threshold": 4.0 B.2

Dataset Construction.

We construct the training set from the C4-RealNewslike subset [25] and the test set from Reddit WritingPrompts [26], LFQA [27], and the BookReport and FakeNews subsets of MMW [28], following [11]. To build training data, we first prompt Qwen3-8B to generate 500-token human-like unwatermarked texts. We then prompt watermarked Llama3.1-8B-Instruct to rewrite each text while preserving its semantics, yielding paired data of the form (human-like text, watermarked rewrite). For RLSpoofer, we use 100 such pairs for training and 20 for validation on each watermark. For DPO, we additionally obtain a rejected response by prompting Qwen3-8B to produce a nonwatermarked rewrite of the same human-written text. The resulting preference pair consists of the watermarked rewrite as the chosen response and the non-watermarked rewrite as the rejected response. For distillation, we cast the task as supervised rewriting: the human-like text is formatted with the prompt template shown below, following [17], and the corresponding watermarked rewrite is used as the target output for SFT. Prompt template used ######Target Text: [human-written text] ######Instruction: Rewrite the target text above using different words but keeping the same meaning and similar length. ######Your Response: For evaluation, we sample 400 unwatermarked test instances evenly from the four benchmark datasets. All training and test samples are standardized to 500 tokens. B.3

Baselines.

Spoof Methods. Under the black-box threat model, in addition to RLSpoofer, we consider three spoofing baselines to further expose vulnerabilities in watermarking schemes. Distill [14] learns to imitate the rewriting distribution of the watermarked model via supervised fine-tuning on human– watermarked rewritten pairs. DITTO [13] further exploits the distilled model by analyzing its output distribution and reproducing the statistical preferences induced by the target watermark. DPO [29] formulates watermark spoofing as a preference alignment problem, where the attacker is optimized to prefer watermarked rewrites over non-watermarked ones. Distill. We implement Distill using LLaMA-Factory† [37] and fine-tune each attacker model on 10,000 human–watermarked rewritten pairs. The task is formulated as a supervised rewriting problem: given an input human-written text wrapped with the prompt template shown above, the model is trained to generate the corresponding watermarked rewrite. We perform full-parameter fine-tuning for each model, using a learning rate of 2 × 10−5 and a batch size of 128. DITTO. We implement DITTO using its official codebase † and follow its default settings. In particular, DITTO is applied on top of the model obtained from the Distill stage, rather than being trained from scratch. That is, it starts from the distilled spoofing model and further enhances spoofing performance by matching the statistical preferences induced by the target watermark. DPO. We also implement DPO using LLaMA-Factory. Following prior work, we construct 7,000 preference pairs, where the chosen response is the watermarked rewrite and the rejected response is a non-watermarked rewrite of the same human-written text. We perform full-parameter fine-tuning † https://github.com/hiyouga/LLaMA-Factory † https://github.com/hsannn/ditto

for each model with β = 0.1, a batch size of 64, and a learning rate of 5 × 10−6 . We additionally tune the learning rate over {2 × 10−5 , 2 × 10−6 , 2 × 10−7 } and find that 5 × 10−6 yields the best overall performance. Evaluation Details. For evaluation, all spoofing methods use deterministic decoding with temperature = 0. For Distill and DPO, we conduct inference with the vLLM engine to accelerate generation. For DITTO, we use the inference pipeline provided in its official implementation rather than vLLM. In all cases, the spoofing model takes the original input text and produces a rewritten response of similar semantics and length, which is then evaluated by the corresponding watermark detector. All reported spoofing results are obtained under this deterministic decoding setting. B.4

Implementation Details of RLSpoofer.

RLSpoofer is implemented on top of the GRPO [19] module in the TRL [38] library. The training data consists only of human–watermarked rewrite pairs (X, X′wm ), where X is a human-written text and X′wm is its semantics-preserving rewrite produced by the target watermarked model. Given X, the attack policy πθ samples a group of G candidate rewrites {x′1 , . . . , x′G }, and is optimized by jointly combining a sequence-level semantic reward, a capacity-aware token-level reward, a token-wise KL regularizer, and a teacher-forced cross-entropy anchor. Next, we introduce the implementation details of each of the components in RLSpoofer. Reward Components. We incorporate two reward signals to encourage semantic preservation and effective watermark spoofing: • Semantic Reward. The semantic reward Ai ∈ [−1, 1] is computed based on the P-SP score [20] between the generated output and both the original watermarked response X′wm and the original human-written text X. Specifically, we first compute the semantic similarity between the generated output and each reference, and then take the minimum of the two scores as the final semantic score. To enhance gradient flow and emphasize semantic fidelity, we apply a sigmoid-based scaling with a threshold of 0.85 following [17]. This maps semantic scores in the range [0.7, 1.0] to reward values between −1 and 1. Specifically, 0.975 2 P-SP score − 0.85 − 1, where x = log . Ai = · 1 + e−x 0.025 1 − 0.85 Ai −mean(A) The advantage term is normalized as Âi = std(A)+10 −6 . • Capacity-aware Token-level Reward. This reward encourages each sampled token to align with the watermark-induced surrogate distribution while staying aware of the locally available semantics-preserving redistribution room. In implementation, to instantiate the surrogate humanlike and watermarked distributions, we query the same reference model πref with the rewriting template in Appendix B.2, following [17]. Specifically, the human-written text X and the watermarked rewrite X′wm are respectively filled into the same template as the target text, yielding two conditioning contexts that share the same rewriting instruction. For each sampled token x′i,t , we then compute its log-probability under the human-conditioned surrogate distribution, log ph (x′i,t ) = log πref (x′i,t | X, x′i,<t ), and under the watermarked-conditioned surrogate distribution, log pwm (x′i,t ) = log πref (x′i,t | X′wm , x′i,<t ). We further define the local capacity mass as ci,t = 1 − max πref (v | X, x′i,<t ), v∈V

which measures the probability mass outside the dominant human-like continuation. Based on this quantity, the capacity-aware token-level reward is defined as ri,t = ci,t log pwm (x′i,t ) − log ph (x′i,t ) . Here, log pwm (x′i,t ) − log ph (x′i,t ) measures whether the sampled token is more preferred by the watermark-conditioned surrogate than by the human-conditioned surrogate, while ci,t downweights positions where the human-like distribution is already sharply concentrated. As a result, the reward is amplified only at positions with sufficient local flexibility for semantics-preserving redistribution. Cross-entropy anchor. To stabilize optimization, we further introduce a cross-entropy anchor. Specifically, we condition the policy on the human-written input X and teacher-force it to predict the 17

P watermarked rewrite X′wm , yielding LCE = − t log πθ (xwm | X, xwm t <t ). This term anchors the policy toward the target watermarked rewriting distribution and prevents unstable distributional drift during RL training. Putting these components together, the implemented objective can be written as |x′i | " G πθ (x′i,t | X, x′i,<t ) 1 X 1 X w1 Âi + w2 ri,t J (θ) ≈ E{x′i }∼πθold ′ ′ ′ G i=1 |xi | t=1 πθold (xi,t | X, xi,<t ) # − βDKL πθ ∥ πref − w3 LCE πθ , {(X, X′wm )} , In implementation, the KL regularizer is directly adopted from TRL-GRPO, and we set β = 0.04 following the OpenR1 [39] setting. Unless otherwise specified, the reference model πref is chosen as the initial policy model before RL training. Hyperparameters. For each watermarking scheme, we train a dedicated RLSpoofer model. The training set consists of 100 human–watermarked rewrite pairs, each standardized to 500 tokens. In addition, we use a validation set of 20 data pairs to select the hyperparameters w1 , w2 , and w3 . We train the model for 10 epochs with a batch size of 48 and a group size of G = 12, using a cosine learning rate scheduler. Empirically, we find that a relatively large learning rate is important for inducing a sufficient distribution shift. Specifically, we use a learning rate of 2 × 10−5 for PMark and 2 × 10−4 and 1 × 10−4 for the other watermarking schemes. Training takes approximately 1.5 hours for Qwen3-4B and 45 minutes for Qwen3-0.6B on four NVIDIA Pro 6000 Blackwell GPUs. We find that RLSpoofer is sensitive to the relative weights of its three optimization components, w1 , w2 , and w3 , and different choices of these weights lead to substantial variation in performance. To study this effect, we train Qwen3-4B to spoof the EWD watermark and analyze, in Figure 5, how the spoof rate and P-SP score change as each weight varies. Here, the spoof rate refers to the proportion of rewritten outputs detected as watermarked, without imposing any semantic similarity constraint. In the sensitivity analysis, we vary one weight at a time while fixing the other two: for w1 , we fix w2 = 2 and w3 = 1; for w2 , we fix w1 = 3 and w3 = 1; and for w3 , we fix w1 = 3 and w2 = 2. Overall, we find that (w1 , w2 , w3 ) = (3, 2, 1) provides the best trade-off between spoofing strength and semantic fidelity for EWD. For completeness and reproducibility, we report the final weight settings for each watermarking scheme and attack model in Table 8. (a) Sensitivity on 𝑤 1

0.90

(b) Sensitivity on 𝑤 2

1.00

100

0.90

0.80

0.70

0.60

0.50

0.40

0.30

0.20

Spoof Rate(%) P-SP Score P-SP Threshold

40 30 20

0.5

𝑤1

Spoof Rate(%) P-SP Score P-SP Threshold

0.5

𝑤2

1.00

P-SP Score

100

P-SP Score Spoof Rate(%)

1.00

P-SP Score Spoof Rate(%)

Spoof Rate(%)

100

0.50 Spoof Rate(%) P-SP Score P-SP Threshold

0.1

0.5

𝑤3

0.40 0.30 0.20

Figure 5: Sensitivity of RLSpoofer across component weights on validation set.

Experimental Results and Analysis

C.1

Robustness of watermarking schemes against spoofing

We report the full spoofing results across six watermarking schemes, five attacker models, and four spoofing methods in Tables 9, 10, and 11. We evaluate all methods using Spoof Success Rate (SSR), Spoof Rate (SR), P-SP, perplexity (PPL), and GPT-score (GPTS). We treat SSR as the primary metric, since it measures the proportion of rewritten outputs that are both detected as watermarked and satisfy the semantic constraint P-SP > 0.7, whereas SR measures detector success without semantic filtering. We report the results averaged across 2 random seeds, 42 and 1234. For GPTS, we use GPT-5.4-nano as the judge model with the prompt template below, following [17]. 18

Table 8: Hyperparameter configurations of RLSpoofer across watermarking schemes. Model

Hyper.

Qwen3-0.6B

w1 w2 w3 lr

EWD

5 3 5 3 1 1 10 2 1 5 5 2 1 1 1 1 2 1 2 × 10−4 2 × 10−4 2 × 10−4 2 × 10−4 2 × 10−4 2 × 10−5

SWEET

KGW

Unigram

PMark

Qwen3-1.7B

w1 w2 w3 lr

5 5 3 8 0.1 1 5 5 2 5 5 5 1 2 1 1 2 1 2 × 10−4 2 × 10−4 2 × 10−4 2 × 10−4 2 × 10−4 2 × 10−5

Qwen3-4B

w1 w2 w3 lr

3 3 5 8 1 1 2 1 1 5 10 5 1 1 1 1 3 1 2 × 10−4 2 × 10−4 2 × 10−4 2 × 10−4 2 × 10−4 2 × 10−5

Qwen2.5-3B-Instruct

w1 w2 w3 lr

3 3 5 5 3 3 2 2 2 2 5 10 1 1 2 1 2 2 2 × 10−4 2 × 10−4 1 × 10−4 2 × 10−4 2 × 10−4 2 × 10−5

w1 w2 Llama3.2-3B-Instruct w3 lr

5 5 1 1 1 1 2 2 1 2 2 5 2 2 1 1 3 1 2 × 10−4 1 × 10−4 1 × 10−4 1 × 10−4 1 × 10−4 2 × 10−5

Prompt Template for GPT-5.4-nano as a Judge role: system, content: You are an impartial evaluator. You will be provided with an original text and a paraphrased version of that text. Your task is to assess the quality of paraphrased text based on the following criteria: 1. The degree to which the paraphrased text maintains the meaning of the original text. 2. The fluency, coherence, and clarity of the paraphrased text. 3. The extent to which the style and tone of the paraphrased text match those of the original text. Please provide an objective evaluation and rate the paraphrased text on a scale of 1 to 10: - A rating of 1 indicates that the paraphrased text completely alters the meaning of the original text. A rating of 10 indicates that the paraphrased text preserves the exact meaning of the original text, maintaining fluency, clarity, and style. Your response should return the score only. role: user, content: You will receive a original text and a pharased text. Please act as an impartial judge and evaluate the quality of the paraphrased text. You should evaluate the paraphrased text based on the following criteria: 1. How closely the meaning of the paraphrased text aligns with the original text. 2. The fluency and clarity of the paraphrased text. 3. How closely the style and tone of the paraphrased text align with the original text. Rate the paraphrased text on a scale of 1 to 10, where: - A rating of 1 indicates that the paraphrased text deviates significantly from the original meaning. - A rating of 10 indicates that the paraphrased text perfectly preserves the original meaning, fluency, clarity, and tone. Your response should strictly follow this format and return the score only: [Rating]. Here the original text: [human-written text] Here’s the paraphrased text: [rephrased text] As shown in Tables 9, 10 and 11, we observe that RLSpoofer achieves the best or second-best SSR in most settings while maintaining competitive rephrasing quality. In contrast, DITTO often attains high SR but much lower P-SP, suggesting that many of its successful detections come from semantically distorted rewrites. DPO shows the opposite pattern: it typically preserves semantics better, but fails to induce sufficient distribution shift toward the target watermark, resulting in consistently weak SSR. Distill is competitive on several logit-based schemes, but is substantially less effective on the more challenging sampling-based settings. For the logit-based watermarks, including EWD, SWEET, KGW, and Unigram, we find that these schemes are already vulnerable to distribution-matching baselines, although RLSpoofer remains the most consistent method overall. In particular, RLSpoofer achieves SSR above 50% on EWD and 19

Table 9: Spoof Success Rate (SSR, %), Spoof Rate (SR, %), P-SP, Perplexity (PPL), and GPT-score (GPTS) across models. For readability, we report SSR and SR as mean ± standard deviation over two random seeds. Bold indicates the best mean SSR for each model under each watermarking scheme. Higher SSR indicates stronger spoofing performance. EWD Models

Methods

SSR

SWEET

P-SP PPL GPTS

SSR

P-SP PPL GPTS

Distill 42.3 ± 1.2 57.3 ± 2.5 0.75 4.50 DITTO 7.50 ± 0.5 97.3 ± 0.5 0.43 4.78 Qwen3-0.6B DPO 0.25 ± 0.8 0.75 ± 0.0 0.57 1.95 RLSpoofer 54.3 ± 0.8 80.5 ± 1.8 0.73 5.36

6.62 3.00 4.42 5.77

20.0 ± 0.8 33.8 ± 1.2 0.76 4.68 6.75 ± 0.3 95.8 ± 0.6 0.41 4.52 0.00 ± 0.0 0.00 ± 0.0 0.76 2.71 50.5 ± 1.2 66.3 ± 2.0 0.79 4.93

6.62 3.00 4.42 5.77

Qwen3-1.7B

Distill 43.8 ± 1.5 53.8 ± 2.2 0.80 4.59 DITTO 21.5 ± 0.8 96.0 ± 0.2 0.53 5.26 DPO 0.25 ± 0.0 0.25 ± 0.0 0.94 2.35 RLSpoofer 53.5 ± 1.2 68.5 ± 0.7 0.76 5.15

7.27 4.68 8.17 6.50

26.8 ± 2.5 37.8 ± 1.4 0.79 4.63 29.0 ± 0.7 92.8 ± 0.8 0.56 5.22 0.00 ± 0.0 0.00 ± 0.0 0.84 2.46 52.0 ± 0.8 72.0 ± 0.6 0.71 5.92

7.27 4.68 8.17 6.45

Qwen3-4B

Distill 51.3 ± 0.6 62.5 ± 1.5 0.81 4.75 DITTO 56.0 ± 0.3 94.3 ± 0.2 0.68 5.62 DPO 0.25 ± 0.0 0.50 ± 0.0 0.78 2.24 RLSpoofer 56.5 ± 0.6 87.0 ± 0.4 0.73 4.88

7.66 6.66 7.55 6.68

37.3 ± 0.8 43.8 ± 2.5 0.82 4.79 43.3 ± 0.6 94.5 ± 0.3 0.66 5.25 0.00 ± 0.0 0.00 ± 0.0 0.78 2.02 52.3 ± 0.2 71.3 ± 0.3 0.75 5.38

7.66 6.66 7.55 6.78

Distill 55.5 ± 0.5 81.5 ± 1.2 0.76 4.40 Qwen2.5-3B DITTO 14.0 ± 0.0 99.5 ± 0.0 0.50 4.95 DPO 0.00 ± 0.0 0.75 ± 0.0 0.88 2.00 -Instruct RLSpoofer 53.5 ± 0.6 83.0 ± 0.8 0.70 4.66

7.31 5.32 7.76 6.80

49.8 ± 0.3 67.5 ± 1.0 0.77 4.37 22.3 ± 0.2 99.8 ± 0.0 0.54 4.78 0.00 ± 0.0 0.50 ± 0.0 0.87 1.94 54.5 ± 1.2 68.8 ± 1.3 0.75 6.64

7.31 5.32 7.76 6.57

Distill 53.8 ± 0.2 71.3 ± 1.3 0.77 4.49 Llama3.2-3B DITTO 19.3 ± 0.3 99.5 ± 0.0 0.54 4.90 DPO 2.50 ± 0.0 1.25 ± 0.0 0.49 2.00 -Instruct RLSpoofer 54.5 ± 0.8 79.5 ± 0.3 0.70 4.99

7.22 5.13 2.62 6.79

45.5 ± 0.5 62.0 ± 2.5 0.76 4.37 24.3 ± 0.2 100. ± 0.0 0.56 4.76 0.50 ± 0.0 3.00 ± 0.0 0.53 1.86 54.5 ± 1.0 79.5 ± 2.2 0.74 6.15

7.22 5.13 2.62 6.43

SWEET for nearly all attacker models, indicating that it can preserve semantics while aligning with watermark-favored distributions. On KGW, Distill is competitive in some cases, suggesting that its watermark signal is relatively learnable from large corpora. On Unigram, however, the gap becomes much larger: while Distill and DITTO degrade substantially, RLSpoofer consistently maintains strong spoofing performance. The largest gap appears on the sampling-based watermarks PF and PMark. We observe that all three baselines struggle on PF, with uniformly low SSR across models, indicating that they largely fail to reproduce the watermark signal. By contrast, RLSpoofer achieves substantially stronger spoofing performance, most notably reaching 62% SSR on PF with Qwen3-4B. This result is especially notable because PF is distortion-free in expectation, making its signal difficult to capture through standard distillation-based attacks. On PMark, RLSpoofer again achieves the best SSR across all attacker models while maintaining high P-SP, showing that its gain does not come merely from sacrificing rephrasing quality. In conclusion, we observe that logit-based watermarks are vulnerable to attacks that learn their induced distributional bias, but RLSpoofer is more reliable because it better preserves semantics while shifting the output distribution. Moreover, sampling-based watermarks, especially PF, appear much more robust to existing baselines, yet remain highly vulnerable to RLSpoofer, suggesting that standard evaluation pipelines can substantially underestimate spoofing risk. Additionally, we present example visualizations for the logit-based watermarking algorithms EWD, SWEET, KGW, and Unigram, comparing the watermarked rewrites produced by RLSpoofer with the corresponding original unwatermarked texts. The results are shown in Figure 6 to Figure 13, with paired figures showing high semantic similarities (P-SP score>0.9). C.2

Detection Score Distribution

The detection scores in Figure 2(a) demonstrate that RLSpoofer effectively shifts the distribution of rephrased texts toward the watermarked distribution and away from the unwatermarked reference. To establish the unwatermarked distribution, we directly use the test set’s z-scores; for the watermarked distribution, we use test-set rewrites generated by the watermarked Llama-3.1-8B-Instruct. Finally, the 20

Figure 6: Watermarked EWD example

Figure 7: Unwatermarked EWD example

Figure 8: Watermarked EWD example

Figure 9: Unwatermarked EWD example

Figure 10: Watermarked EWD example

Figure 11: Unwatermarked EWD example

Figure 12: Watermarked EWD example

Figure 13: Unwatermarked EWD example

Table 10: Spoof Success Rate (SSR, %), Spoof Rate (SR, %), P-SP, Perplexity (PPL), and GPT-score (GPTS) across models. For readability, we report SSR and SR as mean ± standard deviation over two random seeds. Bold indicates the best mean SSR for each model under each watermarking scheme. Higher SSR indicates stronger spoofing performance. KGW Models

Methods

SSR

Unigram

P-SP PPL GPTS

SSR

P-SP PPL GPTS

Distill 35.8 ± 0.5 66.8 ± 3.5 0.68 4.78 DITTO 1.00 ± 0.0 84.0 ± 0.8 0.33 5.05 Qwen3-0.6B DPO 1.00 ± 0.0 11.0 ± 0.6 0.66 2.12 RLSpoofer 52.0 ± 0.6 82.5 ± 0.8 0.72 5.16

5.91 2.34 5.31 5.32

13.8 ± 0.2 27.5 ± 1.5 0.84 2.62 0.25 ± 0.0 99.3 ± 0.0 0.32 1.67 0.25 ± 0.0 13.8 ± 1.2 0.32 1.47 49.5 ± 0.8 82.3 ± 1.0 0.70 5.16

5.91 2.34 5.31 5.43

Qwen3-1.7B

Distill 44.5 ± 0.5 67.5 ± 2.5 0.75 5.11 DITTO 13.8 ± 0.2 95.5 ± 0.0 0.52 6.36 DPO 1.00 ± 0.0 1.00 ± 0.0 0.96 2.55 RLSpoofer 52.0 ± 1.2 78.8 ± 2.2 0.71 5.62

7.24 4.43 9.12 6.24

19.5 ± 0.8 38.0 ± 0.6 0.81 2.50 1.50 ± 0.0 100. ± 0.0 0.36 1.76 0.50 ± 0.0 2.50 ± 0.0 0.87 2.03 54.8 ± 0.8 83.8 ± 1.3 0.73 4.72

7.24 4.43 9.12 6.53

Qwen3-4B

Distill 57.0 ± 0.5 74.3 ± 1.5 0.79 5.23 DITTO 36.5 ± 0.8 98.3 ± 0.0 0.61 6.64 DPO 1.00 ± 0.0 1.50 ± 0.0 0.93 2.43 RLSpoofer 58.0 ± 0.7 79.5 ± 2.5 0.75 5.80

7.67 6.01 9.30 7.25

28.0 ± 0.2 48.3 ± 3.8 0.80 2.74 2.25 ± 0.0 100. ± 0.0 0.39 2.01 0.75 ± 0.0 13.8 ± 0.7 0.59 1.67 54.8 ± 0.5 88.5 ± 1.7 0.74 4.12

7.67 6.01 9.30 6.66

Distill 60.3 ± 0.3 88.8 ± 0.2 0.74 4.89 Qwen2.5-3B DITTO 9.50 ± 0.5 99.8 ± 0.0 0.48 6.16 DPO 1.25 ± 0.0 6.25 ± 0.0 0.87 2.16 -Instruct RLSpoofer 57.3 ± 0.3 80.5 ± 0.7 0.72 5.81

7.43 4.67 7.15 6.73

25.8 ± 1.0 42.5 ± 2.5 0.80 2.60 0.25 ± 0.3 100. ± 0.0 0.34 2.09 2.50 ± 0.0 2.50 ± 0.0 0.78 1.87 54.5 ± 0.5 76.5 ± 1.3 0.77 4.55

7.43 4.67 7.15 6.64

Distill 56.3 ± 0.8 84.0 ± 0.5 0.75 4.86 Llama3.2-3B DITTO 14.0 ± 0.5 99.3 ± 0.0 0.51 5.72 DPO 0.75 ± 0.0 16.0 ± 0.5 0.36 1.84 -Instruct RLSpoofer 55.3 ± 1.5 80.5 ± 2.5 0.76 6.12

7.58 4.72 1.58 6.53

26.0 ± 0.3 48.5 ± 1.3 0.77 2.55 1.00 ± 0.0 99.5 ± 0.0 0.35 1.81 7.75 ± 0.3 34.3 ± 0.8 0.60 1.52 52.0 ± 1.2 75.5 ± 3.0 0.72 4.60

7.58 4.72 1.58 6.53

rephrased distribution consists of detection scores from outputs paraphrased by our RLSpoofer-trained Qwen3-4B. The clear separation between the RLSpoofer-rephrased and human-like distributions confirms the attack’s effectiveness at stealing watermarks, exposing a fundamental vulnerability in current schemes. C.3

Effect of Local capacity mass

To more directly illustrate the effectiveness of local capacity mass in identifying feasible room for watermark injection during training, we conduct an analysis using the first checkpoint obtained by training Qwen3-4B on the 100-sample SWEET watermark training set. Specifically, we first use this checkpoint to rewrite all 100 training samples and collect the resulting paraphrased outputs. Based on these outputs, we compute the raw token-level reward signal, namely the unweighted P (x′t |ht ) log-likelihood ratio log Pwm , together with the corresponding local capacity mass under the ′ h (xt |ht ) human-like surrogate distribution Ph , which is approximated by the original Qwen3-4B reference model. We then compute the final reward used in RL training, rt , and average the raw reward, the capacity mass, and the final reward over all token occurrences across the 100 samples. After grouping tokens by part-of-speech tags, we obtain the statistics shown in Figure 2(b). As shown in Figure 2(b), the induced reward is small on numerals, which are typically critical for preserving semantic fidelity, but larger on adjectives, verbs, and adverbs, which admit greater lexical flexibility. This suggests that local capacity mass suppresses overly sharp rewards on semantically rigid tokens while amplifying rewards on more substitutable ones. Therefore, rather than uniformly favoring all watermark-preferred tokens, RLSpoofer concentrates watermark injection on positions with genuinely feasible semantic slack. Additionally, we verify that this weighting is not only intuitively aligned with semantic flexibility, but also critical to spoofing performance. Specifically, we replace the original weight 1 − pmax with either a uniform weight of 1 or the reverse weight pmax . We use the same hyperparameter settings as in Table 8 and report the detailed results in Table 12. As discussed in the main text, both replacements consistently degrade spoofing performance, indicating that effective watermark spoofing requires 23

Table 11: Spoof Success Rate (SSR, %), Spoof Rate (SR, %), P-SP, Perplexity (PPL), and GPT-score (GPTS) across models. For readability, we report SSR and SR as mean ± standard deviation over two random seeds. Bold indicates the best mean SSR for each model under each watermarking scheme. Higher SSR indicates stronger spoofing performance. PF Models

Methods

SSR

PMark

P-SP PPL GPTS

SSR

P-SP PPL GPTS

Distill 6.50 ± 0.2 7.25 ± 0.3 0.97 2.06 DITTO 5.50 ± 0.2 11.5 ± 0.3 0.79 2.21 Qwen3-0.6B DPO 2.50 ± 0.2 22.5 ± 0.4 0.57 1.56 RLSpoofer 33.3 ± 0.3 52.0 ± 0.2 0.66 2.19

5.91 2.34 5.31 5.28

20.0 ± 0.5 23.3 ± 0.8 0.90 2.31 11.8 ± 0.3 25.3 ± 0.5 0.63 2.38 6.25 ± 0.2 14.5 ± 0.5 0.63 2.99 29.5 ± 0.4 31.5 ± 0.3 0.92 2.03

5.91 2.34 5.31 5.80

Qwen3-1.7B

Distill 7.00 ± 0.3 8.00 ± 0.3 0.96 2.06 DITTO 7.50 ± 0.4 9.50 ± 0.2 0.86 2.27 DPO 4.25 ± 0.2 17.0 ± 0.4 0.58 2.31 RLSpoofer 29.0 ± 0.3 38.5 ± 0.3 0.73 4.10

7.24 4.43 9.12 6.21

20.3 ± 0.2 22.5 ± 0.2 0.90 2.34 16.5 ± 0.5 30.5 ± 0.6 0.68 2.48 22.5 ± 0.2 24.0 ± 0.3 0.93 2.32 29.5 ± 0.5 31.3 ± 0.4 0.90 2.29

7.24 4.43 9.12 6.93

Qwen3-4B

Distill 6.00 ± 0.2 6.50 ± 0.2 0.96 2.11 DITTO 3.50 ± 0.4 6.25 ± 0.2 0.88 2.47 DPO 5.25 ± 0.2 8.75 ± 0.3 0.88 2.33 RLSpoofer 62.0 ± 0.7 82.8 ± 0.5 0.77 2.58

7.67 6.01 9.30 6.60

21.5 ± 0.3 27.3 ± 0.5 0.92 2.39 16.5 ± 0.2 23.5 ± 0.2 0.71 2.50 17.5 ± 0.4 30.8 ± 0.5 0.68 5.06 36.3 ± 0.3 40.8 ± 0.4 0.91 2.05

7.67 6.01 9.30 7.25

Distill 6.50 ± 0.2 8.75 ± 0.4 0.93 2.04 Qwen2.5-3B DITTO 5.25 ± 0.2 12.3 ± 0.2 0.72 2.13 DPO 6.25 ± 0.2 12.3 ± 0.4 0.83 1.98 -Instruct RLSpoofer 50.3 ± 0.3 79.5 ± 0.3 0.68 1.79

7.43 4.67 7.15 6.61

22.3 ± 0.4 26.0 ± 0.4 0.91 2.01 11.3 ± 0.2 29.3 ± 0.5 0.56 2.00 24.3 ± 0.3 30.3 ± 0.3 0.95 2.22 30.3 ± 0.2 33.3 ± 0.3 0.89 1.94

7.43 4.67 7.15 7.41

Distill 8.75 ± 0.4 9.75 ± 0.2 0.93 4.49 Llama3.2-3B DITTO 6.50 ± 0.2 11.3 ± 0.2 0.79 4.90 DPO 6.25 ± 0.2 11.3 ± 0.4 0.67 2.00 -Instruct RLSpoofer 49.8 ± 0.4 60.5 ± 0.4 0.85 8.99

7.58 4.72 1.58 6.53

23.3 ± 0.6 27.8 ± 0.5 0.89 2.22 18.0 ± 0.4 34.2 ± 0.5 0.65 2.03 25.0 ± 0.3 30.3 ± 0.4 0.87 1.94 33.3 ± 0.2 34.3 ± 0.3 0.92 2.23

7.58 4.72 1.58 7.01

identifying positions with sufficient semantics-preserving redistribution room, rather than uniformly enlarging token-level rewards. Table 12: Results across different reward weighting. EWD

C.4

SWEET

Model

Weight

SSR

P-SP

SSR

P-SP

Qwen3-0.6B

1 − pmax 1 pmax

54.3 42.8 40.5

80.5 67.5 73.0

0.73 0.71 0.67

50.5 39.0 29.8

66.3 68.8 41.0

0.79 0.71 0.75

Qwen3-4B

1 − pmax 1 pmax

56.5 35.5 28.0

87.0 64.3 47.3

0.73 0.68 0.70

52.3 28.8 25.0

71.3 38.3 43.0

0.75 0.78 0.68

Effectiveness of conservative semantic rewards

We further study how the design of the sequence-level semantic reward affects spoofing performance. Specifically, we compare four variants: Min., Avg., Hum., and W.M., where Min. denotes the minimum of the semantic similarities to the human-written text and the watermarked rewrite, Avg. denotes their average, and Hum. and W.M. use only the human-written text or the watermarked rewrite as the semantic reference, respectively. We use the same training hyperparameter settings reported in Table 8, the results are shown in Table 13. We observe that Min. consistently achieves the best SSR across both attacker models and all three watermarking schemes. For Qwen3-0.6B, Min. yields the highest SSR and SR on EWD, SWEET, and PF, outperforming the other reward variants by a clear margin. A similar trend holds for Qwen3-4B, where Min. again gives the best SSR and SSR, most notably improving PF from 50.3% and 51% under Avg. and W.M. to 62%. These results suggest that a conservative semantic reward provides a better optimization target for watermark spoofing. We attribute this improvement to the fact that 24

successful spoofing must simultaneously preserve the meaning of the original text and remain close to the watermarked rewrite, which serves as the surrogate target distribution. Using only one reference may encourage one-sided alignment, while averaging the two scores can still mask weak alignment to one side. In contrast, Min. explicitly enforces both constraints and therefore yields more reliable semantic control during training. Table 13: Ablation on different semantic reward designs. EWD

C.5

SWEET

Model

Sem.R.

SSR

P-SP

SSR

P-SP

SSR

P-SP

Qwen3-0.6B

Min. Avg. Hum. W.M.

54.3 44.8 42.3 48.5

80.5 78.3 69.0 71.3

0.73 0.70 0.70 0.70

50.5 48.8 43.0 46.8

66.3 60.5 59.5 61.3

0.79 0.77 0.75 0.76

33.3 26.3 24.5 25.5

52.0 42.3 40.0 51.3

0.66 0.72 0.76 0.64

Qwen3-4B

Min. Avg. Hum. W.M.

56.5 47.3 47.5 45.5

87.0 78.0 80.0 62.3

0.73 0.70 0.68 0.75

52.3 34.3 47.0 48.5

71.3 49.0 59.3 61.8

0.75 0.76 0.76 0.78

62.0 50.3 48.5 51.0

82.8 76.3 69.3 70.8

0.77 0.78 0.76 0.75

Guidance of cross-entropy anchor

We examine the role of the cross-entropy (CE) anchor in stabilizing RLSpoofer training. We compare the RLSpoofer method (With Anchor) with two alternatives: removing the CE anchor (Without Anchor) and replacing RL training with supervised distillation on the same 100 training pairs (Distillation (100)). We adopt the training hyperparameters reported in Appendix B. Specifically, for Without Anchor setting, we set the learning rate to 1 × 10−4 to stabilize the training process. The results are reported in Table 14. We find that the CE anchor is critical for stable and effective spoofing. Removing it leads to substantial drops in SSR across all settings. For example, on Qwen3-4B, SSR decreases from 56.5% to 33.5% on EWD, and we observe similar degradations across all other settings. This shows that, even with carefully designed token-level and semantic rewards, unconstrained policy optimization can drift away from the desired watermarked rewriting distribution. Moreover, supervised distillation on the same 100 training pairs performs extremely poorly, with nearly zero SSR in all settings despite relatively high P-SP. This indicates that the gain of RLSpoofer does not come merely from imitation on limited data, but from the combination of RL-based distributional alignment and CE-based stabilization. Overall, these results confirm that the CE anchor is essential for maintaining a useful optimization trajectory while still allowing the policy to shift toward watermark-favored generations. Table 14: Ablation on the effect of the cross-entropy anchor. With Anchor

C.6

Without Anchor

Distillation (100)

Model

Scheme

SSR

P-SP

SSR

P-SP

SSR

P-SP

Qwen3-0.6B

EWD SWEET PF

54.3 50.5 33.3

80.5 66.3 52.0

0.73 0.79 0.66

29.3 27.3 12.3

45.5 50.0 28.3

0.69 0.70 0.72

0.25 0.25 0.00

0.50 0.25 0.25

0.86 0.84 0.87

Qwen3-4B

EWD SWEET PF

56.5 52.3 62.0

87.0 71.3 82.8

0.73 0.75 0.77

33.5 26.8 25.5

50.3 44.3 45.3

0.73 0.78 0.70

0.00 0.25 0.00

0.25 0.25 0.25

0.93 0.94 0.84

Training set size ablation

We conduct an empirical study to examine the sensitivity of RLSpoofer to training set size. Specifically, we train the models on data generated by watermarked Llama3.1-8B-Instruct under three watermarking schemes, EWD, SWEET, and PF, using training sets containing 50, 100, and 200 samples, where each sample is standardized to 500 tokens. To ensure a comparable number of update steps across different training set sizes, we train for 20 epochs when using 50 samples and for 5 25

epochs when using 200 samples. For the 100-sample setting, we use the standard training schedule described in Table 8. We adopt the same remaining hyperparameters and report the results in Table 15. We observe that RLSpoofer achieves strong spoofing performance with only 50 training samples. On the logit-based watermarks EWD and SWEET, increasing the training set generally improves SSR for both attacker models, with the best performance typically obtained using 200 samples. In contrast, on the sampling-based distortion-free PF watermark, enlarging the training set does not lead to consistent gains. For both Qwen3-0.6B and Qwen3-4B, the best PF performance is achieved with 100 samples, while using 200 samples leads to a noticeable drop in SSR. These results suggest that RLSpoofer is sample-efficient and that relatively small training sets are already sufficient to expose strong spoofing vulnerabilities, especially for the more challenging distortion-free setting. Table 15: Effect of training set size on spoofing performance. EWD

C.7

SWEET

Model

Samples

SSR

P-SP

SSR

P-SP

SSR

P-SP

Qwen3-0.6B

50 100 200

44.3 54.3 56.8

77.0 80.5 85.0

0.72 0.73 0.74

42.5 50.5 51.3

65.5 66.3 71.5

0.74 0.79 0.79

12.3 33.3 20.5

19.3 52.0 37.0

0.72 0.66 0.76

Qwen3-4B

50 100 200

46.5 56.5 58.5

79.0 87.0 88.0

0.73 0.73 0.74

43.0 52.3 54.3

72.0 71.3 65.3

0.70 0.75 0.77

20.8 62.0 25.3

44.0 82.8 36.3

0.68 0.77 0.81

Sensitivity to surrogate model selection

We examine the sensitivity of RLSpoofer to the choice of surrogate reference model. Specifically, for each attacker, we compare two surrogate choices for approximating the target distributions: Qwen3-0.6B and the attacker itself (Self). The results are reported in Table 16. We find that surrogate choice has a limited impact on Qwen3-4B, where the two settings yield similar performance on EWD and SWEET, and Self performs slightly better on PF. In contrast, Qwen3-8B is much more sensitive: using Qwen3-0.6B consistently gives substantially higher SSR than Self across all three watermarking schemes. In particular, SSR improves from 45.3% to 60.5% on EWD, from 35% to 54.3% on SWEET, and from 29.8% to 58.5% on PF. These results suggest that surrogate selection becomes increasingly important as attacker capacity grows. Table 16: Sensitivity of RLSpoofer to surrogate model choice. Qwen3-0.6B

C.8

Self

Attacker

Scheme

SSR

P-SP

SSR

P-SP

Qwen3-4B

EWD SWEET PF

56.5 53.3 56.5

85.8 73.0 83.8

0.73 0.75 0.76

56.5 52.3 62.0

87.0 71.3 82.8

0.73 0.75 0.77

Qwen3-8B

EWD SWEET PF

60.5 54.3 58.5

78.0 75.5 79.3

0.75 0.75 0.78

45.3 35.0 29.8

67.8 65.5 33.8

0.71 0.68 0.86

Cross watermark transferability of RLSpoofer

We further evaluate the zero-shot cross-watermark transferability of RLSpoofer. Specifically, we directly reuse the RLSpoofer-trained Qwen3-4B models obtained in Section 4.2, and test each model on the other watermarking schemes without any further tuning. The results are reported in Table 17. We observe that transferability is not strictly constrained by the watermark family. In particular, the KGW-style logit-based watermarks EWD and SWEET exhibit substantial bidirectional transfer: training on EWD achieves 50.0% SSR on SWEET, while training on SWEET achieves 47.8% SSR on EWD. By contrast, interactions involving the sampling-based PF watermark are notably directional. Training on PF transfers effectively to EWD, achieving 52.5% SSR, whereas the reverse direction 26

is much weaker, with EWD-to-PF achieving only 6.0% SSR. Similarly, PF-to-SWEET almost completely fails, yielding only 0.25% SSR. These results suggest that transferability is determined not only by broad watermark family similarity, but also by more intricate and directional overlaps in the vulnerabilities induced by different watermarking schemes. Table 17: Cross-watermark transferability of RLSpoofer on Qwen3-4B. Test

C.9

EWD

SWEET

Train

SSR

P-SP

SSR

P-SP

SSR

P-SP

EWD SWEET PF

56.5 47.8 52.5

87.0 58.8 80.5

0.73 0.78 0.75

50.0 52.3 0.25

64.8 71.3 1.00

0.76 0.75 0.61

6.00 47.0 62.0

10.0 70.5 82.8

0.78 0.72 0.77

Limitations and Border Impact.

Limitations. Although RLSpoofer demonstrates efficacy in evaluating the resilience of watermarks against spoofing attacks, it exhibits several notable limitations. Primarily, our method does not directly optimize the true detector-level spoofing objective. Instead, it relies on a tractable distributional surrogate together with reference-model-based local preference proxies for human-like and watermark-conditioned generation. Accordingly, our theory should be interpreted as motivating the reward design rather than providing a detector-agnostic guarantee of spoof success. In addition, the practical effectiveness of the method depends on the quality of the surrogate signals and can be sensitive to design choices such as reward weights, cross-entropy anchoring, and the choice of reference model. Border Impact. Despite these limitations, RLspoofer provides a lightweight and practical stress test for evaluating whether current watermarking schemes remain reliable under realistic black-box attacks, which may help the community design more robust provenance and detection mechanisms. At the same time, because our method can expose concrete weaknesses in existing watermarking systems, it also carries dual-use risk if deployed irresponsibly. To mitigate this concern, we focus on evaluation rather than misuse, provide sufficient implementation details for scientific assessment. C.10

Responsible Release

This work studies an attack method for evaluating the spoofing resilience of LLM watermarking schemes, and therefore carries potential dual-use risk. To mitigate misuse, we do not release the attack code during the review period. We provide implementation details, hyperparameter settings, and references sufficient for scientific assessment, while withholding a directly reusable attack pipeline. If the paper is accepted, we will carefully consider the form and scope of release to balance research transparency with misuse risk. Our goal is to support the evaluation of watermark robustness and the development of more spoofing-resistant defenses, rather than facilitate malicious deployment.

Licenses of Existing Assets

We summarize the existing external assets used in this work, including codebases, datasets, and models, together with their publicly stated licenses or access terms. We cite the original papers and official repositories or model cards throughout the paper. When an asset is distributed under a model-specific license, we use it in accordance with the corresponding license agreement and acceptable use policy. The results are presented in Table 18.

Table 18: Existing assets used in this work and their publicly stated licenses or access terms. Asset

Type

License / Terms of Use

PMark [23]

Code

MarkLLM [24] LLaMA-Factory [37] DITTO [13] TRL [38] OpenR1 [39] C4 (RealNewslike subset) [25] Reddit WritingPrompts [26] LFQA [27] MMW benchmark [28] Qwen3 (0.6B, 1.7B, 4B, 8B) [1] Qwen2.5-3B-Instruct [21]

Code Code Code Code Code Dataset Dataset Dataset Dataset / Benchmark Model Model

Llama3.1-8B-Instruct [2]

Model

Llama3.2-3B-Instruct [2]

Model

No explicit standard license found; research reproduction only Apache-2.0 Apache-2.0 Apache-2.0 Apache-2.0 Apache-2.0 ODC-BY CC BY 3.0 Apache-2.0 Apache-2.0 Apache-2.0 Qwen Research License Llama 3.1 Community License; acceptable use policy applies Llama 3.2 Community License; acceptable use policy applies

Record · ID 10256 · SHA-256 15cdb279a5e6021e
Conceptio Open Knowledge Archive — every document is proof-bundled with source, license, and retrieval metadata.