Toward Stronger Code Watermarking: A Grammar-Driven Approach to Optimizing the Trade-off Between Quality and Detectability

Toward Stronger Code Watermarking: A Grammar-Driven Approach to Optimizing the Trade-off Between Quality and Detectability

1

Licheng Yu1 , Aiwei Liu2 , Songze Li1 * Southeast University 2 Tsinghua University

{lichengyu, songzeli}@seu.edu.cn, liuaiwei20@gmail.com

arXiv:2607.10210v1 [cs.CR] 11 Jul 2026

Abstract With the rapid development of Large Language Models (LLMs), text watermarking has emerged as a crucial technique for identifying machine-generated content. However, directly applying existing logits-based watermarking methods to code generation remains challenging, since the low-entropy nature of code exacerbates the trade-off between code quality and watermark detectability. In this paper, we propose a novel code watermarking approach called Grammar-Driven Watermark (GDW) for LLMs. GDW preserves syntactic validity through a grammar-guided threelevel masking mechanism and injects watermark signals via structural role-aware modulation, assigning a stronger bias to contentbearing tokens while applying a more conservative bias to syntax-critical tokens. Aligning with the generation process, we further design a role-aware weighted detection statistic to improve detectability. Experiments across multiple programming languages, models, and decoding strategies show that GDW establishes a stronger quality-detectability trade-off frontier than existing methods, while maintaining robustness against variable-renaming attacks.

1

Logits-based Watermark

Entropy-based Watermark

Grammar-Driven Watermark

Red-Green Vocab Partition

Red-Green Vocab Partition

Red-Green Vocab Partition

Syntax-Agnostic

Syntactic Validity Check

Without Code-Specific Design

Entropy-Guided Strategy

Structural Role Modulation

Uniform Watermark Strength to All Green Tokens

Uniform Watermark Strength to High-Entropy Green Tokens

Hierarchical Watermark Strength to Syntax-Compliant Green Tokens

+ Standard Detection

+ Role-Aware Weighted Detection Unclear Trade-off Frontier

Optimized Trade-off Frontier

Figure 1: Comparison among existing logits-based watermark, entropy-based watermark, and our GrammarDriven Watermark in terms of the trade-off between watermark detectability and code quality. Our proposed method GDW achieves a better trade-off frontier.

Watermarking has emerged as a promising solution for identifying machine-generated content by embedding imperceptible but statistically detectable signals into model outputs. One dominant paradigm for LLM text watermarking is the logitsbased watermark, which injects signals by modifying the model’s output logits (Kirchenbauer et al., 2023a). Specifically, the vocabulary is pseudorandomly partitioned into a ‘green list’ and a ‘red list’ at each decoding step. By adding a constant bias to the logits of green-list tokens, the model is encouraged to select tokens within the green list, which can later be identified via a statistical z-test. However, while logits-based watermarking methods have shown effectiveness in natural language generation tasks, their application to code is hindered by the low-entropy nature of programming languages. Code is low-entropy text, where the model’s output logits are highly concentrated due to the rigid syntax. In such settings, naively injecting watermark bias into low-entropy tokens can significantly distort the generation process. A watermark bias that is too strong may degrade the code’s functionality or readability, while a weaker watermark bias might be difficult to detect.

Introduction

In recent years, large language models (LLMs) have become indispensable tools for code generation and programming assistance, powering widely used AI coding assistants such as GitHub Copilot (Microsoft and OpenAI, 2021), Cursor (Anysphere, 2023) and Claude Code (Anthropic, 2025). However, as LLM-generated code becomes increasingly integrated into open-source projects and industrial software, concerns about code provenance and potential copyright infringement have increased substantially (Dey et al., 2019). Therefore, algorithms that can effectively identify machine-generated code have become crucial. * Corresponding author.

1

Syntax-Critical Tokens

1.2

Content-Bearing Tokens

watermark bias for syntax-critical tokens. In addition, we further introduce a Three-Level Masking Mechanism to enforce syntactic validity during watermark injection. In this work, we introduce Grammar-Driven Watermark (GDW), a novel watermarking method that optimizes the trade-off between code quality and watermark detectability in code generation. Our method integrates Context-Free Grammar (CFG) into the decoding process to ensure syntactic validity at each step, while applying a structural roleaware modulation. Correspondingly, we design a role-aware weighted detection statistic aligned with the generation mechanism. Note that our method operates entirely at inference time and does not require training or fine-tuning. In summary, our contributions are as follows:

1.014

Mean Token Entropy

1.0

0.8

0.952

0.902

0.744 0.649

0.6

0.554

0.580

0.419

0.4

0.2

0.0

HumanEval

MBPP+

HumanEvalPack-Java HumanEvalPack-Go

Figure 2: Mean token entropy of syntax-critical tokens and content-bearing tokens measured with Qwen2.5Coder-3B across Python, Java, and Go. This observation motivates our structural role-aware modulation strategy.

To address this, researchers propose entropybased watermarking methods (Lee et al., 2024; Lu et al., 2024), applying watermark bias only to tokens with entropy exceeding a specific threshold. There remains room for improvement in the tradeoff between watermark strength and code quality. We argue that effective watermarking for code should be structure-aware. Crucially, not all tokens in code contribute equally to structural integrity or functional correctness. Syntax-critical tokens such as keywords, operators, and delimiters govern grammatical correctness, while content-bearing tokens such as identifiers and literals often exhibit greater semantic redundancy. This observation suggests that watermark injection should not be applied uniformly across tokens, nor should lowentropy tokens be entirely excluded. Instead, the watermark strength should be modulated in a manner that respects both grammatical constraints and token roles. As illustrated by Figure 1, existing watermarking schemes face inherent trade-offs during low-entropy code generation, making an effective solution imperative.

• We propose a grammar-driven watermarking method called GDW, tailored for LLM-based code generation task. GDW supports multiple programming languages, including Python, Java, and Go. • We design a corresponding weighted detection method that aligns with the generation process and improves statistical detectability under low-entropy conditions. • Experiments show that GDW achieves a more favorable trade-off between code quality and watermark detectability compared to existing baselines.

2

Preliminaries

2.1

LLM Text Generation

Consider an auto-regressive Large Language Model M with a vocabulary V. Given a prompt or prefix sequence x1:t = (x1 , . . . , xt ), the LLM will generate a logits vector lt ∈ R|V| at decoding step t. At step t + 1, the probability distribution PLM over the vocabulary V is computed by applying the softmax function to the logits. The probability for selecting the next token xt+1 is given by: (t+1) (t+1) P|V | (t+1) pk = exp(lk )/ i=1 exp(li ). During inference, the LLM samples tokens from the probability distribution PLM using decoding strategies such as top-k, top-p, or beam search. A sequence of length Tgen is generated by iteratively sampling tokens xt+1 ∼ P (xt+1 |x1:t ) and appending them to the prefix until a termination criterion is met.

We further analyze the mean token entropy of syntax-critical tokens and content-bearing tokens across Python, Java, and Go using Qwen2.5-Coder3B. As shown in Figure 2, syntax-critical tokens consistently exhibit lower entropy than contentbearing tokens across all three languages. This suggests that syntax-related tokens are generally more deterministic, whereas content-bearing tokens provide greater flexibility for watermark injection. Motivated by this observation, our method applies stronger watermark modulation to contentbearing tokens, while employing a conservative 2

𝑴𝒘𝒎,𝒕 1 Prefix Code

1

0

0

1

0

1

0

Watermark Greenlist Mask

Hash Function

𝑴𝒆𝒍𝒊𝒈,𝒕 0

1

0

1

1

0

0

1

0

0

0

Eligibility Mask

𝑴𝒔𝒚𝒏,𝒕 0

1

0

0

1

Syntactic Validity Mask Context-Free Grammars

Incremental Parser Syntax-Critical Tokens

0 Content-Bearing Tokens

Vocabulary

Grammar-Constrained Hierarchical Modulation

𝑴𝒓𝒐𝒍𝒆

e.g. def, if, while

0

1

0

1

0

1

1

Structural Role Mask

e.g. variable name, strings

Watermarked Logits

Conservative Bias δ

Role Function

𝑟(𝑣)

for SC Tokens

Stronger Bias λ·δ for CB Tokens

Select Eligible Tokens for Watermark (Fall into the Greenlist and Pass the Syntactic Validity Check). -- Section 3.2.1

Tokens in Greenlist

Modulate the Watermark Strength Based on Structural Roles and Role Function. -- Section 3.2.2

Syntactically Valid Tokens

Syntax-Critical Tokens

Modulate the Original Logits to Obtain the Watermarked Logits. -- Section 3.2.3

Watermark-Eligible Tokens

Content-Bearing Tokens

Figure 3: A Grammar-Driven Watermark Approach for LLM-Generated Code

2.2

3.1

LLM Text Watermarking

The general watermarking scheme used in this paper is called KGW (Kirchenbauer et al., 2023a). Given a prefix sequence x1:t−1 , the watermark process can be described as follows. At each decoding step, a pseudorandom number generator is seeded using the hash of the preceding token xt−1 to generate a random permutation of V. The vocabulary is then split into a green list Gt ⊂ V of size γ|V| and a red list Rt ⊂ V of size (1−γ)|V|, where γ ∈ (0, 1) is the green-list ratio. Watermark signals are injected by adding a small bias δ to the logits of tokens in Gt , making them more likely to be selected during generation. The detection process is formulated as a hypothesis test. Specifically, pthe detector computes a z-score z = (|s|G −γT )/ γ(1 − γ)T , where |s|G is the number of green-list tokens in a text of length T . The text is identified as watermarked when the z-score exceeds a predefined threshold.

3

Watermark Generation Framework

We define the overall generation process of our GDW method in this section. The watermark injection process can be decomposed into the following three steps: 1. Guided by the prompt and the preceding context x1:t−1 , GDW imposes a dual constraint to derive a set of eligible tokens that are both syntactically admissible and assigned to the watermark greenlist. Implementation details are elaborated in Section 3.2.1. 2. Modulate the watermark bias through structural role mask and role function r(v). We will provide more details in Section 3.2.2. 3. Modify original logits using a grammarconstrained hierarchical modulation strategy (integrating the constraints and biases from previous steps), introduced in Section 3.2.3.

Methodology

In this section, we introduce GDW, a grammardriven watermarking method for code generation. In code watermarking, a critical challenge lies in optimizing the trade-off between watermark detectability and the quality of the generated code. Our method is proposed to explicitly mitigate this conflict. We first introduce the overall watermark generation process in Section 3.1 and then detail the grammar-driven watermark injection mechanism in Section 3.2, followed by the detailed watermark detection in Section 3.3.

By iteratively executing these three steps across T decoding steps, GDW will produce the watermarked code that maintains quality while ensuring high detectability. The complete watermark embedding process is shown in the Figure 3.

3.2

Grammar-Driven Watermark Injection

In this section, we detail the implementation of the three steps introduced in Section 3.1. 3

3.2.1 Three-Level Masking Mechanism To inject watermark while preserving the quality of the generated code, we construct a three-level masking mechanism over the vocabulary at each decoding step. This mechanism comprises three binary masks, which jointly determine whether a candidate token is eligible for watermark injection.

serve that tokens with different structural functions exert varying degrees of influence on the executability and functional integrity of the generated code. As a result, we conduct a detailed analysis of the three target programming languages (Python, Java and Go) and categorize tokens into two groups: syntax-critical tokens, such as keywords, operators, and delimiters, which constitute the structural skeleton of the program; and content-bearing tokens, including identifiers and literals, which carry specific logic. To formally incorporate this distinction into our modulation process, we define a structural role mask Mrole ∈ {0, 1}|V| to distinguish between the (v) two categories. Specifically, we assign Mrole = 1 (v) to tokens in the syntax-critical pool and Mrole = 0 to those in the content-bearing pool. This mask serves as the foundation for the subsequent hierarchical modulation strategy.

Watermark Greenlist Mask. Following the KGW framework (Kirchenbauer et al., 2023a), at each decoding step t, we define the watermark greenlist mask Mwm,t ∈ {0, 1}|V| by partitioning the vocabulary based on a pseudorandom hash of the preceding token xt−1 . Tokens assigned to the greenlist are marked as 1, while those in the redlist are marked as 0. Syntactic Validity Mask. For a program to execute successfully, its syntactic structure must be valid. Consequently, to preserve the syntactic integrity of generated code, GDW integrates a filtering mechanism guided by CFG. By taking a CFG represented with extended Backus–Naur form (EBNF) rules, GDW ensures that the LLM output remains syntactically valid. Given the current prefix x1:t−1 , we define At ⊆ V as the set of syntactically admissible next tokens that satisfy the predefined CFG rules C. At each decoding step t, we employ an incremental parser to derive the current admissible set At . The watermark bias will be applied exclusively to tokens within the admissible set At , concentrating the watermark signal along valid syntactic paths. This ensures the syntactic correctness of the generated code. Based on this admissible set At , we construct a syntactic validity mask Msyn,t ∈ {0, 1}|V| . At (v) decoding step t, Msyn,t = 1 if the token v is syntactically admissible given the current prefix, and 0 otherwise. This mask ensures that only syntactically valid tokens are considered for watermarking.

Role Function. The core motivation for distinguishing token roles lies in the disparate sensitivity of code structures to logit perturbation. Since syntax-critical tokens form the grammatical backbone of a program, excessive perturbation to these tokens may easily cause program failure. In contrast, content-bearing tokens reside in higherentropy regions, where the model has greater predictive flexibility and watermark signals can be embedded more aggressively. To exploit this flexibility while preserving structural stability, we introduce a role function r(v) that scales the watermark (v) strength based on Mrole . For each token v ∈ V, r(v) is defined as: ( 1, if v ∈ Vsyntax, r(v) = λ, if v ∈ Vcontent.

where we assign λ > 1 to content-bearing tokens to amplify the watermark signal, while restricting a more conservative watermark bias to syntax-critical tokens.

Eligibility Mask. To consolidate these constraints, we combine the watermark greenlist mask and the syntactic validity mask to form an eligibility mask Melig,t , which identifies tokens that are eligible for watermark injection and syntactically valid at the current decoding step. Melig,t = Mwm,t ⊙ Msyn,t .

(2)

3.2.3

Grammar-Constrained Hierarchical Modulation

Finally, by integrating the previously derived eligibility mask and structural role-aware biases, we modulate the original logits to obtain the final watermarked distribution. The watermarked logits lt′ at step t is computed as:

(1)

3.2.2 Structural Role-Aware Modulation Structural Role Mask. Beyond considering the impact of syntactic validity on code quality, we ob-

lt′ = lt +Melig,t ⊙ Mrole +λ·(1−Mrole ) ·δ. (3) 4

This strategy ensures that, under the prerequisite of guaranteeing syntactic validity, watermark signals are injected more conservatively into syntaxcritical tokens that are essential to the structural integrity of the code, while stronger biases are applied to content-bearing tokens, optimizing the trade-off between watermark detectability and code quality. The pseudocode of the proposed GDW injection procedure is provided in Algorithm 1.

watermarked text from unwatermarked text. Given a generated code sequence x = {x1 , . . . , xT }, we follow the standard practice of discarding the initial token used for seeding. For each subsequent position t > 1, we define an indicator variable: It = 1[xt ∈ Gt ],

where Gt denotes the green list at step t, which is recovered using a hash of the preceding token xt−1 . To ensure symmetry between generation and detection, we assign a contribution weight wt to each token xt , governed by the same role function r(·) employed during watermark injection:

Algorithm 1 Watermark Injection Input: Prefix tokens x1:t−1 , original logits lt , green-list ratio γ ∈ (0, 1), base bias δ > 0, role function r(·), CFG grammar rules C, syntax-critical vocabulary Vsyntax , content-bearing vocabulary Vcontent . Output: Watermarked logits lt′ . 1: Compute a hash of token xt−1 and use it to pseudo-randomly partition the vocabulary V into greenlist Gt and redlist Rt . 2: Construct the watermark greenlist mask Mwm,t ∈ {0, 1}|V| : ( 1, if v ∈ Gt , (v) Mwm,t = 0, otherwise

( 1, if xt ∈ Vsyntax , wt = r(xt ) = λ, if xt ∈ Vcontent

PT P wt It − γ Tt=2 wt t=2 z= q . P γ(1 − γ) Tt=2 wt2

At ← I NCREMENTAL PARSER(C, x1:t−1 ). 4: Construct the syntactic validity mask

(6)

A sequence is classified as watermarked if z > τ , where τ is a predefined threshold. The pseudocode of the proposed detection algorithm is provided in Algorithm 2.

Msyn,t ∈ {0, 1}|V| : ( 1, if v ∈ At , (v) Msyn,t = 0, otherwise 5: Compute the eligibility mask using Eq.1. 6: Construct the structural role mask

4

Experiments

4.1

Experimental Settings

Models. We explore the performance of the watermark using three public LLMs: StarCoder2-3B (Lozhkov et al., 2024), Qwen2.5-Coder-3B (Hui et al., 2024), and Qwen3-4B (Yang et al., 2025). We also report additional results with Qwen2.5Coder-7B (Hui et al., 2024) in Appendix A, which further demonstrate the generalizability of our approach across different model scales.

if v ∈ Vsyntax , if v ∈ Vcontent

7: Calculate role-aware watermark strength

adjustment using the role function Eq.2. 8: Combine all the masks and update the logits

according to Eq.3.

Datasets. Our method is evaluated across three programming languages: Python, Java, and Go. We use HumanEval (Chen et al., 2021) and MBPP+ (Liu et al., 2023b, 2024) for Python, HumanEvalPack (Muennighoff et al., 2023) for Java and Go. For each sample, we use human-written prefixes or questions as prompts, generating T = 200 tokens.

9: return lt′

3.3

(5)

where wt = 1 corresponds to syntax-critical tokens and wt = λ to content-bearing tokens, consistent with the watermark generation step. We define the detection statistic as a weighted z-score:

3: Obtain the syntactically admissible set

Mrole ∈ {0, 1}|V| : ( 1, (v) Mrole = 0,

(4)

Grammar-Driven Watermark Detection

Watermark detection is a binary classification task that assesses the detector’s ability to distinguish 5

Qwen2.5-Coder-3B Methods

HumanEval

KGW SWEET EWD STONE CodeIP SynthID-Text GDW

Qwen3-4B

MBPP+

HumanEval

StarCoder2-3B

MBPP+

HumanEval

MBPP+

Sample

Beam

Sample

Beam

Sample

Beam

Sample

Beam

Sample

Beam

Sample

Beam

0.745 0.784 0.771 0.666 0.805 0.702 0.811

0.832 0.742 0.774 0.677 0.730 0.692 0.847

0.735 0.757 0.763 0.670 0.767 0.676 0.787

0.857 0.768 0.789 0.670 0.768 0.668 0.876

0.714 0.766 0.796 0.666 0.722 0.705 0.830

0.815 0.756 0.831 0.671 0.802 0.701 0.847

0.784 0.719 0.797 0.669 0.809 0.744 0.828

0.840 0.774 0.852 0.683 0.796 0.700 0.875

0.691 0.722 0.712 0.681 0.725 0.700 0.743

0.848 0.743 0.795 0.686 0.760 0.679 0.862

0.750 0.761 0.743 0.668 0.766 0.732 0.833

0.837 0.784 0.820 0.683 0.739 0.754 0.855

Table 1: Area Under the Trade-off Curve (AUTC) of different watermarking methods on HumanEval and MBPP+. The best trade-off performance in each column is highlighted in bold. +XPDQ(YDO4ZHQ%

0%334ZHQ&RGHU%



 

 



3DVV#EHWWHU





+XPDQ(YDO4ZHQ%











)6FRUHEHWWHU



 









3DVV#EHWWHU



+XPDQ(YDO6WDU&RGHU%





•







3DVV#EHWWHU

KGW





 





3DVV#EHWWHU





3DVV#EHWWHU

SWEET







3DVV#EHWWHU











  

▲

EWD





3DVV#EHWWHU

⋆



 









3DVV#EHWWHU







3DVV#EHWWHU





CodeIP





3DVV#EHWWHU





 

 

   

 



0%334ZHQ&RGHU%





STONE











 



 



0%336WDU&RGHU%











0%334ZHQ%

 



 







■







+XPDQ(YDO4ZHQ&RGHU%







 











 

)6FRUHEHWWHU









)6FRUHEHWWHU





)6FRUHEHWWHU



)6FRUHEHWWHU





)6FRUHEHWWHU







)6FRUHEHWWHU







)6FRUHEHWWHU



)6FRUHEHWWHU

)6FRUHEHWWHU

)6FRUHEHWWHU

0%336WDU&RGHU% 







)6FRUHEHWWHU

0%334ZHQ%

+XPDQ(YDO4ZHQ&RGHU%

+XPDQ(YDO6WDU&RGHU% 



 





3DVV#EHWWHU





SynthID-Text





♦





3DVV#EHWWHU





GDW (Ours)

Figure 4: Pass@1-F1 curves for watermarked code generation. We used two different decoding strategies: sampling (top row) and beam search (bottom row). GDW achieves a better quality-detectability trade-off than other methods.

Algorithm 2 Watermark Detection Input: Generated code sequence x1:T , green-list ratio γ ∈ (0, 1), role function r(·), detection threshold τ . Output: Detection result d ∈ {True, False}. 1: for t = 2 to T do 2: Compute a hash of the preceding token xt−1 and use it as the seed for a pseudorandom number generator. Recover the green list Gt and the red list Rt . 3: 4: Set indicator It = 1 if xt ∈ Gt , else It = 0. 5: Assign weight wt ← r(xt ) according to its structural role using Eq.5. 6: end for 7: Calculate the weighted z-score z with Eq.6. 8: if z > τ then 9: return True // WATERMARKED C ODE. 10: else 11: return False // U NWATERMARKED C ODE. 12: end if

Therefore, we adopt pass@1 (Chen et al., 2021) as the main metric for code quality evaluation. We additionally use the F1 score at a fixed false positive rate to measure watermark detectability. To quantify the trade-off between quality and detectability, we introduce a metric called Area Under the Trade-off Curve (AUTC), defined as the normalized area under the Pass@1-F1 trade-off curve over the common quality interval [Qstart , Qend ] shared by all methods. Formally, AUTC is computed as:

AUTC =

1 Qend − Qstart

Z Qend F1(q)dq

(7)

Qstart

where q denotes the pass@1 quality and F1(q) represents the corresponding watermark detectability. A higher AUTC indicates a more favorable quality-detectability trade-off. Baselines. We compare GDW with six representative watermarking methods, including the pioneering logits-based watermark KGW (Kirchenbauer et al., 2023a), entropy-based watermarking

Metrics. In code generation tasks, executability and functional correctness are the primary concerns. 6

Qwen2.5-Coder-3B Methods

KGW SWEET EWD STONE CodeIP SynthID-Text GDW

StarCoder2-3B

HumanEvalPack-Java

HumanEvalPack-Go

HumanEvalPack-Java

HumanEvalPack-Go

Sample

BeamSearch

Sample

BeamSearch

Sample

BeamSearch

Sample

BeamSearch

0.768 0.806 0.865 0.675 0.687 0.721 0.887

0.870 0.781 0.854 0.781 0.750 0.813 0.910

0.911 0.842 0.881 — 0.726 0.725 0.951

0.910 0.836 0.879 — 0.768 0.729 0.949

0.797 0.813 0.834 0.679 0.667 0.783 0.871

0.857 0.828 0.859 0.714 0.682 0.702 0.900

0.725 0.779 0.770 — 0.668 0.689 0.831

0.861 0.721 0.748 — 0.723 0.680 0.881

Table 2: Area Under the Trade-off Curve (AUTC) of different watermarking methods on HumanEvalPack. Table 3: AUTC of different λ on HumanEvalPack-Java using Qwen2.5-Coder-3B with beam search.

methods SWEET (Lee et al., 2024) and EWD (Lu et al., 2024), the code watermarking methods STONE (Kim et al., 2025) and CodeIP (Guan et al., 2024), as well as the non-red-green watermarking method SynthID-Text (Dathathri et al., 2024). Decoding Strategies. We evaluate watermarking performance under two decoding strategies: sampling and beam search. Beam search is adopted because deterministic decoding often enhances the stability and reliability of code generation, while its interaction with code watermarking methods remains underexplored in prior work.

λ=2

λ=3

λ=4

λ=5

AUTC

0.942

0.922

0.910

0.897

4.3

Effect of the role scaling factor λ

We further study the effect of the role scaling factor λ, which controls the relative watermark strength assigned to content-bearing tokens. Specifically, we vary λ from 2 to 5 and visualize the resulting trade-off curves of GDW in Figure 5a, with the corresponding AUTC scores reported in Table 3. Overall, GDW remains relatively stable across different λ values, meaning that the proposed structural role-aware modulation is not overly sensitive to this hyperparameter. Among all settings, λ = 2 achieves the most favorable trade-off, showing that moderately increasing the watermark bias on content-bearing tokens is beneficial, whereas overly aggressive modulation can harm the overall balance between detectability and code quality.

Hyperparameters. For red-green watermarking methods, we set the green-list ratio γ to 0.5, the zscore threshold τ to 4.0, and report the F1 score at a fixed false positive rate of 1%, following Kirchenbauer et al., 2023a. To comprehensively evaluate the trade-off between code quality and watermark detectability, we conduct experiments under multiple watermark strength values δ within the range (0, 5.0] for each method. We set λ to 2.0 for GDW. 4.2

λ

Overall Trade-off Performance 4.4

We evaluate GDW and six baselines across benchmarks covering Python, Java, and Go. Tables 1 and 2 report the AUTC results on Python benchmarks and multilingual HumanEvalPack benchmarks respectively. GDW achieves the highest AUTC across all settings, which shows that GDW establishes a more favorable trade-off frontier between code quality and watermark detectability. Figure 4 further visualizes the trade-off curves. Compared with all baselines, GDW consistently maintains a superior trade-off frontier, with its curves remaining consistently positioned above all baselines. Additional visualizations are provided in Appendix B.

Beam Search vs Sampling

As shown in Figure 8, beam search consistently exhibits a more favorable quality-detectability tradeoff than stochastic sampling in code generation tasks. We attribute this to the inherently lowentropy nature of code generation, which makes it tend to favor decoding strategies with lower diversity. We further validate the generalizability of this conclusion across various baselines, with supplementary results provided in Appendix C. 4.5

Ablation Study

To better understand the contribution of each component in GDW, we conduct an ablation study by 7

0.95 0.90

0.95

Detection F1 Score (better )

Detection F1 Score (better )

1.00

0.90 0.85 0.80 0.75 =2 =3 =4 =5

0.70 0.65 0.1

0.85 0.80 0.75

GDW (Full) w/o Weighted Detection w/o Syntactic Validity Mask w/o Structural Role Mask

0.70

0.2

0.3

0.4

0.5

0.6

Code Quality Pass@1 (better )

0.65

0.7

0.075 0.100 0.125 0.150 0.175 0.200 0.225 0.250 0.275

Code Quality Pass@1 (better )

(a)

(b)

Figure 5: Subfigure (a) illustrates the trade-off curves of GDW for different λ values. Subfigure (b) shows the trade-off curves of GDW and its ablated variants on HumanEval and StarCoder2-3B, using beam search decoding. Table 4: Ablation study of GDW on HumanEval and StarCoder2-3B, using beam search decoding.

Table 5: Quality-detectability trade-off under variable renaming attack (measured by AUTC).

Variant

AUTC ↑

Method

GDW (Full) w/o Weighted Detection w/o Syntactic Validity Mask w/o Structural Role Mask

0.860 0.832 0.817 0.800

KGW SWEET EWD STONE CodeIP SynthID-Text GDW

Before Attack

After Attack

0.832 0.742 0.774 0.677 0.730 0.692 0.847

0.811 0.685 0.758 0.673 0.696 0.684 0.819

progressively removing the Structural Role Mask, the Syntactic Validity Mask, and the weighted detection module. As shown in Table 4, the full GDW method achieves the highest AUTC, indicating the best overall trade-off between code quality and watermark detectability. Removing either the Structural Role Mask or the Syntactic Validity Mask leads to noticeable performance degradation, demonstrating the importance of both grammarguided constraints and structural role-aware modulation. In addition, removing the weighted detection module also leads to performance degradation, indicating that the proposed weighted detection statistic provides additional benefits for detectability. We further provide visualizations of the qualitydetectability curves for each variant in Figure 5b.

watermark modulation as content-bearing tokens. As shown in Table 5, GDW decreases from 0.847 to 0.819 after the attack, but still achieves the highest AUTC among all compared methods. In contrast, SWEET and CodeIP exhibit relatively larger performance drops after the attack, indicating weaker robustness to identifier-level perturbations. Although methods KGW, EWD, STONE, and SynthID-Text remain relatively stable, their overall AUTC performance is consistently lower than GDW before and after the attack. This reflects that our method maintains the best qualitydetectability balance under syntax-preserving code transformations.

4.6

5

Robustness Analysis

Related Work

Text Watermarking for LLMs. Text watermarking methods for LLMs have been widely applied to the protection of intellectual property (Wang et al., 2023; Wu et al., 2023; Li et al., 2023a; Yao et al., 2024; Zhang et al., 2024; Huo et al., 2024; Feng

We evaluate the robustness of GDW under variable renaming attack (Suresh et al., 2024), which modifies variable identifiers while preserving program functionality. This setting is particularly challenging for GDW, since identifiers receive stronger 8

et al., 2025; Qu et al., 2025), the deterrence of unauthorized copying (Li et al., 2023b; Sato et al., 2023; Mao et al., 2024; Wang et al., 2025), and the verification of authenticity (Takezawa et al., 2023; Liu et al., 2023a; Kirchenbauer et al., 2023b).

potential limitation is that while GDW consistently outperforms baselines across diverse models and benchmarks, it introduces a slight computational overhead during the inference stage due to the necessity of incremental grammar parsing. Future optimization could involve the use of more efficient, specialized incremental parsers or parallelization techniques to further minimize this overhead.

Code Watermarking. Early works on static code watermarking focused on program structures or binary representations (Collberg and Thomborson, 1999; Collberg and Sahoo, 2005; Balachandran et al., 2014; Chen et al., 2017), but proved brittle for LLM-generated code. Researchers have explored selecting watermark embedding positions through token entropy. SWEET (Lee et al., 2024) filters tokens by entropy thresholds, while Lu et al., 2024 employs entropy-weighted detection. Although these methods mitigate quality degradation, their rigid exclusion of low-entropy tokens leaves room for further improvement. Recent studies have improved watermarking methods from the perspective of code structure. Guan et al., 2024 introduces a grammar-guided watermarking method by training a type predictor to predict the next token, ensuring that the generated code remains grammatically correct. However, this method demands substantial computational resources, which restricts its applicability in large-scale code generation tasks. Meanwhile, Kim et al., 2025 presented a method that embeds watermarks only in non-syntactic tokens during code generation. In contrast, the approach of GDW moves beyond this static exclusion scheme.

6

Ethical Considerations We use open-source artifacts in accordance with their licenses and intended research use. The data resources used in this study are publicly available.

References Anthropic. 2025. Claude code. https://claude.com /product/claude-code. [Accessed 22-05-2026]. Anysphere. 2023. Cursor. https://www.cursor.com /. [Accessed 22-05-2026]. Vivek Balachandran, Ng Wee Keong, and Sabu Emmanuel. 2014. Function level control flow obfuscation for software security. In 2014 Eighth International Conference on Complex, Intelligent and Software Intensive Systems, pages 133–140. IEEE. Jianping Chen, Kui Li, Wanzhi Wen, Weixu Chen, and Chenxue Yan. 2017. Software watermarking for java program based on method name encoding. In International Conference on Advanced Intelligent Systems and Informatics, pages 865–874. Springer. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé, Jared Kaplan, Harrison Edwards, Yura Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 34 others. 2021. Evaluating large language models trained on code. ArXiv, abs/2107.03374.

Conclusion

In this paper, we propose GDW to watermark the LLMs for code generation. Without requiring additional training or fine-tuning, GDW preserves syntactic validity through a three-level masking mechanism and injects watermark signals via structural role-aware modulation. Correspondingly, we design a role-aware weighted detection statistic to improve watermark detectability under low-entropy scenarios. Experiments show that GDW establishes a stronger quality-detectability trade-off frontier than existing methods, while maintaining robustness against variable-renaming attacks. Our findings contribute to more trustworthy code generation in the era of LLM-assisted software development.

Christian Collberg and Tapas Ranjan Sahoo. 2005. Software watermarking in the frequency domain: implementation, analysis, and attacks. Journal of Computer Security, 13(5):721–755. Christian Collberg and Clark Thomborson. 1999. Software watermarking: Models and dynamic embeddings. In Proceedings of the 26th ACM SIGPLANSIGACT symposium on Principles of programming languages, pages 311–324. Sumanth Dathathri, Abigail See, Sumedh Ghaisas, PoSen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, and 1 others. 2024. Scalable watermarking for identifying large language model outputs. Nature, 634(8035):818–823.

Limitations In this work, we proposed GDW, a grammar-driven watermarking approach that achieves a superior trade-off between code quality and detectability. A 9

Ayan Dey, Sukriti Bhattacharya, and Nabendu Chaki. 2019. Software watermarking: Progress and challenges. INAE Letters, 4(1):65–75.

Aiwei Liu, Leyi Pan, Xuming Hu, Shuang Li, Lijie Wen, Irwin King, and Philip S. Yu. 2023a. An unforgeable publicly verifiable watermark for large language models. In International Conference on Learning Representations.

Xiaoyan Feng, He Zhang, Yanjun Zhang, Leo Yu Zhang, and Shirui Pan. 2025. Bimark: Unbiased multilayer watermarking for large language models. arXiv preprint arXiv:2506.21602.

Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023b. Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code generation. In Thirty-seventh Conference on Neural Information Processing Systems.

Batu Guan, Yao Wan, Zhangqian Bi, Zheng Wang, Hongyu Zhang, Pan Zhou, and Lichao Sun. 2024. Codeip: A grammar-guided multi-bit watermark for large language models of code. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 9243–9258.

Jiawei Liu, Songrun Xie, Junhao Wang, Yuxiang Wei, Yifeng Ding, and Lingming Zhang. 2024. Evaluating language models for efficient code generation. In First Conference on Language Modeling.

Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, and 1 others. 2024. Qwen2.5-coder technical report. arXiv preprint arXiv:2409.12186.

Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, and 1 others. 2024. Starcoder 2 and the stack v2: The next generation. arXiv preprint arXiv:2402.19173.

Mingjia Huo, Sai Ashish Somayajula, Youwei Liang, Ruisi Zhang, Farinaz Koushanfar, and Pengtao Xie. 2024. Token-specific watermarking with enhanced detectability and semantic coherence for large language models. arXiv preprint arXiv:2402.18059.

Yijian Lu, Aiwei Liu, Dianzhi Yu, Jingjing Li, and Irwin King. 2024. An entropy-based text watermarking detection method. arXiv preprint arXiv:2403.13485.

Jungin Kim, Shinwoo Park, and Yo-Sub Han. 2025. Marking code without breaking it: Code watermarking for detecting llm-generated code. ArXiv, abs/2502.18851.

Minjia Mao, Dongjun Wei, Zeyu Chen, Xiao Fang, and Michael Chau. 2024. Watermarking low-entropy generation for large language models: An unbiased and low-risk method. arXiv preprint arXiv:2405.14604.

John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023a. A watermark for large language models. In International Conference on Machine Learning, pages 17061–17084. PMLR.

Microsoft and OpenAI. 2021. Github copilot. https: //github.com/features/copilot. [Accessed 22-05-2026].

John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. 2023b. On the reliability of watermarks for large language models. arXiv preprint arXiv:2306.04634.

Niklas Muennighoff, Qian Liu, Qi Liu, Armel Randy Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro von Werra, and S. Longpre. 2023. Octopack: Instruction tuning code large language models. ArXiv, abs/2308.07124. Wenjie Qu, Wengrui Zheng, Tianyang Tao, Dong Yin, Yanze Jiang, Zhihua Tian, Wei Zou, Jinyuan Jia, and Jiaheng Zhang. 2025. Provably robust multi-bit watermarking for {AI-generated} text. In 34th USENIX Security Symposium (USENIX Security 25), pages 201–220.

Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim. 2024. Who wrote this code? watermarking for code generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4890– 4911.

Ryoma Sato, Yuki Takezawa, Han Bao, Kenta Niwa, and Makoto Yamada. 2023. Embarrassingly simple text watermarks. arXiv preprint arXiv:2310.08920.

Shuai Li, Kejiang Chen, Kunsheng Tang, Wen Huang, Jie Zhang, Weiming Zhang, and Nenghai Yu. 2023a. Functionmarker: Watermarking language datasets via knowledge injection. arXiv preprint arXiv:2311.09535.

Tarun Suresh, Shubham Ugare, Gagandeep Singh, and Sasa Misailovic. 2024. Is the watermarking of llm-generated code robust? arXiv preprint arXiv:2403.17983.

Zongjie Li, Chaozheng Wang, Shuai Wang, and Cuiyun Gao. 2023b. Protecting intellectual property of large language model-based code generation apis via watermarks. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pages 2336–2350.

Yuki Takezawa, Ryoma Sato, Han Bao, Kenta Niwa, and Makoto Yamada. 2023. Necessary and sufficient watermark for large language models. arXiv preprint arXiv:2310.00833.

10

Lean Wang, Wenkai Yang, Deli Chen, Hao Zhou, Yankai Lin, Fandong Meng, Jie Zhou, and Xu Sun. 2023. Towards codable watermarking for injecting multi-bits information to llms. arXiv preprint arXiv:2307.15992. Zongqi Wang, Tianle Gu, Baoyuan Wu, and Yujiu Yang. 2025. Morphmark: Flexible adaptive watermarking for large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4842–4860.

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. ArXiv, abs/2505.09388.

Java

Go

KGW SWEET EWD STONE CodeIP SynthID-Text GDW

0.797 0.765 0.792 0.666 0.738 0.687 0.873

0.887 0.800 0.910 0.761 0.801 0.705 0.941

0.896 0.844 0.895 – 0.839 0.701 0.914

are positioned above those of the baselines, indicating that our method maintains higher watermark detectability at comparable levels of code quality, or achieves better code quality under similar detection performance. These results further demonstrate that the effectiveness of GDW is robust across different decoding strategies. In particular, the combination of syntactic-validity filtering and structural-roleaware modulation remains beneficial under beam search, enabling more stable watermark embedding without compromising functional correctness.

Hongwei Yao, Jian Lou, Zhan Qin, and Kui Ren. 2024. Promptcare: Prompt copyright protection by watermark injection and verification. In 2024 IEEE Symposium on Security and Privacy (SP), pages 845–861. IEEE. Ruisi Zhang, Shehzeen Samarah Hussain, Paarth Neekhara, and Farinaz Koushanfar. 2024. {REMARK-LLM}: A robust and efficient watermarking framework for generative large language models. In 33rd USENIX Security Symposium (USENIX Security 24), pages 1813–1830.

C

The Impact of Decoding Strategy on Code Generation

To further validate the consistency of our findings, we provide the quality–detectability trade-off curves for our baselines under both beam search and sampling. For conciseness, we show only representative hyperparameter settings for each method in Figure 7 and 8. Nevertheless, the same qualitative trend is observed across the other settings: beam search consistently shifts the trade-off curves toward the upper-right region. This reinforces the conclusion that the structural consistency provided by deterministic decoding strategies is fundamentally beneficial for code watermarking.

Experimental Results on Qwen2.5-Coder-7B

To further demonstrate the generalizability of GDW, we additionally conduct experiments on Qwen2.5-Coder-7B under beam search decoding. We evaluate GDW across three programming languages: Python on HumanEval dataset, Java and Go on HumanEvalPack dataset. Table 6 presents the AUTC results of different watermarking methods. GDW consistently achieves the best performance across all baselines.

B

Python

Table 6: AUTC of different watermarking methods using Qwen2.5-Coder-7B with beam search decoding. The best result in each column is highlighted in bold.

Yihan Wu, Zhengmian Hu, Hongyang Zhang, and Heng Huang. 2023. Dipmark: A stealthy, efficient and resilient watermark for large language models.

A

Methods

Supplementary Visualization of Trade-off Curves

We supplement corresponding Pass@1-F1 tradeoff curves on the HumanEvalPack benchmark, as shown in Figure 6. Consistent with the observations under sampling in the main text, GDW continues to achieve a stronger trade-off frontier across all evaluated models and datasets. In most cases, the curves of GDW 11

+XPDQ(YDO3DFN-DYD6WDU&RGHU%

+XPDQ(YDO3DFN*R4ZHQ&RGHU%









 

  















3DVV#EHWWHU



 











3DVV#EHWWHU







 

  

 

•





KGW







SWEET

■













3DVV#EHWWHU







+XPDQ(YDO3DFN*R6WDU&RGHU%

 





3DVV#EHWWHU





 



















3DVV#EHWWHU



)6FRUHEHWWHU

)6FRUHEHWWHU













+XPDQ(YDO3DFN*R4ZHQ&RGHU%







 

+XPDQ(YDO3DFN-DYD6WDU&RGHU%











+XPDQ(YDO3DFN-DYD4ZHQ&RGHU%

)6FRUHEHWWHU



















3DVV#EHWWHU



⋆

EWD

▲



)6FRUHEHWWHU



)6FRUHEHWWHU



)6FRUHEHWWHU

)6FRUHEHWWHU



+XPDQ(YDO3DFN*R6WDU&RGHU%



)6FRUHEHWWHU

+XPDQ(YDO3DFN-DYD4ZHQ&RGHU% 



    









3DVV#EHWWHU

STONE



CodeIP













3DVV#EHWWHU

SynthID-Text



GDW (Ours)

♦

Figure 6: Pass@1-F1 trade-off curves on HumanEvalPack for Qwen2.5-Coder-3B and StarCoder2-3B under two decoding strategies: sampling decoding (top row) and beam search decoding (bottom row). GDW establishes a better trade-off frontier than other methods. 0%334ZHQ&RGHU%.*:

+XPDQ(YDO4ZHQ&RGHU%.*:

6DPSOLQJ

%HDP6HDUFK

6DPSOLQJ

0%336WDU&RGHU%(:'

%HDP6HDUFK

6DPSOLQJ











 

 

   







3DVV#EHWWHU



+XPDQ(YDO4ZHQ%&RGH,3 6DPSOLQJ

  

 













3DVV#EHWWHU





 



6DPSOLQJ

%HDP6HDUFK











)6FRUHEHWWHU

    

)6FRUHEHWWHU





    







3DVV#EHWWHU











3DVV#EHWWHU







3DVV#EHWWHU



 

 









3DVV#EHWWHU





6DPSOLQJ

%HDP6HDUFK

 

     



0%336WDU&RGHU%6:((7

%HDP6HDUFK



 



6DPSOLQJ







+XPDQ(YDO4ZHQ%6\QWK,'7H[W

+(3DFN*R4ZHQ&RGHU%&RGH,3

%HDP6HDUFK



%HDP6HDUFK

 

)6FRUHEHWWHU



)6FRUHEHWWHU



)6FRUHEHWWHU





)6FRUHEHWWHU

+XPDQ(YDO6WDU&RGHU%(:'

%HDP6HDUFK



)6FRUHEHWWHU

)6FRUHEHWWHU

6DPSOLQJ









3DVV#EHWWHU







    









3DVV#EHWWHU





Figure 7: Trade-off curves for representative baselines under sampling and beam search decoding strategies.

(a) Trade-off curves on HumanEval with Qwen2.5-Coder-3B.

(b) Trade-off curves on MBPP+ with Qwen2.5-Coder-3B.

Figure 8: Impact of Decoding strategies on the quality-detectability trade-off of GDW.

12

Related documents

• Post-Hoc Attention Steering of Large Language Models for Robust Code Understanding under Obfuscation#1062005
• TrajMark: Ownership Attribution and Segment-Level Tamper Localization for Coding-Agent Trajectories#997300
• Multi-Channel Spread-Spectrum Code Watermarking#346445
• Can Code Evaluation Metrics Detect Code Plagiarism?#141527
• KeyBound: Keyed and Host-Bound Learned Audio Watermarking for Speech Provenance#1062135