Toward Stronger Code Watermarking: A Grammar-Driven Approach to Optimizing the Trade-off Between Quality and Detectability
1
Licheng Yu1 , Aiwei Liu2 , Songze Li1 * Southeast University 2 Tsinghua University
{lichengyu, songzeli}@seu.edu.cn, [email protected]
arXiv:2607.10210v1 [cs.CR] 11 Jul 2026
Abstract With the rapid development of Large Language Models (LLMs), text watermarking has emerged as a crucial technique for identifying machine-generated content. However, directly applying existing logits-based watermarking methods to code generation remains challenging, since the low-entropy nature of code exacerbates the trade-off between code quality and watermark detectability. In this paper, we propose a novel code watermarking approach called Grammar-Driven Watermark (GDW) for LLMs. GDW preserves syntactic validity through a grammar-guided threelevel masking mechanism and injects watermark signals via structural role-aware modulation, assigning a stronger bias to contentbearing tokens while applying a more conservative bias to syntax-critical tokens. Aligning with the generation process, we further design a role-aware weighted detection statistic to improve detectability. Experiments across multiple programming languages, models, and decoding strategies show that GDW establishes a stronger quality-detectability trade-off frontier than existing methods, while maintaining robustness against variable-renaming attacks.
1
Logits-based Watermark
Entropy-based Watermark
Grammar-Driven Watermark
Red-Green Vocab Partition
Red-Green Vocab Partition
Red-Green Vocab Partition
Syntax-Agnostic
Syntactic Validity Check
Without Code-Specific Design
Entropy-Guided Strategy
Structural Role Modulation
Uniform Watermark Strength to All Green Tokens
Uniform Watermark Strength to High-Entropy Green Tokens
Hierarchical Watermark Strength to Syntax-Compliant Green Tokens
+ Standard Detection
+ Role-Aware Weighted Detection Unclear Trade-off Frontier
Optimized Trade-off Frontier
Figure 1: Comparison among existing logits-based watermark, entropy-based watermark, and our GrammarDriven Watermark in terms of the trade-off between watermark detectability and code quality. Our proposed method GDW achieves a better trade-off frontier.
Watermarking has emerged as a promising solution for identifying machine-generated content by embedding imperceptible but statistically detectable signals into model outputs. One dominant paradigm for LLM text watermarking is the logitsbased watermark, which injects signals by modifying the model’s output logits (Kirchenbauer et al., 2023a). Specifically, the vocabulary is pseudorandomly partitioned into a ‘green list’ and a ‘red list’ at each decoding step. By adding a constant bias to the logits of green-list tokens, the model is encouraged to select tokens within the green list, which can later be identified via a statistical z-test. However, while logits-based watermarking methods have shown effectiveness in natural language generation tasks, their application to code is hindered by the low-entropy nature of programming languages. Code is low-entropy text, where the model’s output logits are highly concentrated due to the rigid syntax. In such settings, naively injecting watermark bias into low-entropy tokens can significantly distort the generation process. A watermark bias that is too strong may degrade the code’s functionality or readability, while a weaker watermark bias might be difficult to detect.
Introduction
In recent years, large language models (LLMs) have become indispensable tools for code generation and programming assistance, powering widely used AI coding assistants such as GitHub Copilot (Microsoft and OpenAI, 2021), Cursor (Anysphere, 2023) and Claude Code (Anthropic, 2025). However, as LLM-generated code becomes increasingly integrated into open-source projects and industrial software, concerns about code provenance and potential copyright infringement have increased substantially (Dey et al., 2019). Therefore, algorithms that can effectively identify machine-generated code have become crucial. * Corresponding author.
1
Syntax-Critical Tokens
1.2
Content-Bearing Tokens
watermark bias for syntax-critical tokens. In addition, we further introduce a Three-Level Masking Mechanism to enforce syntactic validity during watermark injection. In this work, we introduce Grammar-Driven Watermark (GDW), a novel watermarking method that optimizes the trade-off between code quality and watermark detectability in code generation. Our method integrates Context-Free Grammar (CFG) into the decoding process to ensure syntactic validity at each step, while applying a structural roleaware modulation. Correspondingly, we design a role-aware weighted detection statistic aligned with the generation mechanism. Note that our method operates entirely at inference time and does not require training or fine-tuning. In summary, our contributions are as follows:
1.014
Mean Token Entropy
1.0
0.8
0.952
0.902
0.744 0.649
0.6
0.554
0.580
0.419
0.4
0.2
0.0
HumanEval
MBPP+
HumanEvalPack-Java HumanEvalPack-Go
Figure 2: Mean token entropy of syntax-critical tokens and content-bearing tokens measured with Qwen2.5Coder-3B across Python, Java, and Go. This observation motivates our structural role-aware modulation strategy.
To address this, researchers propose entropybased watermarking methods (Lee et al., 2024; Lu et al., 2024), applying watermark bias only to tokens with entropy exceeding a specific threshold. There remains room for improvement in the tradeoff between watermark strength and code quality. We argue that effective watermarking for code should be structure-aware. Crucially, not all tokens in code contribute equally to structural integrity or functional correctness. Syntax-critical tokens such as keywords, operators, and delimiters govern grammatical correctness, while content-bearing tokens such as identifiers and literals often exhibit greater semantic redundancy. This observation suggests that watermark injection should not be applied uniformly across tokens, nor should lowentropy tokens be entirely excluded. Instead, the watermark strength should be modulated in a manner that respects both grammatical constraints and token roles. As illustrated by Figure 1, existing watermarking schemes face inherent trade-offs during low-entropy code generation, making an effective solution imperative.
• We propose a grammar-driven watermarking method called GDW, tailored for LLM-based code generation task. GDW supports multiple programming languages, including Python, Java, and Go. • We design a corresponding weighted detection method that aligns with the generation process and improves statistical detectability under low-entropy conditions. • Experiments show that GDW achieves a more favorable trade-off between code quality and watermark detectability compared to existing baselines.
2
Preliminaries
2.1
LLM Text Generation
Consider an auto-regressive Large Language Model M with a vocabulary V. Given a prompt or prefix sequence x1:t = (x1 , . . . , xt ), the LLM will generate a logits vector lt ∈ R|V| at decoding step t. At step t + 1, the probability distribution PLM over the vocabulary V is computed by applying the softmax function to the logits. The probability for selecting the next token xt+1 is given by: (t+1) (t+1) P|V | (t+1) pk = exp(lk )/ i=1 exp(li ). During inference, the LLM samples tokens from the probability distribution PLM using decoding strategies such as top-k, top-p, or beam search. A sequence of length Tgen is generated by iteratively sampling tokens xt+1 ∼ P (xt+1 |x1:t ) and appending them to the prefix until a termination criterion is met.
We further analyze the mean token entropy of syntax-critical tokens and content-bearing tokens across Python, Java, and Go using Qwen2.5-Coder3B. As shown in Figure 2, syntax-critical tokens consistently exhibit lower entropy than contentbearing tokens across all three languages. This suggests that syntax-related tokens are generally more deterministic, whereas content-bearing tokens provide greater flexibility for watermark injection. Motivated by this observation, our method applies stronger watermark modulation to contentbearing tokens, while employing a conservative 2
𝑴𝒘𝒎,𝒕 1 Prefix Code
1
0
0
1
0
1
0
Watermark Greenlist Mask
Hash Function
𝑴𝒆𝒍𝒊𝒈,𝒕 0
1
0
1
1
0
0
1
0
0
0
Eligibility Mask
𝑴𝒔𝒚𝒏,𝒕 0
1
0
0
1
Syntactic Validity Mask Context-Free Grammars
Incremental Parser Syntax-Critical Tokens
0 Content-Bearing Tokens
Vocabulary
Grammar-Constrained Hierarchical Modulation
𝑴𝒓𝒐𝒍𝒆
e.g. def, if, while
0
1
0
1
0
1
1
Structural Role Mask
e.g. variable name, strings
Watermarked Logits
Conservative Bias δ
Role Function
𝑟(𝑣)
for SC Tokens
Stronger Bias λ·δ for CB Tokens
Select Eligible Tokens for Watermark (Fall into the Greenlist and Pass the Syntactic Validity Check). -- Section 3.2.1
Tokens in Greenlist
Modulate the Watermark Strength Based on Structural Roles and Role Function. -- Section 3.2.2
Syntactically Valid Tokens
Syntax-Critical Tokens
Modulate the Original Logits to Obtain the Watermarked Logits. -- Section 3.2.3
Watermark-Eligible Tokens
Content-Bearing Tokens
Figure 3: A Grammar-Driven Watermark Approach for LLM-Generated Code
2.2
3.1
LLM Text Watermarking
The general watermarking scheme used in this paper is called KGW (Kirchenbauer et al., 2023a). Given a prefix sequence x1:t−1 , the watermark process can be described as follows. At each decoding step, a pseudorandom number generator is seeded using the hash of the preceding token xt−1 to generate a random permutation of V. The vocabulary is then split into a green list Gt ⊂ V of size γ|V| and a red list Rt ⊂ V of size (1−γ)|V|, where γ ∈ (0, 1) is the green-list ratio. Watermark signals are injected by adding a small bias δ to the logits of tokens in Gt , making them more likely to be selected during generation. The detection process is formulated as a hypothesis test. Specifically, pthe detector computes a z-score z = (|s|G −γT )/ γ(1 − γ)T , where |s|G is the number of green-list tokens in a text of length T . The text is identified as watermarked when the z-score exceeds a predefined threshold.
3
Watermark Generation Framework
We define the overall generation process of our GDW method in this section. The watermark injection process can be decomposed into the following three steps: 1. Guided by the prompt and the preceding context x1:t−1 , GDW imposes a dual constraint to derive a set of eligible tokens that are both syntactically admissible and assigned to the watermark greenlist. Implementation details are elaborated in Section 3.2.1. 2. Modulate the watermark bias through structural role mask and role function r(v). We will provide more details in Section 3.2.2. 3. Modify original logits using a grammarconstrained hierarchical modulation strategy (integrating the constraints and biases from previous steps), introduced in Section 3.2.3.
Methodology
In this section, we introduce GDW, a grammardriven watermarking method for code generation. In code watermarking, a critical challenge lies in optimizing the trade-off between watermark detectability and the quality of the generated code. Our method is proposed to explicitly mitigate this conflict. We first introduce the overall watermark generation process in Section 3.1 and then detail the grammar-driven watermark injection mechanism in Section 3.2, followed by the detailed watermark detection in Section 3.3.
By iteratively executing these three steps across T decoding steps, GDW will produce the watermarked code that maintains quality while ensuring high detectability. The complete watermark embedding process is shown in the Figure 3.
3.2
Grammar-Driven Watermark Injection
In this section, we detail the implementation of the three steps introduced in Section 3.1. 3
3.2.1 Three-Level Masking Mechanism To inject watermark while preserving the quality of the generated code, we construct a three-level masking mechanism over the vocabulary at each decoding step. This mechanism comprises three binary masks, which jointly determine whether a candidate token is eligible for watermark injection.
serve that tokens with different structural functions exert varying degrees of influence on the executability and functional integrity of the generated code. As a result, we conduct a detailed analysis of the three target programming languages (Python, Java and Go) and categorize tokens into two groups: syntax-critical tokens, such as keywords, operators, and delimiters, which constitute the structural skeleton of the program; and content-bearing tokens, including identifiers and literals, which carry specific logic. To formally incorporate this distinction into our modulation process, we define a structural role mask Mrole ∈ {0, 1}|V| to distinguish between the (v) two categories. Specifically, we assign Mrole = 1 (v) to tokens in the syntax-critical pool and Mrole = 0 to those in the content-bearing pool. This mask serves as the foundation for the subsequent hierarchical modulation strategy.
Watermark Greenlist Mask. Following the KGW framework (Kirchenbauer et al., 2023a), at each decoding step t, we define the watermark greenlist mask Mwm,t ∈ {0, 1}|V| by partitioning the vocabulary based on a pseudorandom hash of the preceding token xt−1 . Tokens assigned to the greenlist are marked as 1, while those in the redlist are marked as 0. Syntactic Validity Mask. For a program to execute successfully, its syntactic structure must be valid. Consequently, to preserve the syntactic integrity of generated code, GDW integrates a filtering mechanism guided by CFG. By taking a CFG represented with extended Backus–Naur form (EBNF) rules, GDW ensures that the LLM output remains syntactically valid. Given the current prefix x1:t−1 , we define At ⊆ V as the set of syntactically admissible next tokens that satisfy the predefined CFG rules C. At each decoding step t, we employ an incremental parser to derive the current admissible set At . The watermark bias will be applied exclusively to tokens within the admissible set At , concentrating the watermark signal along valid syntactic paths. This ensures the syntactic correctness of the generated code. Based on this admissible set At , we construct a syntactic validity mask Msyn,t ∈ {0, 1}|V| . At (v) decoding step t, Msyn,t = 1 if the token v is syntactically admissible given the current prefix, and 0 otherwise. This mask ensures that only syntactically valid tokens are considered for watermarking.
Role Function. The core motivation for distinguishing token roles lies in the disparate sensitivity of code structures to logit perturbation. Since syntax-critical tokens form the grammatical backbone of a program, excessive perturbation to these tokens may easily cause program failure. In contrast, content-bearing tokens reside in higherentropy regions, where the model has greater predictive flexibility and watermark signals can be embedded more aggressively. To exploit this flexibility while preserving structural stability, we introduce a role function r(v) that scales the watermark (v) strength based on Mrole . For each token v ∈ V, r(v) is defined as: ( 1, if v ∈ Vsyntax, r(v) = λ, if v ∈ Vcontent.
where we assign λ > 1 to content-bearing tokens to amplify the watermark signal, while restricting a more conservative watermark bias to syntax-critical tokens.
Eligibility Mask. To consolidate these constraints, we combine the watermark greenlist mask and the syntactic validity mask to form an eligibility mask Melig,t , which identifies tokens that are eligible for watermark injection and syntactically valid at the current decoding step. Melig,t = Mwm,t ⊙ Msyn,t .
(2)
3.2.3
Grammar-Constrained Hierarchical Modulation
Finally, by integrating the previously derived eligibility mask and structural role-aware biases, we modulate the original logits to obtain the final watermarked distribution. The watermarked logits lt′ at step t is computed as:
(1)
3.2.2 Structural Role-Aware Modulation Structural Role Mask. Beyond considering the impact of syntactic validity on code quality, we ob-
lt′ = lt +Melig,t ⊙ Mrole +λ·(1−Mrole ) ·δ. (3) 4
This strategy ensures that, under the prerequisite of guaranteeing syntactic validity, watermark signals are injected more conservatively into syntaxcritical tokens that are essential to the structural integrity of the code, while stronger biases are applied to content-bearing tokens, optimizing the trade-off between watermark detectability and code quality. The pseudocode of the proposed GDW injection procedure is provided in Algorithm 1.
watermarked text from unwatermarked text. Given a generated code sequence x = {x1 , . . . , xT }, we follow the standard practice of discarding the initial token used for seeding. For each subsequent position t > 1, we define an indicator variable: It = 1[xt ∈ Gt ],
where Gt denotes the green list at step t, which is recovered using a hash of the preceding token xt−1 . To ensure symmetry between generation and detection, we assign a contribution weight wt to each token xt , governed by the same role function r(·) employed during watermark injection:
Algorithm 1 Watermark Injection Input: Prefix tokens x1:t−1 , original logits lt , green-list ratio γ ∈ (0, 1), base bias δ > 0, role function r(·), CFG grammar rules C, syntax-critical vocabulary Vsyntax , content-bearing vocabulary Vcontent . Output: Watermarked logits lt′ . 1: Compute a hash of token xt−1 and use it to pseudo-randomly partition the vocabulary V into greenlist Gt and redlist Rt . 2: Construct the watermark greenlist mask Mwm,t ∈ {0, 1}|V| : ( 1, if v ∈ Gt , (v) Mwm,t = 0, otherwise
( 1, if xt ∈ Vsyntax , wt = r(xt ) = λ, if xt ∈ Vcontent
PT P wt It − γ Tt=2 wt t=2 z= q . P γ(1 − γ) Tt=2 wt2
At ← I NCREMENTAL PARSER(C, x1:t−1 ). 4: Construct the syntactic validity mask
(6)
A sequence is classified as watermarked if z > τ , where τ is a predefined threshold. The pseudocode of the proposed detection algorithm is provided in Algorithm 2.
Msyn,t ∈ {0, 1}|V| : ( 1, if v ∈ At , (v) Msyn,t = 0, otherwise 5: Compute the eligibility mask using Eq.1. 6: Construct the structural role mask
4
Experiments
4.1
Experimental Settings
Models. We explore the performance of the watermark using three public LLMs: StarCoder2-3B (Lozhkov et al., 2024), Qwen2.5-Coder-3B (Hui et al., 2024), and Qwen3-4B (Yang et al., 2025). We also report additional results with Qwen2.5Coder-7B (Hui et al., 2024) in Appendix A, which further demonstrate the generalizability of our approach across different model scales.
if v ∈ Vsyntax , if v ∈ Vcontent
7: Calculate role-aware watermark strength
adjustment using the role function Eq.2. 8: Combine all the masks and update the logits
according to Eq.3.
Datasets. Our method is evaluated across three programming languages: Python, Java, and Go. We use HumanEval (Chen et al., 2021) and MBPP+ (Liu et al., 2023b, 2024) for Python, HumanEvalPack (Muennighoff et al., 2023) for Java and Go. For each sample, we use human-written prefixes or questions as prompts, generating T = 200 tokens.
9: return lt′
3.3
(5)
where wt = 1 corresponds to syntax-critical tokens and wt = λ to content-bearing tokens, consistent with the watermark generation step. We define the detection statistic as a weighted z-score:
3: Obtain the syntactically admissible set
Mrole ∈ {0, 1}|V| : ( 1, (v) Mrole = 0,
(4)
Grammar-Driven Watermark Detection
Watermark detection is a binary classification task that assesses the detector’s ability to distinguish 5
Qwen2.5-Coder-3B Methods
HumanEval
KGW SWEET EWD STONE CodeIP SynthID-Text GDW
Qwen3-4B
MBPP+
HumanEval
StarCoder2-3B
MBPP+
HumanEval
MBPP+
Sample
Beam
Sample
Beam
Sample
Beam
Sample
Beam
Sample
Beam
Sample
Beam
0.745 0.784 0.771 0.666 0.805 0.702 0.811
0.832 0.742 0.774 0.677 0.730 0.692 0.847
0.735 0.757 0.763 0.670 0.767 0.676 0.787
0.857 0.768 0.789 0.670 0.768 0.668 0.876
0.714 0.766 0.796 0.666 0.722 0.705 0.830
0.815 0.756 0.831 0.671 0.802 0.701 0.847
0.784 0.719 0.797 0.669 0.809 0.744 0.828
0.840 0.774 0.852 0.683 0.796 0.700 0.875
0.691 0.722 0.712 0.681 0.725 0.700 0.743
0.848 0.743 0.795 0.686 0.760 0.679 0.862
0.750 0.761 0.743 0.668 0.766 0.732 0.833
0.837 0.784 0.820 0.683 0.739 0.754 0.855
Table 1: Area Under the Trade-off Curve (AUTC) of different watermarking methods on HumanEval and MBPP+. The best trade-off performance in each column is highlighted in bold.