Hidden in Plain Tokens: Simply Robust, Gradient-Free Watermark for Synthetic Audio
Georgios Milis 1 Yubin Qin 1 Yihan Wu 1 Heng Huang 1
arXiv:2605.25967v1 [cs.LG] 25 May 2026
Abstract
cerns and restoring trust in digital media. In-generation watermarks for autoregressive models are a one-fits-all solution due to their integration at the token sampling level. Their detection requires identifying subtle statistical signals in sequences of discrete tokens retrieved from the generated content. While highly effective in discrete domains like text, the inevitable retokenization mismatch in continuous modalities seriously degrades the raw watermark signals, making detection error-prone and revealing fundamental limits of watermarking reliability.
As policy catches up with the capabilities of generative AI, watermarking is central to content provenance efforts. Inference-time watermarks for autoregressive models are unfit for continuous modalities due to discretization inconsistencies. Existing methods overcome this by finetuning the modality tokenizers, nullifying the watermark’s training-free advantage. In this work, motivated by the vocabulary redundancy of discretization, we propose an elegant solution for powerful and robust watermarking of synthetic audio. We theoretically analyze the impact of token errors on watermark detection, and effectively mitigate them using a reduced vocabulary obtained via community detection. Thorough experiments showcase that our gradient-free method can boost detectability by several orders of magnitude, while also achieving built-in robustness to audio modifications. Broadly, we discover a new state-of-the-art for token-level watermarks in multimedia, which simply arises from the nature of discrete representation learning.
To mitigate this, existing approaches have relied on finetuning the codec modules towards idempotence, which requires computationally expensive training and white-box access to the codec. In this work, we propose a lightweight alternative that preserves the gradient-free paradigm. We observe that retokenization errors are not random, but rather exhibit a degree of structural consistency, where tokens are mostly confused with a small set of semantic neighbors. By modeling the codec vocabulary as a graph where edges represent confusion probabilities, we can apply community detection to identify stable clusters. By applying the watermark bias at the cluster level rather than the token level, we render the watermark invariant to retokenization noise, confirming that structural alignment in the latent space translates to robustness in the signal domain. We release some audio samples and the code used for our experiments on our project page: https://g-milis.github.io/projects/ nograd-audio-wm.html.
1. Introduction The rise of generative AI has brought attention to watermarking techniques, which imperceptibly mark AI-generated content in a way that is easily detectable algorithmically. Autoregressive models have had a central role in the development of contemporary generative AI systems. Audio models of such architecture enable the synthesis of lifelike synthetic audios, unlocking many possibilities of misuse ranging from misinformation to fraud. The development of robust imperceptible watermarks that are algorithmically detectable is a crucial requirement for addressing these con-
Our main contributions are as follows: • We analyze the impact of retokenization on the watermark detectability from a statistical perspective, quantifying how sequence and watermark parameters contribute to the watermark’s strength. • We introduce a novel, gradient-free method that discovers the underlying vocabulary structure under retokenization and distills it before applying the watermark rule, significantly enhancing discretization stability and subsequently watermark detectability.
1
Department of Computer Science, University of Maryland, College Park, USA. Correspondence to: Georgios Milis <[email protected]>, Heng Huang <[email protected]>.
• We evaluate our method on models of different codec architectures and various tasks, showcasing that it is strongly detectable and robust to attacks, while minimally degrading the perceived quality.
rd
Proceedings of the 43 International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026. Copyright 2026 by the author(s).
1
Hidden in Plain Tokens
2. Related Work
audio quality, lack formal detection guarantees, are vulnerable to waveform perturbations, and require additional embedding steps. Notably, O’Reilly et al. (2025) revealed that state-of-the-art post-hoc audio watermarks can be totally erased with modern neural codecs. This highlights the need for a new watermarking paradigm, with token-based watermarks being considered as a promising direction.
2.1. Audio Generative Models Lakhotia et al. (2021) pioneered the use of discrete language modeling for audio applications, enabling models like AudioLM (Borsos et al., 2022). This discrete paradigm has also been adopted by interactive models like Kimi-Audio (Ding et al., 2025) and the streaming architecture Moshi (Défossez et al., 2024). The audio tokenizers can function as neural codecs independently of their coupled autoregressive models, for instance Moshi’s Mimi codec has been adopted by LLaMA-Mimi (Sugiura et al., 2025). Broader multimodal frameworks like SEED-LLaMA (Ge et al., 2024) generalize this by interleaving audio, image, and text tokens.
3. Statistical Analysis 3.1. Preliminaries We adopt the KGW watermarking scheme of Kirchenbauer et al. (2023). At each step, a hash of the preceding h tokens partitions the vocabulary into a green set Gi (with fixed ratio γ) and a red set. Watermarking is implemented by biasing the logits of green tokens with δ prior to sampling. Detection tracks the total count of green tokens, Gsum , which follows Binomial(N, γ) under the unwatermarked null hypothesis (H0 ). We detect the watermark using the statistic
2.2. Watermarking Text watermarking. Building on the framework introduced by Aaronson (2022), Kirchenbauer et al. (2023) significantly advanced statistical watermarking and demonstrated its effectiveness through extensive experiments on large language models. The latter method partitions the vocabulary into red and green lists and biases generation toward green tokens by adding a fixed offset δ to their logits. To improve robustness, Zhao et al. (2023) proposed a unigram watermark that derives watermark keys via one-gram hashing, while Liu et al. (2023b) further enhanced robustness by using semantic features of the generated content as watermark keys. In a different direction, Liu et al. (2023a) introduced an unforgeable watermarking scheme that leverages neural networks to directly modify token distributions, rather than relying on explicit watermark keys.
Gsum − γN z=p , γ(1 − γ)N
(1)
which for large N approximately follows standard normal distribution under the null hypothesis H0 , showing absence of a watermark. However, the presence of a watermark would increase the number of green tokens, leading to large positive z values. 3.2. Corruption Scenarios Any benign or malicious modification to the generated audio waveform will affect the token sequence. In addition to that, retokenization itself introduces mismatch in continuous modalities, due to the non-idempotent nature of the encoderdecoder pair in discrete representation learning. We quantify this via the token match rate between the original sequence x1:N and the retokenized y1:N = E(x1:N ), as
Continuous Modalities. Tong et al. (2025) first proposed the use of a token-based watermark for images, a method fully developed later by Jovanović et al. (2025). Both approaches rely on finetuning of the discretization modules in order to reduce the retokenization errors. Wu et al. (2025a;b) identified retokenization errors in autoregressive audio models and subsequently proposed a distortion-free watermarking method based on k-means clustering. However, their experiments revealed lower detectability than the Kirchenbauer et al. (2023) baseline, in line with the literature from text watermarking, where strength is traded in order to achieve distortion-freeness.
N
TM(y1:N , x1:N ) =
1 X 1[yi = xi ]. N i=1
(2)
While TM ≈ 1 for text, it is significantly weaker in continuous modalities like images (Jovanović et al., 2025) or audio (O’Reilly et al., 2025), where tokenizers prioritize reconstruction quality over exact index preservation. Instead of modifying the tokenizer to enforce consistency, we address these corruptions by clustering frequently confused tokens into a reduced and robust vocabulary. We visualize the watermarked generation and retokenization of an audio sequence in Figure 1.
Post-hoc audio watermarking. Audio watermarking has a long history, traditionally exploiting human insensitivity to mid-frequency bands (Lie & Chang, 2006). Recent deep-learning methods encode payloads invisibly via autoencoders in the frequency domain (Liu et al., 2024; Chen et al., 2023), with extensions for temporally localized detection (San Roman et al., 2024) and robustness via error-correcting codes (Wu et al., 2023). However, these approaches degrade
3.3. Statistical Limits of the Baseline We model the transition from the generated watermarked sequence x1:N to its reconstruction y1:N as a noisy chan2
Hidden in Plain Tokens
Watermark
N times
D
E
p-value
Count biased tokens
Figure 1. Illustration of a token-level watermarking mechanism in the audio domain. During generation, the autoregressive model computes a probability distribution over the vocabulary at each time step. A logit bias is pseudorandomly applied to a specific subset of tokens, encouraging their selection, and the resulting token sequence is synthesized into a waveform by the decoder D. For detection, the waveform is re-encoded by the encoder E to recover the token sequence. The detector then performs a statistical hypothesis test based on the number of biased tokens, returning a p-value representing the probability of random generation. A sufficiently low p-value is evidence towards the alternative hypothesis, meaning that a watermark signal is present.
This highlights an exponential decay rh+1 . Since r < 1 in continuous modalities, the watermark signal vanishes rapidly as the context window h increases.
nel. Despite potential structural dependencies in encoder features, we assume conditional independence to derive tractable statistical limits. This assumption strictly holds for tokenizers with a non-overlapping sliding window, but we may assume that it approximately holds in general. We treat correct retokenization of tokens as independent Bernoulli events with a fixed success probability r ∈ (0, 1), which we will later estimate empirically as the average Token Match (TM) across the dataset.
3.4. Semantic Clustering To alleviate the high retokenization error, we introduce vocabulary clustering, and define the cluster match rate rcl as the probability that a retokenized token falls into the same semantic cluster as the original, formally
To quantify the limits of watermarking under corruption, we first derive the expected detection statistic for the standard KGW method. We assume that watermarking increases the expected proportion of green tokens from γ to some g > γ in the absence of corruption. Formally, under the alternative hypothesis H1 , g = P[xi ∈ Gi |H1 ]
rcl = P[C(yi ) = C(xi )],
(6)
with C being the clustering map. Since clusters aggregate nearest neighbors, rcl > r. We leverage this robustness by simply replacing tokens with clusters for both the context history and biasing the current token generation. The hh gram context immediately becomes more robust (rcl ). For the current token, since the partition is defined on clusters, exact matches and cluster matches are indistinguishable. Then the expected z-score becomes
(3)
is the probability that the generative model samples a token from the green partition, in other words the realized bias that the model exhibits. This value is implicitly controlled by the green list size γ and the logit bias δ.
√ E[z|H1 ] =
Conditioned on a correct context, the current token xi preserves the watermark if it survives retokenization (yi = xi ), which happens with probability r. If corrupted, it will randomly fall into the green list with probability γ. Thus, the expected probability p1 of observing a green token under H1 is p1 = γ + rh+1 (g − γ), (4)
Np
g−γ γ(1 − γ)
h+1 rcl .
(7)
As rcl > r, this effectively mitigates the exponential decay and boosts the z-score. We briefly extend this analysis to multiple channels in Section C.2. 3.5. Comparison & Trade-offs The main contribution of this method is the improvement in the exponential base. This robustness introduces a trade-off between detectability and generative entropy. By clustering, the vocabulary size shrinks from |V | to c|V | for some c ∈ (0, 1). However, the risk is key collision, especially if the key space is too small (Wu et al., 2024). Therefore, for an
as detailed in Section C.1. We can calculate the expected z-score under H1 , which gives us the relationship between detectability and different variables of interest as follows √ g−γ E[z|H1 ] = N p rh+1 . (5) γ(1 − γ) 3
Hidden in Plain Tokens
E
64
77
55
12
61
E
64
75
55
19
61
64 19 55
4
75
3 77
9 61
D
64
4.1. Vocabulary Reduction
Community detection
5
12
gradient-free method. We observe that successful watermark detection does not necessarily require the current token xi to be retokenized correctly, it is sufficient for its retokenized counterpart yi to merely belong to the same list, either Gi or Ri . Motivated by this, we formulate a sampling-based approach that captures the distribution of retokenization errors and appropriately distills the vocabulary via community detection. The techniques in the following subsections are applied to each RVQ channel separately, due to codebook independence.
55
75 77
12
We obtain a dataset of paired original and retokenized token sequences, by passing an audio dataset through the codec twice. The dataset should be large enough to cover the entire token vocabulary. We then define the confusion matrix M ∈ N|V |×|V | , where |V | is the vocabulary size, and the entry Mij represents the number of times that token i was confused with token j. We interpret M as the adjacency matrix of a weighted graph G = (V, M ), where the node set V represents the tokens.
19 61
1
Figure 2. Illustration of how our method captures and explicitly mitigates the retokenization errors. First, we use the encoder and decoder modules from the codec of interest to encode, decode, and re-encode a dataset of waveforms (top). We use the confusion counts between tokens as edge weights in a graph where the vertices correspond to tokens. Then, we perform community detection on that graph, effectively reducing the vocabulary size by a many-to-one mapping from tokens to clusters (bottom). Singleton vertices represent tokens that have never been confused in the dataset. Notice that we only require black box access to the codec components.
Our goal is to find a partition P = {c1 , c2 , . . . , cK } that minimizes the edge weights between clusters, and maximizes the edge weights within each cluster. This is exactly the definition of modularity, thus we explore unsupervised community detection algorithms such as the Louvain (Blondel et al., 2008) and Leiden (Traag et al., 2019) methods. Both optimize for modular communities, and accept a resolution parameter ρ that controls the cluster granularity.
h-gram context key, there should exist some threshold Kmin such that (c|V |)h ≥ Kmin (8)
Unlike Louvain, the Leiden method takes into account the edge directionality and provides guarantees for connected communities and faster convergence. We also empirically find that it leads to better watermark detectability, thus we encompass it into our method. Due to the multi-channel setting, we are able to enforce different resolution on different channels, leading to a strong multi-scale watermark. We visualize the vocabulary distillation process in Figure 2.
leads to an acceptably diverse key space. If c becomes too small in order to boost rcl , then the context length h should be appropriately increased to satisfy the key space threshold. Still, the issue of handling key collisions remains, with deferral to unwatermarked sampling as a practical solution (Hu et al., 2024).
4.2. Cluster-Level Watermarking
Therefore, the theoretically optimal cluster configuration must balance maximizing rcl for detection while maintaining a sufficiently large key space to avoid key collisions. In this work, we do not fully characterize a Pareto-optimal solution to this trade-off. Rather, we use a practical datadriven heuristic to discover strong semantic clusters, yielding rcl > r, and show that this is sufficient for significant improvements in the watermark strength without a compromise in quality.
In standard KGW, the token vocabulary is shuffled and split into the lists Gi and Ri at each time step. We simply modify this to encompass the clustering information, by shuffling and splitting the cluster vocabulary, and subsequently biasing all tokens that belong to clusters in Gi . Regarding the context for h > 0, since we require stability of the hash value, we simply replace the token indices with their cluster indices before hashing, as explained in Section 3.4. Both modifications contribute to increased watermark robustness to modifications, provided that the clustering captures well the codec’s retokenization inconsistencies. This also ensures negligible computational overhead at inference time, since the only additional step (compared to standard token
4. Methodology In this section we present how we implement the clustering discussed in Section 3.4 via a data-driven but black-box and 4
Hidden in Plain Tokens
Figure 4. Even with h > 0, our watermark still achieves unprecedented detectability even in very low FPR settings. Experiments with the Moshi model with h = 1 (top) and h = 2 (bottom), both prompted by conversational audio.
Figure 3. Our watermark achieves high detectability under extremely low FPR settings. Experiment on the Moshi model with h = 0, prompted by conversational audio inputs (top) and LibriSpeech samples (bottom).
use different audio codecs for discretization. First, we test Moshi (Défossez et al., 2024) which uses the Mimi codec and is highly capable at conversational speech. Second, we test the music generator MusicGen (Copet et al., 2023), which encompasses a 32kHz version of EnCodec (Défossez et al., 2022).
biasing) is an efficient table lookup. The trade-offs are a reduction in the entropy of the autoregressive model distribution, as well as potential key collisions. We acknowledge that KGW already distorts the model distribution, but our experiments, as well as previous work discussed in Section 2.2 suggest that the KGW logit biasing scheme can be applied to RVQ models (with moderate bias values δ), without significantly deteriorating the audio quality. We hypothesize that, unlike text, continuous modalities can afford slight distortion in their token sequences, assuming that the decoder is expressive enough to still create a meaningful waveform.
We evaluate by sampling 500 audios from each model per dataset. We prompt Moshi with the conversational prompts used in WMAR, as well as excerpts from the LibriSpeech dataset (Panayotov et al., 2015). For MusicGen, we create a custom dataset of short music descriptions by randomly shuffling descriptive words and music genres. We compare our method against a plain KGW reweighting scheme on the raw models, referred to as “Base”, as well as the two variants of the WMAR method which encompasses finetuning of the codecs towards idempotence. We present the experimental results for Moshi in the following subsections, and include experiments with MusicGen in Appendix D. In Section 5.4, we also experiment with text-to-speech
5. Experiments For our main audio generation experiments, we consider two state-of-the-art audio generative models with autoregressive architecture, that span different applications and 5
Hidden in Plain Tokens Table 1. Audio quality scores for audios generated by Moshi prompted with conversational audio (top) and LibriSpeech (bottom). Method
FAD ↓ VGGish CLAP
None
0.080
0.023
3.54 ± 0.49
4.43 ± 0.57
0
Base WMAR (aug) WMAR Ours
0.128 0.267 0.407 0.133
0.020 0.067 0.032 0.027
3.46 ± 0.51 3.43 ± 0.50 3.28 ± 0.48 3.53 ± 0.51
4.46 ± 0.48 4.45 ± 0.48 4.41 ± 0.48 4.42 ± 0.54
1
Base WMAR (aug) WMAR Ours
0.068 0.218 0.357 0.051
0.014 0.055 0.024 0.015
3.56 ± 0.49 3.54 ± 0.48 3.37 ± 0.44 3.58 ± 0.44
4.48 ± 0.40 4.46 ± 0.40 4.43 ± 0.40 4.50 ± 0.30
2
Base WMAR (aug) WMAR Ours
0.111 0.189 0.336 0.110
0.021 0.062 0.030 0.016
3.50 ± 0.53 3.47 ± 0.52 3.30 ± 0.48 3.53 ± 0.46
4.44 ± 0.54 4.43 ± 0.54 4.39 ± 0.53 4.46 ± 0.45
h
Method
FAD ↓ VGGish CLAP
None
1.921
0.063
3.15 ± 0.77
3.79 ± 1.02
0
Base WMAR (aug) WMAR Ours
1.858 2.153 2.195 1.670
0.072 0.113 0.093 0.074
3.00 ± 0.82 2.98 ± 0.82 2.89 ± 0.77 3.07 ± 0.77
3.72 ± 1.11 3.69 ± 1.11 3.66 ± 1.10 3.79 ± 0.99
1
Base WMAR (aug) WMAR Ours
1.948 2.036 2.018 1.921
0.079 0.109 0.092 0.074
3.23 ± 0.68 3.19 ± 0.68 3.12 ± 0.66 3.22 ± 0.62
4.09 ± 0.73 4.06 ± 0.74 4.03 ± 0.73 4.03 ± 0.77
2
Base WMAR (aug) WMAR Ours
1.966 1.985 2.089 1.823
0.073 0.096 0.089 0.059
3.11 ± 0.76 3.10 ± 0.77 3.01 ± 0.72 3.15 ± 0.75
3.81 ± 1.02 3.78 ± 1.02 3.75 ± 0.99 3.80 ± 1.00
h
Figure 5. Experiments with the Moshi model with h = 1 (top) and h = 2 (bottom), both prompted by LibriSpeech samples.
NISQA
NISQA
MOS ↑ DNSMOS
MOS ↑ DNSMOS
5.2. Audio Quality as a sequence-to-sequence audio task.
The superior detectability of our watermark, especially given that it relies on KGW, which is not unbiased in terms of sequence modeling, raises the question of whether it compromises the audio quality of the generated samples.
5.1. Detectability We evaluate detectability by showing the true positive rate (TPR) by thresholding the p-values at a desired false positive rate (FPR). This allows us to visualize the sensitivity of the watermark in very low FPR scenarios. We present some results for h = 0 in Figure 3. For higher-order h-grams, the results are presented in Figure 4 and Figure 5. We attribute the generally worse performance on LibriSpeech due to the Moshi model’s training distribution and scope as a conversational model, while LibriSpeech prompts are audiobook excerpts.
Given the in-generation nature of our watermark and the baselines, we rely on non-intrusive (reference-free) methods for quality evaluation. We use the statistical Fréchet Audio Distance (FAD) (Kilgour et al., 2019), computing the FAD score between sets of watermarked audios and their unwatermarked counterparts generated from identical prompts. We employ two different feature extractors under the hood, namely the convolutional VGGish (Hershey et al., 2017) and the transformer-based CLAP (Elizalde et al., 2023). We also report the predicted mean opinion score (MOS) from the NISQA (Mittag et al., 2021) and DNSMOSPro (Cumlin et al., 2024) models. We present the full results in Table 1, demonstrating that our method does not severely impact the audio quality compared to the baselines.
All figures showcase that our watermark is several orders of magnitude stronger than the baselines, even finetuned methods like WMAR. The distilled vocabulary is extremely effective at alleviating the errors, which contribute to discounted green tokens not only in the watermark stream, but also in the context used to determine the pseudo-randomness.
Our method consistently outperforms the baselines in every 6
Hidden in Plain Tokens Table 2. Mean − log p-values and Loss (relative to Identity) for Moshi using conversational audio (top) and LibriSpeech (bottom).
Base − log(p) Loss
WMAR − log(p) Loss
WMAR (aug) − log(p) Loss
Ours − log(p) Loss
0.00
17.44
0.00
13.72
0.00
42.47
0.00
1.99 5.82 1.11 2.23
6.52 2.69 7.40 6.28
1.61 9.23 1.50 0.61
15.84 8.21 15.95 16.83
3.73 10.52 1.94 8.01
9.99 3.20 11.79 5.71
32.68 41.51 25.59 20.59
9.80 0.96 16.89 21.89
Compression
MP3 DAC EnCodec SpeechTok FaCodec
7.47 6.62 2.59 4.48 4.73
1.04 1.89 5.92 4.03 3.77
15.31 8.12 2.82 4.29 5.36
2.14 9.32 14.62 13.15 12.08
12.66 10.51 2.78 4.28 4.75
1.06 3.22 10.95 9.44 8.97
41.26 40.13 32.64 35.92 38.47
1.22 2.35 9.84 6.55 4.00
Temporal
Crop Shift Speedup
1.51 1.86 1.52
7.00 6.65 6.99
1.27 1.81 1.20
16.18 15.63 16.25
1.51 2.36 1.35
12.22 11.37 12.37
16.48 27.67 26.49
26.00 14.80 15.99
Group
Transformation
Base − log(p) Loss
Baseline
Identity
3.61
0.00
7.95
0.00
5.98
0.00
22.77
0.00
Signal Proc.
Smooth Lowpass Highpass Noise
1.50 2.99 0.97 1.49
2.11 0.61 2.63 2.12
1.22 4.60 1.16 0.47
6.72 3.35 6.79 7.48
2.02 4.86 1.69 3.74
3.96 1.11 4.29 2.23
16.96 21.05 14.29 9.19
5.82 1.72 8.49 13.58
MP3 DAC EnCodec SpeechTok
3.30 3.10 1.65 2.43
0.30 0.50 1.96 1.18
7.27 4.19 1.78 2.46
0.68 3.76 6.16 5.49
5.67 4.90 1.79 2.27
0.31 1.08 4.18 3.70
22.29 22.10 17.56 19.50
0.49 0.68 5.22 3.28
Crop Shift Speedup
1.24 1.36 1.06
2.37 2.24 2.55
1.24 1.51 1.07
6.71 6.44 6.88
1.39 1.79 1.23
4.58 4.19 4.75
9.70 16.09 15.54
13.08 6.68 7.23
Group
Transformation
Baseline
Identity
8.51
Signal Proc.
Smooth Lowpass Highpass Noise
Compression
Temporal
WMAR − log(p) Loss
WMAR (aug) − log(p) Loss
Ours − log(p) Loss
Table 3. Audio quality in the text-to-speech setting, with audios generated by the CosyVoice3 and Spark-TTS models.
Model
Method
FAD ↓ VGGish CLAP
MOS ↑ NISQA DNSMOS
ASR ↓ WER CER
CosyVoice3
None Base Ours
0.0964 0.1954 0.1942
0.0235 0.0267 0.0294
3.86 3.75 3.81
2.73 2.70 2.74
0.0323 0.0586 0.0519
0.0178 0.0366 0.0308
Spark-TTS
None Base Ours
0.2057 0.2221 0.3506
0.0401 0.0484 0.0472
3.37 3.30 3.46
2.94 2.97 2.96
0.0097 0.0098 0.0099
0.0018 0.0012 0.0024
5.3. Robustness
attack scenario, although all token-based methods are reasonably robust to modifications that do not cause temporal misalignment. Such modifications, like cropping or speed changes, can inherently break token-level watermarks. We address this limitation in Appendix B.
A watermark’s robustness to modifications is crucial for practical applications, since even benign modifications can be detrimental to the watermarking signal. Codecs are ubiquitously used for efficiency, and everyday users use editing
7
Hidden in Plain Tokens Table 4. Watermark detectability metrics for the CosyVoice3 and Spark-TTS models.
Model
Method
p↓
− log(p) ↑
Cosyvoice3
None Base Ours
0.1885 0.03394 4.89 · 10−14
0.863 1.564 13.927
Spark-TTS
None Base Ours
0.4953 2.061 · 10−9 5.466 · 10−18
0.400 9.237 17.806
These results suggest that our method is highly versatile, generalizing to the flow-matching enhancement paradigm, without severely compromising quality.
6. Conclusion In this work, we addressed the fundamental instability of inference-time watermarking for autoregressive audio generation. Rather than resorting to invasive and costly codec finetuning of the discretization module towards idempotence, we propose an elegant, lightweight, and gradient-free alternative that exploits the intrinsic redundancy of the discrete vocabulary. By modeling the retokenization errors as stochastic transitions in a graph topology, we derive robust token communities that significantly alleviate the discretization mismatch. Our evaluations on state-of-the-art models spanning different audio generative tasks, demonstrate that this simple structural intervention yields unprecedented detectability, without degrading generation quality. Ultimately, our findings suggest that the perceived fragility of tokenlevel audio watermarking is solvable by merely distilling the models’ learned discrete representation space.
tools built into social media platforms to edit their content for viewership purposes. Furthermore, users may maliciously attempt to post-process AI-generated audios to erase potential watermarks, even if they have no knowledge of the watermarking mechanism. To evaluate the robustness of our method under realistic conditions, we apply a diverse suite of transformations, including signal processing modifications (smoothing, lowpass filtering, highpass filtering, noise addition), diverse audio codecs (MP3, DAC (Kumar et al., 2023), EnCodec (Défossez et al., 2022), SpeechTokenizer (Zhang et al., 2023), and FaCodec (Ju et al., 2024)), and temporal modifications (cropping, shift, speedup). The detailed settings are presented in Appendix E.
Acknowledgements This work was partially supported by NSF IIS 2347592, 2348169, DBI 2405416, CCF 2348306, CNS 2347617, RISE 2536663.
The results are summarized in Table 2, while additional results are included in the Appendix D. We report the average − log p-value for each setting, as well as the reduction (Loss) relative to the identity, to highlight which attacks have a stronger impact on the token-level watermarks. We emphasize that the 1% FPR threshold corresponds to − log p = 2.
Impact Statement Our work provides a valuable contribution to the field of watermarking for AI-generated content, which is essential for provenance efforts and trust in digital media. AI governance requires transparency and accountability, and explicitly marking synthetic content in a statistically robust way is extremely useful. For instance, given adoption by large model providers, the watermark detection mechanism can be deployed in social media in order to automatically flag synthetic content. Moreover, a very low p-value may serve as sufficient grounds to dismiss fabricated evidence in court. Potential negative societal impact would be concerns regarding surveillance, loss of anonymity, and nonconsensual tracking of a user’s digital activity. Furthermore, deployment of watermarking without clear disclosure policies could lead to erosion of trust in generative AI tools. Altogether, the impacts highlight the importance of ethical deployment, transparency, and user literacy regarding provenance methods.
5.4. Extension to Flow-Enhanced Text-to-Speech We examine our method’s generalization to a sequence-tosequence task rather than pure generative modeling. We perform a case study on mainstream text-to-speech (TTS) models. Specifically, we examine whether our method can generalize to state-of-the-art models with autoregressive core and subsequent flow-matching for acoustic refining. In this paradigm, flow matching can be considered part of the decoder shown in Figure 2. Our method relies on the effective clustering of retokenization errors, which are also available in such architectures. We experiment with CosyVoice3 (Du et al., 2025) and SparkTTS (Wang et al., 2025), using the conversational prompts as text input. We present the watermark detectability in Table 4 as median p-values and average − log p, demonstrating orders of magnitude better detectability than the KGW method. We present the corresponding audio quality in Table 3, where we also include automatic speech recognition (ASR) errors using the Whisper model (Radford et al., 2022). 8
Hidden in Plain Tokens
References
Du, Z., Gao, C., Wang, Y., Yu, F., Zhao, T., Wang, H., Lv, X., Wang, H., Shi, X., An, K., et al. Cosyvoice 3: Towards in-the-wild speech generation via scaling-up and post-training. arXiv preprint arXiv:2505.17589, 2025.
Aaronson, S. My AI safety lecture for UT effective altruism,. 2022. URL https://scottaaronson. blog/?p=6823.
Elizalde, B., Deshmukh, S., Al Ismail, M., and Wang, H. Clap learning audio concepts from natural language supervision. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5. IEEE, 2023.
Agostinelli, A., Denk, T. I., Borsos, Z., Engel, J., Verzetti, M., Caillon, A., Huang, Q., Jansen, A., Roberts, A., Tagliasacchi, M., Sharifi, M., Zeghidour, N., and Frank, C. Musiclm: Generating music from text. arXiv preprint arXiv:2301.11325, 2023.
Ge, Y., Zhao, S., Zeng, Z., Ge, Y., Li, C., Wang, X., and Shan, Y. Making llama see and draw with seed tokenizer. In ICLR, 2024.
Blondel, V. D., Guillaume, J.-L., Lambiotte, R., and Lefebvre, E. Fast unfolding of communities in large networks. Journal of statistical mechanics: theory and experiment, 2008(10):P10008, 2008.
Hershey, S., Chaudhuri, S., Ellis, D. P., Gemmeke, J. F., Jansen, A., Moore, R. C., Plakal, M., Platt, D., Saurous, R. A., Seybold, B., et al. Cnn architectures for largescale audio classification. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 131–135. IEEE, 2017.
Borsos, Z., Marinier, R., Vincent, D., Kharitonov, E., Pietquin, O., Sharifi, M., Teboul, O., Grangier, D., Tagliasacchi, M., and Zeghidour, N. Audiolm: a language modeling approach to audio generation. arXiv preprint arXiv:2209.03143, 2022.
Hu, Z., Chen, L., Wu, X., Wu, Y., Zhang, H., and Huang, H. Unbiased watermark for large language models. In International Conference on Learning Representations, volume 2024, pp. 45408–45436, 2024.
Chen, G., Wu, Y., Liu, S., Liu, T., Du, X., and Wei, F. Wavmark: Watermarking for audio generation. arXiv preprint arXiv:2308.12770, 2023. Copet, J., Kreuk, F., Gat, I., Remez, T., Kant, D., Synnaeve, G., Adi, Y., and Défossez, A. Simple and controllable music generation. Advances in Neural Information Processing Systems, 36:47704–47720, 2023.
Jovanović, N., Labiad, I., Souček, T., Vechev, M., and Fernandez, P. Watermarking autoregressive image generation. The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025.
Cumlin, F., Liang, X., Ungureanu, V., Reddy, C. K., Schüldt, C., and Chatterjee, S. Dnsmos pro: A reduced-size dnn for probabilistic mos of speech. In 25th Interspeech Conferece 2024, Kos Island, Greece, Sep 1 2024-Sep 5 2024, pp. 4818–4822. International Speech Communication Association, 2024.
Ju, Z., Wang, Y., Shen, K., Tan, X., Xin, D., Yang, D., Liu, Y., Leng, Y., Song, K., Tang, S., et al. Naturalspeech 3: zero-shot speech synthesis with factorized codec and diffusion models. In Proceedings of the 41st International Conference on Machine Learning, pp. 22605– 22623, 2024.
Defferrard, M., Benzi, K., Vandergheynst, P., and Bresson, X. FMA: A dataset for music analysis. In 18th International Society for Music Information Retrieval Conference (ISMIR), 2017.
Kilgour, K., Zuluaga, M., Roblek, D., and Sharifi, M. Fréchet audio distance: A reference-free metric for evaluating music enhancement algorithms. In Interspeech, 2019.
Défossez, A., Copet, J., Synnaeve, G., and Adi, Y. High fidelity neural audio compression. Transactions on Machine Learning Research, 2022.
Kirchenbauer, J., Geiping, J., Wen, Y., Katz, J., Miers, I., and Goldstein, T. A watermark for large language models. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 17061–17084. PMLR, 2023.
Défossez, A., Mazaré, L., Orsini, M., Royer, A., Pérez, P., Jégou, H., Grave, E., and Zeghidour, N. Moshi: a speech-text foundation model for real-time dialogue. arXiv preprint arXiv:2410.00037, 2024.
Kumar, R., Seetharaman, P., Luebs, A., Kumar, I., and Kumar, K. High-fidelity audio compression with improved rvqgan. Advances in Neural Information Processing Systems, 36:27980–27993, 2023.
Ding, D., Ju, Z., Leng, Y., Liu, S., Liu, T., Shang, Z., Shen, K., Song, W., Tan, X., Tang, H., et al. Kimi-audio technical report. arXiv preprint arXiv:2504.18425, 2025. 9
Hidden in Plain Tokens
Lakhotia, K., Kharitonov, E., Hsu, W.-N., Adi, Y., Polyak, A., Bolte, B., Nguyen, T.-A., Copet, J., Baevski, A., Mohamed, A., et al. On generative spoken language modeling from raw audio. Transactions of the Association for Computational Linguistics, 9:1336–1354, 2021.
Sugiura, I., Kurita, S., Oda, Y., and Higashinaka, R. Llamamimi: Exploring the limits of flattened speech language modeling. arXiv preprint arXiv:2509.14882v2, 2025. Tong, Y., Pan, Z., Yang, S., and Zhou, K. Training-free watermarking for autoregressive image generation. arXiv preprint arXiv:2505.14673, 2025.
Lie, W.-N. and Chang, L.-C. Robust and high-quality timedomain audio watermarking based on low-frequency amplitude modification. IEEE transactions on multimedia, 8(1):46–59, 2006.
Traag, V. A., Waltman, L., and Van Eck, N. J. From louvain to leiden: guaranteeing well-connected communities. Scientific reports, 9(1):1–12, 2019.
Liu, A., Pan, L., Hu, X., Li, S., Wen, L., King, I., and Yu, P. S. An unforgeable publicly verifiable watermark for large language models. arXiv preprint arXiv:2307.16230, 2023a.
Wang, X., Zhang, X., Luo, Z., Sun, Q., Cui, Y., Wang, J., Zhang, F., Wang, Y., Li, Z., Yu, Q., et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024.
Liu, A., Pan, L., Hu, X., Meng, S., and Wen, L. A semantic invariant robust watermark for large language models. arXiv preprint arXiv:2310.06356, 2023b. Liu, C., Zhang, J., Zhang, T., Yang, X., Zhang, W., and Yu, N. Detecting voice cloning attacks via timbre watermarking. In Network and Distributed System Security Symposium, 2024. doi: 10.14722/ndss.2024.24200.
Wang, X., Jiang, M., Ma, Z., Zhang, Z., Liu, S., Li, L., Liang, Z., Zheng, Q., Wang, R., Feng, X., Bian, W., Ye, Z., Cheng, S., Yuan, R., Zhao, Z., Zhu, X., Pan, J., Xue, L., Zhu, P., Chen, Y., Li, Z., Chen, X., Xie, L., Guo, Y., and Xue, W. Spark-tts: An efficient llmbased text-to-speech model with single-stream decoupled speech tokens, 2025. URL https://arxiv.org/ abs/2503.01710.
Mittag, G., Naderi, B., Chehadi, A., and Möller, S. Nisqa: A deep cnn-self-attention model for multidimensional speech quality prediction with crowdsourced datasets. In Proc. Interspeech 2021, pp. 2127–2131, 2021.
Wu, S., Liu, J., Huang, Y., Guan, H., and Zhang, S. Adversarial audio watermarking: Embedding watermark into deep feature. In 2023 IEEE International Conference on Multimedia and Expo (ICME), pp. 61–66. IEEE, 2023.
O’Reilly, P., Jin, Z., Su, J., and Pardo, B. Deep audio watermarks are shallow: Limitations of post-hoc watermarking techniques for speech. In The 1st Workshop on GenAI Watermarking, 2025.
Wu, Y., Chen, R., Hu, Z., Chen, Y., Guo, J., Zhang, H., and Huang, H. Distortion-free watermarks are not truly distortion-free under watermark key collisions. arXiv preprint arXiv:2406.02603, 2024.
O’Reilly, P., Seetharaman, P., Su, J., Jin, Z., and Pardo, B. Code drift: Towards idempotent neural audio codecs. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5. IEEE, 2025.
Wu, Y., Chen, R., Milis, G., Guo, J., and Huang, H. A watermark for auto-regressive speech generation models. In Proc. Interspeech, pp. 3474–3478, 2025a. Wu, Y., Milis, G., Chen, R., and Huang, H. Robust distortion-free watermark for autoregressive audio generation models. The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025b.
Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 5206–5210. IEEE, 2015.
Zen, H., Dang, V., Clark, R., Zhang, Y., Weiss, R. J., Jia, Y., Chen, Z., and Wu, Y. Libritts: A corpus derived from librispeech for text-to-speech. In Proc. Interspeech 2019, pp. 1526–1530, 2019.
Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., and Sutskever, I. Robust speech recognition via large-scale weak supervision. arxiv 2022. arXiv preprint arXiv:2212.04356, 10, 2022.
Zhang, X., Zhang, D., Li, S., Zhou, Y., and Qiu, X. Speechtokenizer: Unified speech tokenizer for speech language models. In The Twelfth International Conference on Learning Representations, 2023.
San Roman, R., Fernandez, P., Elsahar, H., Défossez, A., Furon, T., and Tran, T. Proactive detection of voice cloning with localized watermarking. In International Conference on Machine Learning, volume 235, 2024.
Zhao, X., Ananth, P., Li, L., and Wang, Y.-X. Provable robust watermarking for ai-generated text. arXiv preprint arXiv:2306.17439, 2023. 10
Hidden in Plain Tokens
A. Extensibility Our vocabulary distillation method could in principle be applied to other multimodal autoregressive models, such as Emu3 (Wang et al., 2024). Since we consider the RVQ architecture of modern audio codecs, where each channel typically has a much smaller vocabulary size than image discretizers, we hypothesize that an extension to the image domain could require different, perhaps much more diverse sampling of the codec, or tailored hyperparameter settings in order to be effective. Indeed, Jovanović et al. (2025) already acknowledge that some image codebooks suffer from low utilization, requiring stratification to balance the red and green lists. We thus choose to focus on audio and provide a comprehensive audio-specific evaluation, leaving applications with different modality-specific technicalities for future work.
B. Limitations Our watermark’s main limitation is its brittleness to temporal modifications, which are common in social media settings, especially in short form content where cropping, speedup, or slowdown are typically used for user engagement. We argue that this limitation is inherent to any token-based in-generation watermark scheme due to a misaligned waveform being separated into non-overlapping frames, leading to different tokenization. However, it does not negate the superiority over post-hoc watermarks in sophisticated codecs, and is addressable via post-processing. For instance, cropping can be easily bypassed by zero-padding the start of the waveform by uniformly spaced crop offsets between zero and the tokenizer’s window size, performing detection on all padded versions. If the waveform is indeed watermarked but cropped, padding by the right amount will align it back to the original tokenization. A similar linear search strategy can be applied with different speed modifications that resynchronize the modified audio by matching it back to the original speed. A lightweight speed estimator trained on the adopted codec would also be a viable solution for industry practitioners. Furthermore, watermark synchronization methods can be used for explicitly encoding alignment information, such as a post-hoc watermark modulated by a square wave (Jovanović et al., 2025). Policymakers and industry practitioners may decide to use an ensemble of different watermarking approaches to ensure robustness to any type of in-the-wild modifications. Finally, while our method is gradient-free, capture and clustering of the retokenization errors is required. Thus, adaptation to a new model still requires some light computational work, as well as black-box access to the model’s tokenizer.
C. Extended Theoretical Analysis C.1. Green Tokens After Corruption Under the null hypothesis, H0 , the expected percentage of green tokens in a generated sequence would be γ. Then, the expected ratio of green tokens after retokenization would be γr + γ(1 − r)γ + (1 − γ)(1 − r)γ = γ.
(9)
Under H1 , assuming correct context h, and with an expected ratio of green tokens g > γ, we would have gr + g(1 − r)γ + (1 − g)(1 − r)γ = gr + (1 − r)γ = γ + r(g − γ).
(10)
If the context is correct, which happens with probability rh , the expectation of green tokens is rh (γ + r(g − γ)). If it’s corrupted, with probability (1 − rh ), the partition becomes random with green probability of γ. Therefore, the final expected ratio is rh (γ + r(g − γ)) + (1 − rh )γ = γ + rh+1 (g − γ), (11) which is exactly Equation (4). C.2. Extension to Multi-Channel To formalize the benefit of our proposed clustering approach in this a multi-channel setting like RVQ, we derive the expected z-score for the combined multi-channel statistic. For a single channel c, the expected shift in the green count is determined by the robustness of the matching mechanism. The signal is preserved if the retokenized vector falls into the correct semantic cluster. The total statistical deviation is the 11
Hidden in Plain Tokens Table 5. Theoretical and empirical z-scores with correlation coefficients for different h.
h=0
h=1
h=2
γ
Theoretical
Empirical
Theoretical
Empirical
Theoretical
Empirical
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
6.1123 7.0953 6.8406 6.3152 5.9394 5.0018 4.3240 3.4383
6.7318 7.0225 6.8239 5.6013 5.3666 4.5129 4.1575 3.5239
4.9797 5.7805 5.5730 5.1450 4.8389 4.0749 3.5228 2.8012
9.2911 7.5169 7.5557 6.3814 5.6507 4.9890 3.9197 3.0123
4.0570 4.7094 4.5404 4.1916 3.9422 3.3199 2.8700 2.2821
7.6467 6.8360 6.0868 5.7638 4.6116 4.2382 2.9864 2.3622
Correlation
0.9465
0.8535
0.8872
sum across all channels C X
E[Gsum,c − µ0 ] =
c=1
C X
N (gc − γ)(rcl,c )h+1 .
(12)
c=1
Due to independence across channels, the variances sum linearly as 2 σtotal =
C X
σ02 = CN γ(1 − γ).
(13)
c=1
Thus, much like in the single-channel case, the expected statistic E[ztotal ] is the total signal standardized by the total variance PC E[ztotal ] =
h+1 c=1 N (gc − γ)(rcl,c )
p CN γ(1 − γ)
,
(14)
which is the statistic we empirically measure through the actual Gsum,c of every channel, and use it to report the watermark p-value. C.3. Validation of Analysis In order to strengthen the validity of our analysis, we empirically verify the expected z-score for single-channel watermarking (derived in Section 3.3). We use MusicGen, since we were able to embed robust watermarks on only one of its RVQ channels (unlike Moshi, where watermarking on only one of the 8 channels was not strong). First, we estimate r and g from an independent set of samples using Equations 2 and 3. Then we proceed to estimate Equation 5 for a fixed generation length N , comparing it with the actual empirical z-score from a set of samples (independent from the previous ones) in Table 5. The high correlation indicates that our modeling, while simplified, captures very well the interplay of token-level watermarking with an unstable tokenizer. Thus, boosting detectability via artificially increasing r via clustering is theoretically sound. Unfortunately, the clustering effect is not as easily predictable, and we are unable to verify Equation 7 with equally high correlation for all values of γ. We attribute this to g (the aggregate effect of watermarking) implicitly relying on the clustering map used, and thus being intractable to model. However, our derivation is still valid as a tractable simplified model indicating how watermark parameters are expected to impact the detectability. C.4. Clustering Effectiveness We finally examine how the clustering-based vocabulary reduction actually tackles the retokenization errors, leading to a strong watermarking signal. The variable of interest is rcl , defined in Section 3.4 as the probability of a token maintaining its cluster identity after retokenization, as compared to the baseline r which measures the token match. In other words, rcl is the cluster-level match that bypasses the brittleness of the token-level resolution. In Figure 6, we plot rcl across hyperparameter sweeps of the resolution ρ and the noise threshold for the Leiden community detection algorithm in the first Mimi channel. 12
Hidden in Plain Tokens
Figure 6. Clustering effectiveness of the Leiden community detection method for different hyperparameter sweeps. As expected, the cluster preservation is much stronger than the baseline token preservation r. We observe that the effectiveness increases in small resolutions (where larger clusters are encouraged, thus fewer chances of inter-cluster error), and in small noise thresholds (meaning that even rare token substitutions are important). However, both of these directions also decrease the vocabulary size measured on the horizontal axis.
Despite the effectiveness being optimal in small resolutions and small noise thresholds, the decrease in the vocabulary size can lead to distorted output and key collisions. We thus perform a per-channel selection of hyperparameters by activating the watermark only on the tokens of one channel, performing a grid search, and selecting the optimal configurations. Table 6. Mean − log p-values and Loss (relative to Identity) for MusicGen with h = 0.
Base − log(p) Loss
WMAR − log(p) Loss
Ours − log(p) Loss
Group
Transformation
Baseline
Identity
9.05
0.00
9.06
0.00
28.46
0.00
Signal Proc.
Smooth Lowpass Highpass Noise
1.01 6.31 1.27 6.88
8.04 2.74 7.78 2.16
0.85 5.47 1.36 5.53
8.21 3.59 7.70 3.53
14.39 25.55 0.72 24.92
14.08 2.91 27.74 3.54
Compression
MP3 DAC SpeechTok FaCodec
8.98 1.67 0.98 0.62
0.07 7.37 8.07 8.43
8.94 1.78 0.96 0.62
0.12 7.28 8.09 8.44
27.86 19.27 13.75 14.92
0.60 9.19 14.72 13.54
Temporal
Crop Shift Speedup
0.93 0.69 0.51
8.11 8.35 8.54
1.10 0.88 0.72
7.96 8.17 8.34
11.73 16.41 9.66
16.73 12.05 18.81
D. Supplementary Experiments We experiment with the MusicGen model in a caption-to-music task, and show the results in Figure 7. Table 7 shows the quality metrics obtained in the music generation task. We only evaluate with the FAD metrics, since MOS is tailored for speech. Finally, robustness experiments for the music task are shown in Table 6. 13
Hidden in Plain Tokens
Figure 7. Experiments with the MusicGen model with h = 0, prompted by captions describing music. Our proposed method is still superior to the baselines despite the different architecture and task.
Table 7. Audio quality scores for audios generated by MusicGen with music prompts.
h-gram
h=0
Method
FAD ↓ VGGish CLAP
None
0.247
0.039
Base WMAR Ours
0.330 1.193 1.256
0.043 0.132 0.082
E. Experimental Details We first create the vocabulary reductions for our proposed method by performing community detection on the retokenization errors of audios. For the Mimi codec and the TTS models in the flow-matching case study, we use 2.7k audios from the full development set of LibriSpeech, while for EnCodec we use 2.1k music segments from the MusicCaps dataset (Agostinelli et al., 2023). To account for potential noise and search for more robust communities, we perform a grid search of clustering hyperparameters for each channel. Namely, we perform experiments on a validation set with single channel watermarks (keeping the other channels unwatermarked) with different ρ and a noise threshold m that filters small entries in the adjacency matrix M , and eventually keep the pair that results in the highest detectability without deviating too much from the pair (ρ, m) = (1, 1). We empirically make sure that each configuration does not result in very large monolithic clusters or a low cluster vocabulary. We present these empirical selections in Table 8 for reproducibility, and present the full ablation results in Table 9. Thus, we select the optimal configuration per channel, and finally apply watermarking to all channels, each with its own optimal clustering. We use the finetuned Moshi checkpoints released by WMAR, but for MusicGen we finetune the 32kHz version of EnCodec. To finetune MusicGen’s codec, we use a balanced mixture of music and speech data from Free Music Archive (Defferrard et al., 2017) and LibriTTS (Zen et al., 2019), respectively. We use the medium sized checkpoint for generation, and finetune in the non-augmented setting. As watermarking parameters, we follow Jovanović et al. (2025) and use γ = 0.25 everywhere and δ = 2 for Moshi and Spark-TTS. We empirically attenuated the watermark strength to achieve a better detectability-quality trade-off, namely we used δ = 1 for MusicGen, and δ = 0.5 for CosyVoice3. Regarding h > 0, we choose to use the corresponding h-gram from the first channel as a hash key at each time step, since the lower channels have better robustness to retokenization. We set the generation length to 200, which leads to roughly 10 sec of audio for Moshi, and 4 sec of audio for MusicGen. As in 14
Hidden in Plain Tokens Table 8. Selected hyperparameters (ρ, m) for each model, channel-specific where applicable.
Model (codec)
Channel
Selected pair
Moshi (Mimi)
0 1 2 3
(0.8, 1) (0.8, 10) (0.8, 10) (0.8, 1)
MusicGen (EnCodec)
0 1 2 3
(0.8, 1) (0.8, 1) (1.2, 1) (1.2, 1)
CosyVoice3
0
(0.8, 1)
Spark-TTS (BiCodec)
0
(1.0, 10)
Jovanović et al. (2025), we apply the watermark on the first 4 audio streams of Moshi, leaving the semantic (text) stream untouched. Similarly, we apply the watermark on all 4 streams of MusicGen. For the robustness experiments, we use the following diverse attack suite: • Speed perturbation: Resamples the audio to increase playback speed by a factor of 1.1. • Noise injection: Adds white Gaussian noise with a standard deviation of σ = 0.01. • Lowpass filter: Attenuates frequencies above a cutoff of 3 kHz. • Highpass filter: Attenuates frequencies below a cutoff of 1 kHz. • Signal smoothing: Applies a smoothing filter to reduce high-frequency variations. • MP3 compression: Applies standard lossy compression at a bitrate of 64 kbps. • Time shift: Shifts the waveform start time by fractional frame offsets (1/8 and 1/2 of a frame) to test synchronization. • Temporal crop: Truncates the audio to retain only 50% of the original duration. • Neural compression: Transcodes the output using external neural codecs (DAC, Encodec, SpeechTokenizer, and FaCodec).
15
Hidden in Plain Tokens
Table 9. Clustering metrics and p-values evaluations across different channels.
Channel
(ρ, m)
Max cluster size
No. of clusters
− log(p)
c=0
(0.8, 1) (1.0, 5) (0.8, 5) (1.0, 10) (1.2, 10) (1.2, 5) (0.5, 10) (0.5, 5) (1.2, 1) Base
883 182 247 121 89 153 165 349 325 1
151 659 655 1071 1072 664 1067 649 164 2048
39.049 29.025 26.946 14.836 14.517 12.427 10.731 9.601 9.412 9.044
c=1
(0.5, 1) (0.8, 1) (1.2, 5) (0.8, 10) (0.5, 5) (1.2, 10) (1.0, 5) Base
1795 1611 206 128 646 103 201 1
254 256 722 1371 727 1374 718 2048
78.916 68.069 17.506 16.086 13.475 13.322 10.059 9.876
c=2
(1.2, 5) (0.8, 5) (0.8, 10) (1.2, 1) (1.2, 10) (1.0, 5) (1.0, 10) Base (0.5, 10)
330 394 164 421 85 391 119 1 282
880 899 1552 375 1536 889 1540 2048 1548
29.952 25.835 22.198 17.655 17.185 14.934 13.391 12.099 8.505
c=3
(1.0, 5) (1.2, 5) (0.8, 10) (1.2, 10) Base (0.5, 10) (1.2, 1) (0.8, 5) (1.0, 10)
427 306 116 56 1 308 691 469 122
947 950 1635 1628 2048 1640 341 946 1626
31.253 20.810 16.888 15.843 15.230 10.961 10.609 9.493 7.657
16