Probing Low Frame Rate Degradation in Neural Audio Codecs Alex Gichamba ID 1 , Moise Busogi ID 1 1
Carnegie Mellon University Africa, Rwanda
[email protected], [email protected]
arXiv:2606.16969v1 [cs.SD] 15 Jun 2026
Abstract Low frame rates in neural audio codecs are attractive for autoregressive speech synthesis, where the generation cost scales linearly with the sequence length. Recent work has demonstrated that codecs can operate at 12.5 Hz and below, but the mechanisms underlying low frame rate degradation remain insufficiently understood. We investigate these mechanisms through a controlled frame rate ablation. We reproduce a quality cliff at 6.25 Hz reported in previous works and evaluate candidate explanations: phonemic collisions and codebook saturation, neither of which shows evidence of a fundamental barrier. The cliff is instead caused by suboptimal training configuration: fixed clip duration during training yields too few tokens at low frame rates, starving the decoder of inter-token context. Once corrected, WER degrades smoothly with phonemic load down to 3.1 Hz and 1.6 Hz, suggesting the inference-time efficiency gains of low frame rate codecs are more accessible than previously assumed. Index Terms: neural audio codec, speech tokenization
1. Introduction Neural audio codecs [1] compress continuous audio waveforms into sequences of discrete tokens. Beyond their applications in audio compression, modern speech processing pipelines rely on neural audio codecs as speech tokenizers, serving as a bridge between raw audio and the language modeling frameworks that underpin text-to-speech synthesis systems [2, 3] and spoken dialogue systems [4]. The modern neural audio codec paradigm can be largely attributed to SoundStream [1] and EnCodec [5], both pairing a fully convolutional encoder-decoder with Residual Vector Quantization (RVQ) and training end-to-end with adversarial, feature matching, and multi-scale spectral reconstruction losses. DAC [6] addressed codebook collapse through factorized codes and L2-normalized quantization, and introduced Snake activations and a multi-band STFT discriminator to push fidelity to state-of-the-art levels. Subsequent work considerably diversified the design space: SpeechTokenizer [7] distilled HuBERT semantics into the first RVQ layer to disentangle content from paralinguistic information; Mimi [4] adopted a split-RVQ design with transformer bottleneck layers to achieve streamingcompatible 12.5 Hz tokenization; SNAC [8] introduced multiscale RVQ operating at multiple temporal resolutions simultaneously; and BigCodec [9] abandoned RVQ in favour of a single 8192-entry codebook, demonstrating that scaling model capacity beyond 150M parameters can compensate for the representational constraints of single-level VQ at ultra-low bitrates. Practitioners specifying a codec for speech synthesis face several key design decisions. The frame rate fr , the duration
represented by each token, has a direct and significant impact on downstream generation cost. Recent speech synthesis systems such as Moshi [4] and Qwen3-TTS [3] adopt a two-stage autoregressive pipeline: a primary decoder autoregressively generates the token sequence for the 0th codebook, after which a secondary decoder fills the remaining residual levels conditioned on it. The cost of the primary decoder scales linearly with sequence length and therefore linearly with frame rate. At 12.5 Hz a 10-second utterance requires 125 autoregressive steps; at 50 Hz the same utterance requires 500 steps. Reducing frame rate is therefore a direct lever on inference latency and throughput, independent of model size, architecture, or quantization depth.
Despite this incentive, very low frame rates have been associated with significant quality degradation, and the mechanisms behind this degradation have remained unclear. [10] trained codecs at 25, 12.5, and 6.25 Hz and observed that while the 25 Hz and 12.5 Hz models performed comparably, the 6.25 Hz model showed a significant decline particularly in intelligibility. They hypothesized that this was because at 6.25 Hz the model frequently compresses two distinct phonemes within a single frame , consistent with English speakers producing approximately 10–12 phonemes per second , and concluded that phonemic collision sets a practical lower bound on codec frame rates. Subsequent work addressed low frame rate quality through increasingly complex architectural solutions: FlexiCodec [11] introduced dynamic frame rate allocation with ASR feature-assisted dual-stream encoding and Transformer bottlenecks to preserve semantic information at rates as low as 3 Hz.
In this work we ask a more fundamental question: why does standard codec training fail at low frame rates, and is the failure intrinsic to the frame rate itself? Through a controlled ablation of the DAC framework [6] spanning 1.6 to 100 Hz, we identify a training misconfiguration as the primary cause: standard training maintains a fixed clip duration regardless of frame rate, yielding too few tokens per example at low frame rates and preventing the decoder from learning inter-token coherence. We evaluate phonemic collision and codebook saturation as alternative explanations and find no evidence of a fundamental barrier in either case (Figure 1). Once the misconfiguration is corrected, reconstruction intelligibility degrades smoothly with phonemic load, suggesting that residual degradation at very low frame rates reflects expected information loss due to the limited representational capacity. This extends to 3.125 Hz and 1.6 Hz without architectural modifications, confirming that the inference-time efficiency gains of low frame rate codecs are more accessible than previously assumed.
Fixed Tclip Fixed K
WER (%) ↓
100 75
R = fr · nq · log2 |V|
bits per second
(1)
2.2. Codebook Utilization and Entropy Efficiency Let Z(q) = {zt,q } denote the multiset of all level-q code indices emitted by the encoder across the evaluation corpus. The codebook utilization at level q is the fraction of vocabulary entries assigned to at least one frame, expressed as:
50 25
phones/frame
a vocabulary of size |V|, the bit rate is:
0 6
Uq = phones/frame
5
|{zt,q }| |V|
(2)
Let p̂q (v) = |{t : zt,q = v}| / |Z(q) | be the empirical probability of code v at level q. The entropy efficiency is:
4 3 2
ηq = 0 10
50
25
12 .5
6. 25
1 3.
1.
6
1
fr (Hz) Figure 1: Top: WER of DAC variants under fixed clip duration (solid blue) and fixed token sequence length (dashed orange). The quality cliff at 6.25 Hz under fixed clip duration disappears when sequence length is matched across frame rates, after which WER degrades smoothly and monotonically with phonemic load (bottom). Bottom: mean phonemes per frame from MFA forced alignments on LibriSpeech test-clean, extending to 3.125 Hz and 1.6 Hz where intelligible speech persists at bitrates as low as 192 bps.
2. Preliminaries 2.1. General RVQ Codec Formulation We borrow from the notation in [12] in our codec modeling below. Consider an input waveform of single channel audio to be encoded as a one dimensional signal w ∈ RT fs sampled at a frequency fs Hertz and of duration T seconds. Neural audio codecs encode input waveforms into compressed discrete representations through a two-stage process. We first define the encoder as a function E : RT fs → RT fr ×d that maps the input waveform to a continuous hidden representation, where fr is the frame rate in Hertz (typically much smaller than fs ), T fr is the number of frames in the compressed representation, and d is the dimensionality of the hidden representation. The hidden representation h is then processed by a Residual Vector Quantizer (RVQ) Q : RT fr ×d → V T fr ×nq that discretizes the representations in the latent space. V := {1, 2, . . . , |V|} is the discrete vocabulary of size |V|, and nq is the number of codebook levels. The RVQ operates residually letting each successive level quantize the residual of the previous level so that level 0 captures the coarsest representation and each subsequent level refines the residual error. Correspondingly, we define the decoder as a function D : V T fr ×nq → RT fs that approximately reconstructs the encoded waveform. The theoretical bit rate is the minimum information rate required to transmit the quantized representation. Since we have fr frames per second, each containing nq quantized codes from
H(p̂q ) log2 |V|
where
H(p̂q ) = −
X
p̂q (v) log2 p̂q (v)
v∈V
(3) A value of ηq = 1 indicates uniform usage of the full codebook, representing maximum information capacity per token. A value approaching 0 indicates codebook collapse, where the encoder maps the majority of inputs to a small set of codes.
3. Experimental Setup 3.1. Reference Codecs To situate our ablation within the broader landscape of neural audio codecs, we evaluate a set of published pretrained models without any retraining. For 16 kHz speech we include DAC16k [6] and BigCodec [9], which replaces RVQ with a single large codebook at 80 Hz. For 24 kHz speech we evaluate DAC24k, Mimi [4], SNAC [8], WavTokenizer [13], and the Qwen3TTS 12.5 Hz speech tokenizer [3]. All models are evaluated on LibriSpeech test-clean [14]. Model outputs are resampled to the target frequency for each metric that operates at a specific frequency. 3.2. Evaluation Metrics Reconstructed audio is assessed across four dimensions. 3.2.1. Intelligibility Word Error Rate (WER): is computed by transcribing reconstructed audio with MMS-1B [15] and measuring word error rate against reference transcriptions, serving as a model-based measure of intelligibility. Short-Time Objective Intelligibility (STOI) [16]: predicts speech intelligibility by computing the average correlation between clean and processed speech across short-time timefrequency regions. 3.2.2. Voice Preservation Speaker Similarity (SPK-SIM): We compute the cosine similarity between the speaker embeddings of the reference and decoded audios. We use a WAVLM-based [17] model finetuned for speaker recognition via Espnet-SPK [18]. 3.2.3. Distortion Mel Cepstral Distortion (MCD) [19]: measures spectral distance between reference and synthesized speech by computing the Euclidean distance between mel-frequency cepstral coefficients (MFCCs).
3.2.4. Pseudo Mean Opinion Scores UTMOS [20]: is a learned mean opinion score predictor that estimates perceptual naturalness without a reference signal, scored 1–5. 3.3. Frame Rate Study All models in our controlled frame rate study are based on the 16 kHz Descript Audio Codec (DAC) [6]. The encoder consists of L strided convolutional blocks, each containing three dilated residual units, followed by a Snake activation [21]. The decoder mirrors this structure with transposed convolutions. The frame rate fr is determinedQby the total stride product of the encoder, which must satisfy L l=1 sl = fs /fr for fs = 16,000 Hz. We construct codec variants spanning a wide range of frame rates by varying the per-block strides while keeping all other architectural parameters fixed. The decoder strides are the reverse of the encoder strides in all cases. Bitrate is computed from Equation 1 with nq = 12 and |V| = 1024 held constant, so that R = 120fr bps and bitrate varies solely as a function of frame rate. 3.4. Training All ablation models are trained on LibriSpeech train-clean-100 [14], a read English speech dataset sampled at 16 kHz. All models are trained for 100,000 iterations on a single NVIDIA H100-80 GPU using the Adam optimizer with the learning rate schedule from the original DAC implementation. Following the DAC training method, clips are drawn by randomly cropping each utterance to a fixed duration Tclip seconds, yielding K = ⌊Tclip · fr ⌋ tokens per training example. In the baseline configuration we follow DAC and set Tclip = 0.38 seconds for all variants. This results in substantially fewer tokens per clip at lower frame rates: as few as K = 2 at 6.25 Hz compared to K = 19 at the 50 Hz baseline. To isolate the effect of this disparity, we additionally retrain models with a fixed K.
4. Results 4.1. Performance of Reference Codecs Table 1 reports performance metrics for published and publicly available neural audio codecs evaluated on LibriSpeech test-clean. Among 16 kHz models, DAC-16k at its default 50 Hz configuration achieves a WER of 5.25% at 6,000 bps, serving as a strong upper-bound reference. BigCodec, which replaces RVQ with a single 8192-entry codebook, achieves a WER of 7.77% at a substantially lower bitrate of 1,040 bps, demonstrating that single-codebook designs can remain competitive at low bitrates. Among 24 kHz models, DAC-24k at 75 Hz achieves nearperfect STOI of 0.99 and WER of 5.01% at 24,000 bps. Mimi, which was specifically engineered for low frame rate tokenization via a transformer bottleneck and split-RVQ design, achieves STOI of 0.96 and WER of 5.52% at just 1,100 bps and 12.5 Hz, the strongest result among low-bitrate models. WavTokenizer-75 achieves comparable STOI of 0.90 at a similar bitrate of 900 bps, though with higher WER of 11.37%. Models operating below 50 Hz without architectural accommodations for low frame rates show clear degradation: SNAC at its multi-scale rates and WavTokenizer-40 at 40 Hz both exhibit substantially higher WER (13.14% and 24.36% respectively) and reduced speaker similarity, consistent with the quality cliff
we investigate in the following sections. Notably, Qwen3-TTSTokenizer achieves competitive WER of 5.43% and high SPKSIM of 0.91 despite an anomalously low STOI of 0.65, reflecting its design objective of preserving semantic content and speaker identity over waveform fidelity. 4.2. Frame Rate Ablation Table 2 reports reconstruction quality across all frame rate variants trained with the standard clip duration Tclip = 0.38s. Performance degrades gradually as frame rate decreases from 100 Hz to 12.5 Hz, with modest but consistent reductions across all metrics. At 12.5 Hz the codec remains competitive, achieving STOI of 0.89 and WER of 10.62% at 1,500 bps. However, reducing the frame rate by a further factor of two to 6.25 Hz produces a catastrophic collapse: STOI drops to 0.46, WER increases to 107.4%, MCD rises to 20.17, and SPK-SIM falls to 0.09. This sharp discontinuity, a 10× increase in WER for a 2× reduction in frame rate, motivates the analysis in the following subsections. 4.3. Phoneme Collisions A natural first hypothesis is that the quality cliff reflects a fundamental theoretic limit. At 6.25 Hz each frame spans 160 ms of audio, and given an average phoneme rate of 12.2 phonemes per second in the test set, each frame must encode, on average, 2.6 distinct phonemes, a potential cause hypothesized by [10] as the primary cause of intelligibility collapse. The bottom panel of Figure 1 confirms that phonemes per frame increases monotonically with decreasing frame rate, consistent with this hypothesis. However, the top panel reveals a strong decoupling between phonemic load and WER. Under fixed token sequence length, K training, the 6.25 Hz model encodes the same 2.6 phonemes per frame yet achieves WER of 15.37% rather than 107.4%, and intelligible speech persists down to 1.6 Hz where each frame spans over six phonemes on average. Phonemic collision is therefore a correlate of the performance cliff rather than its cause. 4.4. Codebook Utilization A second hypothesis is that codebook saturation explains the cliff: at low frame rates the encoder must map diverse 160 ms acoustic contexts into a fixed vocabulary of 1024 codes, potentially exhausting the codebook’s representational capacity. We test this by computing codebook utilization Uq and entropy efficiency ηq at each frame rate as defined in Section 2. Table 3 shows that both metrics are essentially flat across all frame rates. Utilization remains above 98.7% and entropy efficiency varies by less than 0.05 across the full range of frame rates, including at 6.25 Hz. The encoder produces welldistributed codes regardless of frame rate, and the codebook operates near capacity in all conditions. Codebook saturation is therefore ruled out as an explanation for the performance cliff. 4.5. Training Sequence Lengths as the Root Cause Having ruled out phoneme collision and codebook saturation as fundamental barriers, we turn to the training procedure. Under the standard configuration, all models are trained on clips of Tclip = 0.38 seconds, following the DAC baseline. At 50 Hz this yields K = 19 tokens per training clip, sufficient for the decoder to observe multiple token boundaries and learn intertoken coherence. At 6.25 Hz the same clip duration yields only
Table 1: Reconstruction quality of published neural audio codecs on LibriSpeech test-clean. fr denotes frame rate in Hz, nq is the number of quantization levels, |V| is codebook size, and R is bitrate. Bold denotes best within each frequency group. fr (Hz)
Model 16 kHz DAC-16k [6] BigCodec [9]
nq
|V|
R (bps) STOI ↑ WER (%) ↓ MCD ↓ SPK-SIM ↑ UTMOS ↑
12 1024 1 8192
6000 1040
0.97 0.93
5.25 7.77
2.04 3.01
0.90 0.81
4.00 4.11
24 kHz DAC-24k [6] 75 32 1024 Qwen3-TTS-Tok. [3] 12.5 16 2048 Mimi [4] 12.5 8 2048 SNAC [8] 12,23,47 3 4096 WavTokenizer-75 [13] 75 1 4096 WavTokenizer-40 [13] 40 1 4096
24000 2200 1100 984 900 480
0.99 0.65 0.96 0.88 0.90 0.85
5.01 5.43 5.52 13.14 11.37 24.36
1.02 7.11 2.49 4.07 3.92 4.99
0.94 0.91 0.88 0.59 0.66 0.52
4.06 4.13 3.92 3.02 3.79 3.57
50 80
Table 2: Performance metrics across frame rate variants trained with standard clip duration Tclip = 0.38s. † denotes the DAC baseline trained with the same parameters as variants for a controlled comparison. fr (Hz)
STOI ↑
WER (%) ↓
MCD ↓
SPK-SIM ↑
UTMOS ↑
100 50† 25 12.5 6.25
0.98 0.97 0.95 0.89 0.46
5.10 5.38 5.90 10.62 107.40
1.33 2.00 2.61 4.06 20.17
0.97 0.93 0.86 0.62 0.09
4.03 3.98 3.88 3.02 1.27
Table 3: Codebook utilization U0 and entropy efficiency η0 at RVQ level 0 across frame rate variants. At levels 1–11, all models achieve ηq > 0.90 with differences across frame rates of less than 0.01. fr (Hz)
U0
η0
6.25 12.5 25 50 100
0.987 0.997 0.999 1.000 1.000
0.822 0.817 0.837 0.866 0.868
K = 2 tokens per clip, meaning the decoder is trained almost exclusively on single-token reconstruction and never learns to produce coherent audio across token boundaries. At inference on full-length utterances, the decoder must produce sequences of 40–50 tokens, a regime it has never encountered during training. To test this hypothesis, we retrain the models with a fixed sequence length (K = 19), matching the 50 Hz baseline. Comparing the 6.25 Hz row in Table 2 against the corresponding row in Table 4, this single change recovers performance substantially: STOI improves from 0.46 to 0.89, WER drops from 107.40% to 15.37%, MCD falls from 20.17 to 3.72, and SPKSIM recovers from 0.09 to 0.62, bringing the 6.25 Hz model to near parity with the 12.5 Hz baseline at half the bitrate. The performance cliff is therefore not a consequence of any information-theoretic limit imposed by low frame rates. 4.6. Extension to Ultra-Low Frame Rates The recovery of 6.25 Hz performance raises a natural question: how low can the frame rate go while still producing intelligible
speech, provided the training clip duration is matched appropriately? We train two additional models at 3.125 Hz and 1.6 Hz using the matched-token protocol (K = 19 tokens per clip). Table 4 reports results for all matched-token models including the ultra-low frame rate variants. At 3.125 Hz, where each token spans 320 ms and the bitrate falls to just 375 bps, the model achieves STOI of 0.84 and WER of 29.36%, a substantial degradation relative to 6.25 Hz but far from the catastrophic collapse observed under standard training. At 1.6 Hz, where each token spans 625 ms and the bitrate falls to 192 bps, STOI remains at 0.76 and WER at 63.22%, indicating that the codec retains meaningful intelligibility at compression ratios that would have been considered entirely infeasible under the standard training protocol.
Table 4: Reconstruction quality across frame rate variants trained with matched sequence length K = 19 (Tclip = 19/fr ). † denotes the DAC baseline. fr (Hz)
STOI ↑
WER (%) ↓
MCD ↓
SPK-SIM ↑
UTMOS ↑
100 50† 25 12.5 6.25 3.125 1.6
0.98 0.97 0.95 0.93 0.89 0.84 0.76
5.02 5.38 5.79 7.17 15.37 29.36 63.22
1.45 2.00 2.50 3.06 3.72 4.63 5.67
0.96 0.93 0.90 0.82 0.62 0.48 0.32
3.99 3.98 3.99 4.00 3.80 3.24 2.67
5. Conclusion We investigated why neural audio codecs degrade at low frame rates, asking whether the failure is intrinsic to the frame rate itself. Phonemic collision and codebook saturation show no evidence of a fundamental barrier. The degradation instead resulted from a training misconfiguration. However, when sequence length is fixed during training, reconstruction intelligibility degrades smoothly with phonemic load, suggesting that residual degradation at very low frame rates reflects expected information loss due to the limited representational capacity. This extends to 3.125 Hz and 1.6 Hz at bitrates as low as 192 bps, showing that the inference-time efficiency gains of low frame rate codecs are more accessible than previously assumed.
6. Acknowledgments This publication was developed as part of the African Engineering and Technology Network (Afretec), which is managed by Carnegie Mellon University Africa and receives financial support from the Mastercard Foundation. The views expressed in this document are solely those of the authors and do not necessarily reflect those of Carnegie Mellon University Africa or the Mastercard Foundation. This work used Bridges2 at Pittsburgh Supercomputing Center through allocation CIS250700 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program [22], which is supported by U.S. National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296.
7. Generative AI Use Disclosure Generative AI tools were used in two limited capacities in this work. First, they were used to develop portions of the code that supported our experiments. All generated code was reviewed and validated by the authors. Second, they were used to identify grammatical errors in the manuscript and suggest corrections. All outputs were reviewed and approved by the authors.
8. References [1] N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasacchi, “SoundStream: An End-to-End Neural Audio Codec,” IEEE/ACM Trans. Audio, Speech and Lang. Proc., vol. 30, pp. 495–507, Nov. 2021. [Online]. Available: https: //doi.org/10.1109/TASLP.2021.3129994 [2] C. Wang, S. Chen, Y. Wu, Z. Zhang, L. Zhou, S. Liu, Z. Chen, Y. Liu, H. Wang, J. Li, L. He, S. Zhao, and F. Wei, “Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers,” Jan. 2023, arXiv:2301.02111 [cs]. [Online]. Available: http://arxiv.org/abs/2301.02111 [3] H. Hu, X. Zhu, T. He, D. Guo, B. Zhang, X. Wang, Z. Guo, Z. Jiang, H. Hao, Z. Guo, X. Zhang, P. Zhang, B. Yang, J. Xu, J. Zhou, and J. Lin, “Qwen3-TTS Technical Report,” Jan. 2026, arXiv:2601.15621 [cs]. [Online]. Available: http://arxiv.org/abs/2601.15621 [4] A. Défossez, L. Mazaré, M. Orsini, A. Royer, P. Pérez, H. Jégou, E. Grave, and N. Zeghidour, “Moshi: a speech-text foundation model for real-time dialogue,” Oct. 2024, arXiv:2410.00037 [eess]. [Online]. Available: http://arxiv.org/abs/2410.00037 [5] A. Défossez, J. Copet, G. Synnaeve, and Y. Adi, “High Fidelity Neural Audio Compression,” Transactions on Machine Learning Research, Apr. 2023. [Online]. Available: https: //openreview.net/forum?id=ivCd8z8zR2 [6] R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, “High-Fidelity Audio Compression with Improved RVQGAN,” Advances in Neural Information Processing Systems, vol. 36, pp. 27 980–27 993, Dec. 2023. [Online]. Available: https://proceedings.neurips.cc/paper files/paper/2023/hash/ 58d0e78cf042af5876e12661087bea12-Abstract-Conference. html [7] X. Zhang, D. Zhang, S. Li, Y. Zhou, and X. Qiu, “SpeechTokenizer: Unified Speech Tokenizer for Speech Language Models,” Oct. 2023. [Online]. Available: https: //openreview.net/forum?id=AF9Q8Vip84 [8] H. Siuzdak, F. Grötschla, and L. A. Lanzendörfer, “SNAC: MultiScale Neural Audio Codec,” Oct. 2024. [Online]. Available: https://openreview.net/forum?id=PFBF5ctj4X [9] D. Xin, X. Tan, S. Takamichi, and H. Saruwatari, “BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codec,” Sep. 2024, arXiv:2409.05377 [eess]. [Online]. Available: http: //arxiv.org/abs/2409.05377
[10] E. Casanova, P. Neekhara, R. Langman, S. Hussain, S. Ghosh, X. Yang, A. Jukić, J. Li, and B. Ginsburg, “NanoCodec: Towards High-Quality Ultra Fast Speech LLM Inference,” Aug. 2025, arXiv:2508.05835 [eess]. [Online]. Available: http://arxiv.org/abs/2508.05835 [11] J. Li, Y. Qian, Y. Hu, L. Zhang, X. Wang, H. Lu, M. Thakker, J. Li, S. Zhao, and Z. Wu, “FlexiCodec: A Dynamic Neural Audio Codec for Low Frame Rates,” Oct. 2025, arXiv:2510.00981 [cs]. [Online]. Available: http://arxiv.org/abs/2510.00981 [12] S.-L. Wu, A. Lahoti, A. D. Desai, K. Goel, C. Donahue, and A. Gu, “Towards Codec-LM Co-design for Neural Codec Language Models,” in Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 4: Student Research Workshop), A. Ebrahimi, S. Haider, E. Liu, S. Haider, M. Leonor Pacheco, and S. Wein, Eds. Albuquerque, USA: Association for Computational Linguistics, Apr. 2025, pp. 55–65. [Online]. Available: https: //aclanthology.org/2025.naacl-srw.6/ [13] S. Ji, Z. Jiang, W. Wang, Y. Chen, M. Fang, J. Zuo, Q. Yang, X. Cheng, Z. Wang, R. Li, Z. Zhang, X. Yang, R. Huang, Y. Jiang, Q. Chen, S. Zheng, and Z. Zhao, “WAVTOKENIZER: AN EFFICIENT ACOUSTIC DISCRETE CODEC TOKENIZER FOR AUDIO LANGUAGE MODELING,” 2025. [14] V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An ASR corpus based on public domain audio books,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Apr. 2015, pp. 5206– 5210. [Online]. Available: https://ieeexplore.ieee.org/document/ 7178964/ [15] V. Pratap, A. Tjandra, B. Shi, P. Tomasello, A. Babu, S. Kundu, A. Elkahky, Z. Ni, A. Vyas, M. Fazel-Zarandi, A. Baevski, Y. Adi, X. Zhang, W.-N. Hsu, A. Conneau, and M. Auli, “Scaling Speech Technology to 1,000+ Languages,” May 2023, arXiv:2305.13516 [cs]. [Online]. Available: http://arxiv.org/abs/2305.13516 [16] C. H. Taal, R. C. Hendriks, R. Heusdens, and J. Jensen, “A short-time objective intelligibility measure for time-frequency weighted noisy speech,” in 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, Mar. 2010, pp. 4214–4217. [Online]. Available: https://ieeexplore.ieee.org/ document/5495701/ [17] S. Chen, C. Wang, Z. Chen, Y. Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y. Qian, Y. Qian, J. Wu, M. Zeng, X. Yu, and F. Wei, “WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing,” IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1505–1518, Oct. 2022. [Online]. Available: https://ieeexplore.ieee.org/document/9814838 [18] J. weon Jung, W. Zhang, J. Shi, Z. Aldeneh, T. Higuchi, A. Gichamba, B.-J. Theobald, A. Hussen Abdelaziz, and S. Watanabe, “ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf models,” in Interspeech 2024, 2024, pp. 4278–4282. [19] R. Kubichek, “Mel-cepstral distance measure for objective speech quality assessment,” in Proceedings of IEEE Pacific Rim Conference on Communications Computers and Signal Processing, vol. 1, May 1993, pp. 125–128 vol.1. [Online]. Available: https://ieeexplore.ieee.org/document/407206/ [20] T. Saeki, D. Xin, W. Nakata, T. Koriyama, S. Takamichi, and H. Saruwatari, “UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022,” in Interspeech 2022. ISCA, Sep. 2022, pp. 4521–4525. [Online]. Available: https://www. isca-archive.org/interspeech 2022/saeki22c interspeech.html [21] L. Ziyin, T. Hartwig, and M. Ueda, “Neural Networks Fail to Learn Periodic Functions and How to Fix It,” in Advances in Neural Information Processing Systems, vol. 33. Curran Associates, Inc., 2020, pp. 1583–1594. [Online]. Available: https://proceedings.neurips.cc/paper/2020/ hash/1160453108d3e537255e9f7b931f4e90-Abstract.html
[22] T. J. Boerner, S. Deems, T. R. Furlani, S. L. Knuth, and J. Towns, “Access: Advancing innovation: Nsf’s advanced cyberinfrastructure coordination ecosystem: Services & support,” in Practice and Experience in Advanced Research Computing 2023: Computing for the Common Good, ser. PEARC ’23. New York, NY, USA: Association for Computing Machinery, 2023, p. 173–176. [Online]. Available: https: //doi.org/10.1145/3569951.3597559