A Dominant Self-Conditioning Direction Drives Repetition in Unconditional Continuous Diffusion Language Models

Continuous diffusion language models offer an alternative to autoregressive generation, but their generations may suffer from repetition. We find that unconditional generations from ELF, a recent family of continuous diffusion language models, are more repetitive than human text, while Gen-PPL, a co