A Mechanistic Study of Algorithmic Generalization in Diffusion Models on Modular Arithmetic

Despite their empirical success, how diffusion models generalize remains poorly understood from a mechanistic perspective. In this work, we provide a controlled mechanistic account of algorithmic learning in diffusion models. By utilizing the grokking phenomenon on an image-based modular addition ta