Official Review · ICLR 2026

This paper investigates the dynamic transition from generalization to memorization of diffusion models during the training process. The authors demonstrate across image and language models that generalization is, in fact, achieved progressively during training before the onset of memorization, finding an empirical law that the memorization time is proportional to the dataset size. Ultimately, the results suggest that generalization and memorization are distinct temporal phases, implying that a principled, dataset-size-aware early-stopping criterion can be an optimal strategy for preserving generalization and avoiding memorization in large diffusion models Strengths: * Besides image generation, they also study the memorization of the masked diffusion model in the text modality, which is novel in the generalization-memorization field to me. * They use the reproducibility of two different models trained over two disjoint datasets to show that the score function attempted to learn the real underlying distribution at the early stage. * The random hierarchy model further provides some interesting insights to the learning process, such as the partial generation. Weaknesses: * In Section 3.1, you showed that the transition point $\tau_{mem}$ scales approximately linearly with the training set size. Do you think the distribution complexity, such as the intrinsic dimension of the data and the entropy of the data, also influences the transition point? Besides, for the latent diffusion model and pixel diffusion model, is there any differences on the transition point? Including more factors into your study would make this work more thorough and robust. * The key claim of this paper is that the model first generalizes at an early stage but then memorizes after $\tau_{mem}$. But I am concerned about calling the first stage generalization. Although the validation loss indeed decreased in the first stage, it may still be too high, and the score function hasn’t learned a good distribution. As you also visualized in Figure 2 (right), the generated images before $\tau_{mem}$ have bad quality and thus cannot be treated as good generalizations. Then, the early stopping strategy fails in this case. I feel that the early stop is effective only when both the sample size and network size are large, which is also supported by Figure 6. * The dynamic transition and the linear relation between $\tau_{mem}$ and dataset size have also been revealed in prior work [1]. Could you compare the novelty of your work? [1]: Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training. https://arxiv.org/abs/2505.17638 Do you also visualize the partial generalization in real images? Is there any prior work proposing hierarchy data frameworks for images?