Official Review · ICLR 2024

This paper integrates the diffusion process into Generative Adversarial Networks (GANs) training, striving to achieve the fast image generation capabilities of GANs with the enhanced generation diversity proffered by diffusion models. The authors introduce the diffusion forward process into both the real images and the latent code of the generator, subsequently employing a neural ordinary differential equation (ODE) to facilitate the mapping from $h(0)$ to $h(u)$ for generator's input. With the diffused latent code, the generator is trained to produce images conditioned on time $u$, while the discriminator receives diffused real images. Empirical investigations conducted across three benchmark datasets—CIFAR-10, CelebA-HQ, and LSUN-Church—underscore the approach’s efficacy, showcasing noticeable improvements. Furthermore, the authors demonstrate the generator’s ability in generating images across varied $u$ values. 1. Combining diffusion models with GANs for enhanced diversity and a fast generation process presents an intriguing research topic. 2. Employing Neural Ordinary Differential Equations (NODEs) to map embeddings, while conditioning the latent code input to the generator on time, is an interesting approach. 3. The experimental section demonstrates performance improvements over the baselines. 4. The presentation is clear and easy to follow. 1. The rationale behind integrating the diffusion process into the generator, particularly in terms of applying it to the latent code input, remains unclear. This might be attributable to the discriminator being exposed to a variety of augmented images, potentially helping to avert overfitting. Meanwhile, the results in Table 3 depict a noticeably inferior performance of SPI-GAN in comparison to both Diffusion-GAN and StyleGANs, raising questions about the soundness of the approach of employing the diffusion process on the generator. 2. An ablation study could be beneficial, exploring alternatives to using the Neural Ordinary Differential Equation (NODE) on the latent code. Investigating the generator’s performance when directly conditioned on the diffused latent code $h(u)$—without the NODE—or simply conditioned on $u$, could provide valuable insights into the viability of such designs. 3. The observed performance gains over the baseline are relatively modest. Particularly in the case of LSUN-Church-256, the proposed method trails behind both Diffusion GANs and StyleGAN2. It's also inconvincing that the paper does not include a comparison of SPI-GAN with DiffusionGAN on CelebA-HQ-256, which could have provided a more comprehensive evaluation. 4. Including details on training time, particularly in comparison to DiffusionGAN, would enhance the paper’s completeness. 1. Is $u$ applied directly to the image, or is it integrated into an intermediate layer within the discriminator’s architecture? 2. Given that $h(u)$ can be derived from $i(u)$ in the current design, I'm curious to know if the gener