Official Review · ICLR 2024

The researchers are introducing a new approach called DiffusionNAG to overcome limitations in Neural Architecture Search (NAS) methods. Instead of traditional NAS, they propose a conditional Neural Architecture Generation (NAG) framework based on diffusion models. This framework, utilizing graph diffusion models, allows for efficient generation of task-optimal neural architectures with desired properties. DiffusionNAG incorporates parameterized predictors to guide the generation process, making it more efficient than previous NAS methods. The effectiveness of DiffusionNAG is demonstrated through experiments in Transferable NAS and Bayesian Optimization-based NAS scenarios. 1. This work treats neural architectures as directed graphs and introducing a graph diffusion model. It enables flexible generation of task-optimal architectures guided by parameterized predictors, surpassing the efficiency of traditional NAS methods. 2. DiffusionNAG outperforms baselines in Transferable NAS, achieving up to 20x speedups. Additionally, when integrated into a BO-based algorithm, it outperforms existing approaches, especially in the extensive MobileNetV3 search space on the ImageNet 1K dataset. 1. This work only reports results on two NAS benchmark search spaces. It is unclear whether it can achieve similar performance on much larger search spaces, such as the ones proposed in [1] and [2], which are widely used in previous works. 2. Fig. 3 and 4 compare results on existing AO strategies and various acquisition functions. However, there is no comparison to other methods on ImageNet, as shown in Tab. 1 and 2. 3. In Tab. 5, the authors include robust accuracy against the APGD attack, which is good. However, the corruption evaluation is insufficient, considering only glass blur. It would be better to include more comprehensive corruptions, such as those in ImageNet-C or CIFAR-10-C. [1] Learning Transferable Architectures for Scalable Image Recognition, CVPR 2018. [2] DARTS: Differentiable Architecture Search, ICLR 2019. Please address questions in "Weaknesses".