Official Review · ICLR 2025

This paper introduces POMONAG, an extension to DiffusionNAG that applies a many-objective diffusion model to optimize neural architecture generation for many-objective optimization. By incorporating additional performance predictors for hardware efficiency metrics such as number of parameters, multiply-accumulate operations (MACs), and inference latency, POMONAG aims to provide a more balanced approach to architecture optimization across accuracy and computational efficiency. Experiments validate POMONAG’s efficacy on two major CNN search spaces (NASBench201 and MobileNetV3). The motivation to extend DiffusionNAG to a many-objective setting is valid and POMONAG does so by incorporating both accuracy and efficiency metrics like latency and MACs, which are critical for resource-constrained environments. The paper provides extensive experimental comparisons with DiffusionNAG, including evaluations across multiple datasets and search spaces, which helps demonstrate the general applicability of POMONAG. Balancing the different objectives being optimized is also very important in my opinion. The authors do so by proposing a pareto front filtering and stretching subroutine. I have the following main concerns related to this submission, which I believe were crucial in the final decision: - **Incremental Contributions**: Although POMONAG claims to extend DiffusionNAG’s capabilities by addressing more objectives, the modifications appear incremental and lack substantial theoretical advancement. More specifically, I see the adaptation of diffusion models to accommodate multiple objectives, as described in section 3.1, more as a technical modification rather than a novel conceptual framework. I would recommend the authors to reiterate over their methodology and pinpoint the main contributions of their approach. - **Experimental Evaluation**: The benchmarks that POMONAG was evaluated contain only CNN spaces. It would be beneficial for the paper if the authors would demonstrate the efficacy of POMONAG in Transformer search spaces, such as the one from HW-GPT-Bench [1]. Most importantly, in the multi-(many-)objective experiments, the proposed method is not compared to any baseline. I would recommend the authors to add baselines in their experimental evaluation and report hypervolume indicator together with the individual objective values, as well as the search time. Ultimately, I would also be interested in visualizing the pareto front plots in the main paper. As for baselines, you can find a non-exhaustive list of simple ones in SyneTune (https://syne-tune.readthedocs.io/en/latest/getting_started.html#supported-multi-objective-optimization-methods). Finally, the experiments lack a thorough ablation study that demonstrates the impact of POMONAG’s unique contributions independently of DiffusionNAG’s foundational structure. - **Clarity and Presentation**: The paper seems to have a somehow fragmented structure, making it challenging for readers to follow the