Official Review · ICLR 2026

The paper introduces PhyRL, a Physics-Guided Reinforcement Learning framework that improves the robustness and out-of-distribution generalization of spatio-temporal Graph Neural Networks. A Physics Reward Oracle provides rewards based on physical consistency, robustness, and uncertainty reliability, guiding the model via reinforcement learning rather than standard loss minimization. Experiments on N-body, Navier–Stokes, and sea surface temperature tasks show that PhyRL significantly enhances accuracy, stability, and uncertainty calibration, offering a new reward-driven approach to physics-informed modeling. - The paper presents a novel perspective by reframing physics-informed learning as a reinforcement learning problem, where physical laws are not imposed as soft constraints but embedded as verifiable reward signals. - The work addresses the limitation of current spatio-temporal GNNs—poor OOD generalization—and introduces a broadly applicable framework for improving the physical fidelity and reliability of data-driven dynamical models. - The paper is well-organized and clearly written, with intuitive figures (e.g., Fig. 1) that effectively illustrate the dual-phase framework. - Key implementation and training information are missing. The GNN architecture, baseline configurations, and RL training details (e.g., learning rate, steps, reward curves) are not provided. Important hyperparameters ($w_c, w_r, w_u, \lambda_c, \lambda_r, \lambda_u$) and their sensitivity are also unreported, making the results less convincible. - The framework’s computational cost is not discussed. Reinforcement learning loops can be significantly more expensive than supervised training, and the paper provides no analysis of training efficiency or scalability. - Missing related work[1], which also integrates reinforcement learning into GNN-based physical simulations through adaptive remeshing. - Although the paper claims to release code, the provided anonymous link only contains a README without actual implementation. [1] Learning Controllable Adaptive Simulation for Multi-resolution Physics. Tailin Wu, Takashi Maruyama, Qingqing Zhao, Gordon Wetzstein, Jure Leskovec. ICLR 2023 1. In line 192, you mention computing the consistency loss by integrating the squared PDE residual over the spatio-temporal domain, with differential operators approximated on the graph. Could you provide the explicit computational formula or an example of how this residual is implemented on discrete graph structures? 2. The reward terms (e.g., R = $\exp(-\lambda v)$ ) all adopt an exponential transformation. What is the rationale behind this specific functional form? Have you compared it with other mappings such as linear or logistic functions? 3. In line 203, when defining the robustness reward, how is the perturbation noise intensity ($\sigma$) determined? Is it fixed across datasets or tuned per task? 4. The uncertainty reward relies on Monte Carlo Dropout, which requires dropout layers.