Official Review · ICLR 2025

The paper proposes a novel activation function based on Hadamard representations to address the limitations of traditional activation functions in reinforcement learning. It highlights issues like vanishing gradients and dying neurons, demonstrating that the proposed method improves learning efficiency and representation capacity in deep Q-networks and proximal policy optimization. 1. The proposed Hadamard representation effectively addresses dead neuron issue with existing activation functions, particularly in reinforcement learning contexts. 2. The experimental results show clear improvements in learning speed and performance metrics, providing solid empirical evidence for the claims made. 3. The paper presents a well-defined problem and clearly articulates its contributions, making the paper easy to understand. 1. The lack of a diverse range of network architectures tested limits the generalizability of the findings. The paper should test HR on various networks and test HR on different layers. It would be better if HR were a universal activation function suitable for all layers, rather than only suitable for the last layer. 2. Absence of detailed comparisons with other novel activation functions that have emerged recently. 3. The paper did not conduct tests on diverse tasks, e.g. MuJoCo, DMControl. using only Atari does not demonstrate that this is a universal issue in reinforcement learning. 4. As mentioned in Line 279, the experiments are all performed on 8 Atari games. Testing on only 8 Atari games does not sufficiently demonstrate a stable improvement in performance. 5. In the PREVENTING DYING NEURONS section in Line 191, the analysis is not related to reinforcement learning task at all. So, claiming that HR is an activation function particularly suitable for reinforcement learning feels somewhat forced. 1. How do Hadamard representations compare with other recent activation functions not covered in the paper? 2. What potential challenges might arise when applying this method to different neural network architectures? 3. What is the result if HR is tested on various tasks, e.g. MuJoCo, DMControl, and more tasks on Atari?