Official Review · ICLR 2025

The paper attempts to address the lack of standardization in reinforcement learning environments. The paper’s library is able to be used in conjunction with a number of libraries, showing that it has been adopted across the field. The paper provides abstraction for environments through a set of fundamental spaces. The paper implements vectorization to enable parallelization across environments. The paper analyzes the increase in environment throughput with vectorization to show that this can provide substantial speedups to using the library. Gym and now gymnasium is an essential part of reinforcement learning with many packages and works being built on top of or inspired by a similar methodology. The abstraction provided by the method is commonly used across reinforcement learning and is easy to use. The vectorization increases throughput of running an environment without significant overhead. The writing is fairly straightforward and it is easy to understand the methodology of the work. Without outside comparison, the results show that the vectorization is able to provide significant speedups. The use of vectorization is key to enable speed-up in performance. Despite the use of the name recognition of the library, it is unclear if the improvements from OpenAI Gym to Gymnasium are comparable to the contributions from other RL libraries 2015-present. My main issue with the work is the lack of discussion and comparison with similar work. (In section 2 the paper details a number of comparable works that can use their framework but this is not sufficient to my point.) The paper should complement technical report details with meaningful comparisons to help researchers choose which library to use. Reinforcement learning is going a massive change in environments due to parallelization but these benefits are from Jax and other GPU engines such as Madrona engine [1], not work created by this library so I do not think it is a fair claim that they can claim credit for those works. Other works have better vectorization such as Puffer [2] and I would expect a mention or comparison to similar libraries. The paper should explain why one should use Gymnasium versus competitors. It would be helpful to see comparisons across the same environments for each library. RLlib [3], which is in the related work of the paper, shows substantial deployment and scalability for reinforcement learning on distributed training. Each of these papers regarding reinforcement learning libraries provide substantial merits for scaling reinforcement learning training. Though I think it is important to encourage RL infrastructure and environment development in reinforcement learning, it is unclear that a set of abstractions provided by Gymnasium have the same level of contribution as the other papers empirical efforts. [1] Shacklett, B., Rosenzweig, L. G., Xie, Z., Sarkar, B., Szot, A., Wijmans, E., ... & Fatahalian, K. (2023). An extensible, data-oriented architecture for hig