Official Review · ICLR 2026

The paper proposes a novel method for learning distance metrics between states in reinforcement learning. The approach is to learn mappings from observation modalities to latent states, and measure distance using a well-define metric in the latent state space. The learned state representation can then be used as the basis for reinforcement learning algorithms. The method is tested in a series of benchmarks. State representation learning is an important problem in reinforcement learning, and in this sense the paper makes a timely contribution. It also looks as if the proposed approach performs well in practice compared to other algorithms. I believe that several concepts are not clearly explained, which makes it difficult to accurately evaluate the contribution. Mainly for this reason my opinion regarding acceptance at ICLR is on the negative side. Apart from the cited paper by Wang et al., there are other works that explicitly learn a distance estimate between pairs of states. Concretely, the first work also measures distance as the minimum number of actions required to transition from one state to another. State Representation Learning for Goal-Conditioned Reinforcement Learning Steccanella & Jonsson, ECML 2022 Park, Kreiman & Levine, ICML 2024 Foundation Policies with Hilbert Representations In the definition of POMDPs, do you assume that the underlying state space S is known to the learner? (even if it is not observable) The proposed approach is to map *each* observation modality to the same latent space, in the hope that all observation modalities derived from the same state will map to approximately the same latent state. However, the mapping from states to observation modalities is stochastic, so a deterministic map is not likely to map all modalities to the same latent state. Why do you map all observation modalities to the same latent space? An alternative would be to have a different latent space for each observation modality and aggregate the distance measures in another way. From Equation (3) it appears as if you assume that the function \varphi_T is known, is this indeed the case? I would have expected you to *learn* an approximate transition function in the latent space. If you do not assume that \varphi_T is known, then I do not see how you can make an assumption regarding the transition error of \varphi_T, since this depends on the quality of learning. What is the motivation for the exact form of Equation (3)? The description of the experimental setup leaves a lot to be desired. What is the training pipeline? Do you train a distance estimate first, and then a deep RL policy, or is learning simultaneous? Is distance estimation done online or from offline data? What is the input to the deep RL algorithm (SAC), i.e. exactly how are states and state features defined? From where did you take the seven corruption families? Can you provide a reference?