Wasserstein Formulation of Reinforcement Learning. An Optimal Transport Perspective on Policy Optimization

We present a geometric framework for Reinforcement Learning (RL) that views policies as maps into the Wasserstein space of action probabilities. First, we define a Riemannian structure induced by stationary distributions, proving its existence in a general context. We then define the tangent space o