Almost Sure Convergence of Networked Policy Gradient over Time-Varying Networks in Markov Potential Games

We propose networked policy gradient play for solving Markov potential games with continuous and/or discrete state-action pairs. During the game, agents use parametrized and differentiable policies that depend on the current state and the policy parameters of other agents. During training, agents up