Physics-Inspired Optimizers beyond the Edge of Stability: Period-Two Orbits and Delayed Instability

Gradient descent with momentum can be viewed as a particle moving with friction in the loss landscape. In deep learning the learning rate is usually large, so training is a discrete dynamical system: in many full-batch experiments the sharpness grows to the optimizer's stability limit, the edge of s