Overfitting of Spectral Gradient Descent: How Matrix Geometry shapes Generalization and Implicit Bias

We study the generalization of spectral gradient descent (SpecGD) in overparameterized matrix classification with corrupted labels. Each input combines a shared low-rank signal with a rank-one sample-specific perturbation, referred to as a shortcut, that enables memorization but does not generalize.