Implicit Bias of (Stochastic) Gradient Descent on Deep Linear Diagonal Networks

Modern machine learning models are often overparameterized, i.e., have more parameters than training data. This typically implies that the loss function admits multiple minimizers so that training methods (and their hyperparameter choices) have significant influence on the computed solution. We stud