Do Flatter Minima Drive Better Generalization? An Algorithmic Separation in Grokking

Flat loss landscapes have long been linked to better generalization in neural networks. However, its role as a causal mechanism for generalization is less established. Grokking provides an unique testbed to understand this distinction: models are prone to fit observed data using non-generalizing str