Generalization and Memorization along the Learning Trajectory of Neural Language Models: A Geometric Account of Categorization

We investigate how generalization and memorization develop along the learning trajectory of neural language models. Using controlled synthetic grammars, we examine both the geometry of representation space and model behaviour over training. We find that continuous regions of representation space not