The Bottleneck Regime: Capacity Scaling Reveals an Intermediate Sweet Spot for Cross-Modal Alignment in Recurrent Neural Networks

When do neural networks trained on multiple related prediction tasks converge to aligned internal dynamics rather than segregated representations? We present a minimal, fully controlled computational study demonstrating that strong cross-modal alignment emerges from a simple architectural constraint: a shared latent bottleneck trained bidirectionally on forward and inverse prediction (Wolpert, Ghahramani, & Jordan, 1995; Kawato, 1999). We train gated recurrent units (GRUs) in a synthetic sensorimotor environment with two self-supervised objectives: predicting sensory consequences from actions (forward model) and inferring actions from sensory streams (inverse model). When both objectives share a single recurrent core, we observe substantial unitwise alignment between perform and observe modes. Across 2,000 sequences and 20 epochs in a canonical setting (hidden size 96), 25–30% of units exhibit |r| > 0.75 between perform and observe activations, ~35–40% exceed |r| > 0.6, and peak units reach r ≈ 0.9. Ablations using separate recurrent cores, forward-only training, or shuffled observation labels eliminate this alignment, establishing that it arises from architectural and training constraints rather than task structure alone. We then conduct a systematic capacity scaling analysis, varying GRU hidden size from 4 to 512 with strict evaluation controls (independent test sequences and untrained baselines). We identify a pronounced intermediate-capacity regime (h ≈ 32–48) in which four metrics co-optimize: (i) alignment gain Δr (difference between shared- and separate-core maximum |r|), (ii) compression ratio between perform and observe state manifolds, (iii) linear separability of mode identity, and (iv) structured developmental trajectories during training. Beyond this regime, larger models achieve lower prediction error but reduced alignment and weaker compression, deviating from standard monotonic scaling trends observed in predictive performance (Kaplan et al., 2020; Wei et al., 2022). These results suggest that cross-modal alignment is not a byproduct of scale, but a consequence of architectural bottlenecks interacting with bidirectional predictive objectives. More broadly, they indicate that representational compression, mode alignment, and structured developmental dynamics can arise from constraint-driven learning in recurrent systems, with implications for multi-task learning, multimodal modeling, and representation geometry in neural networks.