From Interpolation to Extrapolation: How Language Models Theorize in Data-Sparse Regimes

When large language models (LLMs) generate citations in sparse domains, they frequently produce non-existent but theoretically coherent references. We argue these are not random errors but instances of abductive extrapolation: extending learned structures to infer what research should exist. Using a 2×2×2 factorial design (N=192 citations), we manipulated domain density (dense/sparse), framing (conservative/creative), and awareness (aware/unaware) while models generated academic manuscripts. Domain density showed a massive effect (Cohen's h=0.93, 35.5 percentage points), dwarfing framing and awareness manipulations. Critically, creative framing achieved 100% validity in dense domains, while awareness warnings provided no benefit. These findings suggest "hallucination" reflects theoretically coherent extrapolation under data sparsity—a substrate-independent mechanism shared with human theory formation—rather than stochastic error.