Beyond the Next Token: Geodesic Decoding and the Gravitational Landscape of Semantic Space

Contemporary large language models (LLMs) generate text through autoregressive next-token prediction —a fundamentally local process. This paper presents an inference-time framework for steering a frozen model’s hidden states using a small set of semantically anchored “attractor” and “repulsor” concepts, together with a dynamical extension in which the same field defines a Riemannian metric on the hiddenstate space. We give two variants: (a) potential-field steering, a first-order update along the gradient of a softened multi-anchor potential, mechanically close to Plug-and-Play Language Models (PPLM); and (b) conformal-geodesic steering, in which the potential defines a conformal metric g = e2Φδ and the hidden state instead follows the associated second-order geodesic equation, subject to a trustregion projection that we state explicitly does not preserve exact geodesic-ness. We do not claim that either mechanism recovers “the” semantic geometry of a language model; the metric is an engineered decoding aid, not a discovered property of the model. We do not claim reduced hallucination, increased coherence, or any practical advantage over cheaper alternatives such as PPLM or prompting; these are open questions requiring the controlled comparison in Section 5. Section 7 discusses, explicitly, that the current evidence does not support deployment value and that the paper’s more defensible contribution at this stage is methodological: an experimental template — particularly the random/shuffled-anchor controls — for evaluating any future inference-time semantic-steering claim. No experiments have been run.