Autoregressive Decoding as a Martingale: Characterizing the Stochastic Dynamics of LLM Generation

Large language models (LLMs) have achieved remarkable success across a wide range of language tasks. At their core, autoregressive LLMs are conditional probability models that learn to approximate the next-token distribution given the preceding context. Despite this probabilistic nature, the stochas