Semantic Collapse in Embedding Space: Neighborhood Semantics and Entropic Drift

Large Language Models (LLMs) encode linguistic meaning in high-dimensional embedding spaces, where geometric proximity is treated as a proxy for semantic similarity. While effective for fluency, this architecture introduces a recurrent failure mode: embedding collapse, the systematic loss of distinctions essential for reasoning. Collapse occurs when a misleading decoy representation is treated as more similar to an anchor than its correct semantic substitute, thereby flattening distinctions such as epistemic versus factual, modal versus consensus, or indexical versus general. This paper develops a diagnostic framework for analyzing collapse. First, we formalize embeddings as neighborhood semantics and show how the clustering of tokens resembles plausibility models from modal logic. Second, we quantify collapse through entropy, interpreting drift as the dispersion of semantic boundaries within embedding neighborhoods. Third, we introduce graph-based triplet diagnostics, where anchors, substitutes, and decoys are visualized as nodes, and collapse is detected when misleading edges dominate intended ones. This analysis reframes hallucination as collapse: not incidental noise, but a structural failure of embedding-based similarity to preserve logical operators. We conclude that constraint-aware interventions—modal kernels, neighborhood orderings, and operator-fidelity layers—are necessary to move from fluency without fidelity to semanticsaware AI.