Word embeddings from text corpora: a simulation study on the representation of underlying structures

This study investigates whether word embeddings -- language models that transform words into numerical vectors based on distributional patterns -- constitute genuine semantic representations grounded in structural reality, or merely sophisticated statistical artifacts. To address this fundamental question in computational linguistics and philosophy of mind, we developed a novel microsimulation methodology using a virtual `supermarket' with known spatial topology comprising four departments (fruit, vegetables, beverages, extras) across forty networked shelves. Computer simulations of 80,000 `shoppers' navigating this space generated unique sentences from their shopping trajectories, creating a corpus where language users metaphorically `shop' for words during sentence formulation. When the GloVe embedding algorithm was applied to this artificial corpus -- derived purely from distributional co-occurrence patterns without external semantic annotation -- the model successfully reconstructed the original (hidden) supermarket layout with remarkable fidelity, correctly identified departmental clusters, and preserved spatial relationships between products. Dimensionality reduction of the 50-dimensional embedding space revealed striking one-to-one correspondence with the ground-truth network topology across all product variants, demonstrating distributional robustness. From a philosophical perspective, these findings contribute empirically to debates about statistical pattern recognition versus genuine semantic understanding. The successful structural recovery suggests the distributional hypothesis may capture more profound relational properties than critics acknowledge, though questions remain about whether such correspondence constitutes authentic grounding in real-world conceptual frameworks or sophisticated pattern matching disconnected from experiential meaning dimensions. The theoretical framework draws upon Shannon's information theory and minimum description length principles, suggesting that optimal data compression underlying embedding generation may naturally align with discovering generative structures. This implies Occam's razor may operate as a computational pathway toward semantic representation, though whether this achieves genuine intentionality remains philosophically contested. These results have implications for contemporary debates about artificial intelligence, computational semantics, and machine understanding.