What Does an Embedding Mean? A Semiotic Account of Learned Representation

Language models learn embeddings from text; predictive world models learn internal states from observations and, in some systems, interaction. What do these different sources allow an embedding to mean? An embedding is a numerical representation that preserves some differences among inputs while making others difficult to distinguish. Its significance depends on which differences the system uses and what makes that use appropriate or mistaken. This paper develops a functional, relational account that brings learning history, use, and correction together. It calls the arrangement connecting them a regime of relevance. Language supplies distinctions developed through human observation, reasoning, and social practice. Perception and action provide additional measurements and consequences that can challenge or extend those distinctions. Neither source guarantees that an embedding preserves what a task requires. The comparison includes language embeddings, image–text models, and joint-embedding predictive architectures (JEPAs), which predict learned representations rather than reconstructing inputs in detail. Transformation tests and interventions make the account concrete by examining which distinctions an embedding supports and how the system uses them. A code-search example then separates correcting an answer, learning to use fixed embeddings differently, and changing the encoding itself. These changes can improve performance while supporting different claims about the embedding. The resulting implication for general intelligence is conditional: broader competence may require both a wide range of learned distinctions and the ability to revise which ones guide activity when tasks or evidence change.