Language Models Without a Trainable Input Embedding Table: Learning from Fixed Minimal Binary Token Codes

We study whether a decoder-only language model requires an independently trainable input vector for every token. For a vocabulary of size $V$, an injective fixed-length binary identifier requires $K=\lceil\log_2 V\rceil$ bits. We replace the usual trainable $V\times d_{\mathrm{model}}$ input table w