arXiv CSopen access

Convergent Evolution: How Different Language Models Learn Similar Number Representations

2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

machine learning, deep learning, neural networks

April 22, 2026

Convergent Evolution: How Different Language Models Learn Similar Number Representations Deqing Fuσ Tianyi Zhouσ Mikhail Belkinψ Vatsal Sharanσ Robin Jiaσ σ University of Southern California ψ UC San Diego {deqingfu,tzhou029,vsharan,robinjia}@usc.edu, [email protected]

arXiv:2604.20817v1 [cs.CL] 22 Apr 2026

Abstract Language models trained on natural text learn to represent numbers using periodic features with dominant periods at T = 2, 5, 10. In this paper, we identify a two-tiered hierarchy of these features: while Transformers, Linear RNNs, LSTMs, and classical word embeddings trained in different ways all learn features that have period-T spikes in the Fourier domain, only some learn geometrically separable features that can be used to linearly classify a number mod-T. To explain this incongruity, we prove that Fourier domain sparsity is necessary but not sufficient for mod-T geometric separability. Empirically, we investigate when model training yields geometrically separable features, finding that the data, architecture, optimizer, and tokenizer all play key roles. In particular, we identify two different routes through which models can acquire geometrically separable features: they can learn them from complementary co-occurrence signals in general language data, including text-number co-occurrence and cross-number interaction, or from multi-token (but not single-token) addition problems. Overall, our results highlight the phenomenon of convergent evolution in feature learning: A diverse range of models learn similar features from different training signals. Models: https://hf.co/collections/deqing/convergent-evolution

Introduction

Language models trained on natural language develop periodic representations for number tokens. For many Transformer-based language models, Zhou et al. (2024) show that the embeddings of integer tokens have consistent spikes in the Fourier domain at periods T = 2, 5, and 10. Such periodic features have also been well-documented in models’ intermediate representations (Levy & Geva, 2025) and in the model mechanisms that implement addition (Zhou et al., 2024; Kantamneni & Tegmark, 2025). Engels et al. (2025) even find analogous periodic structures for other cyclical concepts such as days of the week and months of the year. These findings have been broadly interpreted as evidence that language models learn structured numerical representations through next-token prediction. In this paper, we first demonstrate that this phenomenon is far more general than previously recognized. Figure 1 shows that the same T = 2, 5, 10 spikes appear not only in Transformers (Vaswani et al., 2017) of varying scale (GPT-2 (Radford et al., 2019), GPT-OSS (OpenAI, 2025), Llama-3 (Meta, 2024), Llama-4 (Meta, 2025), and DeepSeek-V3 (DeepSeek, 2025)), but also in non-Transformer LLMs (Mamba (Gu & Dao, 2024), Falcon-Mamba (Zuo et al., 2024), xLSTM (Beck et al., 2024), Kimi-Linear (Kimi, 2025)) and classical word embeddings (GloVe (Pennington et al., 2014) and FastText (Bojanowski et al., 2017)). Even the raw token frequency distribution of numbers in the training corpus, with no model at all, exhibits the same periodic spectrum (see Figure 2). We view this universality as a case of convergent evolution: different systems independently develop the same representation because they share the same constraints from training data and tokenization. In biology, convergent evolution refers to the independent emergence of similar traits in unrelated organisms facing shared environmental pressures, such as the independent evolution of eyes in vertebrates and cephalopods (McGhee, 2011). Fourier features in number embeddings are analogous: a shared trait that arises from shared constraints on the learning process.

Convergent Evolution

Fourier Spectrum on Number Embeddings of Pretrained Models Transformer LLMs

GPT-2-1.5B GPT-OSS-120B Llama-3.1-8B Llama-4-Scout-109B DeepSeek-V3-671B

Mamba-2.8B Falcon-Mamba-7B xLSTM-7B Kimi-Linear-48B

Transformer (Muon) Transformer (AdamW)

Autoregressive Language Models

Linear RNNs

Gated DeltaNet Mamba-2

Gated RNNs Learned Models

Non-Transformer LLMs

Word Embeddings

LSTM Word2Vec PPMI

Transformer Trained on Addition

Word Embeddings

GloVe-Dolma FastText-WikiNews 0.0 1 2 3 4 Norm. Magnitude

Transformer

0.1

0.2

3-digit Addition

Counting Only

0.3

Fourier Frequency

0.4

0.5

9-digit Addition Token Distribution

Both Spectral & Geometric Convergence Spectral & Partial Geometric Convergence

Spectral Convergence Only Neither

Figure 1: Universality of Fourier Features and Convergent Evolution. (Left) Fourier spectrum of number embeddings across three architecture families: Transformer LLMs, non-Transformer LLMs, and classical word embeddings. Each row shows the mediannormalized magnitude at each Fourier frequency. All models exhibit consistent spikes at frequencies of period T = 2, 5, 10, etc. (Right) Convergent Evolution of various models studied in this paper. It shows two types of convergence: spectral convergence where models learn Fourier spikes, and geometric convergence where models learn modular probes.

But do these Fourier spikes indicate that models have learned functional numerical structure? We identify a two-tiered hierarchy of periodic features: only some systems with Fourier spikes cleanly encode modular arithmetic properties in their embeddings. By this we mean that the residue class n mod T is linearly decodable from the embedding e(n). Period-T features naturally group numbers by their value mod T, and the linear representation hypothesis (Park et al., 2023) conjectures that such structure should be accessible via linear probes. We call the emergence of Fourier spikes spectral convergence and the emergence of linearly separable mod-T classes geometric convergence. Spectral convergence appears in almost every system we examine, but geometric convergence does not: Transformers and linear RNNs trained on 10 billion tokens develop embeddings where mod-T classes are linearly separable, while LSTMs trained on identical data develop more prominent Fourier spikes but achieve chance-level probing. Understanding what separates these two levels of convergence is the central question of this paper. We summarize our contributions below. Fourier spikes are universal but probing is not. We show that Fourier spikes at T = 2, 5, 10 appear in every system we examine, spanning Transformer and non-Transformer LLMs, classical word embeddings, and even the raw token frequency distribution (Figure 1). We demonstrate both theoretically (Theorem 1) and empirically (Figure 2) that Fourier spikes are necessary but not sufficient for mod-T probing, and explain how models with similar spectra can have vastly different probing accuracy (§3). Geometric convergence requires data, architecture, and optimizer to align. Through controlled experiments on 300M-parameter models trained on identical data, we isolate three factors that jointly determine whether mod-T classes become linearly separable in the embeddings. Our experimental methodology can be viewed as a form of structure attribution. Analogous to how influence functions (Koh & Liang, 2017) or Shapley values (Ghorbani & Zou, 2019) attribute model predictions to individual training examples, our controlled perturbations attribute the emergence of learned representations to specific structural properties of the data distribution. We find that geometric convergence depends on several complementary data signals: perturbations that progressively remove textnumber co-occurrence, cross-number interaction, or context length each degrade probing, while Fourier spikes persist across all conditions (§4.1). The architecture plays a critical role: Transformers and linear RNNs achieve strong probing while LSTMs trained on the same data remain at chance (§4.2). Regardless of the optimizer, both Transformers and linear RNNs learn the same Fourier spectrum but different probing performance.

Fu, Zhou, Belkin, Sharan, & Jia

Convergent evolution takes a different form under arithmetic task pressure. In models trained on addition from scratch, the tokenizer determines whether Fourier structure emerges (§5). Multi-token addition requires computing each output digit as a sum modulo 1000, forcing the model to solve modular subproblems that produce circular representations. Single-token addition admits multiple strategies, and the resulting representations randomly depend on the optimizer.

Related Work

Fourier Features. Fourier features have been used for many years in computer vision as edge and orientation detectors (Olshausen & Field, 1997; Olah et al., 2020; Fiquet & Simoncelli, 2023). The original Transformer applies sinusoidal position encodings (Vaswani et al., 2017), and several works have found that explicitly injecting high-frequency components into inputs helps with spatial and numerical tasks (Tancik et al., 2020; He et al., 2023; Hua et al., 2024). Recently, this structure is also found to emerge without being designed in. Transformers trained on modular addition embed numbers on a circle and rotating to compute the answer (Nanda et al., 2023; Zhong et al., 2023; Gromov, 2023). The same holds in pretrained LLMs that number token embeddings break into Fourier components, and there are recognizable addition circuits in the attention and MLP layers (Zhou et al., 2024; Kantamneni & Tegmark, 2025; Levy & Geva, 2025). Zhou et al. (2025) show that hard-coding these Fourier features improves arithmetic learning. These studies document spectral structure but do not test whether it implies geometric separability — a distinction our work shows is critical. In this paper, we further study the question these papers leave open, where the structure comes from in the first place. Mechanistic Interpretability. Mechanistic interpretability aims to reverse-engineer the representations and algorithms learned by language models. The linear representation hypothesis (Park et al., 2023) conjectures that high-level concepts, if learned, should be linearly decodable from model representations. Probing (Orgad et al., 2025; Kossen et al., 2024) is the standard tool for this. However, number representations are not linearly encoded (Nanda et al., 2023; Zhong et al., 2023; Gromov, 2023). Karkada et al. (2026) also find circular representations for days of the week, which suggests this is a fairly general solution to any problem with rotational symmetry. Allen-Zhu (2025) use controlled synthetic pretraining to isolate which capabilities emerge from which architectural and data choices. Separately, Huh et al. (2024) argue that representations across models and modalities are converging toward a shared statistical model of reality, measured via global kernel alignment. We ask a different question: whether models that converge on the similar representation of a specific concept have learned the same functional structure, and show that they can diverge fundamentally. In this paper, we vary tokenization, architecture, optimizer, and task to understand the convergent evolution of number representations.

Problem Setup and Preliminary Analysis

We study the token embeddings of numbers 0 through N − 1 in language models, where N = 1000 corresponds to the set of numbers that receive single-token representations in the Llama-3 tokenizer (Meta, 2024). Let e(n) ∈ Rd denote the token embedding of number n. To detect periodic structure in these embeddings, following Zhou et al. (2024), we compute the discrete Fourier transform along the token index: 1 N −1 Fν = √ ∑ e(n) e−2πiνn ∈ Cd , N n =0

ν=

k , k = 0, . . . , N − 1. N

( j)

The power at frequency ν is ∥ Fν ∥2 = ∑dj=1 | Fν |2 . A Fourier spike at period T refers to a visible peak in ∥ F1/T ∥2 relative to neighboring frequencies. For a period T dividing N, define the mod-T residue classes Cr = {n : n ≡ r (mod T )} for r = 0, . . . , T − 1, each of size |Cr | = N/T. To evaluate whether the embeddings encode modular arithmetic at period T, we train a linear probe (T-class logistic regression) to predict n mod T from e(n). A natural question is whether the presence of a Fourier spike at period T guarantees good mod-T probing accuracy. As we will show in Figure 2, the answer is strikingly no: an

Convergent Evolution

LSTM

Gated DeltaNet

Transformer

Fourier Magnitude Spectrum

100

Cohen's of Linear Probe (%) 96

Eigenvalues of SW1SB (T=10) max = 23.314 Fourier 10 = 33.8 cond(SW) = 122.1

64 T=10

4 0 0.0 8 4

0.1

0.2

0.3

0.4

0.5 100

T=10

0.2

T=5

0.3

0.4

0.5 100

-9

-5

2 3 4 5 6 7 8 9 10

-5

-7

-5

2 3 4 5 6 7 8 9 10

50 0 0.3

0.4

0.5

T=2 100

0 30

0 0.1

0.2

0.3

Fourier Frequency

0.4

0.5

1 2 3 4 5 6 7 8 9 max = 0.290 Fourier 10 = 337.4 cond(SW) = 8569.0

15 9

-1

2 3 4 5 6 7 8 9 10

1 2 3 4 5 6 7 8 9 max = 19.963 Fourier 10 = 24.6 cond(SW) = 126.3

T=2

0 0.0

0 30

78 55

0 0.1

-4

T=2

T=5

T=10

0 0.0 8

T=2

T=5

0 0.0 0.1 0.2 8 T=10 T=5

Token Dist.

0 30

1 2 3 4 5 6 7 8 9 max = 0.046 Fourier 10 = 94.0

15 4

-1

2 3 4 5 6 7 8 9 10

Modulus T

1 2 3 4 5 6 7 8 9

Eigenvalue Rank i

Figure 2: A “Spiky” Fourier Spectrum Does Not Imply Good Feature Learning. (Left) Token embeddings of Transformer, Gated DeltaNet and LSTM, and even simply the Number Token Distribution Frequency exhibit distinct Fourier spikes at T = 2, 5, and 10. (Middle) Linear probes reveal only the Transformer and Gated DeltaNet learned functional modular arithmetic with high Cohen’s κ, while others remain at chance. (Right) Theorem 1 explains this discrepancy through the internal noise structure of the T = 10 embeddings.

LSTM trained on the same data as a Transformer develops larger Fourier power at T = 10 yet achieves chance-level probing. The following result shows how this is possible and demonstrates that the presence of a Fourier spike is a necessary but not a sufficient condition for learning modular probes. Theorem 1. Given embeddings {e(n)}nN=−01 and residue classes {Cr }rT=−01 defined above, let the class means µr , grand mean µ, between-class scatter matrix SB , and within-class scatter matrix SW be µr =

1 e ( n ), |Cr | n∑ ∈C r

T −1

SB =

1 (µr − µ)(µr − µ)⊤ , T r∑ =0

SW =

µ=

1 N −1 e ( n ), N n∑ =0

1 T −1 ∑ (e(n) − µr )(e(n) − µr )⊤ . N r∑ =0 n ∈ C r

T −1 2 Let Φ T = ∑ℓ= 1 ∥ Fℓ/T ∥ be the total power at harmonics of period T, and HT {0, 1/T, 2/T, . . . , ( T −1)/T } be the set of harmonic frequencies.

(i) If Φ T = 0, then SB = 0 and no linear probe can classify n mod T above chance. (ii) For any T ≥ 2, C > 0, and ε > 0, there exist N divisible by T and embeddings e(n) satisfying Φ T > C yet no T-class linear classifier achieves accuracy above 1/T + ε, i.e., no more than ε above random guessing.

Fu, Zhou, Belkin, Sharan, & Jia

A=5, B=0.2,

T=1250, best linear acc=100.0%

e(n) = A(n mod T) + B n/T 0

10 0 20

16 6

12 2 22

18 8

5 15

1 21 11

7 17

3 23 13

mod 5 2

14 4 24 9 19

e(n)

10 0

2 1

3 5

4 6

20 11

24 23

15 0

1000 800 600 400 200 0

T=5

0.0

0.1

0.2

e(n)

1000 800 600 400 200 0

0.3

frequency

T=1250, best linear acc=36.0%

F 2

A=5, B=11,

F 2

N=25 Points. Period T=5

0.4

0.5

0.4

0.5

T=5

0.0

0.1

0.2

0.3

frequency

Figure 3: Examples for the Proof of Theorem 1 Part (ii) in § A.3. In the proof, every number n ∈ {0, . . . , N − 1} has a unique decomposition n = r + mT with residue r = n mod T ∈ {0, . . . , T −1} and block index m = ⌊n/T ⌋ ∈ {0, . . . , K −1}, where K = N/T. We set the embedding e(n) = Ar + Bm, so that A controls the between-class scatter SB (and hence Φ T = N · SB ) while B controls only the within-class scatter SW , with neither parameter affecting the other. As illustrated in the figure, fixing A produces a persistent Fourier spike at period T regardless of B: when B is small the residue classes cluster into linearly separable groups, but as B grows the embeddings interleave across classes so that the best linear classification accuracy goes closer to random guessing 1/T. Theorem 1 establishes that Φ T > 0 is necessary but not sufficient. A natural follow-up is: what quantitatively determines whether a spiky Φ T translates into geometric separability? Remark 2. The gap between two parts of Theorem 1 can be made more precise through the lens of Fisher’s Linear Discriminant Analysis (Fisher, 1936). Assume d ≥ T − 1 and SW is invertible. Assume d > T − 1 and SW is invertible. While a linear probe for T classes separates data across a ( T − 1)-dimensional subspace, the absolute ceiling on its performance is dictated by its principal axis of separation: the single direction v that maximizes the ratio of between-class to within-class ⊤

variance, given by the Rayleigh quotient vv⊤ SSB vv . The maximum achievable separability along this W

−1 optimal axis is exactly the largest generalized eigenvalue of the scatter matrices, λmax (SW SB ). We show (see § A.4) that this maximum discriminant satisfies: 1 ΦT ΦT −1 · ≤ λmax (SW SB ) ≤ . ( T − 1) · cond(SW ) N · λmin (SW ) N · λmin (SW ) The Fourier power Φ T only guarantees a large total variance among the class means, which drives Φ T = Tr(SB )/N in the numerator. It ensures the class centers are dispersed in space. However, actual linear separability depends on how this signal aligns with the within-class scatter, SW . A large condition number, cond(SW ) = λmax (SW )/λmin (SW ), drastically lowers the minimum bound. If the periodic signal aligns with the dimensions of maximum within-class noise, the massive Φ T is entirely swallowed by within-class variance. This drives λmax toward zero and forces the entire ( T − 1)-dimensional subspace to collapse into overlapping classes.

The proof of Theorem 1 is given in Appendix A, where part (i) follows naturally from Fourier identities in Lemma 4 which shows that Tr(SB ) = Φ T /N, and part (ii) constructs examples (see Figure 3) so that Φ T can be arbitrarily large but residue classes are not linearly separable. We now ask two empirical questions: how common are Fourier spikes in practice, and do they actually predict mod-T probe accuracy? Throughout this paper, unless stated otherwise, we train models with around 300 million parameters on 10B tokens from FineWeb-Edu (Lozhkov et al., 2024) using the Llama-3 tokenizer, which assigns single tokens to each integer from 0 to 999 such that N = 1000. We include architecture details in the Appendix B.1. We evaluate number representations

Convergent Evolution

using two complementary measurements: (1) the Fourier magnitude spectrum, computed as the power ∥ Fν ∥2 normalized by the median across frequencies ν, and (2) the mod-T probe accuracy, measured by Cohen’s κ of a classifier trained to predict n mod T from the token embedding e(n). Cohen’s κ adjusts for the baseline accuracy of random guessing (1/T for balanced classes), so that κ = 0 corresponds to chance and κ = 100% to perfect classification regardless of T. We report accuracy-based results in the appendix. We use three probe types: linear (logistic regression), MLP, and RFM kernel (Radhakrishnan et al., 2024); unless noted, we report linear probe results, with the others in § B.5. We report probe performance averaged over 30 runs: 3 random seeds, each with 10-fold cross-validation. The first measurement detects periodic structure while the second tests whether that structure supports mod-T classification. Fourier spikes are universal. Figure 1 shows that every pretrained model we examine, spanning Transformer LLMs, non-Transformer LLMs, and classical word embeddings, exhibits peaks at ν = 1/10, 1/5, 1/2. Spectral convergence is universal as long as natural languages are used for training. This holds across architectures across fundamentally different learning algorithms and across models never explicitly trained on numerical tasks. Fourier spikes do not imply modular arithmetic. Figure 2 presents a controlled comparison under our training setup: a Transformer, a Gated DeltaNet (Yang et al., 2025), an LSTM (Hochreiter & Schmidhuber, 1997) (all around 300M parameters), and the raw number token frequency distribution from the training corpus, where each number n is represented by its scalar corpus frequency pn via counting rather than a learned embedding. The left column shows that all four produce qualitatively similar Fourier spikes at periods T = 2, 5, 10. The middle column shows mod-T probe accuracy: the Transformer and Gated DeltaNet achieve κ = 96 and 95 at T = 2, and κ = 85 and 78 at T = 10, while the LSTM and the token distribution remain at chance across all moduli. The right column explains this gap through Theorem 1. The LSTM has larger Fourier power Φ10 than the Transformer, yet its Fisher −1 discriminant λmax (SW SB ) is two orders of magnitude smaller. The difference lies in the condition number cond(SW ): the LSTM’s within-class scatter is highly anisotropic, so the periodic signal is buried under within-class variance and the classes overlap despite visible Fourier spikes. Several recent studies have inspected Fourier spectra and concluded that models have learned modular structure (Nanda et al., 2023; Zhou et al., 2024). Our analysis shows why this inference is unreliable: a visible spike at period T guarantees Tr(SB ) > 0, −1 but probe accuracy depends on the eigenspectrum of SW SB , which the Fourier power spectrum alone does not determine. We call this arrangement of embeddings into linearly separable mod-T classes Geometric Convergence. Unlike spectral convergence, geometric convergence is selective: only certain combinations of data, architecture, and optimizer produce it. Both are instances of convergent evolution: different systems arriving at similar representations because of shared constraints. The next sections identify what drives each.

Convergent Evolution in Language Model Pretraining

Spectral convergence requires only the periodic frequency distribution of number tokens, but geometric convergence is selective. We now ask which constraints must be present for geometric convergence to emerge. Through controlled experiments that vary one factor at a time, we identify three: the data signal the model receives (§4.1), the architecture, and the optimizer (§4.2). All experiments use 300M-parameter models trained on 10B tokens from FineWeb-Edu (Lozhkov et al., 2024) with the Llama-3 tokenizer, unless stated otherwise. 4.1

Structural Attribution to Data

To isolate the environmental pressures driving convergence, we fix the architecture (300M Transformer) and optimizer (Muon, Jordan et al., 2024) and vary only the training data. We apply controlled perturbations (Table 1) that each remove a specific type of co-occurrence while leaving others intact, attributing the emergence of spectral and geometric convergence to specific structural properties of the data.

Fu, Zhou, Belkin, Sharan, & Jia

Table 1: Data perturbation used in §4.1. All models on the 10B tokens from FineWeb-Edu. Configuration

Perturbation

Structure of Data Removed

Original Isolate-k

− Each sequence contains at most k numbers token via packing. We use k = 1, 2, and 8. Sequences split into windows of ℓ. We use ℓ = 2, 4, 8, and 64 Number token sequence replaced by that of another sequence, keeping number n-gram Every number token resampled i.i.d. from marginal distribution to replace the original

− Interaction across numbers. k = 1 indicates no interaction. Role of broad context

Unigram Replace

Context Length 64 Context Length 8 Context Length 4 Context Length 2 Isolate (k=8) Isolate (k=2) Isolate (k=1) Swap Numbers Unigram Replace

Co-occurrence v.s. frequency

Cohen's of Linear Probe (%)

Fourier Magnitude Spectrum

Original

Number ↔ text association

Transformer LM (Muon)

96.1

-4.4

34.7

63.5

10.9

-9.0

9.9

-5.2

85.4

93.5

-7.1

31.4

46.6

11.7

-7.8

10.7

-6.3

72.0

3.0

88.7

-4.1

29.8

34.9

10.1

-7.3

10.0

-3.7

51.7

2.0

87.3

-7.2

27.7

34.7

8.1

-5.6

10.2

-4.1

47.4

1.5

71.7

-6.3

26.2

29.7

7.6

-3.9

12.1

-1.0

40.6

1.0

94.1

-4.4

32.2

57.2

9.4

-8.7

11.9

-6.4

77.2

100

89.6

-4.7

31.6

36.1

11.2

-5.9

12.1

-5.3

53.0

85.9

-0.4

30.1

34.3

14.1

-4.7

11.8

-2.4

45.0

70.7

-10.2

20.6

22.8

5.2

-6.3

8.7

-5.2

28.8

-5.9

-0.7

-1.8

1.2

-2.2

0.6

-1.0

0.2

1.2

4.0

Reducing Context Length

Reducing Cross-Number Interaction

0.0

0.1

60 40

Number Replacement

0.2

0.3

Fourier Frequency

0.4

Norm. Magnitude

Swap Numbers

Cohen's (%)

ContextLength-ℓ

20 0

0.5 mod 2 mod 3 mod 4 mod 5 mod 6 mod 7 mod 8 mod 9 mod 10

Figure 4: Spectral convergence is universal but geometric convergence depends on the data signal. (Left) Fourier spectra of Transformer embeddings trained under data perturbations in Table 1. All perturbations produce similar spikes to the original at periods T = 2, 5, 10, including Unigram Replace, which destroys all co-occurrence structure among number tokens. (Right) Cohen’s κ of linear probes for mod-T classification tells a different story: Original, Isolate-8, and Context Length 64 achieve strong probing, but with shorter context length or fewer numbers within each sequence (e.g. Isolate-2) is much weaker at T = 5 and 10. Swap Numbers drops substantially, and Unigram Replace falls to chance.

Spectral convergence requires only token frequencies. All perturbations produce nearly identical Fourier spectra (Figure 4, left), including Unigram Replace, which destroys all co-occurrence structure by independently resampling every number token from its marginal distribution. As predicted by the universality observed in §3, the periodic frequency distribution of number tokens is sufficient to produce Fourier spikes, and no co-occurrence information is needed. Geometric convergence draws on several complementary data signals. The right panel of Figure 4 reveals that geometric convergence degrades gradually as different types of co-occurrence information are removed. The probing power is measured with Cohen’s κ for balanced classes, the metric that removes random guessing from accuracy for Tway classifications defined as κ = (Accuracy − T1 )/(1 − T1 ). When κ = 0, the probe is at chance and κ = 100% is perfect classification regardless of T. Swap Numbers, which preserves number n-gram statistics but destroys the association between specific numbers and their text contexts, drops probing from κ = 85.4 to 28.8 at T = 10, making text-number co-occurrence an important signal. But it is not the only one. Longer context provides a second signal. At Context Length 2, where each token sees only one neighbor, mod-10 probing already reaches κ = 40.3, well above Swap Numbers (28.8). Increasing the window to ℓ = 4, 8, and 64 steadily improves mod10 probes (κ = 47.3, 51.7,

Convergent Evolution

Cohen's of Linear Probe (%) 96.1

-4.5

34.9

63.5

11.0

-9.1

9.9

-5.2

85.4

95.1

-5.3

37.3

55.2

10.7

-6.9

13.6

-5.2

77.8

Mamba-2

93.8

-2.1

29.3

50.7

10.7

-8.4

12.9

-4.8

76.7

Transformer

89.1

-2.8

32.4

47.7

12.3

-8.1

11.3

-5.3

72.1

Gated DeltaNet

91.1

-5.8

28.4

46.4

11.7

-9.8

10.6

-4.2

69.7

Mamba-2

95.0

-6.0

31.2

55.3

11.1

-7.6

10.8

-3.7

80.1

LSTM

9.0

-0.3

0.3

2.7

0.4

-1.2

0.3

-0.8

2.0

PPMI

54.4

-14.5

19.8

20.8

-0.1

-10.6

7.5

-7.5

27.1

word2vec

40.7

-2.3

18.9

25.0

7.4

-2.4

6.0

-2.0

29.3

Language Models (AdamW)

Classical Word Embeddings

0.0

0.1

0.2

4.0 3.0 2.0 1.5 1.0 100 80 60 40

0.3

Fourier Frequency

0.4

0.5

Norm. Magnitude

Transformer Gated DeltaNet

Cohen's (%)

Fourier Magnitude Spectrum

Language Models (Muon)

20 0

mod 2 mod 3 mod 4 mod 5 mod 6 mod 7 mod 8 mod 9 mod 10

Figure 5: Geometric convergence depends on architecture and optimizer. (Left) Fourier magnitude spectra of number embeddings across architectures and optimizers; all exhibit spikes at periods T = 2, 5, 10, including the LSTM and classical word embeddings. All three architectures produce similar Fourier spectra under both optimizers. (Right) Mod-T probe accuracy separates the models into tiers. With both Muon and AdamW, Transformer, Gated DeltaNet, and Mamba-2 all achieve strong geometric convergence, while the LSTM remains near zero. Muon outperforms AdamW for Transformer and Gated DeltaNet, but Mamba-2 with AdamW slightly outperforms its Muon counterpart. PPMI and word2vec fall in between.

and 72.0 respectively), showing that the model accumulates richer co-occurrence statistics from broader context. Cross-number interaction provides an additional signal as well. Isolate-k directly controls cross-number interaction by restricting each packed sequence to contain at most k number tokens. The extreme limit of k = 1 isolates text-number co-occurrence completely, ensuring no two number tokens can interact within the same attention window. Under this setting, it achieves κ = 45.0 at T = 10 and 85.9 at T = 2. Notably, even k = 1 with a Transformer surpasses PPMI (κ = 27.1) and word2vec (κ = 29.3), suggesting that autoregressive language modeling with text-number co-occurrence alone extracts richer modular structure than classical embedding methods. Allowing more numbers to co-occur within each attention block improves probing: κ = 53.0 at k = 2 and 77.2 at k = 8 for T = 10. The fact that Isolate (k = 1) still outperforms Swap Numbers confirms that text-number co-occurrence alone provides strong signal, but the gap to Original almost closed by Isolate (k = 8) shows that cross-number interaction contributes on top of it. In all cases, Fourier spikes are fully preserved while probing degrades, reinforcing the two-tiered hierarchy between spectral and geometric convergence and they are driven by different mechanisms. Probing accuracy varies sharply across moduli. Across all conditions that achieve geometric convergence, the probing accuracy depends strongly on the modulus. Mod 2, 5, and 10 are consistently the easiest (κ = 96.1, 63.5, 85.4 for Original), mod 4 achieves nontrivial probing (κ = 34.9), while moduli that share no common factor with 10 (e.g., 3, 7, 9) remain near chance. This pattern is stable across all perturbations that preserve geometric convergence. In Section 5, we show that this structure can be traced to the tokenizer. 4.2

Structural Attribution to Architecture and Optimizer

We now fix the pretraining data, and vary the architecture and optimizer. We train a 300M parameter Transformer model and two linear RNNs (Gated DeltaNet and Mamba-2 (Dao & Gu, 2024)) whose architectural and training details are shown in § B.1. We vary two different optimizers for training LLMs: AdamW and Muon. Figure 5 shows the results. Transformers and linear RNNs achieve geometric convergence; LSTMs do not. Under both Muon and AdamW, Transformer, Gated DeltaNet, and Mamba-2 all achieve strong probing, with the Transformer performing best. All three produce nearly identical Fourier spectra regardless of the optimizer, confirming that spectral convergence is architecture-

Fu, Zhou, Belkin, Sharan, & Jia

Cohen's of Linear Probe (%) -10.2

59.4

100.0

7.0

-9.9

7.7

-9.2

100.0 -24.2

99.7

53.6

100.0

-6.6

-13.1

3.4

-11.5 100.0

Muon (seed 42)

5.9

-2.3

-0.5

-10.4

-4.4

-2.1

-5.4

-6.6

-4.5

Muon (seed 123)

-4.9

-12.0

-8.4

2.8

-5.3

-6.6

-7.2

-5.9

0.3

AdamW (seed 42)

3.7

-10.1

-10.5

-14.3

-9.9

-12.2

-11.1

-10.7

-4.8

1.9

-0.4

-2.1

0.4

0.9

1.1

-1.9

-1.4

-3.3

Muon AdamW

100.0

3-Digit Addition

0.1

3.0 2.0 1.5

AdamW (seed 123) 0.0

4.0

0.2

0.3

Fourier Frequency

0.4

0.5

1.0

Norm. Magnitude

Fourier Magnitude Spectrum

100 80 60 40 20 0

Cohen's (%)

9-Digit Addition

mod 2 mod 3 mod 4 mod 5 mod 6 mod 7 mod 8 mod 9 mod 10

Figure 6: Tokenization determines convergence in arithmetic. (Top) In 9-digit addition, both Muon and AdamW converge to the same spectral structure with sharp Fourier peaks and near-perfect κ for mod 2, 5, and 10, showing both spectral and geometric convergence. (Bottom) In 3-digit addition, where every operand and sum fits in a single token, the Fourier spectra vary across optimizers and random seeds, and κ remains near chance for all moduli. Multi-token tokenization forces modular subproblems that produce convergent representations; single-token tokenization leaves the representation unconstrained.

independent and optimizer-independent in language pretraining task. The 12-layer LSTM, trained with AdamW, develops even more prominent Fourier spikes but its probing accuracy remains near chance across all moduli. A 4-layer LSTM shows no improvement nor degradation, indicating that the failure is architectural rather than a matter of capacity (see Figure 8). We note that as shown in Figure 2, the Fourier spectrum of the LSTM embeddings closely resembles that of the number token marginal distribution, suggesting that the LSTM embeddings capture little beyond unigram frequency statistics for numbers. Classical word embeddings fall in between: PPMI and word2vec, trained on the same 10B tokens, achieve moderate probing (κ = 27.1 and 29.3 at T = 10) with clear Fourier spikes yet weaker probing, illustrating the dissociation between spectral and geometric convergence. The effect of optimizer is architecture-dependent. Comparing the top and middle blocks of Figure 5 isolates the effect of the optimizer. Muon produces stronger probing for the Transformer (κ = 85.4 vs 72.1 at T = 10) and for Gated DeltaNet (77.8 vs 69.7), but Mamba-2 trained with AdamW actually outperforms Mamba-2 trained with Muon (80.1 vs 76.7). We find that the Transformer trained with Muon has the best probing performance. The optimizer’s effect on geometric convergence thus depends on the architecture, and we do not observe a universal advantage for either optimizer in language pretraining task. Spectral and geometric convergence co-emerge gradually. Figure 9 tracks Φ T and probe accuracy throughout Transformer pretraining for T = 2, 5, 10. Both increase smoothly with no phase transition, unlike the grokking observed in modular arithmetic (Nanda et al., 2023). We will discuss more on model behavior when trained directly on arithmetic in Section 5.

Convergent Evolution in Training on Arithmetic

Sections 3 and 4 studied models trained on general language, where Fourier features emerge from the statistics of number tokens in natural text. We now ask whether convergent evolution also occurs when models are trained directly on arithmetic, where the training signal is purely numerical, and the prior given by human language is absent. Experimental setup. We train 300M Transformers from random initialization on integer addition using the same architecture as in §3. Each example has the form a + b = c, with loss masked on prompt tokens up to = sign. We train for 3B tokens under both Muon and AdamW. In 9-digit addition, operands have 1-9 digits with stratified digit-count sampling. Each operand may span multiple number tokens. In 3-digit addition, we enumerate all pairs ( a, b) with a, b ∈ [0, 999] and a + b ≤ 999. Every operand and sum is a single token. Training for 3 billion tokens amounts to roughly 1,000 epochs; we run two seeds per optimizer. We additionally train circular probes that project embeddings onto the unit circle (see Figure 13).

Convergent Evolution

9-digit addition: convergence across optimizers. Both Muon and AdamW converge to the same spectral structure (Figure 6, top), with sharp Fourier peaks at the expected harmonics and near-perfect κ for mod 2, 5, and 10. The two optimizers produce nearly identical Fourier spectra and probing accuracy, suggesting that the multi-token setting imposes constraints strong enough for both spectral and geometric convergence. Models determine the representation regardless of optimizer. 3-digit addition: no convergence without modular pressure. The single-token setting produces a different outcome (Figure 6, bottom). With 500K unique pairs repeated over 1,000 epochs, the learned representations vary across optimizer and random seed. Muon develops Fourier peaks at frequencies that do not align with modular periods, and κ remains near chance. AdamW exhibits grokking under one seed: training accuracy reaches 100% early while test accuracy remains low until a phase transition after training achieves roughly 1.6B to 2B tokens (Figure 10); under another seed, generalization never occurs. Unlike Nanda et al. (2023), who train on mod-113 addition where modular structure is explicit, single-token addition imposes no modular constraint: the sequences a + b = c are identical whether interpreted as mod-1000 or mod-1111. Without this constraint, convergent evolution does not occur and the learned representation is seed-dependent. Why tokenization determines learned representation. In 9-digit addition after tokenization [ a2 , a1 , a0 ] + [b2 , b1 , b0 ] = [c2 , c1 , c0 ], each output token satisfies ci = ( ai + bi + γi ) mod 1000 where γi ∈ {0, 1} is the carry. Each output position is therefore a mod-1000 classification problem, especially the least significant position with no carry, i.e. γ0 = 0. Since all our models use tied embeddings (Table 2), the output logits depend directly on the input embedding matrix, creating pressure for the embeddings to develop high Φ1000 to distinguish residue classes at each output position. This in turn implies non-trivial Φ T for all T | 1000 = 23 · 53 . In single-token addition, no modular constraint is imposed, so Φ T is unconstrained and depends on the optimizer and random seed. This reveals a second route through the two-tiered hierarchy: multi-token tokenization creates modular subproblems that produce both spectral and geometric convergence through carry propagation, while single-token tokenization guarantees neither.

Conclusion and Discussion

We have shown that periodic number representations in language models exhibit a twotiered convergence: Fourier spikes are universal, but linearly separable mod-T classes emerge only when data, architecture, and optimizer align. The central lesson is that visible structure in representations does not guarantee functional organization: the LSTM and even the raw token distribution develop more prominent Fourier spikes than the Transformer yet achieve chance-level probing. More generally, any representation-level diagnostic could mistake statistical artifacts of the training distribution for learned structure. Our controlled perturbation approach offers a complementary lens to instance-level attribution methods such as influence functions (Koh & Liang, 2017): rather than attributing predictions to individual training examples, we attribute learned representations to structural properties of the data distribution. Analogous periodic representations have been found for days of the week and months of the year (Engels et al., 2025; Karkada et al., 2026); whether the spectral-geometric dissociation extends to these and other cyclic concepts is a natural next step. More broadly, the spectralgeometric hierarchy introduced here provides a concrete framework for distinguishing superficial from functional feature learning. This distinction may prove important well beyond numerical representations as we increasingly rely on representation-level diagnostics to understand large language models.

Acknowledgments The authors acknowledge the Center for Advanced Research Computing (CARC) at the University of Southern California for providing computing resources that have contributed to the research results reported within this publication. We also acknowledge the use of the USC NLP cluster provided by the USC NLP Group. DF and RJ were also supported by a

Fu, Zhou, Belkin, Sharan, & Jia

gift from the USC-Capital One Center for Responsible AI and Decision Making in Finance (CREDIF). RJ was supported in part by the National Science Foundation under Grant No. IIS-2403436. VS was supported by National Science Foundation award CCF-2239265, an Amazon Research Award, a Google Research Scholar Award and an Okawa Foundation Research Grant. The work was done in part while some of the authors were visiting the Simons Institute for the Theory of Computing. This work used the Delta system at the National Center for Supercomputing Applications through allocation CIS250737 from the Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS) program, which is supported by National Science Foundation grants #2138259, #2138286, #2138307, #2137603, and #2138296. This work was supported in part by the NVIDIA Academic Grant Program. The GPU resources provided by NVIDIA were essential for training and analyzing the 300M-parameter models examined in this study. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of the funding agencies.

References Zeyuan Allen-Zhu. Physics of language models: Part 4.1, architecture design and the magic of canon layers. arXiv preprint arXiv:2512.17351, 2025. Maximilian Beck, Korbinian Pöppel, Markus Spanring, Andreas Auer, Oleksandra Prudnikova, Michael Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. xlstm: Extended long short-term memory, 2024. URL https://arxiv.org/abs/2405.04517. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5:135–146, 2017. doi: 10.1162/tacl_a_00051. URL https://aclanthology.org/Q17-1010/. Tri Dao and Albert Gu. Transformers are SSMs: Generalized models and efficient algorithms through structured state space duality. In Forty-first International Conference on Machine Learning, 2024. URL https://openreview.net/forum?id=ztn8FCR1td. DeepSeek. Deepseek-v3 technical report, 2025. URL https://arxiv.org/abs/2412.19437. Joshua Engels, Eric J Michaud, Isaac Liao, Wes Gurnee, and Max Tegmark. Not all language model features are one-dimensionally linear. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=d63a4AM4hb. Pierre-Étienne H Fiquet and Eero P Simoncelli. A polar prediction model for learning to represent visual transformations. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=hyPUZX03Ks. Ronald Aylmer Fisher. The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7(2):179–188, 1936. doi: https://doi.org/10.1111/j.1469-1809.1936.tb02137.x. URL https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1469-1809.1936.tb02137. x. Amirata Ghorbani and James Zou. Data shapley: Equitable valuation of data for machine learning. In International Conference on Machine Learning, pp. 2242–2251, 2019. Andrey Gromov. Grokking modular arithmetic. arXiv preprint arXiv:2301.02679, 2023. Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces, 2024. URL https://arxiv.org/abs/2312.00752. Keji He, Chenyang Si, Zhihe Lu, Yan Huang, Liang Wang, and Xinchao Wang. Frequency-enhanced data augmentation for vision-and-language navigation. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 4351–4364. Curran Associates, Inc., 2023. URL https://proceedings.neurips.cc/paper_files/paper/2023/ file/0d9e08f247ca7fbbfd5e50b7ff9cf357-Paper-Conference.pdf.

Convergent Evolution

Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735–1780, 11 1997. ISSN 0899-7667. doi: 10.1162/neco.1997.9.8.1735. URL https: //doi.org/10.1162/neco.1997.9.8.1735. Ermo Hua, Che Jiang, Xingtai Lv, Kaiyan Zhang, Ning Ding, Youbang Sun, Biqing Qi, Yuchen Fan, Xuekai Zhu, and Bowen Zhou. Fourier position embedding: Enhancing attention’s periodic extension for length generalization. arXiv preprint arXiv:2412.17739, 2024. Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. Position: The platonic representation hypothesis. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp (eds.), Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 20617–20642. PMLR, 21–27 Jul 2024. URL https://proceedings.mlr.press/ v235/huh24a.html. Keller Jordan, Yuchen Jin, Vlado Boza, Jiacheng You, Franz Cesista, Laker Newhouse, and Jeremy Bernstein. Muon: An optimizer for hidden layers in neural networks, 2024. URL https://kellerjordan.github.io/posts/muon/. Subhash Kantamneni and Max Tegmark. Language models use trigonometry to do addition. arXiv preprint arXiv:2502.00873, 2025. Dhruva Karkada, Daniel J Korchinski, Andres Nava, Matthieu Wyart, and Yasaman Bahri. Symmetry in language statistics shapes the geometry of model representations. arXiv preprint arXiv:2602.15029, 2026. Kimi. Kimi linear: An expressive, efficient attention architecture, 2025. URL https://arxiv. org/abs/2510.26692. Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 1885–1894. PMLR, 06–11 Aug 2017. URL https://proceedings.mlr.press/v70/koh17a.html. Jannik Kossen, Jiatong Han, Muhammed Razzak, Lisa Schut, Shreshth Malik, and Yarin Gal. Semantic entropy probes: Robust and cheap hallucination detection in llms. arXiv preprint arXiv:2406.15927, 2024. Amit Arnold Levy and Mor Geva. Language models encode numbers using digit representations in base 10. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pp. 385–395, 2025. Anton Lozhkov, Loubna Ben Allal, Leandro von Werra, and Thomas Wolf. Fineweb-edu: the finest collection of educational content, 2024. URL https://huggingface.co/datasets/ HuggingFaceFW/fineweb-edu. George R. McGhee. Convergent Evolution: Limited Forms Most Beautiful. The MIT Press, 2011. ISBN 9780262016421. URL http://www.jstor.org/stable/j.ctt5hhhwt. Meta. The llama 3 herd of models, 2024. URL https://arxiv.org/abs/2407.21783. Meta. The llama 4 herd: The beginning of a new era of natively multimodal ai innovation, 2025. URL https://ai.meta.com/blog/llama-4-multimodal-intelligence/. Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id= 9XFSbDPmdW. Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. An overview of early vision in inceptionv1. Distill, 5(4):e00024–002, 2020.

Fu, Zhou, Belkin, Sharan, & Jia

Bruno A Olshausen and David J Field. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vision research, 37(23):3311–3325, 1997. OpenAI. gpt-oss-120b & gpt-oss-20b model card, 2025. URL https://arxiv.org/abs/2508. 10925. Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, and Yonatan Belinkov. LLMs know more than they show: On the intrinsic representation of LLM hallucinations. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=KRnsX5Em3W. Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. In Causal Representation Learning Workshop at NeurIPS 2023, 2023. URL https://openreview.net/forum?id=T0PoOJg8cK. Jeffrey Pennington, Richard Socher, and Christopher Manning. GloVe: Global vectors for word representation. In Alessandro Moschitti, Bo Pang, and Walter Daelemans (eds.), Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543, Doha, Qatar, October 2014. Association for Computational Linguistics. doi: 10.3115/v1/D14-1162. URL https://aclanthology.org/D14-1162/. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019. Adityanarayanan Radhakrishnan, Mikhail Belkin, and Dmitriy Drusvyatskiy. Linear recursive feature machines provably recover low-rank matrices, 2024. URL https: //arxiv.org/abs/2401.04553. Matthew Tancik, Pratul Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. Advances in neural information processing systems, 33:7537–7547, 2020. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper_files/paper/2017/ file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf. Songlin Yang, Jan Kautz, and Ali Hatamizadeh. Gated delta networks: Improving mamba2 with delta rule. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=r8H7xhYPwz. Ziqian Zhong, Ziming Liu, Max Tegmark, and Jacob Andreas. The clock and the pizza: Two stories in mechanistic explanation of neural networks. Advances in neural information processing systems, 36:27223–27250, 2023. Tianyi Zhou, Deqing Fu, Vatsal Sharan, and Robin Jia. Pre-trained large language models use fourier features to compute addition. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id= i4MutM2TZb. Tianyi Zhou, Deqing Fu, Mahdi Soltanolkotabi, Robin Jia, and Vatsal Sharan. Fone: Precise single-token number embeddings via fourier features. arXiv preprint arXiv:2502.09741, 2025. Jingwei Zuo, Maksim Velikanov, Dhia Eddine Rhaiem, Ilyas Chahed, Younes Belkada, Guillaume Kunsch, and Hakim Hacid. Falcon mamba: The first competitive attention-free 7b language model, 2024. URL https://arxiv.org/abs/2410.05355.

Convergent Evolution

Appendix A Proof of Theorem 1

A.1 Lemmas Fourier-variance identity . . . . . . . . . . . . . . . . . . . . . . . .

A.2 Necessary condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

A.3 Insufficiency condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

A.4 Lower and Upper Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

B Experiments

B.1 Model and Training Details . . . . . . . . . . . . . . . . . . . . . . . . . . . .

B.2 LSTM ablation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

B.3 Data Perturbation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

B.4 Training Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

B.5 Modular Probe Results with MLP and RFM Probes . . . . . . . . . . . . . .

Fu, Zhou, Belkin, Sharan, & Jia

Proof of Theorem 1

We first restate the theorem here. Theorem 1. Given embeddings {e(n)}nN=−01 and residue classes {Cr }rT=−01 defined above, let the class means µr , grand mean µ, between-class scatter matrix SB , and within-class scatter matrix SW be µr =

1 e ( n ), |Cr | n∑ ∈C

µ=

T −1

SB =

1 (µr − µ)(µr − µ)⊤ , T r∑ =0

SW =

1 N −1 e ( n ), N n∑ =0

1 T −1 ∑ (e(n) − µr )(e(n) − µr )⊤ . N r∑ =0 n ∈ C r

T −1 2 ∑ℓ= 1 ∥ Fℓ/T ∥

Let Φ T = be the total power at harmonics of period T, and HT {0, 1/T, 2/T, . . . , ( T −1)/T } be the set of harmonic frequencies.

Lemmas Fourier-variance identity

We first establish a lemma connecting the class means µr to the Fourier coefficients Fν . Lemma 3. For each ℓ = 0, . . . , T − 1, define

Then µ̂[ℓ] =

√

1 T −1 µ̂[ℓ] = √ ∑ µr e−2πiℓr/T . T r =0 T/N · Fℓ/T .

Every n ∈ {0, . . . , N − 1} belongs to exactly one class Cr with r = n mod T, so we can write n = mT + r for some non-negative integer m. Then e−2πiℓr/T = e−2πiℓn/T , since the additional mℓ full periods contribute an integer multiple of 2π to the exponent. Re-indexing the double sum as a single sum over n: √ N −1 T µ̂[ℓ] = e(n) e−2πiℓn/T . (2) N n∑ =0 Because T | N, the ratio ℓ/T is one of the N Fourier frequencies, so comparing with Fℓ/T = √1 ∑n e(n) e−2πi(ℓ/T )n gives N r √ T √ T µ̂[ℓ] = · N · Fℓ/T = ·F . N N ℓ/T Next, we state our second lemma on the identities for Tr(SB ) and Tr(SW ). Lemma 4. Let the between-class and within-class variances SB and SW be defined as in Theorem 1, they satisfy Φ 1 Tr(SB ) = T , Tr(SW ) = ∑ ∥ Fν ∥2 . N N ν/ ∈H T

Convergent Evolution

Proof. Define the T × d matrix M whose rows are the class means:   — µ0⊤ —  — µ⊤ —  1   M =   ∈ RT ×d , ..   . — µ⊤ T −1 —

and let U be the T × T matrix with entries Uℓr = √1 e−2πiℓr/T . Define M̂ = U M ∈ CT ×d , T

whose rows are µ̂[ℓ]⊤ :    M̂ =  

— µ̂[0]⊤ — — µ̂[1]⊤ — .. .



— µ̂[ T − 1]⊤ —

   ∈ CT ×d . 

Since U is the DFT matrix in T dimensions, U is unitary. Therefore, T −1

T −1

ℓ=0

r =0

∑ ∥µ̂[ℓ]∥2 = ∥ M̂ ∥2F = ∥U M ∥2F = ∥ M ∥2F = ∑ ∥µr ∥2 .

This is simply the fact that a unitary change of basis preserves the sum of squared norms. √ √ √ For ℓ = 0: µ̂[0] = T/N · F0 . Since F0 = √1 ∑n e(n) = N µ, we have µ̂[0] = T µ, and N therefore T1 ∥µ̂[0]∥2 = ∥µ∥2 .

The between-class variance is: 1 1 Tr(SB ) = ∑ ∥µr − µ∥2 = T r T 1 = T

∑ ∥ µr ∥2 − ∥ µ ∥2

1 = T

T −1

∑ ∥µr ∥ − 2µ 2

1 T

T −1

1 µr T∑ r !

+ ∥µ∥

(3)

∑ ∥µ̂[ℓ]∥2

(4)

1 T −1 Φ ∥ Fℓ/T ∥2 = T , ∑ N ℓ=1 N

(5)

T ∑ N ∥ Fℓ/T ∥2 ℓ=1

⊤

ℓ=1

where the second line uses T1 ∑r µr = µ, which holds because the classes {Cr } partition {0, . . . , N − 1} into equal-sized groups. For Tr(SW ), the total variance decomposes as V = Tr(SB ) + Tr(SW ). Define the N × d matrix E whose rows are e(0)⊤ , . . . , e( N − 1)⊤ , and let UN be the N × N DFT matrix with entries (UN )kn = √1 e−2πikn/N . Then Ê = UN E has rows Fν⊤ , and since UN is unitary, N

N −1

∑ ∥e(n)∥2 = ∥E∥2F = ∥Ê∥2F = ∑ ∥ Fν ∥2 .

Since F0 =

√

n =0

N µ, we have ∥ F0 ∥2 = N ∥µ∥2 , and therefore

1 N −1 1 V= ∥ e ( n ) − µ ∥2 = N n∑ N =0

∑ ∥e(n)∥ − N ∥µ∥ 2

! 2

1 ∥ Fν ∥2 . N ν∑ ̸ =0

Therefore: Tr(SW ) = V − Tr(SB ) =

1 1 T −1 1 ∥ Fν ∥2 − ∑ ∑ ∥ Fℓ/T ∥2 = N ∑ ∥ Fν ∥2 , N ν ̸ =0 N ℓ= 1 ν/ ∈H T

where HT = {0, 1/T, 2/T, . . . , ( T −1)/T } collects the zero-frequency and harmonic frequencies.

Fu, Zhou, Belkin, Sharan, & Jia

A.2

Necessary condition

Proof of Part (i) of Theorem 1. If Φ T = 0, then Tr(SB ) = 0 by Part (i). Since SB is positive semidefinite, Tr(SB ) = 0 implies SB = 0, which requires µr = µ for all r = 0, . . . , T − 1. When all class means coincide, the class-conditional distributions of e(n) share the same first moment, so no linear probe (or any probe relying on mean separation) can distinguish the T classes above the chance rate 1/T.

A.3

Insufficiency condition

Proof of Part (ii). We construct embeddings whose residue classes interleave periodically on the real line, then show that the geometry of linear decision boundaries prevents any T-class linear classifier from exceeding chance-level accuracy by more than ε. Fix T ≥ 2, C > 0, and ε > 0. Set K = ⌈( T −1)/( Tε)⌉ and N = KT. Every index n ∈ {0, . . . , N −1} has a unique decomposition n = mT + r with residue r ∈ {0, . . . , T −1} and block index m ∈ {0, . . . , K −1}. Define jnk e(n) = A · (n mod T ) + B · , | {z } T |{z} residue r

block index m

with A, B > 0 to be chosen. Intuitively, A controls n mod T: enlarging A pulls numbers sharing the same residue class Cr together. The parameter B controls ⌊n/T ⌋, which measures how many full copies of T fit below n: increasing B clusters numbers of similar magnitude together, regardless of their residue. We now show how A determines the Fourier power while B controls the interleaving that defeats linear classifiers. Fourier power. Within class Cr = {r, r + T, . . . , r +(K −1) T }, the term Ar is constant and only the block term varies, so the class mean is µr =

1 K −1 B ( K −1) ( Ar + Bm) = Ar + . K m∑ 2 =0

The grand mean is µ = A( T −1)/2 + B(K −1)/2, giving µr − µ = A r − ( T −1)/2 . Note that the block term B⌊n/T ⌋ contributes nothing to SB : its class mean B(K −1)/2 is identical across all classes and cancels in µr − µ. Therefore 1 T −1 A 2 T −1 T −1 2 A2 ( T 2 − 1) SB = ( µr − µ )2 = r− = , ∑ ∑ T r =0 T r =0 2 12 where the last equality uses the standard identity for the second central moment of T consecutive integers. By Lemma 4, Φ T = N · SB = A2 KT ( T 2 − 1)/12. Setting A = p 12C/(KT ( T 2 − 1)) gives Φ T = C. Periodic interleaving. Choose B > ( T −1) A so that consecutive blocks separate on the real line. The largest value in block m is e(( T −1) + mT ) = ( T −1) A + Bm, and the smallest value in block m+1 is e(0 + (m+1) T ) = B(m+1). Since B > ( T −1) A, we have B(m+1) > ( T −1) A + Bm, so block m and block m+1 occupy disjoint intervals on the real line. Within each block, the T points are sorted by residue: e(mT ) = Bm < e(mT +1) = A + Bm < · · · < e(mT + T −1) = ( T −1) A + Bm, since e(n) is increasing in n mod T for fixed ⌊n/T ⌋. Combining both observations, sorting all N embeddings by value yields the natural ordering: the i-th smallest embedding (0-indexed) is x(i) = e(i ) = A · (i mod T ) + B · ⌊i/T ⌋, i = 0, 1, . . . , N −1, whose class label is i mod T. The sorted class label sequence is therefore (0, 1, . . . , T −1) repeated K times: 0, 1, . . . , T −1, 0, 1, . . . , T −1, . . . , 0, 1, . . . , T −1 . | {z } | {z } | {z } block 0

block 1

block K −1

Any contiguous subsequence of length T or longer contains at least one complete cycle and therefore includes at least one point from every class.

Convergent Evolution

N=1000 Points. Period T=10

e(n) = A(n mod T) + B n/T

T=206250, best linear acc=100.0%

F 2

A=5, B=0.03,

e(n)

T=206250, best linear acc=10.9%

F 2

A=5, B=31,

1000

2000

e(n) 0

3000

mod 10 4 5

125000 100000 75000 50000 25000 0

T=10

0.0

0.1

0.2

0.3

0.4

0.5

0.3

0.4

0.5

frequency

T=10

0.0

0.1

0.2

frequency

Figure 7: Emergence of Fourier structure in a constructed embedding e(n) = A(n mod T ) + B⌊n/T ⌋ with T = 10, A = 5, N = 1000. Each dot is a number n placed at its scalar embedding value on the horizontal axis and colored by its class n mod T; since e(n) is onedimensional, the vertical coordinate carries no information and is jittered purely for visibility so that overlapping points of different classes remain distinguishable. Top (B = 0.03): within-block drift is small relative to between-class spacing, so the sorted line separates into 10 clean color bands: a linear classifier on e(n) recovers the mod-10 residue at near-perfect accuracy, and the DFT concentrates energy at the fundamental frequency ν = 1/T = 0.1. Bottom (B = 21): the between-block drift B dominates, interleaving classes along the line so that every mod-10 lane spans the full range; best linear accuracy collapses to ≈ 1/T + ε with ε = 0.9%. The Fourier spectrum keeps the same peak at ν = 0.1 (same Φ T ), showing that periodic energy at the fundamental is a property of the construction itself, independent of whether class identity is linearly decodable.

Classification bound. A T-class linear classifier assigns each instance x ∈ R to argmaxc∈{0,...,T −1} (wc x + bc ) for parameters wc , bc ∈ R. This is the hypothesis class of multiclass logistic regression. Note that this hypothesis class is expressive enough to perfectly classify T contiguous groups: when B is small, the T classes cluster near 0, A, 2A, . . . , ( T −1) A, and setting wc = c with biases cascaded via b0 = 0, bc+1 = bc − (cA + A/2) places the T −1 decision boundaries between consecutive clusters, achieving 100% accuracy. Next we exploit the limitation that linear classifiers partition R into at most T contiguous intervals: any two of the T lines x 7→ wc x + bc intersect in at most one point, yielding at most T − 1 breakpoints and hence at most T intervals, each assigned to a single class. Now consider any such interval. Each complete cycle (0, 1, . . . , T −1) contained in the interval contributes exactly one point from every class, so the assigned label matches exactly a 1/T fraction of those points. Only at interval boundaries can a partial cycle contribute additional correct predictions: at most one per boundary, for a total of at most T − 1 extra correct points across all T − 1 boundaries. It follows that N/T + T − 1 1 T−1 accuracy ≤ = + . N T KT The choice K = ⌈( T −1)/( Tε)⌉ guarantees ( T −1)/(KT ) ≤ ε, completing the proof. Figures 3 and 7 illustrates this construction for T = 5, N = 25, and for T = 10, N = 1000, −1 , at 16% and 0.9% respectively. where in both cases ε achieves its minimum at TN

The token frequency distribution in Figure 2 provides an empirical counterexample: LSTM exhibits clear Fourier spikes at T = 2, 5, 10 yet achieves chance-level probing for all moduli, showing that the construction above captures a phenomenon that occurs in practice.

Fu, Zhou, Belkin, Sharan, & Jia

A.4

Lower and Upper Bounds

Lemma 5. Assume d ≥ T − 1 and SW is invertible (which holds when d ≤ N − T, or after projecting to a subspace of dimension at most N − T). In Fisher’s Linear Discriminant Analysis (Fisher, 1936), the optimal linear probe maximizes the ratio of between-class to within-class variance along a projection direction. The separability of this optimal discriminant is characterized by −1 λmax (SW SB ), the largest generalized eigenvalue of the scatter matrix pair, which satisfies 1 ΦT ΦT −1 SB ) ≤ · ≤ λmax (SW . ( T − 1) · cond(SW ) N · λmin (SW ) N · λmin (SW ) The ratio between the upper and lower bounds is ( T − 1) · cond(SW ), where cond(SW ) = λmax (SW )/λmin (SW ). Proof. The optimal linear discriminant for T-class classification projects the data onto the direction maximizing the generalized Rayleigh quotient: v⊤ SB v . v̸=0 v⊤ SW v

−1 λmax (SW SB ) = max

Upper bound. For any v ̸= 0, the numerator satisfies v⊤ SB v ≤ λmax (SB )∥v∥2 ≤ Tr(SB )∥v∥2 , where the second inequality holds because SB is positive semidefinite and λmax (SB ) ≤ Tr(SB ). The denominator satisfies v⊤ SW v ≥ λmin (SW )∥v∥2 . Therefore: ΦT Tr(SB ) −1 = . λmax (SW SB ) ≤ λmin (SW ) N · λmin (SW ) Lower bound. The matrix SB has rank at most min(d, T − 1); since d ≥ T − 1 by assumption, this simplifies to T − 1. This holds because the T vectors {µr − µ}rT=−01 satisfy ∑r (µr − µ) = 0 and therefore span a subspace of dimension at most T − 1. It follows that λmax (SB ) ≥ Tr(SB )/( T − 1) = Φ T /( N ( T − 1)). Let v∗ be the unit eigenvector of SB corresponding to λmax (SB ). Then: −1 λmax (SW SB ) ≥

λmax (SB ) λmax (SB ) v∗ ⊤ SB v∗ ΦT = ∗⊤ ≥ ≥ . ⊤ ∗ ∗ ∗ λmax (SW ) N · ( T − 1) · λmax (SW ) v SW v v SW v

Gap between the bounds. The ratio of the upper to lower bound is: Φ T /( N · λmin (SW )) λmax (SW ) = ( T − 1) · = ( T − 1) · cond(SW ). Φ T /( N · ( T − 1) · λmax (SW )) λmin (SW ) The Fourier power spectrum fully determines Φ T via Part (i), but cond(SW ) depends on the directional structure of within-class variation, which the power spectrum {∥ Fν ∥2 } does not capture. Specifically, ∥ Fν ∥2 aggregates power across all d embedding dimensions at frequency ν, discarding any information about which dimensions carry the periodic signal versus which carry within-class noise. As a result, two embeddings with identical power spectra (and hence identical Φ T ) but different within-class covariance structures can yield −1 λmax (SW SB ) values differing by up to a factor of ( T − 1) · cond(SW ), producing vastly different probe accuracies.

Convergent Evolution

Experiments

B.1

Model and Training Details Table 2: Model architectures and training configurations. Architecture Total parameters → Embedding → Non-embedding Tied embeddings Layers Hidden dim d Heads Head dim MLP intermediate MLP activation Positional encoding Normalization Sequence mechanism Short convolution SSM expand factor Dropout

Transformer

Gated DeltaNet

320M 131M 189M ✓ 12 1024 16 (8 KV) 64 4096 SwiGLU RoPE RMSNorm GQA — — —

318M 316M 232M 131M 131M 131M 186M 185M 101M ✓ ✓ ✓ 12 28 12 1024 1024 1024 16 32 — 64 64 — 4096 — — SwiGLU — — None None None RMSNorm RMSNorm — Linear Attn + Gate SSM (dstate =128) Recurrence size 4 size 4 — 1.5× (value) 2× — — — 0.1

Muon optimizer 2D weight LR Embed/norm/bias LR Weight decay

B.2

Mamba-2

LSTM

3 × 10−3 , momentum = 0.95 3 × 10−4 (AdamW, β 2 =0.95) 0.01 (2D weights only)

AdamW optimizer Learning rate ( β1 , β2 ) Weight decay

3 × 10−4 (0.9, 0.95) 0.01

Shared training Context length Batch size LR schedule Training tokens Precision

1024 512 sequences (∼524K tokens/step) Cosine decay, 500 warmup steps, min = 10% of peak ∼9.4B (1 epoch of FineWeb-Edu 10BT) bfloat16 mixed precision

LSTM ablation

We ablate the number of layers of LSTMs and find reducing the number of layers to 4 instead of 12 does not change the phenomenon: both will learn fourier spikes but no probing performance. Both have huge condition number on SW yet huge Φ T as well. B.3

Data Perturbation Details

In this section, we describe in detail how we perturb data for each configurations in Table 1. Isolate-k configuration. We design an isolate configuration to test whether Fourier features and mod-T probes can emerge when reducing the interactive between number tokens, even indirectly through intermediate text tokens across multiple layers. The key idea is to enforce a block-diagonal causal attention mask that partitions each sequence into segments, where each segment contains at most k token. Concretely, given a tokenized sequence, we locate all positions containing number tokens and place segment boundaries at the midpoint of the text span between each consecutive pair of k number tokens. This way, every number token still sees some surrounding context on both sides, but can never interact with any other number token, if they are not in the same segment. Within each segment, standard causal attention applies: position i attends to position j only if j ≤ i and both positions belong to the same segment. We also reset RoPE position IDs to zero at segment boundaries to avoid leaking positional information across segments, and mask the loss at boundaries so

Fu, Zhou, Belkin, Sharan, & Jia

Fourier Magnitude Spectrum T=5

T=10

100

Cohen's of Linear Probe (%)

LSTM (4-layer)

LSTM

T=2

0 0.0 0.1 0.2 8 T=10 T=5

0 0.3

0.4

0.5 100

0 0.0

-1

2 3 4 5 6 7 8 9 10

50 0 0.1

0.2

0.3

Fourier Frequency

0.4

0.5

max = 0.290 Fourier 10 = 337.4 cond(SW) = 8569.0

T=2

Eigenvalues of SW1SB (T=10)

0 30

1 2 3 4 5 6 7 8 9 max = 0.698 Fourier 10 = 736.3 cond(SW) = 20297.1

15 5

2 3 4 5 6 7 8 9 10

Modulus T

1 2 3 4 5 6 7 8 9

Eigenvalue Rank i

Figure 8: Ablation on the depth of LSTM models. We find that reducing the number of layers to 4 (green) instead of 12 (blue) does not change the phenomenon: both will learn fourier spikes but no probing performance.

the model is not trained to predict across segment breaks. Importantly, we do not modify the training data at all. The tokenized sequences are identical to those used in the standard configuration; only the attention mask differs. Context length ℓ. To test the role of broad context in Fourier feature formation and geometric emergence, we train models whose effective context is limited to a fixed window of ℓ consecutive tokens. Each 1024-token sequence is reshaped into ⌊1024/ℓ⌋ independent subsequences of length ℓ, each processed with its own standard causal attention mask. Tokens in one window cannot attend to tokens in any other window. This is equivalent to training on a corpus of short documents of length ℓ. We experiment with ℓ ∈ {2, 4, 8, 64}: a window of ℓ = 2 reduces the model to learning bigram statistics (each token sees only the immediately preceding token), while ℓ = 4 and 8 permit short-range dependencies but still prevents any long-range co-occurrence patterns. ℓ = 64 permits longer range dependencies but is still much shorter than Original’s context length of 1024. As with the isolate configuration, the underlying token sequence is unchanged; only the effective context window differs. Swap numbers. This configuration dissociates number tokens from their original textual context while preserving natural number n-gram statistics. For each training sequence, we keep every text token in its exact position and replace the entire subsequence of number tokens with a contiguous, order-preserving slice of number tokens drawn from other documents in the corpus. Concretely, we pre-extract all number tokens from the full training set into a single in-memory stream, and for each sequence we substitute a randomly chosen contiguous segment from this pool. The replaced number tokens therefore retain realistic sequential patterns but lose their association with the surrounding text. This tests whether the text-number co-occurrence structure, rather than the number token statistics alone, drives Fourier feature and modular probe emergence. Unigram replace. In the unigram configuration, every number token in the training data is independently replaced by a random draw from the corpus-wide marginal (unigram) distribution over number tokens. This destroys all sequential structure among numbers, such as their relationship to surrounding text and their co-occurrence with other numbers. But this type of perturbation exactly preserves the marginal frequency of each number token. If a number token n appears with probability pn in the original corpus, it appears with the same probability in the perturbed data. By comparing to the original and swap-numbers configurations, the unigram ablation isolates whether co-occurrence statistics beyond simple token frequency are necessary for Fourier features and modular arithmetic to emerge.

Convergent Evolution

B.4

Training Dynamics

Figure 9 shows that during language pretraining, both Fourier power Φ T and linear probe accuracy increase smoothly from the start of training for T = 2, 5, 10, with no sudden phase transition. This contrasts with grokking in modular arithmetic (Nanda et al., 2023), where structured representations appear abruptly after prolonged memorization. In language pretraining, the model continuously encounters diverse co-occurrence statistics rather than memorizing a fixed set of examples, so both tiers of convergence emerge gradually. In contrast, Figure 10 shows training dynamics for addition.

1.5 0.0

Fourier Power 2 Linear Probe Acc

Training Tokens (B)

4 0

Fourier Power 5 Linear Probe Acc

Training Tokens (B)

100

Fourier Power 10 Linear Probe Acc

Training Tokens (B)

Linear Probe Accuracy (%)

T = 10

Fourier Power 10

3.0

Fourier Power 5

Fourier Power 2

4.5

T=5

Linear Probe Accuracy (%)

100

Linear Probe Accuracy (%)

T=2

6.0

Figure 9: Spectral and geometric convergence co-emerge gradually during pretraining. Fourier power Φ T (left axis) and linear probe accuracy (right axis) for a 300M Transformer trained with Muon, shown for T = 2, 5, 10. Both metrics increase smoothly throughout training with no phase transition, in contrast to the sudden emergence observed in grokking on modular arithmetic tasks in Nanda et al. (2023). 9-Digit Addition: Muon vs AdamW (seed 42)

100

Accuracy (%)

80 60 40 20 0

3-Digit Addition: Muon vs AdamW (seed 42)

100

Muon AdamW 0.0

0.5

1.0

1.5

2.0

Training Tokens (B)

2.5

60 40 20

Test Train

3.0

Muon AdamW 0.0

0.5

1.0

1.5

2.0

Training Tokens (B)

2.5

Test Train 3.0

3-Digit Addition: Muon vs AdamW (seed 123)

100

Accuracy (%)

80 60 40 20 0

Muon AdamW 0.0

0.5

1.0

1.5

2.0

Training Tokens (B)

2.5

Test Train 3.0

Figure 10: Training dynamics for Transformers trained on addition (two seeds). (Left) In 9-digit addition, both Muon and AdamW converge smoothly to near-perfect train and test accuracy, with no grokking phase. (Right) In 3-digit addition, under seed 42, training accuracy reaches 100% for both optimizers, but generalization is optimizer- and seeddependent. AdamW exhibits a quick grokking under seed 42 (test accuracy jumps around 1.5–2B tokens) but not under seed 123 (where training accuracy can’t reach 100% and test accuracy remains random). This confirms that single-token addition imposes no consistent pressure toward structured representations.

Fu, Zhou, Belkin, Sharan, & Jia

B.5

Modular Probe Results with MLP and RFM Probes

In this section, we present the modular probes similar to Figures 4 to 6 but with RFM probes and 2-layer MLP probes with hidden layer of size 64. Cohen's of RFM (%)

Fourier Magnitude Spectrum

65.4

11.0

-7.7

13.3

-3.9

90.0

94.4

-8.1

34.4

47.5

11.9

-8.0

14.4

-4.8

79.1

3.0

88.8

-7.1

30.8

34.7

11.8

-7.5

9.9

-2.8

55.9

2.0

86.9

-9.1

29.8

33.1

8.8

-6.4

12.2

-3.0

53.7

1.5

70.8

-7.2

28.2

29.9

7.7

-4.8

11.5

-0.3

39.5

1.0

94.7

-9.0

35.6

57.6

10.3

-8.7

13.6

-6.0

82.4

100

91.7

-6.0

32.9

37.2

11.2

-6.3

14.8

-3.6

59.4

86.9

1.3

32.0

35.3

15.8

-4.8

12.3

-0.8

48.5

73.0

-12.4

22.5

24.6

5.8

-5.7

9.9

-4.5

33.4

-1.5

-0.8

-4.5

0.8

-2.0

1.5

0.2

0.3

0.6

0.1

0.2

0.3

0.4

Fourier Frequency

Transformer Gated DeltaNet Mamba-2 LSTM PPMI word2vec

Cohen's of RFM (%)

Language Models (Muon)

97.2

-7.0

41.2

65.4

11.0

-7.7

13.3

-3.9

90.0

96.1

-7.8

40.1

55.3

10.4

-7.3

17.3

-3.6

85.0

95.1

-5.0

34.1

51.1

10.5

-8.3

15.9

-3.2

82.4

91.1

-4.5

35.8

48.2

14.8

-8.2

14.3

-4.2

80.9

92.4

-6.7

32.2

47.8

13.1

-9.1

12.5

-4.3

77.9

95.3

-7.1

35.8

55.5

10.6

-8.0

13.3

-2.8

85.8

2.0

-2.0

1.1

2.0

-1.3

-0.5

0.8

-0.1

2.7

50.5

-9.3

18.1

19.0

4.7

-5.8

7.2

-4.0

29.3

48.0

-4.5

22.5

25.8

8.0

-1.6

8.5

-1.4

32.4

Language Models (AdamW)

Classical Word Embeddings

0.0

0.1

Muon AdamW Muon (seed 42) Muon (seed 123) AdamW (seed 42) AdamW (seed 123)

0.2

0.3

0.4

2.0 1.5 1.0

100 80 60

0.2

0.3

Fourier Frequency

20 0

Cohen's of RFM (%) -12.5

60.1

100.0

7.7

-9.9

10.1

-8.1

100.0 -23.2

60.0

100.0

-5.1

-13.8

9.1

-11.2 100.0

100.0

3-Digit Addition

0.1

3.0

0.5 mod 2 mod 3 mod 4 mod 5 mod 6 mod 7 mod 8 mod 9 mod 10

Fourier Magnitude Spectrum 99.6

0.0

4.0

Fourier Frequency

9-Digit Addition

20 0

0.5 mod 2 mod 3 mod 4 mod 5 mod 6 mod 7 mod 8 mod 9 mod 10

Fourier Magnitude Spectrum

Transformer Gated DeltaNet Mamba-2

60 40

Number Replacement

0.0

Cohen's (%)

Reducing Cross-Number Interaction

Norm. Magnitude

4.0

Norm. Magnitude

Swap Numbers Unigram Replace

41.2

Cohen's (%)

Isolate (k=8) Isolate (k=2) Isolate (k=1)

-7.0

0.4

22.1

-3.7

-0.8

-10.8

-4.4

-0.8

-4.9

-6.1

-10.0

-14.0

-8.9

3.6

-4.0

-6.0

-6.6

-5.7

-3.4 1.1

3.2

-15.7

-11.6

-14.1

-10.4

-11.5

-10.1

-10.7

-3.5

-1.2

-0.8

-2.7

1.0

0.4

0.1

-2.2

-1.9

-2.7

0.5 mod 2 mod 3 mod 4 mod 5 mod 6 mod 7 mod 8 mod 9 mod 10

Figure 11: Structural Attribution Results with RFM probes.

4.0 3.0 2.0 1.5 1.0 100 80 60 40 20 0

Norm. Magnitude

Context Length 64 Context Length 8 Context Length 4 Context Length 2

97.2

Reducing Context Length

Cohen's (%)

Original

Transformer LM (Muon)

Convergent Evolution

Cohen's of MLP (%)

Fourier Magnitude Spectrum

62.5

7.6

-9.6

7.2

-5.1

83.8

90.9

-11.2

28.1

44.2

8.0

-7.7

8.6

-5.6

69.7

3.0

86.1

-4.8

24.5

34.1

9.0

-7.5

7.3

-4.3

49.7

2.0

86.5

-9.6

26.1

31.3

7.5

-6.7

10.6

-4.5

44.9

1.5

67.7

-7.3

24.5

27.7

7.0

-5.5

11.0

-1.8

36.9

1.0

92.3

-10.9

27.9

57.2

9.0

-9.7

8.8

-5.4

75.6

100

89.5

-7.6

28.3

35.0

9.3

-7.0

10.0

-5.1

51.0

84.7

-0.8

26.9

32.9

12.9

-4.8

10.5

-2.6

43.6

65.1

-13.3

18.7

22.3

3.8

-8.5

5.9

-5.5

29.8

-4.5

0.7

-3.2

3.0

-2.4

1.1

-1.6

-0.2

0.1

0.2

0.3

0.4

Fourier Frequency

Transformer Gated DeltaNet Mamba-2 LSTM PPMI word2vec

Cohen's of MLP (%)

Language Models (Muon)

94.8

-10.9

32.0

62.6

7.6

-9.8

7.8

-5.5

83.9

93.6

-9.9

33.4

53.2

7.5

-8.6

11.0

-5.8

76.1

92.0

-8.0

26.8

51.0

5.6

-9.8

10.0

-3.8

74.5

86.8

-7.9

28.4

46.6

11.4

-8.7

9.2

-5.4

68.5

88.4

-9.6

25.2

47.7

5.9

-9.0

7.8

-4.7

67.5

92.3

-11.1

29.0

54.1

6.2

-8.5

7.4

-4.0

78.4

4.3

-3.3

1.6

2.2

-0.4

-2.0

1.2

-0.2

2.5

45.9

-11.3

11.8

15.2

0.8

-8.1

2.8

-6.6

22.2

44.7

-6.1

19.2

25.3

4.5

-3.8

4.7

-3.7

32.3

Language Models (AdamW)

Classical Word Embeddings

0.0

0.1

Muon AdamW Muon (seed 42) Muon (seed 123) AdamW (seed 42) AdamW (seed 123)

0.2

0.3

0.4

2.0 1.5 1.0

100 80 60

0.2

0.3

Fourier Frequency

20 0

Cohen's of MLP (%)

Fourier Magnitude Spectrum -17.0

55.0

100.0

4.8

-10.1

4.2

-9.3

100.0 -28.3

69.1

100.0

-2.1

-13.7

11.3

-11.5 100.0

100.0

3-Digit Addition

0.1

3.0

0.5 mod 2 mod 3 mod 4 mod 5 mod 6 mod 7 mod 8 mod 9 mod 10

99.7

0.0

4.0

Fourier Frequency

9-Digit Addition

20 0

0.5 mod 2 mod 3 mod 4 mod 5 mod 6 mod 7 mod 8 mod 9 mod 10

Fourier Magnitude Spectrum

Transformer Gated DeltaNet Mamba-2

60 40

Number Replacement

0.0

Cohen's (%)

Reducing Cross-Number Interaction

Norm. Magnitude

4.0

Norm. Magnitude

Swap Numbers Unigram Replace

31.9

Cohen's (%)

Isolate (k=8) Isolate (k=2) Isolate (k=1)

-10.3

0.4

14.1

-5.5

-0.4

-11.3

-4.5

0.6

-4.5

-6.0

-4.2

-10.4

-11.8

-9.9

18.0

-6.4

-7.2

-6.6

-6.3

2.9

9.2

-15.2

-13.0

-14.1

-6.4

-12.3

-11.0

-10.2

-4.9

-2.9

0.9

-3.0

-0.3

0.4

0.8

-2.0

-2.3

-3.1

0.5 mod 2 mod 3 mod 4 mod 5 mod 6 mod 7 mod 8 mod 9 mod 10

Figure 12: Structural Attribution Results with 2-layer MLP probes.

4.0 3.0 2.0 1.5 1.0 100 80 60 40 20 0

Norm. Magnitude

Context Length 64 Context Length 8 Context Length 4 Context Length 2

95.0

Reducing Context Length

Cohen's (%)

Original

Transformer LM (Muon)

Fu, Zhou, Belkin, Sharan, & Jia

9-digit addition (84% acc)

3-digit addition (11% acc)

3 1

4 0

9 7

2 1

0 6

9 7

0 1 2 3 4 5 6 7 8 9

Figure 13: Circular probe projections of token embeddings onto the unit circle for mod-10 classification. Each point is a number token n ∈ [0, 999] projected to 2-D by the probe and normalized; color indicates n mod 10. (Left) embeddings from a model trained on 9-digit addition cluster sharply by residue class (84% test accuracy), indicating that the token embedding layer has learned a geometrically organised, clock-like representation. (Right) embeddings from a model trained on 3-digit addition show no angular structure (11% test accuracy, near chance), consistent with the absence of Fourier peaks in Figure 6. Cicular Probes for Transformers Trained on Addition. To probe how number tokens are represented in the embedding layer, we train circular probes on the token embeddings for numbers. Beyond T-class modular probes, which test linear separability in a ( T −1)dimensional subspace, a circular probe tests angular separability in 2-D: it learns a linear map W ∈ Rd×2 projecting each embedding onto the unit circle, and classifies by cosine similarity to m anchor directions at θk = 2πk/m.

Record · ID 124036 · SHA-256 373e27c0b7997f5b
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.