The Missing Primitive: Diagnosing and Repairing Mathematical Reasoning in Large Language Models

Preprint

T HE M ISSING P RIMITIVE : D IAGNOSING AND R EPAIR ING M ATHEMATICAL R EASONING IN L ARGE L AN GUAGE M ODELS Shuo Xing1 Zilin Dai2 Chengyuan Qian1 Fangzhou Lin1 Wenjing Chen1 Ping He3 Pan Lu4 Alvaro Velasquez5 Mohit Bansal6 Zhengzhong Tu1∗ 1

arXiv:2610.02191v1 [cs.LG] 1 Oct 2026

4

Texas A&M University 2 Harvard University 3 Vanderbilt University Stanford University 5 DARPA 6 University of North Carolina at Chapel Hill {shuoxing,tzz}@tamu.edu

€ Model & Code: https://taco-group.github.io/Math-Primitive/

A BSTRACT While Large Language Models (LLMs) have demonstrated striking capabilities on frontier mathematical problems, it remains unclear whether they possess the structural mathematical understanding underlying their solutions. In this paper, we take a first step toward systematically studying mathematical understanding in LLMs, from diagnosing its distinct capabilities to leveraging these findings to improve post-training. First, we introduce the notion of Mathematical Primitive to probe structural mathematical understanding and propose P RIM, a novel benchmark that evaluates mathematical reasoning along four distinct dimensions: Discovery, Generation, Digestion, and Execution. Second, our systematic diagnosis shows that solution accuracy masks distinct capability profiles, primitives unlock substantial latent execution capacity, and Discovery is the dominant bottleneck in mathematical reasoning. Our post-training analysis further shows that discovery-limited failures are particularly amenable to repair. Finally, building on these findings, we introduce A BSORB, a primitive-privileged self-distillation framework that selectively transfers primitive-guided reasoning into the student model. Extensive experiments demonstrate that A BSORB consistently improves mathematical reasoning over baselines across model scales and challenging benchmarks.

1

I NTRODUCTION

Recent advances in Large Language Models (LLMs) have driven rapid progress in mathematical problem solving, with frontier models increasingly capable of tackling research-level questions and, in several recent cases, resolving or making substantive progress on long-standing open problems (OpenAI, 2026a;b; Claude, 2026; Bubeck et al., 2025; Jang & Ryu, 2025). Yet these striking successes leave unresolved a more foundational question: what models actually understand about the problems they solve and the mathematical content of the solutions they produce. Successful problem solving does not necessarily imply such understanding: models may arrive at correct answers through specious deductions (Boye & Moell, 2025), while conversely identifying sound high-level proof structures yet failing in low-level technical execution (Lu et al., 2024; Qin et al., 2025). Existing works provide limited insight into this disconnect, as mathematical reasoning is still assessed primarily through outcome-level metrics such as final-answer accuracy or formal proof verifiability, obscuring whether success reflects the discovery of the mathematical structure needed for a solution or merely the successful execution of a derivation. Understanding mathematical reasoning therefore requires more than verifying that a derivation is correct; it requires exposing the ideas that organize the argument—why it works, which components are essential, and what can be generalized or reused. This need becomes increasingly salient as ∗

Corresponding author.

1

Preprint

Problem Are the mug and donut homeomorphic?

Structural Understanding

Primitive

Four Dimensions of Mathematical Understanding

Both have one hole (genus 1).

PrimitiveGuided Reasoning

Mathematical Primitive Teacher

Solution

1. Construct a map 𝑓: M → D. 2. Verify that 𝑓 is bijective.

3. Verify continuity of 𝑓 and 𝑓⁻¹

④ Digestion

① Discovery

③ Execution

Problem

② Generation

Internalize

Problem StudentGenerated Reasoning

Solution Student

Figure 1: Diagnosing and internalizing mathematical primitives. We introduce Mathematical Primitives to probe structural mathematical understanding in LLMs across four dimensions: Discovery, Generation, Digestion, and Execution. Our diagnosis further motivates A BSORB, which selectively transfers primitive-guided reasoning into the student without requiring primitives at inference time. LLM-generated proofs grow in scale and complexity: lengthy chains of valid steps may establish correctness while leaving the underlying mathematical structure difficult to identify. Recent work has begun to emphasize precisely these aspects of mathematical reasoning, calling for representations that reveal the motivation, structure, and key ideas underlying an argument rather than only its final outcome (Frieder et al., 2024). In his ICM 2026 lecture, Terence Tao similarly highlighted the importance of making machine-generated mathematics conceptually digestible (Tao, 2026), warning that optimizing primarily for formal correctness risks producing an increasing volume of valid but opaque proofs without a corresponding increase in mathematical understanding. For LLMs, this perspective shifts the focus from merely producing valid proofs to determining whether the structural knowledge organizing those proofs is actually accessible to the model. In this paper, we systematically study structural mathematical understanding in LLMs, as illustrated in Figure 1: the ability to identify the compact, load-bearing mathematical structure that explains why a solution works, distinct from the procedural ability to generate the derivation itself. We operationalize this notion through the Mathematical Primitive, a concise, non-procedural representation of the foundational idea organizing a solution, and introduce P RIM, a benchmark spanning four complementary dimensions: Discovery, Generation, Digestion, and Execution. Our diagnosis reveals that ❶ similar solving accuracy can mask markedly different capability profiles, ❷ correct primitives unlock substantial latent execution capacity, and, most importantly, ❸ independent Discovery is the dominant bottleneck to mathematical reasoning. We further find that discovery-limited failures are substantially more amenable to post-training, but existing methods can introduce regressions on problems the model already solves. Motivated by these findings, we introduce A BSORB, a post-training paradigm that uses Mathematical Primitives as privileged information and a bounded override mechanism to selectively transfer their guidance into the student’s reasoning process, enabling primitive-guided reasoning to be internalized without access to primitives at inference time. Our main contributions are as follows: • Benchmarking structural mathematical understanding. We introduce the notion of Mathematical Primitive and propose P RIM, a benchmark that evaluates mathematical reasoning across four complementary dimensions: Discovery, Generation, Digestion, and Execution. • Systematic Diagnosis. Our analysis reveals distinct capability profiles hidden by answer accuracy, substantial latent execution capacity unlocked by correct primitives, and independent Discovery as the dominant bottleneck in mathematical reasoning. We further show that discovery-limited failures are substantially more amenable to post-training. • Internalizing primitive-guided reasoning. Building on these findings, we introduce A BSORB, a primitive-privileged post-training paradigm that selectively transfers primitive-guided reasoning into the student model. Across multiple model scales and challenging mathematical reasoning benchmarks, A BSORB consistently improves average performance by 2.42 – 4.48 points.

2

M ATHEMATICAL P RIMITIVES AS S TRUCTURAL U NDERSTANDING

Rather than viewing mathematical problem solving as a sequence of steps, we separate structural mathematical understanding from procedural execution. We formalize the distinction through the 2

Preprint

Mathematical Primitive (§2.1) and introduce P RIM, a benchmark that evaluates LLM mathematical reasoning across four dimensions: Generation, Discovery, Digestion, and Execution (§2.3). 2.1

M ATHEMATICAL P RIMITIVE

A mathematical solution often begins before the first line of a proof, when an opaque problem is recognized through the structure that makes it solvable. We term this load-bearing structural understanding the Mathematical Primitive (Definition 1). Unlike generic advice such as use “induction” or “apply a theorem”, a primitive identifies the structural property that makes such a strategy applicable. It therefore serves as an intermediate reasoning object between problem understanding and procedural solution construction. Definition 1 (Mathematical Primitive). For a mathematical problem, a primitive is the essential conceptual observation that reveals why the problem can be solved. It captures the latent mathematical structure shared by the problem and its solution, such as an invariant, a theorem condition, a representation, a reduction, or a reformulation. A primitive should be concise, problem-specific, and explanatory: it must make clear what property is being exploited and how that property unlocks the solution. It is not a routine calculation, a piece of generic advice, or a full proof. 2.2

F OUR D IMENSIONS OF M ATHEMATICAL U NDERSTANDING

Taking the mathematical primitive as the fundamental concept of structural understanding, we systematically decompose mathematical reasoning into four complementary dimensions. Let x denote the input problem, p the corresponding mathematical primitive, y a complete proof or solution, and π an LLM solver. The four dimensions are defined as follows: • Discovery (π(x) → p̂) measures whether the model can independently identify the load-bearing mathematical primitive from the input problem alone. • Generation (π(x) → ŷ) evaluates whether the model can solve the problem directly under zeroshot CoT prompting, with performance measured by exact match on the final answer. • Digestion (π(x, y) → p̂) instead provides the model with a complete correct solution and asks it to extract the mathematical primitive underlying the derivation, measuring whether the model can identify the underlying structure once a valid reasoning trajectory is exposed. • Execution (π(x, p) → ŷ) conditions the solver on the mathematical primitive and evaluates whether it can successfully solve the problem once the relevant structure is explicitly provided. Both Discovery and Digestion are evaluated using the scoring protocol introduced in § 2.3. 2.3

T HE P RIM B ENCHMARK

Data Curation. We build on Humanity’s Last Exam (HLE) (Center for AI Safety et al., 2026), focusing on its mathematics, text-only, free-form subset with exactMatch evaluation. To reduce annotation noise in the original benchmark, we draw from the HLE-V ERIFIED release (Zhai et al., 2026), retaining its Gold and Revision subsets, from which we randomly sample 200 problems. For each problem, GPT-5.4-High generates an initial primitive annotation conditioned on the problem, reference answer, and gold rationale, following Definition 1. Three human experts with graduate-level training in mathematics independently review the primitive and verify the validity of the problem, answer, and rationale. This process excludes 18 unsuitable problems, yielding a final evaluation set of 182. Further curation and annotation details are provided in Appendix A.1. Primitive Scoring. We evaluate each predicted primitive against the human-annotated gold primitive, using the reference solution only to recognize mathematically equivalent formulations. To account for different forms of mathematical structure, each problem is assigned to one of three primitive families, including R ECAST, W ITNESS, and A RGUMENT, with full definitions provided in Appendix A.2. Our scoring rubric evaluates three aspects of a predicted primitive: V determines whether the response constitutes a valid primitive, σgate measures whether it identifies the essential mathematical structure, and σmech measures whether it explains how that structure enables the solution. We define
Score = V · σgate · 0.6 + 0.4 · σmech , PrimitiveAcc = 1 [Score ≥ τ ] , with τ = 0.8. This formulation treats validity and structural identification as necessary conditions, while using mechanism quality to refine the score. Further details are provided in Appendix A.3. 3

Preprint

3

D IAGNOSING M ATHEMATICAL R EASONING

In this section, we conduct an empirical study of a diverse set of LLMs, including both openand closed-source models, on P RIM. Moving beyond standard answer correctness, our evaluation systematically disentangles procedural execution from structural understanding. We report model performance across four dimensions: Generation, Discovery, Digestion, and Execution, following the evaluation protocol described in § 2.3. The main results are summarized in Table 1. 3.1

E VALUATION S ETUP

We evaluate 12 models spanning three lineages and a broad capability range. The OpenAI lineage includes the closed-source GPT-5.4 (OpenAI, 2026a), GPT5.4-Mini (OpenAI, 2026b), and GPT-5.4-Nano (OpenAI, 2026b), alongside the open-weight GPT-OSS-20B (OpenAI, 2025). The Qwen lineage comprises Qwen-3.6-27B (Qwen Team, 2026b) and Qwen-3.5-{27B, 9B, 4B} (Qwen Team, 2026a). The DeepSeek lineage features DeepSeek-R1-0528-Qwen38B (DeepSeek-AI, 2025) and DeepSeek-R1-Distill-Qwen{32B, 14B, 7B} (DeepSeek-AI, 2025). We use high reasoning effort when available and a maximum generation budget of 120k tokens per problem. 3.2

Model

Discovery Generation

OpenAI gpt-5.4 82.42 gpt-5.4-mini 59.34 gpt-5.4-nano 41.76 gpt-oss-20B 34.62 Qwen Qwen3.6-27B 24.73 Qwen3.5-27B 28.57 Qwen3.5-9B 13.19 Qwen3.5-4B 6.59 DeepSeek-R1 Distill R1-0528-8B 6.59 R1-Distill-32B 6.04 R1-Distill-14B 4.40 R1-Distill-7B 7.14

Digestion

Execution

Dig. − Disc.

Exec. − Gen.

67.03 50.55 44.51 43.41

100.00 (+17.58) 86.81 (+19.78) 97.80 (+38.46) 71.98 (+21.43) 97.25 (+55.49) 68.68 (+24.17) 91.21 (+56.59) 60.99 (+17.58)

52.75 47.80 38.46 26.37

92.31 (+67.58) 93.41 (+64.84) 79.67 (+66.48) 70.88 (+64.29)

78.57 (+25.82) 71.98 (+24.18) 61.54 (+23.08) 56.04 (+29.67)

17.03 18.13 15.93 13.19

68.68 (+62.09) 65.38 (+59.34) 67.03 (+62.63) 48.90 (+41.76)

39.56 (+22.53) 43.41 (+25.28) 39.01 (+23.08) 32.42 (+19.23)

Table 1: Overall performance on the four dimensions of P RIM. Detailed results are presented in Table 7 in Appendix B.3.

R ESULTS A NALYSIS

Table 1 presents the comprehensive evaluation results of all models on P RIM. Performance in the Discovery and Digestion dimensions is assessed by the model’s capacity to isolate the underlying mathematical primitive, utilizing the scoring protocol detailed in § 2.3. Meanwhile, Generation and Execution are evaluated based on standard final-answer accuracy, following the HLE benchmark (Center for AI Safety et al., 2026). Further details on the diagnostic evaluation and additional analyses are provided in Appendix B.

Figure 2: Joint composition of Generation and Discovery outcomes across models. Finding 1: Answer accuracy masks distinct capability profiles. Final-answer accuracy conflates structural discovery with downstream execution. As shown in Figure 2, Discovery and Generation are positively associated overall, yet models with similar solving accuracy can exhibit sharply different structural capabilities. For example, Qwen3.6-27B and gpt-5.4-mini achieve comparable 4

Preprint

Generation performance, while differing by over 30 percentage points in Discovery. Conversely, GPT-5.4 achieves substantially higher Discovery than Generation. The problem-level decomposition in Figure 2 reveals similarly distinct profiles across model families. The OpenAI lineage is more primitive-forward, with successful Generation frequently accompanied by correct Discovery and a substantial fraction of cases with correct Discovery but failed Generation (D+ G− ). In contrast, Qwen exhibits more cases with failed Discovery but successful Generation (D− G+ ), despite competitive Generation performance. The DeepSeek-R1 distilled models are more heavily concentrated in joint Discovery–Generation failures. Thus, similar top-line accuracy can arise from markedly different underlying capability profiles. Qualitative analysis further clarifies these asymmetries. For gpt-5.4, D+ G− cases primarily reflect two postprimitive failure modes: ❶ technical knowledge gaps, where the correct structure is identified but a required lemma, theorem condition, or domain-specific fact is missing; and ❷ procedural execution failures, such as algebraic errors, missed edge cases, or unresolved logical gaps. For Qwen3.6-27B, among D− G+ cases, the decisive structural idea often emerges after an initial attempt without using it, typically around one quarter into the reasoning trace; in most remaining cases it appears at the outset, and only rarely is it absent entirely. The corresponding cold Discovery output often captures a partial Figure 3: Discovery versus Generation form of the correct idea but misses its decisive component. across all evaluated models on P RIM. Together with the longer reasoning traces in Figure 12, this pattern is consistent with a more grind-first profile, where the relevant structure tends to emerge during extended procedural exploration rather than being isolated before execution. Finding 2: Correct primitives unlock latent execution capacity. To examine whether models possess downstream capabilities that are not realized during direct Generation, we evaluate Execution when the correct Mathematical Primitive is explicitly provided. As shown in Table 1, primitive guidance improves performance by 17.58 to 29.67 percentage points across all 12 models. Notably, under the same primitive-guided setting, the Qwen models achieve Execution performance competitive with the closed-source gpt-5.4 variants despite substantially weaker Discovery. This gap shows that difficulty independently identifying the relevant mathematical structure can coexist with strong downstream problem-solving capability once that structure is available. More broadly, these gains indicate that many Generation failures mask downstream solving capacity that models can successfully exercise when the relevant mathematical structure is supplied. Model

Execution

Generation

Self-Generated Primitive Teacher Plan Teacher Primitive Gold Primitive gpt-5.4-mini gpt-5.4-nano Qwen3.6-27B

50.55 44.51 52.75

48.90 (-1.65) 46.15 (+1.64) 41.21 (-11.54)

60.44 (+9.89) 54.40 (+9.89) 56.04 (+3.29)

63.74 (+13.19) 60.99 (+16.48) 64.84 (+12.09)

71.98 (+21.43) 68.68 (+24.17) 78.57 (+25.82)

Table 2: Generation and Execution performance on P RIM under different forms of reasoning support. Detailed results are presented in Table 8 in Appendix B.3. We further investigate whether this rescue can be attributed simply to task decomposition or generic procedural scaffolding. Table 2 compares four intervention settings. Conditioning the model on its own generated primitive provides little benefit: it yields only a marginal gain for gpt-5.4-nano and substantially degrades Qwen3.6-27B, indicating that a forced discover-then-execute decomposition alone is insufficient. We then compare two forms of external guidance generated by the same teacher model (gpt-5.4) from the problem alone: a step-by-step Teacher Plan, which serves as a procedural hint by specifying the major solution steps, and a Teacher Primitive, which is generated according to the definition of primitive. Although the Teacher Plan improves over direct Generation, the Teacher Primitive yields substantially larger gains across the evaluated models. The Gold Primitive further maximizes this effect, pushing accuracy to 71.98%, 68.68%, and 78.57% for the representative models. This contrast provides crucial empirical validation for our premise in §2.1: a mathematical primitive is fundamentally distinct from, and vastly more effective than, a generic plan or hint. The 5

Preprint

rescue effect therefore cannot be attributed to task decomposition or the procedural scaffold evaluated here alone; exposing the load-bearing mathematical structure provides a substantially stronger signal for downstream reasoning. Finding 3: Independent structural discovery is the dominant bottleneck. As shown in Table 1, models perform substantially better on Digestion, recovering the Mathematical Primitive from a correct solution, than on Discovery, where the same primitive must be identified from the problem alone. This asymmetry is particularly pronounced outside the strongest OpenAI models. Even models with weak Discovery can often recover the organizing primitive once the relevant reasoning is exposed through a completed solution; for instance, Qwen3.6-27B rises from 24.73% Discovery to 92.31% Digestion and R1-Distill-32B from 6.04% to 65.38%. These results reveal a pronounced gap between retrospective recognition and prospective discovery of mathematical structure: models can often recognize the key idea once it is instantiated in a valid solution, yet struggle to identify it independently before the derivation is known. To further localize this bottleneck, we decompose Generation failures according to the joint outcomes of Discovery (D) and Execution (E), as shown in Figure 4. A striking 83.6% of all failures fall into the D− regime, indicating that failed primitive discovery dominates the failure landscape. Importantly, a substantial fraction of these cases are discoverylimited failures (D− E + ): the model fails to identify the correct primitive Figure 4: Decomposition independently, yet succeeds once it is supplied. This reveals that, for of Generation failures by many failures, latent execution capacity is already present but remains Discovery (D) and Exeinaccessible because of the upstream bottleneck in primitive discovery. cution (E). Detailed results can be found in Ta3.3 F URTHER A NALYSIS : W HICH FAILURES A RE R EPAIRABLE ? ble 9 in Appendix B.3. While our preceding diagnosis reveals the failure modes of LLMs during inference, a natural followup question arises: how does post-training impact these mathematical reasoning failures? Of particular interest are discovery-limited failures, which exhibit substantial room for improvement (§3.2). We hypothesize that these failures lie just at the boundary of the models’ reasoning capabilities, making them highly amenable to repair through post-training. Therefore, we utilize Supervised Fine-Tuning (SFT) and On-Policy Self-Distillation (OPSD) (Zhao et al., 2026) to investigate how different failure modes uniquely respond to these post-training paradigms. Setup. Our training corpus contains 709 Mathematics Ph.D. qualifying-examination problems from 1991–2026, each paired with a human-authored proof (median length: 118 words); additional details are provided in Appendix C. We use qualifying-examination problems because their open-ended, proof-oriented nature closely matches the difficulty and structural reasoning demands of P RIM. Since such problems generally lack algorithmically verifiable short answers, we focus on text-supervised post-training rather than outcome-verified methods such as RLVR. We evaluate SFT and OPSD on Qwen-3.5-27B, 9B, 4B (Qwen Team, 2026a), using the default OPSD configuration, bfloat16 precision, a fixed seed of 42, and NVIDIA RTX 6000 Ada GPUs. On-Policy Self-Distillation. Given training examples {x, y}, where x denotes the input problem and y its reference solution, both the teacher and student policies are initialized from the same base model π0 . At each decoding step t, the student autoregressively generates an on-policy response, inducing the token distribution Pt := πS (· | ŷ<t , x), while the teacher distribution Qt := πT (· | ŷ<t , x, y) additionally conditions on the reference solution. Notably, both policies are conditioned on the same student prefix ŷ<t , allowing the teacher to provide token-level supervision along the student’s own reasoning trajectory. OPSD minimizes the divergence between these distributions: LOPSD =

T T 1X 1X D(Pt , Qt ) = D (πS (· | ŷ<t , x), πT (· | ŷ<t , x, y)) , T t=1 T t=1

where D is instantiated as Kullback–Leibler (KL), reverse KL (rKL), or Jensen–Shannon (JSD). Specifically, KL uses DKL (Qt ∥ Pt ), rKL uses DKL (Pt ∥ Qt ), and JSD is symmetric in Pt and Qt . Results. Across the 27B, 9B, and 4B models, the base models leave 341 failure instances in P RIM. We focus on the 294 cases with failed Discovery: 147 D− E + and 147 D− E − cases. We 6

Preprint

define repair as an incorrect baseline Generation response becoming correct after post-training. As shown in Table 3, SFT and OPSD repair 20.4% and 21.1% of D− E + cases, roughly three times their respective rates of 6.8% and 8.2% for D− E − cases. Failure type SFT OPSD This ordering holds at each model scale, identifying discovery− + D E 20.4% 21.1% limited failures as a promising target for post-training. However, D− E − 6.8% 8.2% net gains in Generation remain modest (as shown in Table 4) because some initially correct cases become incorrect, offsetting Table 3: Post-training repair rates part of these repairs. These regressions are consistent with post- of SFT and OPSD across initial training drift and suggest that supervision from reference proofs P RIM failure quadrants. should be transferred more selectively.

4

I NTERNALIZING P RIMITIVE -G UIDED R EASONING

In this section, we introduce A BSORB, a novel variant of on-policy self-distillation that improves the mathematical problem-solving capability of LLMs by internalizing primitive-guided reasoning, motivated by the observations in §3. Extensive experiments with Qwen models as backbones show that A BSORB consistently outperforms strong baselines on a diverse set of mathematical reasoning benchmarks (§4.4). Further implementation details and ablation results are provided in Appendix E. 4.1

P RIMITIVES AS P RIVILEGED I NFORMATION

Our diagnosis in §3 identifies missing mathematical Kept Capped at π primitives as the primary Teacher Problem bottleneck in discoverŷ , x, p) τT ( ⋅ | y<t x limited problems. A natPrimitive ural remedy is to directly token distribution supervise primitive generPrefix Teacher Policy Student Policy ation. In practice, howStudent ̂y<t Absorb keeps the teacher's pull toward better choices, ever, this objective transfers ̂ , x) τT ( ⋅ | y<t and caps how strongly it can overturn the student's own. poorly to solution reasoning, likely reflecting a misFigure 5: Illustration of A BSORB paradigm. match between structural understanding and step-by-step generation, as further supported by the ablations in Appendix E. More importantly, §3.3 shows that post-training can repair discovery-limited failures while leaving Discovery largely unchanged. This dissociation suggests that effective reasoning does not require primitives to be explicitly generated; instead, they can guide the reasoning trajectory implicitly. Primitive Privileged

Bounded Override

Motivated by this observation, we provide mathematical primitives as privileged information to the teacher rather than as prediction targets for the student. This mirrors our Execution evaluation, where providing the primitive unlocks latent execution capability, and uses the same primitive-conditioned reasoning to guide post-training. Compared with full reference proofs, primitives provide more targeted supervision by supplying the missing structural information without prescribing execution steps the student can already perform. This design transfers primitive-guided reasoning into the student’s own generation process without requiring primitives at inference time. 4.2

B OUNDED OVERRIDE

Given the reasoning drift observed with existing post-training methods, indiscriminate teacher intervention should be avoided. Moreover, introducing mathematical primitives as privileged information can cause substantial privilege leakage under naive OPSD on complex reasoning problems, as shown in Appendix E. This motivates a selective transfer mechanism that preserves useful privileged guidance while limiting how strongly it can override the student’s own reasoning. Intuitively, the teacher should pull the student toward better choices without abruptly displacing preferences that the student has already formed. The primitive should refine the student’s existing reasoning space rather than force it to imitate decisions that are only accessible under privileged context. This leads to two design principles: ❶ when the privileged teacher assigns higher preference to a choice than the student, this positive guidance should be transferred in full, as it directly reflects 7

Preprint

P RIM

HLE Math

HMMT25

Omni-MATH Avg.

Model

Discovery

Generation

Qwen3.5-4B + SFT + OPSD + A BSORB

6.59 5.49 (-1.10) 8.79 (+2.20) 9.89 (+3.30)

26.37 26.70 76.67 25.27 (-1.10) 27.36 (+0.65) 83.33 (+6.66) 30.22 (+3.85) 29.58 (+2.88) 83.33 (+6.66) 31.32 (+4.95) 31.02 (+4.32) 83.33 (+6.66)

78.00 79.33 (+1.33) 79.33 (+1.33) 80.00 (+2.00)

51.94 53.82 (+1.89) 55.62 (+3.68) 56.42 (+4.48)

Qwen3.5-9B + SFT + OPSD + A BSORB

13.19 38.46 35.60 90.00 12.64 (-0.55) 32.42 (-6.04) 34.55 (-1.05) 90.00 (+0.00) 10.99 (-2.20) 33.52 (-4.95) 35.08 (-0.52) 90.00 (+0.00) 14.29 (+1.10) 43.96 (+5.49) 38.35 (+2.75) 93.33 (+3.33)

78.67 80.67 (+2.00) 81.33 (+2.67) 82.00 (+3.33)

60.68 59.41 (-1.27) 59.98 (-0.70) 64.41 (+3.73)

Qwen3.5-27B 28.57 47.80 42.67 96.67 87.33 + SFT 28.02 (-0.55) 48.35 (+0.55) 43.59 (+0.92) 96.67 (+0.00) 86.67 (-0.67) + OPSD 28.57 (+0.00) 44.51 (-3.30) 42.80 (+0.13) 96.67 (+0.00) 88.00 (+0.67) + A BSORB 28.02 (-0.55) 48.90 (+1.10) 46.60 (+3.93) 100.00 (+3.33) 88.67 (+1.33)

68.62 68.82 (+0.20) 67.99 (-0.62) 71.04 (+2.42)

Table 4: Comparison of SFT, OPSD, and A BSORB across three Qwen3.5 model scales. information contributed by the primitive; and ❷ when the teacher strongly suppresses a choice favored by the student, the resulting pressure should be bounded, since large disagreement may arise from privileged information unavailable to the student rather than from an error it can independently recognize. Importantly, bounding this negative pressure does not freeze incorrect student preferences: positive guidance toward better alternatives remains fully active, progressively shifting probability mass away from suboptimal choices over training. These desiderata admit a simple implementation through reverse KL with one-sided clamping, which preserves positive teacher guidance while bounding excessive suppression of student-preferred choices. 4.3

OVERALL O BJECTIVE

Combining the two design principles above, we introduce A BSORB, illustrated in Figure 5. A BSORB conditions the teacher on the mathematical primitive p to provide targeted guidance for the missing discovery signal, while bounding how strongly this can override the student’s own reasoning. Let St denote the teacher’s top-K token support at step t, and let π̄S and π̄T denote the student and privileged-teacher distributions renormalized over St , respectively. The resulting objective is:   T 1XX π̄S (v | ŷ<t , x) LA BSORB = ,τ . min π̄S (v | ŷ<t , x) log T t=1 π̄T (v | ŷ<t , x, p) v∈St

Here, the privileged teacher injects primitive-guided information along the student’s on-policy reasoning trajectory, while the one-sided clamp preserves its positive guidance toward better alternatives and bounds excessive suppression of choices favored by the student. 4.4

E XPERIMENTAL R ESULTS

Setup. For implementing A BSORB, we utilize the identical base models and optimization configurations detailed in §3.3, differing only in the substitution of the standard OPSD objective with our primitive-privileged divergence loss (LABSORB ). To comprehensively evaluate internalized reasoning capabilities and generalization, we assess performance across four challenging mathematical benchmarks: our curated P RIM (evaluating both Generation and Discovery), the full mathematics subset of HLE-Verified (Zhai et al., 2026), HMMT25 (Dekoninck et al., 2026), and Omni-MATH (Gao et al., 2025). To account for the extreme difficulty of the competition-level datasets, we report pass@4 accuracy for HMMT25 and Omni-MATH. The overall average (Avg.) is computed exclusively over the Generation scores across all four benchmarks, excluding Discovery. Results. As shown in Table 4, A BSORB consistently outperforms SFT and OPSD across model scales and benchmarks. At the 9B scale, SFT and OPSD reduce Generation accuracy by 6.04 and 4.95 percentage points, respectively, while OPSD incurs a 3.30-point drop at the 27B scale. These regressions explain the modest net gains discussed in §3.3: repairing previously failed problems can be offset 8

Preprint

by newly introduced errors on problems the model already solves. In contrast, A BSORB improves Generation at all three model scales, including a 5.49 point gain at 9B and a 3.73 point improvement in average benchmark performance. This pattern is consistent with the role of bounded override: selective transfer retains useful primitive-guided information while limiting disruptive changes to the model’s existing reasoning behavior. The results also support the intended internalization of primitive-guided reasoning. Improvements from A BSORB are concentrated in downstream reasoning rather than explicit Discovery. At the 9B scale, Generation improves by 5.49 points while Discovery increases by only 1.10; at 27B, Generation improves by 1.10 points and HLE Math by 3.93, despite a 0.55-point decrease in Discovery. Thus, the benefit of the primitive need not manifest as better explicit primitive recovery; instead, its structural guidance is increasingly reflected in the model’s unassisted reasoning process.

5

R ELATED W ORK

Mathematical Reasoning in LLMs. Mathematical reasoning benchmarks have rapidly escalated in difficulty, progressing from early grade-school and competition datasets (e.g., MathQA (Amini et al., 2019), ASDiv (Miao et al., 2020), SVAMP (Patel et al., 2021), GSM8K (Cobbe et al., 2021), MATH (Hendrycks et al., 2021)) to advanced Olympiad- and expert-level frontiers (TheoremQA (Chen et al., 2023), MathBench (Liu et al., 2024), AIME (Dekoninck et al., 2026), OlympiadBench (He et al., 2024), Omni-MATH (Gao et al., 2025), HLE (Center for AI Safety et al., 2026), FrontierMath (Glazer et al., 2024)). Despite this scaling in complexity, evaluation predominantly remains rigidly anchored to final-answer correctness. Recognizing the limitations of outcome-based metrics, recent literature explores process-oriented evaluation: SMART (Hou et al., 2026) decomposes problem-solving across cognitive dimensions, ReasoningMath-Plus (Zheng et al., 2026) verifies minimal procedural skeletons, and DeepInsightTheorem (Li et al., 2026a) leverages proof sketches for intermediate supervision. Executable Functional Abstractions (EFAs) (Khan et al., 2025) represent mathematical structure as executable generative programs, whereas our Mathematical Primitive captures the concise, non-procedural idea underlying a specific solution. Our approach instead isolates the load-bearing mathematical primitive, decoupling structural understanding from procedural execution. We evaluate whether models can discover it from a problem, recover it from a solution, and execute the proof when it is provided. On-Policy Self-Distillation. On-Policy Self-Distillation (OPSD) advances knowledge distillation for LLM reasoning by mitigating the distribution shift inherent in traditional off-policy methods (Zhao et al., 2026; He et al., 2026; Penaloza et al., 2026). Specifically, the student generates its own reasoning trajectories on-policy, while the teacher, conditioned on additional privileged information, provides dense supervision along these rollouts, with both sharing the same base model. However, recent studies show that naive OPSD can suffer from privilege leakage and suppress exploratory or selfcorrective reasoning. To address this, Shen et al. (2026) isolate inference-transferable supervision from reference-induced shortcuts, while Kaur et al. (2026); Peng et al. (2026) focus on preserving high-entropy decision points and epistemic behaviors from aggressive privileged-teacher corrections. Other approaches modulate supervision based on teacher–student disagreement (Li et al., 2026b; Yu et al., 2026; Xie et al., 2026), or utilize RL primarily for token-level credit assignment (Yang et al., 2026). AVSD mitigates privilege leakage by separating shared and view-specific signals across multiple privileged views (Nguyen et al., 2026). A BSORB differs primarily in the privileged signal itself. Rather than conditioning the teacher on a full solution, A BSORB uses the primitive as privileged information and selectively transfers its guidance through a bounded override mechanism.

6

C ONCLUSION

In this paper, we introduced the notion of Mathematical Primitive and P RIM, a benchmark for systematically evaluating structural mathematical understanding in LLMs across Discovery, Generation, Digestion, and Execution. Our diagnosis reveals that similar solution accuracy can mask markedly different capability profiles, correct primitives can unlock substantial latent execution capacity, and independent Discovery constitutes a dominant bottleneck in mathematical reasoning. We further show that discovery-limited failures are substantially more amenable to post-training, while existing methods can introduce regressions on problems the model already solves. Building on these findings, we 9

Preprint

proposed A BSORB, a primitive-privileged post-training paradigm that selectively transfers primitiveguided reasoning through a bounded override mechanism. Experiments on challenging mathematical reasoning benchmarks show that A BSORB consistently improves over strong post-training baselines. Overall, our results highlight the value of explicitly diagnosing structural mathematical understanding and leveraging this structure to improve LLM reasoning. By systematically investigating mathematical understanding in LLMs, we hope this work encourages a broader shift in AI for Science from optimizing benchmark performance alone toward identifying the capabilities required for scientific problem solving and discovery. Given the foundational role of mathematics across STEM disciplines, this structural perspective may extend beyond mathematics. We highlight three promising directions. First, our diagnostic framework could be generalized to other structurally rigorous domains, such as theoretical physics and computational chemistry. Second, the observed discovery-execution asymmetry suggests a potential design principle for mathematical agents: explicitly separating structural search from procedural execution may enable more effective allocation of inference-time computation. Finally, future work could explore integrating mathematical primitives with formal verification systems such as Lean, connecting informal structural reasoning with formal proof generation.

R EFERENCES Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. Mathqa: Towards interpretable math word problem solving with operation-based formalisms. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 2357–2367, 2019. Johan Boye and Birger Moell. Large language models and mathematical reasoning failures. arXiv preprint arXiv:2502.11574, 2025. Sébastien Bubeck, Christian Coester, Ronen Eldan, Timothy Gowers, Yin Tat Lee, Alexandru Lupsasca, Mehtaab Sawhney, Robert Scherrer, Mark Sellke, Brian K. Spears, Derya Unutmaz, Kevin Weil, Steven Yin, and Nikita Zhivotovskiy. Early science acceleration experiments with GPT5. arXiv preprint arXiv:2511.16072, 2025. URL https://arxiv.org/abs/2511.16072. Center for AI Safety, Scale AI, and HLE Contributors Consortium. A benchmark of expert-level academic questions to assess AI capabilities. Nature, 649:1139–1146, 2026. doi:10.1038/s41586025-09962-4. URL https://arxiv.org/abs/2501.14249. Wenhu Chen, Ming Yin, Max Ku, Pan Lu, Yixin Wan, Xueguang Ma, Jianyu Xu, Xinyi Wang, and Tony Xia. Theoremqa: A theorem-driven question answering dataset. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 7889–7901, 2023. Claude. More than two thirds of the zeros of the riemann zeta function are simple and on the critical line. Anthropic manuscript, 2026. URL https://www-cdn.anthropic.com/ 95c246936988e43127bc6b2ceb7077c1dad2d68e.pdf. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948. Jasper Dekoninck, Nikola Jovanović, Tim Gehrunger, Kári Rögnvaldsson, Ivo Petrov, Chenhao Sun, and Martin Vechev. Beyond benchmarks: Matharena as an evaluation platform for mathematics with llms. 2026. URL https://arxiv.org/abs/2605.00674. Simon Frieder, Jonas Bayer, Sam Looi, Jacob Loader, Julius Berner, Katherine M Collins, András Juhász, Fabian Ruehle, Sean Welleck, Gabriel Poesia, et al. Data for mathematical copilots: Better ways of presenting proofs for machine learning. arXiv preprint arXiv:2412.15184, 2024. 10

Preprint

Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Zhengyang Tang, et al. Omni-math: A universal olympiad level mathematic benchmark for large language models. In International Conference on Learning Representations, volume 2025, pp. 100540–100569, 2025. Elliot Glazer, Ege Erdil, Tamay Besiroglu, Diego Chicharro, Evan Chen, Alex Gunning, Caroline Falkman Olsson, Jean-Stanislas Denain, Anson Ho, Emily de Oliveira Santos, et al. Frontiermath: A benchmark for evaluating advanced mathematical reasoning in ai. arXiv preprint arXiv:2411.04872, 2024. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3828–3850, 2024. Yinghui He, Simran Kaur, Adithya Bhaskar, Yongjin Yang, Jiarui Liu, Narutatsu Ri, Liam Fowl, Abhishek Panigrahi, Danqi Chen, and Sanjeev Arora. Self-distillation zero: Self-revision turns binary rewards into dense supervision. arXiv preprint arXiv:2604.12002, 2026. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. Yujie Hou, Mei Wang, Yaoyao Zhong, Ting Zhang, Xuetao Ma, and Hua Huang. SMART: Evaluating LLMs’ mathematical reasoning via a human cognitive process-inspired benchmark. In Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens (eds.), Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 35426–35452, San Diego, California, United States, July 2026. Association for Computational Linguistics. ISBN 979-8-89176-390-6. doi:10.18653/v1/2026.acl-long.1638. URL https: //aclanthology.org/2026.acl-long.1638/. Uijeong Jang and Ernest K. Ryu. Point convergence of nesterov’s accelerated gradient method: An ai-assisted proof. arXiv preprint arXiv:2510.23513, 2025. URL https://arxiv.org/abs/ 2510.23513. Simran Kaur, Narutatsu Ri, Yinghui He, Liam Fowl, and Sanjeev Arora. Rethinking on-policy self-distillation for thinking models. arXiv preprint arXiv:2607.05184, 2026. Zaid Khan, Elias Stengel-Eskin, Archiki Prasad, Jaemin Cho, and Mohit Bansal. Executable functional abstractions: Inferring generative programs for advanced math problems. arXiv preprint arXiv:2504.09763, 2025. Yunhe Li, Hao Shi, Bowen Deng, Wei Wang, Mengzhe Ruan, Hanxu Hou, Zhongxiang Dai, Siyang Gao, Chao Wang, Shuang Qiu, et al. Learning to reason with insight for informal theorem proving. arXiv preprint arXiv:2604.16278, 2026a. Yunhe Li, Hao Shi, Wenhao Liu, Mengzhe Ruan, Hanxu Hou, Zhongxiang Dai, Shuang Qiu, and Linqi Song. Demopsd: Disagreement-modulated policy self-distillation. arXiv preprint arXiv:2607.02502, 2026b. Hongwei Liu, Zilong Zheng, Yuxuan Qiao, Haodong Duan, Zhiwei Fei, Fengzhe Zhou, Wenwei Zhang, Songyang Zhang, Dahua Lin, and Kai Chen. Mathbench: Evaluating the theory and application proficiency of llms with a hierarchical mathematics benchmark. In Findings of the Association for Computational Linguistics: ACL 2024, pp. 6884–6915, 2024. Minghai Lu, Benjamin Delaware, and Tianyi Zhang. Proof automation with large language models. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, pp. 1509–1520, 2024. Shen-yun Miao, Chao-Chun Liang, and Keh-Yih Su. A diverse corpus for evaluating and developing English math word problem solvers. In Dan Jurafsky, Joyce Chai, Natalie Schluter, 11

Preprint

and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 975–984, Online, July 2020. Association for Computational Linguistics. doi:10.18653/v1/2020.acl-main.92. URL https://aclanthology.org/2020. acl-main.92/. Duy Nguyen, Hanqi Xiao, Archiki Prasad, Zaid Khan, Anirban Das, Austin Zhang, Sambit Sahu, Hyunji Lee, Elias Stengel-Eskin, and Mohit Bansal. Avsd: Adaptive-view self-distillation by balancing consensus and teacher-specific privileged signals. arXiv preprint arXiv:2605.20643, 2026. OpenAI. gpt-oss-120b & gpt-oss-20b model card, 2025. URL https://arxiv.org/abs/ 2508.10925. OpenAI. Introducing gpt-5.4, March 2026a. introducing-gpt-5-4/.

URL https://openai.com/index/

OpenAI. Introducing gpt-5.4 mini and nano, March 2026b. URL https://openai.com/ index/introducing-gpt-5-4-mini-and-nano/. OpenAI. Ten advances in mathematics and theoretical computer science. https://openai. com/index/ten-advances-in-mathematics/, 2026a. Astra Model Research Release and Lean 4 Formalizations. OpenAI. Planar point sets with many unit distances. Manuscript, 2026b. URL https://cdn.openai.com/pdf/74c24085-19b0-4534-9c90-465b8e29ad73/ unit-distance-proof.pdf. Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are nlp models really able to solve simple math word problems? In Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies, pp. 2080–2094, 2021. Emiliano Penaloza, Dheeraj Vattikonda, Nicolas Gontier, Alexandre Lacoste, Laurent Charlin, and Massimo Caccia. Privileged information distillation for language models. arXiv preprint arXiv:2602.04942, 2026. Keqin Peng, Chen Li, Yuanxin Ouyang, Yancheng Yuan, and Liang Ding. Diagnosing and mitigating thinking collapse in on-policy self-distillation. arXiv preprint arXiv:2607.10805, 2026. Tian Qin, Core Francisco Park, Mujin Kwun, Aaron Walsman, Eran Malach, Nikhil Anand, Hidenori Tanaka, and David Alvarez-Melis. Decomposing elements of problem solving: What" math" does rl teach? arXiv preprint arXiv:2505.22756, 2025. Qwen Team. Qwen3.5: Accelerating productivity with native multimodal agents, February 2026a. URL https://qwen.ai/blog?id=qwen3.5. Qwen Team. Qwen3.6-27B: Flagship-level coding in a 27b dense model, April 2026b. URL https://qwen.ai/blog?id=qwen3.6-27b. Zhanming Shen, Jintao Tong, Shaotian Yan, Chen Shen, Hao Chen, Wentao Ye, Xiaomeng Hu, Rui Miao, Haobo Wang, Junbo Zhao, et al. Purified opsd: On-policy self-distillation without losing how to think. arXiv preprint arXiv:2607.02234, 2026. Terence Tao. Mathematics in the age of ai. arXiv preprint arXiv:2608.16753, 2026. Zhengpeng Xie, Li Lyna Zhang, Zeke Xie, and Mao Yang. Trust region policy distillation. arXiv preprint arXiv:2607.04751, 2026. Chenxu Yang, Chuanyu Qin, Qingyi Si, Minghui Chen, Naibin Gu, Dingyu Yao, Zheng Lin, Weiping Wang, Jiaqi Wang, and Nan Duan. Self-distilled rlvr. arXiv preprint arXiv:2604.03128, 2026. Zichao Yu, Chengzhi Yu, Shengze Xu, Yujin Han, Bingqing Jiang, Xu Wang, and Difan Zou. Mismatch matters: On-policy distillation beyond token agreement. arXiv preprint arXiv:2608.09836, 2026. 12

Preprint

Weiqi Zhai, Zhihai Wang, Jinghang Wang, Boyu Yang, Xiaogang Li, Xander Xu, Bohan Wang, Peng Wang, Xingzhe Wu, Anfeng Li, et al. Hle-verified: A systematic verification and structured revision of humanity’s last exam. arXiv preprint arXiv:2602.13964, 2026. Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-distilled reasoner: On-policy self-distillation for large language models. arXiv preprint arXiv:2601.18734, 2026. Xiang Zheng, Weiqi Zhai, Wei Wang, Boyu Yang, Wenbo Li, Ruixiang Luo, Haoxiang Sun, Yucheng Wang, Zhengze Li, Meng Wang, et al. Unmasking reasoning processes: A process-aware benchmark for evaluating structural mathematical reasoning in llms. arXiv preprint arXiv:2602.00564, 2026.

13

Preprint

A

DATA C URATION D ETAILS

A.1

DATASET F ILTERING AND C ORRECTION

To reduce annotation noise in the original HLE, we draw from the community-verified HLEV ERIFIED release (Zhai et al., 2026), retaining only its Gold and Revision subsets. We randomly sample 200 text-only problems and conduct an additional expert review for primitive-level evaluation. This process excludes 18 unsuitable problems due to ill-posed or ambiguous questions, incorrect or unverifiable answers, non-self-contained tasks, insufficient rationales, exhaustive-computation requirements, or other quality issues. For 17 of these 18 problems, no valid gold primitive could be established, yielding a final benchmark of 182 problems. Expert review further identifies residual errors in the official answers of 15 retained problems, including 13 mathematical errors and 2 corrupted answer strings, which we correct in our benchmark. Re-evaluating the strongest model on these corrected answers flips 11 predictions from incorrect to correct, with no reverse flips, providing an additional consistency check on the revisions. Because gold primitives are derived with access to reference solutions, we also audit them for answer leakage. We search the concatenated primitive fields for bounded occurrences of the gold answer, using token boundaries designed to avoid spurious numeral matches. This audit initially identifies 4 affected primitives. We manually revise them to remove the leaked answers while preserving their structural content, and verify that no direct answer-string leakage remains in the final benchmark. A.2

P RIMITIVE T YPE AND FAMILY

To make the definition operational and to structure evaluation, we anchor every primitive to a frozen taxonomy of ten primitive types (plus a computation_only escape hatch for problems that genuinely have no kernel primitive in the sense of Definition 1), as presented in Table 5. Type

The essential property is. . .

reformulation

a change of language/encoding/viewpoint makes the problem tractable the objects secretly carry a known structure (group, graph, metric space, . . . ) a preserved or monotone quantity constrains all transformations the task reduces to verifying the hypotheses of a known theorem a dual / spectral / transformed viewpoint simplifies the problem a maximal/minimal/extreme object forces structure an explicit object/certificate/counterexample is built the problem is reduced/embedded into a known canonical one the opposite assumption violates a necessary constraint a recursive/self-similar structure with a strengthened hypothesis

hidden_structure invariant_or_monotonicity theorem_applicability duality_or_representation extremal_principle construction_or_witness reduction_or_embedding obstruction_or_contradiction induction_or_recursion

Table 5: The ten primitive types. Further, since the same problem can often be solved through conceptually different routes, we further assign the primary primitive type as presented in Table 6 of each problem to one of three families: R ECAST, W ITNESS, or A RGUMENT, as summarized below: • R ECAST: The load-bearing object is the target setting Y into which the original problem is moved, such as an equivalent reformulation, a hidden known structure, a canonical target problem, a dual or representational viewpoint, or a named theorem. The gate asks whether the model recasts the problem into the same Y , or a provably equivalent one up to trivial restatement. The mechanism asks how this target setting unlocks the solution, such as the reduction step actually used, or, in theorem-based cases, whether the theorem’s hypotheses are correctly verified. Moving to a different setting, naming the wrong theorem, or remaining entirely within the original framing constitutes a gate mismatch; identifying a coarser or weaker target receives partial credit. • W ITNESS: The load-bearing object is a specific mathematical object: a construction, certificate, witness, extremal object, invariant, or monotone quantity. The gate asks whether the model identifies the same object, or an equivalent object that plays the identical role in the solution. The 14

Preprint

Family

Routed primitive types

Gate (WHAT)

Mechanism (HOW)

R ECAST

reformulation, hidden_structure, reduction_or_embedding, duality_or_representation, theorem_applicability

target setting Y (equiva- how Y unlocks it; for lent form / hidden struc- theorems, hypotheses ture / canonical target / verified dual / named theorem)

W ITNESS

construction_or_witness, extremal_principle, invariant_or_monotonicity

the object (construction its decisive property: / certificate / extremal / built to satisfy / exinvariant–monotone) tremality / invariance forces it

A RGUMENT obstruction_or_contradiction, the load-bearing claim why the opposite fails / induction_or_recursion (specific obstruction / how the recursion closes strengthened hypothesis)

Table 6: The primitive families. Each of the ten frozen primitive types routes to exactly one family.

mechanism asks for its decisive property: for example, why the construction satisfies the required constraint, why extremality forces the conclusion, or why invariance forbids or forces the desired outcome. An object of the correct general kind but without the decisive feature receives partial credit, such as a near-construction that fails the needed property or a monotone quantity that is not the one driving the proof. • A RGUMENT: The load-bearing object is the central claim on which the proof depends, such as the obstruction contradicted by the opposite assumption or the strengthened induction hypothesis needed for the recursive step. The gate asks whether the model’s reasoning hinges on the same claim; merely naming the proof style, such as “by contradiction” or “by induction”, is not sufficient. The mechanism asks for the logical step itself, such as why the opposite assumption violates the constraint or how the strengthened hypothesis closes the induction. Stating only the proof style without the specific load-bearing claim is a gate mismatch, while a related but loosely stated or under-strengthened claim receives partial credit. A.3

P RIMITIVE S CORING P ROTOCOL

Mathematical primitive correctness is evaluated against the annotated gold primitive, with the problem’s full reference solution (rationale) supplied as supporting context. The rationale is provided only so the judge can recognize coarser phrasings, equivalent paraphrases, or alternative routes that the reference solution shows to be valid, i.e. so a correct primitive is not penalized merely for failing to match the terse gold annotation verbatim, while never requiring the model to reproduce the downstream execution. All three primitive families share the same three-verdict decomposition (V, σgate , σmech ) and the deterministic scoring rule in § 2.3. Here, V ∈ {0, 1} is a binary validity indicator (is the output a genuine primitive rather than a generic remark, a bare technique name, or pure computation), while the two graded slots σgate , σmech ∈ {0, 12 , 1} evaluate whether the response identifies the essential WHAT (the gate) and explains the operative HOW (the mechanism). The difference across families is the type of mathematical object over which these two slots are instantiated, ensuring that “the same idea” is matched against the appropriate target. Because V is binary and σgate , σmech ∈ {0, 12 , 1}, the score lies in [0, 1] but is restricted to the finite set, and we have Score ∈ {0, 0.30, 0.40, 0.50, 0.60, 0.80, 1.00}. All primitive judgments are produced by GPT-5.4-High at temperature 0. To validate the judge, we sampled 200 (model, problem) pairs, 100 judged primitive-correct and 100 judged primitive-incorrect, stratified across the 12 evaluated models. Three annotators with graduate-level mathematical training, blind to the judge’s verdict, independently labeled PrimitiveAcc. Judge–human agreement is 95.5% (Cohen’s κ = 0.91), with 9 disagreements. 15

Preprint

Because each problem is scored against a single gold primitive, a response following a genuinely different but valid conceptual route may be under-credited. To assess this limitation, we further examined the 100 primitive-incorrect cases in the same validation sample (8–9 per model, fixed seed): exactly 1 case (1%) contained a valid alternative primitive missed by the single-gold rubric. We additionally audited all 32 problems marked primitive-incorrect for the strongest model (gpt-5.4, Score < 0.8), identifying exactly 1 such case. These audits suggest that, although the single-gold rubric may conservatively underestimate valid primitive generation, under-counting due to alternative conceptual routes is rare in the audited cases. A.4

I LLUSTRATIVE E XAMPLE OF M ATHEMATICAL P RIMITIVES

To make the notion of a mathematical primitive concrete, we provide a representative example in Figure 6. The full rationale contains the complete derivation needed to solve the problem, whereas the primitive isolates the underlying structural idea that makes the derivation possible. Due to space constraints, we show only the core_concept field in the figure. This example illustrates that a primitive is not a shortened proof, but a compact representation of the key mathematical structure underlying the solution. A.5

P ROMPT ON E XTRACTING A M ATHEMATICAL P RIMITIVE AND P RIMITIVE S CORING

During data curation, we use a teacher model to extract a mathematical primitive for each problem. As shown in Figure 7, the teacher is asked to identify the essential mathematical property underlying the problem and explain the solution principle enabled by that property, while explicitly avoiding calculations or solution steps. We additionally require a structured primitive classification, including a primary type, optional secondary types, a classification rationale, and a confidence score. We evaluate each predicted primitive using three complementary signals. First, the validity term V checks whether the prediction contains a substantive mathematical primitive rather than a vacuous response, such as generic advice, a bare technique name, or computation without a conceptual claim (Figure 8). For predictions that pass this check, we separately assess the correctness of the GATE and MECHANISM components using the reference solution and gold primitive (Figure 9). The GATE score σgate measures whether the prediction identifies the essential problem-specific idea, the “what” that unlocks the problem, while the MECHANISM score σmech measures whether it correctly explains the high-level “how,” namely why that idea enables a valid solution principle. Both correctness judgments emphasize semantic agreement rather than exact reproduction and do not penalize missing downstream execution details.

B

A DDITIONAL D ETAILS ON D IAGNOSTIC E VALUATION

B.1

C OMPARISON WITH F ULL -HLE P ERFORMANCE

Notably, the absolute answer accuracies in Table 7 are substantially higher than the commonly reported performance of these model families on the full HLE benchmark. This difference should not be interpreted as a contradiction, for two reasons: ❶ P RIM is a curated scorable subset rather than the full HLE distribution; and ❷ during benchmark construction, human experts removed examples with annotation noise or evaluation ambiguity in the original HLE (Center for AI Safety et al., 2026) and HLE-V ERIFIED (Zhai et al., 2026) sources, including fundamentally incorrect reference answers, logical gaps in rationales, and irregular formatting that hinders automated evaluation. B.2

P ROMPT ON G ENERATING THE P RIMITIVE AND P LAN USING GPT-5.4

We use GPT-5.4 to generate two forms of teacher guidance for the Execution analysis: a Teacher Primitive, which captures the load-bearing mathematical structure following Definition 1, and a Teacher Plan, which provides a step-by-step solution outline. The prompts used to generate these two forms of guidance are provided below in Figures 10 and 11.

16

Preprint

Question Let q L := Q

(2 +

√

2)(3 +

√

3),

√

2,

√

 3 .

What is the Galois group of L/Q?

Rationale Set α =

√

2, β =

√

3, and γ=

q

(2 + α)(3 + β).

Let K = Q(α, β). Then K/Q is biquadratic, hence Galois with [K : Q] = 4 and Gal(K/Q) ∼ = C2 × C2 . Step 1: [L : K] = 2 and [L : Q] = 8 We have γ 2 = (2 + α)(3 + β) ∈ K, so L = K(γ) and [L : K] ≤ 2. To see γ ∈ / K, it suffices to show a := (2 + α)(3 + β) is not a square in K. Consider the relative norm NK/Q(α) . The nontrivial automorphism of K/Q(α) sends β 7→ −β, hence τ

2

2

2

NK/Q(α) (a) = a · a = (2 + α)(3 + β) · (2 + α)(3 − β) = (2 + α) (9 − β ) = 6(2 + α) . 2

If a were a square in K, then its norm down to Q(α) √ would be a square in Q(α). But 6(2 + α) is a square in Q(α) if and only if 6 is a square in Q(α). Now 6 is not a square in Q( 2): if (u + vα)2 = 6 with u, v ∈ Q, then the α-coefficient gives 2uv = 0, so 2 either v = 0 (forcing u = 6, impossible) or u = 0 (forcing 2v 2 = 6, i.e. v 2 = 3, impossible in Q). Hence a is not a square in K, so [L : K] = 2 and therefore [L : Q] = 8. Step 2: Explicit automorphisms and normality Define the involution ρ : α 7→ α, β 7→ β, γ 7→ −γ. This generates Gal(L/K) ∼ = C2 . Next extend the two generators of Gal(K/Q). Let σ0 be the automorphism of K with α 7→ −α, β 7→ β. Note that 2

σ0 (γ ) = (2 − α)(3 + β). But 2−α (2 − α)(3 + β) 2 = = 3 − 2α = (α − 1) , (2 + α)(3 + β) 2+α so σ0 (γ 2 ) = (α − 1)2 γ 2 . Hence we can define an extension σ ∈ Aut(L) by σ : α 7→ −α, β 7→ β, γ 7→ (α − 1)γ, since then σ(γ)2 = (α − 1)2 γ 2 = σ0 (γ 2 ). Similarly let τ0 be the automorphism of K with α 7→ α, β 7→ −β. Then   (2 + α)(3 − β) 3−β β−1 2 = =2−β = (2 + α)(3 + β) 3+β α  2 2 (using (β − 1)2 = 4 − 2β and α2 = 2). Thus τ0 (γ 2 ) = β−1 γ , and we define τ ∈ Aut(L) by α 2

τ0 (γ ) = (2 + α)(3 − β),

and

β−1 γ. α These maps show every sign-change on α, β extends to L, so L/Q is Galois. Since [L : Q] = 8, we now have |Gal(L/Q)| = 8. τ : α 7→ α, β 7→ −β, γ 7→

Step 3: Group structure Compute squares on γ:
2 2 σ (γ) = σ (α − 1)γ = σ(α − 1) σ(γ) = (−α − 1)(α − 1)γ = −(α − 1)γ = −γ, so σ 2 = ρ. Likewise,        β−1 β−1 −β − 1 β−1 β2 − 1 2 2 τ (γ) = τ γ =τ τ (γ) = γ=− γ = − γ = −γ, α α α α α2 2 so τ 2 = ρ. In particular, σ and τ have order 4, and ρ is central of order 2. Finally compare στ and τ σ on γ. Let u = α − 1 and v = (β − 1)/α. Then σ(v) = −v (because σ(α) = −α and σ(β) = β), while τ (u) = u. Hence στ (γ) = σ(vγ) = σ(v)σ(γ) = (−v) uγ = −(uv)γ,

but

τ σ(γ) = τ (uγ) = τ (u)τ (γ) = u vγ = (uv)γ.

Therefore στ (γ) = −τ σ(γ), i.e. στ = ρ τ σ. So the group is generated by σ, τ with relations 2

2

σ = τ = ρ,

2

ρ = 1,

ρ central,

στ = ρ τ σ,

which is exactly the quaternion group of order 8. Equivalently, every nontrivial element other than ρ has order 4, so the group cannot be dihedral; it is Q8 .

Primitive core_concept. The key insight is to track the added radical by its square class over the biquadratic base; preservation of that square class gives the lifts, and the orders of those lifts together with whether they commute are what pin down the isomorphism type.

Figure 6: An example illustrating the distinction between a full solution rationale and the corresponding mathematical primitive.

17

Preprint

You are given a language-only mathematics problem, its answer, and its rationale. Your task is to extract and classify the mathematical PRIMITIVE behind the rationale. ## Definition: A mathematical primitive is the essential mathematical property that makes the problem solvable, together with an explanation of how this property connects the problem to a valid solution principle. The primitive is not merely a theorem name, a technique, or a proof step. For example, “use induction,” “apply Riesz representation,” or “use generating functions” is insufficient unless it explains what property of the problem makes that method applicable. ## Steps: Step 1: Identify the essential mathematical property. Ask: What hidden structure, invariant, relation, decomposition, obstruction, or theorem condition makes the problem tractable? Step 2: Identify the solution principle. Ask: How does that property enable a valid theorem, reduction, construction, reformulation, contradiction, or proof strategy? Name the essential conceptual move(s) -- one OR several distinct key ideas, whichever the problem genuinely needs. Step 3: Write the mathematical primitive. Write 1-3 sentences combining the essential property and its connection to the solution principle. ## CRITICAL CONSTRAINTS: • Strictly conceptual: no algebraic manipulation, equations, or step-by-step proof. • HIGH-LEVEL but COMPLETE: state EVERY essential CONCEPTUAL idea the solution rests on (a problem may need ONE or SEVERAL distinct key observations -- include each one). Do NOT write out calculations, derivations, or intermediate numeric results. Step 4: Classify the primitive. Choose exactly one primary primitive type and zero to three secondary primitive types. CRITICAL CONSTRAINT: You MUST use the exact string keys provided in the taxonomy below (e.g., “reformulation”, “hidden_structure”). Do NOT modify, capitalize, or invent new types. The primary primitive type should describe the main mathematical mechanism that unlocks the solution. The secondary primitive types should describe additional mathematical ideas that support, enable, or refine the primary primitive. Do not classify based only on surface-level technique names. Classify based on the essential mathematical property and how it connects to the solution principle. ## Primitive Taxonomy: • reformulation: The problem becomes tractable by rewriting it in a more useful mathematical language, representation, or equivalent form. • hidden_structure: The key is recognizing that the objects secretly have a known mathematical structure, such as a group, vector space, Hilbert space, graph, lattice, convex set, metric space, probability space, or algebraic structure. • invariant_or_monotonicity: The key is identifying a quantity or property that is preserved, monotone, or constrains all valid transformations. • theorem_applicability: The key is recognizing that the problem reduces to verifying the assumptions of a known theorem. • duality_or_representation: The key is passing to a dual, transformed, spectral, functional, or representational viewpoint where the problem becomes simpler. • extremal_principle: The key is choosing a maximal, minimal, largest, smallest, boundary, or otherwise extreme object and exploiting its extremality. • construction_or_witness: The key is explicitly constructing an object, example, counterexample, certificate, function, sequence, graph, or witness. • reduction_or_embedding: The key is reducing the problem to another known or canonical problem, or embedding it into a setting where a known result applies. • obstruction_or_contradiction: The key is identifying a necessary structural constraint that the opposite assumption would violate. • induction_or_recursion: The key is recognizing a recursive, self-similar, or inductive structure and choosing an induction hypothesis that preserves the essential property. ## DO NOT SOLVE OR SUMMARIZE THE SOLUTION: • Do NOT state or compute the final answer; no specific numeric result, final value, or boxed expression. • Do NOT write a full derivation or proof. • Do NOT write out the calculations or the step-by-step derivation. State all the essential conceptual primitive(s) the solution rests on -- including each distinct key idea it genuinely needs -- but not the execution that follows. Extract and classify the mathematical primitive behind the rationale. ## Output Format: Output ONLY a valid JSON object in the exact format below, with no markdown code blocks formatting (e.g., do not wrap in ‘ ‘ ‘ json), just the raw JSON: { "essential_property": "...", "solution_principle": "...", "core_concept": "...", "primary_primitive_type": "...", "secondary_primitive_types": ["..."], "classification_rationale": "...", "confidence": <integer between 1 and 5> } ## Problem: [PROBLEM] ## Answer: [GOLD_ANSWER] ## Rationale: [GOLD_RATIONALE]

Figure 7: Prompt used during data curation to generate mathematical primitives and their structured annotations.

18

Preprint

You are checking whether a candidate mathematical “primitive” for a mathematics problem is VACUOUS -i.e., not really a primitive at all. Look only at the candidate below. A mathematical primitive names the essential property that makes the problem solvable and explains how it connects to a solution principle. Mark verdict = “invalid” ONLY if the candidate is VACUOUS, i.e., ANY of: • “generic”: pattern”).

vague advice with no problem-specific content (“use a clever substitution”, “find the

• “technique_name_only”: makes it apply. • “computation_only”:

names a method/theorem but never states the property of THIS problem that

only arithmetic/numeric work, with no conceptual claim.

Otherwise verdict = “valid”, failure = “none”. Do NOT mark invalid for stating a conclusion or a final result, and do NOT mark invalid for being a full derivation -- over-completeness is NOT vacuity. A thorough or conclusion-stating answer can still contain a valid primitive. ## Candidate Primitive: essential_property: [MODEL essential_property] solution_principle: [MODEL solution_principle] core_concept: [MODEL core_concept] ## Output Format: Return JSON only: {“verdict”: “valid”|“invalid”, “failure”: “none”|“generic”|“technique_name_only”|“computation_only”, “evidence”: “<one short sentence>”}

Figure 8: Prompt for assessing primitive validity V by detecting vacuous predictions.

You are judging whether a MODEL’s candidate mathematical primitive for a math problem is CORRECT on ONE dimension, using a known-correct REFERENCE SOLUTION. Judge correctness FOR THIS PROBLEM -- not reproduction of any particular wording. ## GROUND RULES (read carefully): • The REFERENCE SOLUTION below is correct and complete. its key idea, written WITH the full solution in hand.

The GOLD PRIMITIVE is a terse distillation of The MODEL only saw the problem.

• Judge whether the MODEL’s primitive correctly captures the [DIM] that unlocks this problem, as evidenced by the reference solution. • CREDIT a correct primitive even if it is (i) phrased more coarsely or less precisely than the gold primitive, or (ii) a different but valid route that the reference solution shows would work. • Do NOT require the model to reproduce the gold primitive’s exact framing, or to name the same theorem if its statement is equivalent. • Do NOT re-solve from scratch; judge the model’s primitive against the reference solution. • Do NOT consider whether the model reached any final answer -- judge the primitive content only. • A primitive here is the SINGLE CORE high-level idea (the dominant “aha”). Everything DOWNSTREAM of it is execution, NOT part of the primitive, and its absence must NOT lower the score: secondary lemmas, second-step ideas, lower-bound / sharpness / optimality arguments, exact constants, specific numeric values, and calculation steps are what the core primitive ENABLES. Judge only whether the model has the correct CORE idea. (A vague, non-problem-specific statement, however, is still not a primitive.) The dimension you are judging is the [DIM]: [DIM_DEF] ## Scoring (CORRECTNESS, not reproduction): • “match”:

[C_MATCH]

• “partial”:

[C_PARTIAL]

• “mismatch”:

[C_MISMATCH]

## Problem: [PROBLEM] ## Reference Solution (correct): [REFERENCE SOLUTION] ## Gold Primitive (terse pointer to the essential idea): essential_property: [GOLD essential_property] solution_principle: [GOLD solution_principle] core_concept: [GOLD core_concept] ## Model Primitive (judge this): essential_property: [MODEL essential_property] solution_principle: [MODEL solution_principle] core_concept: [MODEL core_concept] ## Output Format: Return JSON only: {“agreement”: “match”|“partial”|“mismatch”, “evidence”: “<one short sentence>”}

Figure 9: Prompt for assessing primitive correctness along the GATE and MECHANISM dimensions, corresponding to σgate and σmech .

19

Preprint

You are given a language-only mathematics problem -- the problem ONLY, with no solution. Your task is to produce the single kernel mathematical PRIMITIVE needed to solve it: the essential structural understanding that makes the problem tractable. Do NOT solve the problem. ## Definition: A mathematical primitive is the essential mathematical structure or property that makes the problem solvable, together with an explanation of how this structure connects the problem to a valid solution principle. The primitive is not merely a theorem name, a technique, or a proof step. For example, “use induction,” “apply Riesz representation,” or “use generating functions” is insufficient unless it explains what property of the problem makes that method applicable. ## Steps: Step 1: Identify the essential mathematical property. Ask: What hidden structure, invariant, relation, decomposition, obstruction, or theorem condition makes the problem tractable? Step 2: Identify the solution principle. Ask: How does that property enable a valid theorem, reduction, construction, reformulation, contradiction, or proof strategy? Name the essential conceptual move(s) -- one OR several distinct key ideas, whichever the problem genuinely needs. Step 3: Write the mathematical primitive. Write 1-3 sentences capturing the single high-level “Aha!” -- the central structural understanding that makes the problem tractable. ## CRITICAL CONSTRAINTS: • Strictly conceptual: no algebraic manipulation, equations, or step-by-step proof. • HIGH-LEVEL but COMPLETE: state EVERY essential CONCEPTUAL idea the solution rests on (a problem may need ONE or SEVERAL distinct key observations -- include each one). Do NOT write out calculations, derivations, or intermediate numeric results. • Contrast: “recognize the configuration is secretly a Cayley graph” is a mathematical primitive; “build the Cayley graph, compute its spectrum, then sum the eigenvalues to get the count” is a solution outline -- produce the former, never the latter. ## DO NOT SOLVE OR SUMMARIZE THE SOLUTION: • Do NOT state or compute the final answer; no specific numeric result, final value, or boxed expression. • Do NOT write a full derivation or proof. • Do NOT write out the calculations or the step-by-step derivation. State all the essential conceptual primitive(s) the solution rests on -- including each distinct key idea it genuinely needs -- but not the execution that follows. Produce the essential mathematical primitive(s): the key thing(s) to notice before starting to solve. ## Output Format: Output ONLY a valid JSON object in the exact format below, with no markdown code blocks formatting (e.g., do not wrap in ‘ ‘ ‘ json), just the raw JSON: { "essential_property": "...", "solution_principle": "...", "core_concept": "..." } ## Problem: [PROBLEM]

Figure 10: Prompt for generating teacher mathematical primitives.

You are a math problem solver. Below is a problem. Produce a concise high-level solution plan describing the major procedural steps needed to solve it. ## HARD CONSTRAINTS: • Give 3-6 major steps in the order they should be carried out. • Each step should state the intended mathematical operation, construction, theorem application, or reduction, without carrying out the corresponding derivation. • Do NOT provide algebraic calculations, intermediate numerical results, detailed derivations, or the final answer. • Keep the overall level of detail concise and comparable to a short conceptual guidance. ## Problem: [PROBLEM]

Figure 11: Prompt for generating teacher solution plans from language-only mathematics problems.

20

Preprint

B.3 B.3.1

A DDITIONAL E XPERIMENT R ESULTS P ERFORMANCE ACROSS P RIMITIVE FAMILIES

To provide a more fine-grained view of model behavior, we further break down performance by primitive family. Table 7 reports results on P RIM separately for Recast, Witness, and Argument primitives across all four dimensions of mathematical understanding. This breakdown complements the aggregate results in Table 1 in the main text by showing how the observed capability gaps vary across different forms of structural reasoning. Discovery Model OpenAI gpt-5.4 gpt-5.4-mini gpt-5.4-nano gpt-oss-20b Qwen Qwen3.6-27B Qwen3.5-27B Qwen3.5-9B Qwen3.5-4B

R

W

84.97 61.44 43.79 36.60

72.73 54.55 27.27 13.64

Generation A

R

W

Digestion A

R

W

Execution A

R

W

71.43 68.63 28.57 51.63 42.86 46.41 57.14 42.48

54.55 40.91 31.82 40.91

71.43 100.00 100.00 100.00 86.27 95.45 57.14 97.39 100.00 100.00 70.59 81.82 42.86 96.73 100.00 100.00 69.28 72.73 71.43 91.50 90.91 85.71 60.99 72.73

28.76 0.00 14.29 53.59 29.41 18.18 42.86 47.71 14.38 9.09 0.00 38.56 7.19 0.00 14.29 26.14

45.45 40.91 27.27 27.27

57.14 71.43 71.43 28.57

DeepSeek-R1 Distill R1-0528-8B 5.88 R1-Distill-32B 6.54 R1-Distill-14B 4.58 R1-Distill-7B 7.84

4.55 4.55 4.55 4.55

90.85 92.81 78.43 69.93

28.57 15.69 27.27 14.29 67.97 0.00 15.03 31.82 42.86 64.05 0.00 15.69 18.18 14.29 68.63 0.00 13.73 9.09 14.29 49.02

A 71.43 71.43 42.86 28.57

100.00 100.00 78.43 77.27 85.71 95.45 100.00 70.59 81.82 71.43 86.36 85.71 59.48 72.73 71.43 72.73 85.71 54.25 68.18 57.14 81.82 68.18 54.55 50.00

42.86 85.71 71.43 42.86

37.25 41.83 37.25 30.07

54.55 59.09 54.55 45.45

42.86 28.57 28.57 42.86

Table 7: Performance on P RIM across the four dimensions of mathematical understanding, stratified by primitive family: R = Recast, W = Witness, and A = Argument. B.3.2

R EASONING L ENGTH AND G ENERATION B EHAVIOR

Figure 12 compares the output-token distributions of representative models during Generation. The left panel shows the overall distribution across all rollouts, while the right panel separates rollouts by final-answer correctness, providing a view of how reasoning length varies across models and between successful and unsuccessful solutions.

(a) All rollouts

(b) By correctness

Figure 12: Output-token distributions of representative models from each model family for Generation. Left: all rollouts. Right: rollouts separated by final-answer correctness. B.3.3

E XECUTION UNDER D IFFERENT P RIMITIVE S OURCES

Table 8 provides the full results underlying the Execution analysis in §3, extending the comparison from the representative models in Table 2 to all 12 evaluated models. The same pattern holds 21

Preprint

consistently: conditioning on self-generated primitives provides little or no benefit and often degrades performance, whereas externally supplied teacher primitives substantially improve Execution across model families, with gold primitives yielding the largest gains. These results further support that the rescue effect is not a consequence of simply decomposing generation into a discover-then-execute pipeline; rather, successful execution depends on supplying the relevant mathematical structure that the model often fails to identify on its own.

Model

Execution

Generation

Self-Generated Primitive Teacher Primitive Gold Primitive OpenAI gpt-5.4 gpt-5.4-mini gpt-5.4-nano gpt-oss-20b Qwen Qwen3.6-27B Qwen3.5-27B Qwen3.5-9B Qwen3.5-4B DeepSeek-R1 Distill R1-0528-Qwen3-8B R1-Distill-Qwen-32B R1-Distill-Qwen-14B R1-Distill-Qwen-7B

67.03 50.55 44.51 43.41

74.18 (+7.15) 48.90 (-1.65) 46.15 (+1.64) 38.46 (-4.95)

—† 63.74 (+13.19) 60.99 (+16.48) 58.24 (+14.83)

86.81 (+19.78) 71.98 (+21.43) 68.68 (+24.17) 60.99 (+17.58)

52.75 47.80 38.46 26.37

41.21 (-11.54) 36.81 (-10.99) 23.08 (-15.38) 17.03 (-9.34)

64.84 (+12.09) 58.24 (+10.44) 52.75 (+14.29) 42.31 (+15.94)

78.57 (+25.82) 71.98 (+24.18) 61.54 (+23.08) 56.04 (+29.67)

17.03 18.13 15.93 13.19

10.99 (-6.04) 17.58 (-0.55) 14.84 (-1.09) 10.44 (-2.75)

31.32 (+14.29) 37.91 (+19.78) 31.87 (+15.94) 19.78 (+6.59)

39.56 (+22.53) 43.41 (+25.28) 39.01 (+23.08) 32.42 (+19.23)

Table 8: Generation and Execution performance on P RIM under different settings. Execution is evaluated with self-generated, teacher-provided, and gold primitives. Percentage-point changes are reported relative to Generation. † The teacher-primitive result for gpt-5.4 is omitted because gpt-5.4 serves as the teacher model.

Model

Composition of Generation Failures (%)

nfail −

D− E −

D+ E −

D+ E +

Discovery-limited

Capability-limited

Execution-limited

Utilization-limited

60 90 101 103

15.0 28.9 38.6 35.0

10.0 21.1 31.7 45.6

21.7 26.7 18.8 9.7

53.3 23.3 10.9 9.7

86 95 112 134

48.8 42.1 44.6 42.5

33.7 30.5 42.9 52.2

7.0 15.8 6.2 3.7

10.5 11.6 6.2 1.5

151 149 153 158

28.5 33.6 30.7 26.6

65.6 61.7 68.0 68.4

2.6 2.0 1.3 3.2

3.3 2.7 0.0 1.9

D E OpenAI gpt-5.4 gpt-5.4-mini gpt-5.4-nano gpt-oss-20b Qwen Qwen3.6-27B Qwen3.5-27B Qwen3.5-9B Qwen3.5-4B DeepSeek-R1 Distill R1-0528-Qwen3-8B R1-Distill-Qwen-32B R1-Distill-Qwen-14B R1-Distill-Qwen-7B

+

Table 9: Composition of direct Generation failures on P RIM. Each failed problem is categorized by whether the model discovers the gold primitive from the question (D; τ = 0.8) and whether it can execute the primitive when provided (E).

22

Preprint

B.3.4

FAILURE D ECOMPOSITION AND R ESCUE A NALYSIS

Table 9 provides the full model-wise breakdown underlying Figure 4. Across model families, Generation failures are concentrated in the D− regimes. This decomposition further clarifies the role of primitive discovery: even when overall solving ability is relatively strong, a substantial portion of failures arises because the model cannot independently identify the relevant mathematical structure, despite being able to execute it once supplied. We further examine how Digestion relates to a model’s ability to make use of externally supplied primitive guidance. Table 10 partitions initially failed problems according to whether the model can correctly recover the primitive from the reference solution, and then measures the rescue rate after supplying the teacher primitive. For most models, digestible failures are substantially more likely to be rescued than undigestible ones, suggesting that the ability to recover mathematical structure from a completed solution is closely related to the ability to incorporate that structure into downstream reasoning. This relationship is particularly clear for the Qwen3.5 models, where digestible problems exhibit rescue-rate advantages of 17.6–28.6 percentage points. The pattern is weaker for the DeepSeek-R1 distill models and is noisy for models with very few undigestible examples, indicating that Digestion and successful execution are related but distinct capabilities.

Rescue Rate

Digestion Model OpenAI gpt-5.4-mini† 97.8 gpt-5.4-nano† 97.3 gpt-oss-20b† 91.2 Qwen Qwen3.6-27B† 92.3 Qwen3.5-27B 93.4 Qwen3.5-9B 79.7 Qwen3.5-4B 70.9 DeepSeek-R1 Distill R1-0528-8B 68.7 R1-Distill-32B 65.4 R1-Distill-14B 67.0 R1-Distill-7B 48.9

Gap

Digestible

Undigestible

42.5 40.8 40.4

33.3 0.0 11.1

9.2 40.8 29.3

36.4 37.6 41.6 35.1

66.7 10.0 13.0 17.5

−30.3 27.6 28.6 17.6

23.3 32.6 24.8 19.7

16.7 26.3 23.1 13.4

6.6 6.3 1.7 6.3

Table 10: Relationship between Digestion and rescue under teacher-primitive guidance. Problems are partitioned by whether the model correctly recovers the primitive from the reference solution (digestible) or not (undigestible); rescue rate is the fraction of initially incorrect problems solved after providing the teacher primitive. Gap denotes the digestible–undigestible difference in rescue rate. † Undigestible estimates are based on fewer than 20 problems and are therefore high variance.

B.3.5

R EPAIRABILITY ACROSS FAILURE M ODES

To complement the main-text analysis of discovery-limited and capability-limited failures, Table 11 reports repair rates across all four initial failure quadrants of the base model. The distinction between D− E + and D− E − remains particularly consistent: across SFT, OPSD, and A BSORB, discoverylimited failures are repaired substantially more often than capability-limited failures. The remaining quadrants exhibit different repair profiles, with utilization-limited (D+ E + ) cases showing relatively high repair rates and execution-limited (D+ E − ) cases varying more strongly across post-training methods. These results provide a fuller view of how the initial capability decomposition relates to subsequent repairability, while reinforcing the main-text observation that failures with latent execution capacity are especially amenable to post-training. 23

Preprint

Base failure type −

+

D E : discovery-limited D− E − : capability-limited D+ E − : execution-limited D+ E + : utilization-limited

SFT

OPSD

A BSORB

20.4% 6.8% 11.1% 35.0%

21.1% 8.2% 0.0% 30.0%

23.8% 10.2% 18.5% 45.0%

Table 11: Post-training repair rates across initial P RIM failure quadrants.

C

P OST-T RAINING DATASET D ETAILS

Table 12 presents the detailed composition of the 709 Mathematics Ph.D. qualifying examination problems used for our post-training experiments, categorized by source university and mathematical domain. The problems span 1991–2026 and are strictly parsed into LATEX. We select qualifyingexamination problems because their difficulty and open-ended reasoning demands closely match those of P RIM, requiring substantial mathematical maturity and structural reasoning rather than shortform answer prediction. Accordingly, the dataset consists entirely of open-ended proof problems without algorithmically verifiable short-answer labels. Each problem is paired with a human-authored reference proof (median length: 118 words) and a one-sentence primitive extracted from that proof by GPT-5.4-High using the curation prompt of Figure 7. Source University (1991–2026)

Problem Count

University of Oregon UW Madison Harvard University UC Berkeley

332 131 126 120

Mathematical Domain

Problem Count

Algebra Analysis Topology Geometry Number Theory Logic

318 206 97 82 5 1

Table 12: Detailed composition of the training corpus.

D

P OST-T RAINING S ETUP

All A BSORB and OPSD runs use the same training recipe and differ in the privileged context provided to the teacher: A BSORB uses the mathematical primitive, whereas OPSD uses the reference solution. Training uses the 709 qualifying-examination problems, each paired with a one-sentence primitive. For each problem, the student generates one on-policy rollout from the problem alone using temperature 1.0, top-p 0.95, top-k 20, a maximum of 4096 completion tokens, and a total context length of 6144. The teacher uses the same base model with the adapter disabled and is conditioned on the corresponding privileged context. For A BSORB, we optimize LA BSORB with top-K support K = 128 and clamp threshold τ = 0.06, applied to all generated completion tokens, including truncated rollouts. We train LoRA adapters with rank 64 and α = 128 on all attention and MLP projections, including the linear-attention projections in Qwen3.5. Optimization uses fused AdamW without weight decay, a learning rate of 5 × 10−6 with linear scheduling and no warmup, gradient clipping at 0.1, and an effective batch size of 32 with per-device batch size 1 and gradient accumulation. Training uses bf16, gradient checkpointing, and seed 42. Each model is trained for one epoch, corresponding to 22–23 optimizer steps. On-policy rollouts are generated with a co-located vLLM engine synchronized with the student after every optimizer step. 24

Preprint

E

P OST-T RAINING A BLATION S TUDIES

To validate the design choices of A BSORB, we conduct ablation studies on the Qwen3.5-9B baseline using P RIM (n = 182). We organize the analysis around three questions: what information should be provided to the teacher as privilege, how that privileged guidance should be transferred to the student, and whether the primitive should instead be learned as an explicit prediction target. For all comparisons, we report Generation and Discovery together with absolute changes relative to the base model. Variant

Generation

Discovery

Qwen3.5-9B (base)

38.46

13.19

Solution privilege Primitive privilege (A BSORB)

33.52 (-4.95) 43.96 (+5.49)

10.99 (-2.20) 14.29 (+1.10)

Table 13: Ablation on the source of privileged information using Qwen3.5-9B. The training objective is held fixed while the teacher is conditioned on either the full reference solution or the mathematical primitive.

Teacher Conditioning Table 13 isolates the effect of the teacher’s privileged input. In both variants, the student follows the same on-policy solution rollout and is trained with the same A BSORB objective; the only difference is whether the teacher conditions on the full reference solution or on the mathematical primitive. Under this controlled comparison, solution conditioning reduces Generation from 38.46 to 33.52, whereas primitive conditioning improves it to 43.96. Discovery follows the same trend. This shows that the gain does not arise from privileged conditioning alone: the primitive provides a more targeted supervisory signal than the full solution. Transfer Mechanism Table 14 fixes the mathematical primitive as the teacher’s privileged input and keeps the student rollout in solution space, while varying how the primitive-conditioned teacher signal is transferred. The rKL, no cap variant applies reverse KL without restricting teacher-induced changes, providing the unconstrained baseline. The fKL, capped variant instead uses forward KL on the teacher’s top-K support while retaining the same cap, testing whether the benefit comes simply from limiting large contributions rather than from the asymmetric reverse-KL formulation. Both variants underperform the base model, with Generation dropping to 31.87 and 36.81, respectively. Transfer Mechanism

Generation

Discovery

Qwen3.5-9B (base)

38.46

13.19

rKL, no cap fKL, capped Divergence control Entropy control Bounded override (A BSORB)

31.87 (-6.59) 36.81 (-1.65) 34.62 (-3.85) 31.87 (-6.59) 43.96 (+5.49)

10.44 (-2.75) 12.09 (-1.10) 12.09 (-1.10) 14.29 (+1.10) 14.29 (+1.10)

Table 14: Ablation on mechanisms for transferring primitive-guided teacher supervision using Qwen3.5-9B. All variants use the mathematical primitive as privileged information. Divergence control tests whether harmful transfer can be identified purely by the magnitude of teacher–student disagreement. Let Dt = DKL (π̄S (· | ŷ<t , x) ∥ π̄T (· | ŷ<t , x, p)) , where both distributions are renormalized. Since the divergence distribution is highly concentrated in a small tail, we mask tokens whose divergence exceeds the offline 95th-percentile threshold: mdiv = 1[Dt ≤ τD ] , t 25

τD = 0.05.

Preprint

This removes only about 5% of tokens but approximately 86% of the total divergence mass, partially recovering Generation from 31.87 to 34.62. Entropy control tests a finer hypothesis: whether harmful overrides and useful primitive guidance can be separated within this high-divergence tail using the student’s confidence. We define the student’s full-vocabulary entropy as X Ht = − πS (v | ŷ<t , x) log πS (v | ŷ<t , x), v

and mask only high-divergence, low-entropy positions: ment = 1[Dt ≤ τD ∨ Ht ≥ τH ] , t

τH = 0.86.

This preserves high-divergence positions where the student is uncertain, under the hypothesis that these are more likely to contain useful primitive-specific guidance. However, despite masking fewer tokens, Generation returns to 31.87, indicating that neither divergence magnitude nor student confidence cleanly separates beneficial from harmful teacher intervention. In contrast, A BSORB avoids making a whole-token keep-or-drop decision. Its bounded override operates within each token distribution, retaining supervision at every position while limiting how strongly individual teacher preferences can suppress student-supported choices. This is the only transfer mechanism in the comparison that improves over the base model, reaching 43.96 Generation. Rollout Target Table 15 isolates whether the mathematical primitive should serve as privileged guidance or as the student’s rollout target. In the A BSORB setting, the student continues to generate solutions while the primitive is available only to the teacher. We compare this against variants in which the student instead generates the primitive itself under uncapped reverse KL, capped forward KL, or the same capped reverse-KL objective used by A BSORB. Primitive-target rollouts remain weak across all three objectives. Even under the A BSORB objective, changing only the rollout target from solution to primitive yields 39.01 Generation, compared with 43.96 when the primitive instead conditions the teacher while the student generates solutions. This indicates that the primitive is more effective as privileged guidance that shapes solution reasoning than as the object the student is directly optimized to produce. Rollout Target / Objective

Generation

Discovery

Qwen3.5-9B (base) A BSORB

38.46 43.96 (+5.49)

13.19 14.29 (+1.10)

rKL, no cap fKL, capped rKL, capped

32.97 (-5.49) 33.52 (-4.95) 39.01 (+0.55)

13.19 (+0.00) 9.34 (-3.85) 12.09 (-1.10)

Table 15: Ablation on the student rollout target using Qwen3.5-9B. The A BSORB baseline keeps the student in solution-generation space and uses the primitive only as privileged teacher information; the remaining variants instead train the student to generate the primitive itself under different distillation objectives. Alternative Training Objectives Table 16 compares A BSORB against alternative post-training objectives on the same Qwen3.5-9B backbone. We separate the comparison into teacher-guided distillation objectives and direct supervised fine-tuning objectives. Among the distillation baselines, TOP-D (Xie et al., 2026) is a policy-gradient on-policy distillation method based on a bounded proximal reward. We adapt it to our setting using the same 709-problem training corpus, primitive privilege, and self-teacher as A BSORB, with G = 4 student rollouts per prompt, proximal reward r̃t = log(αρt + 1 − α) , α = 0.1, followed by token-level length-normalized future returns and group-normalized advantages. Despite operating with the same privileged signal, TOP-D reduces Generation to 31.87 and Discovery to 9.34, indicating that the primitive alone is insufficient without an appropriate transfer mechanism. 26

Preprint

Dual-channel is an alternative design motivated by our analysis of rescued trajectories, which suggested that primitive guidance primarily acts through early route selection and later adjudication. The route channel applies reverse KL only to the first 512 completion tokens. The adjudication channel operates at student-generated self-doubt forks (e.g., “Wait”, “But”, “However”, “Actually”) and asks the student to sample candidate continuations. The primitive-conditioned teacher provides only a preference over these student-generated candidates rather than generating tokens itself. Given teacher and frozen-base scores zT and z0 , we form the base-corrected teacher preference   zT − z0 qT = softmax , 0.2 and optimize Ladj = DKL (qT ∥ qS ), with weight 0.3 in addition to the early-route reverse-KL term. Because this variant uses a larger 1,692problem pool, full-length primitive annotations, and 8k-token completions, it should be interpreted as a reference alternative rather than a strict single-variable ablation. It nevertheless substantially underperforms both the base model and A BSORB. We separately compare direct supervised objectives. Solution SFT fine-tunes the model on the human-authored reference solutions, while primitive SFT directly trains the model to generate the mathematical primitive. Solution SFT lowers Generation to 32.42, and primitive SFT performs worst overall, reducing Generation and Discovery to 28.02 and 2.75, respectively. Together, these results show that neither alternative teacher-guided distillation nor direct supervision on solutions or primitives substitutes for the combination used by A BSORB: primitive-conditioned teacher guidance transferred through bounded override while the student remains in solution-generation space. Training Objective

Generation

Discovery

Qwen3.5-9B (base) A BSORB

38.46 43.96 (+5.49)

13.19 14.29 (+1.10)

Teacher-guided distillation TOP-D (Xie et al., 2026) 31.87 (-6.59) Dual-channel† 29.67 (-8.79)

9.34 (-3.85) 10.44 (-2.75)

Direct supervised fine-tuning Solution SFT 32.42 (-6.04) Primitive SFT 28.02 (-10.44)

12.64 (-0.55) 2.75 (-10.44)

Table 16: Comparison of alternative post-training objectives using Qwen3.5-9B. Teacher-guided distillation baselines include TOP-D (Xie et al., 2026) and our dual-channel alternative, while the SFT baselines directly supervise either the reference solution or the mathematical primitive. † Dualchannel uses a larger 1,692-problem training pool, full-length primitive annotations, and 8k-token completions, and is therefore included as a reference alternative rather than a strict single-variable ablation.

27

Related documents

• The Missing Primitive: Diagnosing and Repairing Mathematical Reasoning in Large Language Models#1157539
• INSPIRE: An Internalize-Then-Improve Approach for Example-Driven Mathematical Reasoning#927451
• Do LLMs Overthink Basic Math Reasoning? Benchmarking the Accuracy-Efficiency Tradeoff in Language Models#160912
• MGSM-Pro: A Simple Strategy for Robust Multilingual Mathematical Reasoning Evaluation#1118469
• Grammar-Aligned Decoding#925022