ISO: An RLVR-Native Optimization Stack

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

ISO: An RLVR-Native Optimization Stack

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

machine learning, deep learning, neural networks

ISO: An RLVR-Native Optimization Stack Hanqing Zhu1,♠ , Wenyan Cong1,♠ , Zhizhou Sha1 , Sagnik Mukherjee2 , Xinyuan Song3 , David González-Martínez6 , Xiaoxia Wu4 , Yuandong Tian5 , Shiwei Liu6 , David Z. Pan1 , Zhangyang "Atlas" Wang1,† 1 The University of Texas at Austin 2 UIUC, 3 Emory University, 4 Together AI, 5 Recursive Superintelligence Inc, 6 ELLIS Institute Tübingen

arXiv:2607.19331v1 [cs.LG] 21 Jul 2026

§ Code

Website

Reinforcement learning with verifiable rewards (RLVR) is rapidly advancing the reasoning capabilities of language models, yet the optimization layer that converts reward feedback into weight-space updates remains poorly understood. Building on our prior analysis [1], we study this missing layer through the singular structure of model weights and identify spectral inheritance: RLVR can reuse the base model’s weight spectra while acquiring new behavior through changes in the associated input and output singular frames. We further confirm that close reconstruction of learned endpoints requires both singular frames to remain adaptable: remixing only within the incoming input and output spans, or keeping either incoming subspace fixed, leaves substantially more of the checkpoint change unexplained. We operationalize spectral inheritance as Isospectral Optimization (ISO), an RLVR-native, fixedspectrum optimization framework with complementary offline and online instantiations. Offline, ISOMerger combines the frame changes of shared-base specialists into a single fixed-spectrum model, requiring no post-merge data, rollouts, gradient updates, or on-policy distillation (OPD). It recovers complementary specialist capabilities and achieves the strongest aggregate performance among the compared data-free merging methods. Online, ISO-Optimizer applies a chosen base optimizer, including AdamW and Muon, to the frame variables while keeping the base spectra fixed. Across reasoning and coding tasks ranging from 1.5B to 8B parameters, ISO-Optimizer improves accuracy in the reported runs and reaches matched scores with substantially fewer training steps. On Qwen3-8B-Base, AdamW reaches an aggregate accuracy of 0.495 after 270 training steps. ISO-AdamW reaches the same accuracy after only 100 training steps and improves further to 0.509 at 210 training steps. Together, ISO offers a concrete answer to RLVR’s missing optimization layer: rather than inheriting pre-training optimization wholesale, design post-training around the structure of reward-driven adaptation: inherit the spectrum, optimize the frames.

1. Introduction Reinforcement learning with verifiable rewards (RLVR) has become a major scaling axis for modern reasoning models [2], with rapid progress across data and environments [3], learning objectives [4, 5, 6, 7], and systems [8, 9]. Yet the optimizers and parameterizations that translate reward feedback into weight-space motion remain largely inherited from pre-training, despite the recent proliferation of optimizers designed specifically for that regime [10, 11, 12, 13]. This default is not yet obviously natural: pre-training learns from dense token-level supervision, whereas RLVR adapts an already capable policy using comparatively sparse outcome-level rewards. We study this missing optimization layer and uncover a separation between what RLVR reuses and Work completed when Hanqing Zhu was at UT Austin. This work builds on our prior analysis of RL optimization dynamics (The Path Not Taken) and develops a concrete RL-native optimization stack. ♠ Equal contribution. † Corresponding author.

# [email protected], # [email protected]

ISO: An RLVR-Native Optimization Stack

Figure 1 | From spectral inheritance to Isospectral Optimization. Top. Unconstrained RLVR changes both

singular frames while its spectra remain close to their base values. Restoring Σ0 with the RL-trained frames fixed leaves performance flat at every 𝛼. ISO turns this regularity into an inductive bias: reuse Σ0 and optimize (𝑈, 𝑉 ). Left. ISO-Merger (offline) applies project–mask–merge–retract to shared-base specialists’ frame changes and reconstructs a fixed-spectrum model that recovers their specialist capabilities. Right. ISO-Optimizer (online) applies a conventional optimizer (e.g., AdamW) to (𝑈, 𝑉 ) under fixed Σ0 . On Qwen3-8B-Base, ISO-AdamW reaches AdamW’s end-of-run accuracy with 2.7× fewer training steps.

what it changes. We call this phenomenon spectral inheritance: RLVR can reuse the base model’s weight spectra while acquiring new behavior by changing the associated input and output singular frames. Existing accounts do not expose this separation, but they do suggest that RLVR differs structurally from pre-training and supervised fine-tuning (SFT). At the policy level, RLVR has been described through reverse-KL and KL-proximal views [1, 14]. At the parameter level, its Euclidean updates have been reported to be sparse and off-principal, with strongly overlapping footprints across independent runs initialized from the same base model [15, 1]. These findings identify distinctive structure in RLVR, but not a coordinate system that separates what is reused from what changes. Observation: spectra stay. Motivated by our prior observation of limited spectral drift in RLVR [1], we move from Euclidean weight coordinates to a spectral view. For each weight matrix, write 𝑊𝑡 = 𝑈𝑡 Σ𝑡 𝑉𝑡⊤ : Σ𝑡 specifies the scales of the singular modes, while (𝑈𝑡 , 𝑉𝑡 ) specify their output and input directions. Across the unconstrained RLVR runs studied here, the learned checkpoints remain close 2

ISO: An RLVR-Native Optimization Stack

to the fixed-spectrum families of their base weights, a pattern we call near-isospectrality. Because unconstrained RLVR optimization imposes no spectral constraint, we further ask whether this proximity reflects a genuine optimization preference. A dimension-aware calibration finds no strong additional preference for spectrum-preserving motion beyond what high dimensionality alone predicts, although the contrast with SFT remains pronounced. The calibration therefore refines rather than weakens the observation: RLVR remains near-isospectral, but proximity alone does not establish an intrinsic optimization preference. This leaves the more consequential functional question: are the small spectral changes that do occur necessary for the acquired behavior? Functional regularity: spectral inheritance. We next test a stronger question than spectral proximity: are the small spectral changes produced by unconstrained RLVR functionally necessary? Restoring the base spectra of RLVR checkpoints, while retaining their learned frames, preserves most acquired gains. More stringently, keeping the base spectra fixed throughout training and updating only the associated frames still supports strong RLVR learning. Conversely, a restricted spectrum-only control, which updates the singular values while freezing the base frames, yields only limited improvement. We refer to this functional reuse as spectral inheritance: RLVR can reuse the base model’s weight spectra rather than having to rewrite them to acquire new capabilities. Structure: both frames must remain adaptable. Spectral inheritance identifies what can be reused, but not which variables must remain adaptable. Could the learned endpoint instead be explained by a simpler transformation that either remixes the model only within the incoming input and output spans, or retains one incoming singular subspace while allowing only the other side to change? We find that both alternatives leave a substantially larger portion of the checkpoint change unexplained than retaining the incoming spectrum while allowing both singular frames to adapt. Thus, among the structural choices tested, the spectrum can remain fixed, but both frames must remain adaptable. The same spectral-inheritance and two-frame-adaptability pattern recurs across sequential RL stages with distinct objectives. ISO: Isospectral Optimization for RLVR. These findings suggest a simple design principle: inherit the spectrum, optimize the frames. We operationalize this principle through Isospectral Optimization (ISO), a framework that represents post-training change within the fixed-spectrum families of the base weights while keeping both singular frames adaptable. ISO is not a single numerical optimizer; it is an RLVR-native optimization stack with two complementary instantiations spanning the RLVR workflow: ISO-Merger for checkpoint-only composition of shared-base specialists and ISO-Optimizer for online learning with a chosen base optimizer such as AdamW or Muon. Offline: ISO-Merger. A modular RLVR workflow trains domain specialists from a shared base and later consolidates them, often through an additional on-policy distillation stage [16, 17]. ISOMerger instead consolidates the specialists directly from their checkpoints. Guided by spectral inheritance, it reuses the shared base spectra and directly combines the experts’ singular-frame changes into a single fixed-spectrum model. Without post-merge data, additional rollout generation, gradient updates, or distillation, ISO-Merger recovers complementary specialist capabilities and achieves the strongest aggregate performance among the compared data-free methods. Online: ISO-Optimizer. ISO-Optimizer applies the same principle during RLVR training. Given a conventional base optimizer such as AdamW or Muon, it channels reward feedback into weight-space motion within the corresponding fixed-spectrum families, updating the singular-frame variables while keeping the base spectra fixed. Across reasoning and coding settings and multiple model scales, ISO-Optimizer improves final accuracy and reaches matched accuracy in fewer training steps than the corresponding weight-space optimizers. Our contributions are summarized as follows: 3

ISO: An RLVR-Native Optimization Stack

• Spectral inheritance in RLVR. We formalize near-isospectrality as distance to a fixed-spectrum family, calibrate it against the geometry of high-dimensional weight spaces, and show through endpoint and training-time interventions that the base model’s weight spectra remain functionally reusable. We further show that, among the transformation classes tested, a low-residual reconstruction of learned endpoints requires both singular frames to remain adaptable. • RLVR-Native Isospectral Optimization Framework. We then introduce ISO, a fixed-spectrum framework that reuses the base spectra and represents reward-driven post-training change through the associated singular-frame coordinates. • Data-free offline RL expert composition. We develop ISO-Merger, which directly composes shared-base RLVR specialists in fixed-spectrum coordinates without post-merge data, rollout generation, gradient updates, or distillation. • Fixed-spectrum online RLVR. We develop ISO-Optimizer, which applies a chosen base optimizer, including AdamW or Muon, to the singular-frame variables while preserving the base spectra. Across reasoning and coding tasks ranging from 1.5B to 8B parameters, ISO-Optimizer improves accuracy in the reported runs and reaches matched scores with substantially fewer training steps. On Qwen3-8B-Base, AdamW reaches an aggregate accuracy of 0.495 after 270 training steps. ISO-AdamW reaches the same accuracy after only 100 training steps and improves further to 0.509 at 210 training steps.

2. Spectra Stay: From Near-Isospectrality to Spectral Inheritance From sparse Euclidean motion to spectral structure. Recent analyses suggest that RLVR differs from pre-training and SFT in its weight-space motion. At the policy level, RLVR has been described as a conservative, KL-proximal improvement of the current policy [18, 14, 1]. At the parameter level, its Euclidean updates have been reported to be sparse and off-principal, with strongly overlapping patterns across runs initialized from the same base model [15, 1]. This raises a puzzle: large behavioral gains arise from apparently sparse and off-principal weight motion. These regularities suggest that RLVR motion is structured, but raw Euclidean coordinates do not reveal what is reused and what is rewritten. Building on our prior observation of limited spectral drift in RLVR [1], we ask two questions. First, does the observed spectral proximity reflect a genuine preference for spectrum-preserving motion? A dimension-aware calibration finds no strong preference beyond what ambient dimensionality already predicts. Second, are the small spectral changes that remain functionally necessary? We test this through complementary interventions that restore the base spectra after training or keep them fixed throughout training. We call the resulting functional reuse spectral inheritance. Section 3 then asks which variables must remain adaptable once the spectrum is reused, and whether the same requirement recurs across sequential RL stages. Notation. For a weight matrix 𝑊 ∈ ℝ𝑑out × 𝑑in , let 𝑞 = min{ 𝑑out , 𝑑in } and let 𝜎 (𝑊 ) = ( 𝜎1 (𝑊 ) , . . . , 𝜎𝑞 (𝑊 )) denote all singular values in nonincreasing order. We write 𝑊 = 𝑈Σ𝑉 ⊤ for a thin SVD, with 𝑈 ∈ St( 𝑑out , 𝑞), 𝑉 ∈ St( 𝑑in , 𝑞), and Σ = Diag( 𝜎 (𝑊 )), where St( 𝑑, 𝑟 ) = { 𝑋 ∈ ℝ𝑑 ×𝑟 : 𝑋 ⊤ 𝑋 = 𝐼𝑟 } . The spectrum Σ specifies the scales of the singular modes, while (𝑈, 𝑉 ) specify their output and input directions. Throughout this section, the base model means the checkpoint that initializes the RL stage under study: it may itself be pretrained, SFT-trained, or already RL-trained. Accordingly, 𝑊0 and Σ0 denote a base weight matrix and its spectrum for the transition being analyzed. Section 3 reserves 𝑟 < 𝑞 for an informative top-𝑟 truncation.

ISO: An RLVR-Native Optimization Stack

The fixed-spectrum family.

Equivalently,

For a base matrix 𝑊0 = 𝑈0 Σ0𝑉0⊤ , define its fixed-spectrum family F (𝑊0 ) := 𝑍 ∈ ℝ𝑑out × 𝑑in : 𝜎 ( 𝑍 ) = 𝜎 (𝑊0 ) .

(1)

F (𝑊0 ) = 𝑈Σ0𝑉 ⊤ : 𝑈 ∈ St( 𝑑out , 𝑞) , 𝑉 ∈ St( 𝑑in , 𝑞) .

(2)

Thus, F (𝑊0 ) is simply the set of matrices of the same shape that share the singular values of 𝑊0 while allowing the associated left and right singular frames to change. 2.1. A Precise Observation: Spectra Stay We first make “spectra stay” precise by measuring the distance from a learned weight matrix 𝑊 to the fixed-spectrum family F (𝑊0 ) of its base matrix 𝑊0 . Proposition 2.1 (Exact distance to the fixed-spectrum family). For any 𝑊, 𝑊0 ∈ ℝ𝑑out × 𝑑in , dist𝐹 𝑊, F (𝑊0 ) = ∥ 𝜎 (𝑊 ) − 𝜎 (𝑊0 ) ∥ 2 .

(3)

Moreover, if 𝑊 = 𝑈Σ𝑉 ⊤ and Σ0 = Diag( 𝜎 (𝑊0 )), then 𝑈Σ0𝑉 ⊤ ∈ arg min ∥𝑊 − 𝑍 ∥ 𝐹 .

(4)

𝑍 ∈ F (𝑊0 )

Proposition 2.1 gives singular-value drift a precise checkpoint-level meaning: it is the distance from the learned matrix to F (𝑊0 ), the family of matrices sharing the corresponding base spectrum. The proposition also identifies a closest representative with that spectrum, which motivates the functional intervention in Section 2.2. The proof and the treatment of repeated singular values are given in Appendix B.1. Measurements.

For each matrix ℓ, let Δ𝑊 ( ℓ ) = 𝑊1( ℓ ) − 𝑊0( ℓ ) . We measure (ℓ)

𝛿Σ

(ℓ)

𝜌Σ

∥ 𝜎 (𝑊1( ℓ ) ) − 𝜎 (𝑊0( ℓ ) ) ∥ 2 ∥𝑊0( ℓ ) ∥ 𝐹 ∥ 𝜎 (𝑊1( ℓ ) ) − 𝜎 (𝑊0( ℓ ) ) ∥ 2 ∥ Δ𝑊 ( ℓ ) ∥ 𝐹

(5) .

The first measures spectral drift relative to the base-weight scale. The second compares the exact distance to F (𝑊0 ) with the complete checkpoint displacement. Long-horizon endpoint evidence. To rule out a short-horizon artifact, we analyze the released endpoint of a reasoning RLVR run on DeepSeek-R1-Distill-Qwen-1.5B (DS-1.5B) trained for over 3,000 updates [19, 20]. The checkpoint sequence

Qwen2.5-Math-1.5B → DS-1.5B → Nemotron-Research-Reasoning-Qwen-1.5B contains an SFT transition followed by an RLVR transition. For the RLVR stage, DS-1.5B is the base model. Broader evidence across models, datasets, RL objectives, and training horizons was established in our prior work [1]. Figure 2 shows that the post-RL spectrum nearly overlaps that of its pre-RL base, whereas the illustrative SFT transition produces substantial spectral contraction. Across RLVR layers, 𝛿Σ( ℓ ) is approximately 10−2 %, and the relative spectral residual 𝜌Σ( ℓ ) averages approximately 3% across the analyzed matrices (Figure 2d). Thus, for each analyzed matrix, the distance from the learned endpoint to a closest checkpoint in F (𝑊0 ) is only a few percent of its base-to-RL displacement. 5

ISO: An RLVR-Native Optimization Stack

(b) Per-rank spectral change Δσi /σi

2 SFT base SFT end = RL base RL end

1 1

−2.5 −5.0

16 32 64 128 256

Rank index i

ρΣ (%)

κspec (log scale)

upper bound ρΣ = 100%

10−2

10−4

16 32 64 128 256

Rank index i

(f) Update geometry (schematic) SFT base residual to (W0 ) (W0SFT ) SFT

SFT: ρΣ ≈ 0.35 (shrinks the spectrum)

101

Layer index

RL: ρΣ ≈ 0.03 (stays near (W0RL ))

SFT end = RL base

isotropic reference = 1

Layer index

102

mean over matrices per matrix (q/k/v/o/gate/up/down)

10−3

103

layer in (a,b)

10−1

(e) Dimension-normalized spectrum-changing energy κspec

(d) Relative spectral residual ρΣ 100

SFT RL

−7.5

δΣ (log scale)

0.0

Δσi /σi (%)

Singular value σi

(a) Singular value spectra

(W0RL )

gray: fixed-spectrum families

Layer index

Figure 2 | Spectral dynamics under SFT and RLVR post-training. (a) Representative singular-value profiles (v_proj, layer 14). The SFT endpoint is also the RLVR base model. (b) Rank-wise relative change: SFT reshapes the spectrum, whereas RLVR stays within a fraction of a percent of its pre-RL values. (c) Layer-wise spectral distance 𝛿Σ : the RLVR endpoint is two to three orders of magnitude closer to its fixed-spectrum family than the illustrative SFT endpoint. (d) Relative spectral residual 𝜌Σ : the mean RLVR residual is approximately 3% of the checkpoint displacement, compared with approximately 35% for SFT. (e) Dimension-normalized spectrum-changing energy 𝜅spec (Eq. 6): RLVR remains order-one (≈ 1.0–1.4), whereas SFT lies two to three orders of magnitude above the dimensional reference. (f ) Schematic endpoint view: SFT substantially rewrites the base spectrum, whereas RLVR checkpoints (dots) remain close to the fixed-spectrum family of the pre-RL base. The straight arrow denotes the SFT endpoint displacement, not a continuous optimization path. Evidence across sampled training checkpoints. An endpoint comparison cannot exclude a large intermediate spectral excursion that later cancels. We therefore track an independent unconstrained Qwen3-8B-Base AdamW run using checkpoints saved every 10 training steps. At each saved checkpoint, we measure its distance to the same fixed-spectrum family F (𝑊0 ). As shown in Figure 3, both 𝛿Σ ( 𝑡 ) and 𝜌Σ ( 𝑡 ) remain small throughout the observed checkpoint sequence. Thus, at the resolution of the saved checkpoints, spectral stability persists during training rather than appearing only at the final endpoint. Together, these results establish a descriptive fact: for the analyzed matrices, unconstrained RLVR checkpoints remain close to the fixed-spectrum families of their base weights. We call this property near-isospectrality. Does near-isospectrality reflect a preference? A natural hypothesis of the preceding results is that RLVR preferentially suppresses spectrum-changing directions. High dimensionality, however, makes this interpretation nontrivial. In a 𝑑out 𝑑in -dimensional matrix space, only 𝑞 = min{ 𝑑out , 𝑑in } independent first-order coordinates change the singular values in the generic full-rank, simplespectrum case. At 𝑊0 = 𝑈0 Σ0𝑉0⊤ , the first-order spectrum-changing coordinates of a displacement Δ𝑊 are diag(𝑈0⊤ Δ𝑊𝑉0 ). Thus, even a generically oriented high-dimensional displacement places only a small fraction of its squared norm in spectrum-changing coordinates. We calibrate this dimensional effect with (ℓ)

𝜅spec :=

𝑑out 𝑑in 𝑞

2 diag 𝑈0( ℓ )⊤ Δ𝑊 ( ℓ ) 𝑉0( ℓ ) 2

∥ Δ𝑊 ( ℓ ) ∥ 2𝐹

(6)

An isotropically oriented displacement has expected value one under this normalization. Hence, 6

ISO: An RLVR-Native Optimization Stack

mean over matrices per matrix (q/k/v/o/gate/up/down)

100

150

Training step t

1.5

1.4

1.0

1.2

κspec

10−5

10−6

(b) Relative spectral residual ρΣ (t)

ρΣ (%)

δΣ (log scale)

(a) Spectral distance δΣ (t)

0.5

1.0

0.0

0.8

100

150

isotropic reference = 1

Training step t

100

150

Training step t

Figure 3 | Near-isospectrality along an RLVR training trajectory (Qwen3-8B-Base, checkpoints every 10 steps up to step 190). (a) The mean spectral distance 𝛿Σ ( 𝑡 ) stays below 10−5 . (b) The mean relative spectral residual 𝜌Σ ( 𝑡 ) stays at ≈1.3% of the total displacement. (c) The dimension-normalized spectrum-changing energy 𝜅spec ( 𝑡 ) remains order-one relative to the isotropic dimensional reference. 𝜅spec ≪ 1 would indicate additional suppression of spectrum-changing motion. Order-one values do not reveal such a strong preference. Values of 𝜅spec ≫ 1 indicate concentration in spectrum-changing

coordinates. Across matrix types in the public reasoning run, RLVR yields mean 𝜅spec values between 1.02 and 1.35. Thus, after accounting for the small number of spectrum-changing coordinates, RLVR does not exhibit a strong additional suppression of spectral change. The same order-one pattern persists along the sampled Qwen3-8B-Base trajectory in Figure 3(c). Crucially, this calibration does not erase the contrast with SFT. The illustrative SFT transition yields 𝜅spec values between 89 and 1364, two to three orders of magnitude above the dimensional reference. Thus, even after correcting for ambient dimensionality, RLVR and SFT remain sharply different in how strongly their updates concentrate in spectrum-changing coordinates. 2.2. Spectral Inheritance: Reusing the Base Model’s Spectrum The dimensional calibration changes the interpretation of the initial observation without overturning it. RLVR remains close to F (𝑊0 ), but this proximity alone does not reveal a strong preference for spectrum-preserving updates. The key functional question is whether the small spectral changes that do occur are needed for the acquired behavior. We address this question through two complementary interventions. First, we restore the base spectrum after unconstrained RLVR while retaining the learned frames, testing whether the endpoint still requires its acquired spectral change. Second, we keep the base spectrum fixed throughout training, testing whether strong gains can be acquired without allowing that change at all. ⊤ . We interpolate Restoring the base spectrum after training. Let 𝑊0 = 𝑈0 Σ0𝑉0⊤ , 𝑊RL = 𝑈RL ΣRL𝑉RL only the spectrum: ⊤ e ( 𝛼) = 𝑈RL [(1 − 𝛼) Σ0 + 𝛼ΣRL ] 𝑉RL 𝑊 ,

𝛼 ∈ [0, 1] .

(7)

At 𝛼 = 0, the base spectrum is restored while the RL-trained frames are retained. At 𝛼 = 1, the original e (0) is a closest point in F (𝑊0 ). RL checkpoint is recovered. Proposition 2.1 shows that 𝑊 Figure 4(a) shows that restoring the base spectrum preserves most acquired performance. Conversely, replacing the base spectrum with the RL-trained spectrum while retaining the base frames does not improve the base model (Figure 15). Learning with the base spectrum fixed. A stronger constructive test is to keep Σ0 fixed from the first RL update onward and optimize only the singular frames. For contrast, we freeze the base frames and optimize only the diagonal spectrum Σ. 7

ISO: An RLVR-Native Optimization Stack

Score (%)

70 60 50 40 30 20 base

0.0

0.2

0.4

0.6

0.8

1.0 (RL)

AMC23 accuracy, avg@16

Composite RL model coefficient α AIME24 Minerva

AIME25 Olympia

AMC23

Training step

LCB pass@1 LCB pass@4

ISO-AdamW

(a) Spectrum restoration after training.

Sigma-only

AdamW

(b) Fixed-spectrum and spectrum-only control.

Figure 4 | Functional evidence for spectral inheritance. (a) Restoring the base spectrum while retaining the RL-trained frames preserves most endpoint performance. (b) Keeping the base spectrum fixed throughout training still permits strong gains, whereas the restricted spectrum-only parameterization does not recover comparable performance under the studied recipe.

Figure 4(b) shows that the fixed-spectrum parameterization acquires strong reasoning gains and, in this run, outperforms the AdamW baseline, whereas spectrum-only training does not achieve comparable gains. Because the latter control has restricted capacity,1 its failure shows only that spectral rescaling alone is insufficient in this configuration. Together, these interventions support spectral inheritance: RLVR can functionally reuse the base model’s spectral structure rather than having to rewrite it. Take-away 1: Spectral Inheritance. Unconstrained RLVR checkpoints remain close to the fixedspectrum families of their base weights. More importantly, the interventions show that RLVR can reuse this spectral structure rather than having to rewrite it, a property we call spectral inheritance.

3. Frames Move: What Changes Under Spectral Inheritance? Structure: both frames must remain adaptable. Spectral inheritance identifies what can remain fixed, but not which variables must remain adaptable. Could the endpoint be explained by a simpler transformation that either remixes only within the incoming input and output spans or retains one incoming singular subspace while leaving the other side free? We find that these simpler alternatives leave substantially more of the checkpoint update unexplained than reusing the incoming spectrum while adapting both frames. Thus, among the transformation classes tested, a low-residual fixed-spectrum description requires both singular frames to remain adaptable. Q1: Can a more conservative transformation explain the endpoint? For a transition 𝑊 𝑗 → 𝑊𝑖 , we refer to 𝑊 𝑗 as the incoming checkpoint and to 𝑊𝑖 as the learned endpoint. Let 𝑞 = min{ 𝑑out , 𝑑in } and consider an admissible top-𝑟 truncation (𝑟 )

𝑊𝑡

= 𝑈𝑡( 𝑟 ) Σ𝑡( 𝑟 ) 𝑉𝑡( 𝑟 )

⊤

(𝑟 )

𝑃𝑡

= 𝑈𝑡( 𝑟 ) 𝑈𝑡( 𝑟 )

⊤

(𝑟 )

𝑄𝑡

= 𝑉𝑡( 𝑟 ) 𝑉𝑡( 𝑟 )

⊤

(8)

where 𝑡 ∈ { 𝑖, 𝑗 }. Admissibility requires a positive rank-𝑟 boundary gap at both endpoints, so that the retained projectors are well-defined. We use 𝑟 = ⌊0.9𝑞⌋ in the main text and report rank sensitivity in Appendix B.4. 1 This family has only 𝑞 active degrees of freedom per matrix.

ISO: An RLVR-Native Optimization Stack

Define the rank-𝑟 fixed-spectrum family associated with the incoming checkpoint: n o F𝑗 ( 𝑟 ) := 𝑈Σ (𝑗 𝑟 ) 𝑉 ⊤ : 𝑈 ∈ St( 𝑑out , 𝑟 ) , 𝑉 ∈ St( 𝑑in , 𝑟 ) .

(9)

This family retains the incoming singular values Σ (𝑗 𝑟 ) while allowing both singular frames to vary. Proposition 3.1 (Optimal reconstructions and unexplained-update ratios). For an admissible transition 𝑊 𝑗 → 𝑊𝑖 , the Frobenius-optimal reconstructions under the four structural restrictions are bmix 𝑊 b𝐿 𝑊

= 𝑃 (𝑗 𝑟 ) 𝑊𝑖( 𝑟 ) 𝑄 (𝑗 𝑟 ) ,

remix within both incoming subspaces,

(𝑟 )

= 𝑃 𝑗 𝑊𝑖 ,

retain the incoming output subspace,

(𝑟 )

retain the incoming input subspace,

b𝑅 𝑊

= 𝑊𝑖 𝑄 𝑗 ,

biso 𝑊

= 𝑈𝑖( 𝑟 ) Σ (𝑗 𝑟 ) 𝑉𝑖( 𝑟 )

⊤

(10)

retain the incoming spectrum and adapt both frames.

For 𝑊𝑖( 𝑟 ) ≠ 𝑊 𝑗( 𝑟 ) , define the unexplained-update ratio 𝑢ℎ :=

bℎ ∥ 𝐹 ∥𝑊𝑖( 𝑟 ) − 𝑊 ∥𝑊𝑖( 𝑟 ) − 𝑊 𝑗( 𝑟 ) ∥ 𝐹

ℎ ∈ {mix, 𝐿, 𝑅, iso} .

(11)

Because 𝑊 𝑗( 𝑟 ) is feasible under all four restrictions, optimality gives 0 ≤ 𝑢ℎ ≤ 1. Thus, 𝑢ℎ = 0 denotes exact reconstruction, whereas 𝑢ℎ = 1 means that the best reconstruction under restriction ℎ is no closer to the endpoint than the unchanged incoming checkpoint. Moreover, biso ∈ arg min ∥𝑊 ( 𝑟 ) − 𝑍 ∥ 𝐹 , 𝑊

𝑢iso =

𝑖

(𝑟)

𝑍 ∈ F𝑗

∥ Σ𝑖( 𝑟 ) − Σ (𝑗 𝑟 ) ∥ 𝐹 ∥𝑊𝑖( 𝑟 ) − 𝑊 𝑗( 𝑟 ) ∥ 𝐹

(12)

The remix reconstruction can be written as (𝑟 )

(𝑟 )

(𝑟 ) ⊤

bmix = 𝑈 𝐵 𝑉 𝑊 𝑗 𝑖𝑗 𝑗

(𝑟 )

(𝑟 ) ⊤

𝐵𝑖 𝑗 := 𝑈 𝑗

(𝑟 )

𝑊𝑖 𝑉 𝑗 .

(13)

The core 𝐵𝑖(𝑗𝑟 ) is unconstrained and may rotate, mix, rescale, and change the represented spectrum. Thus, this class fixes only the incoming input and output spans. The one-sided classes are similarly permissive: they retain one incoming span while allowing the remaining mapping and spectrum to refit freely. Their 𝑢ℎ values are therefore optimistic lower bounds on the reconstruction error incurred by freezing the corresponding frame structure. We use 𝑢ℎ to test parameterization sufficiency. Formal class definitions and proofs are provided in Appendix B.3. Q2: Does the same structure recur under a new RL objective? A second question is whether this structure is specific to a single RL stage and objective, or whether it reappears when a new RL stage starts from an already RL-trained checkpoint and targets a distinct capability. A sequential objective-shift stress test. We evaluate the parameterization test on a two-stage visionlanguage-action RL pipeline initialized from Qwen2.5-VL-3B-Instruct [21]. Stage I optimizes spatial reasoning and produces 𝑊1 . Stage II starts from this already RL-trained checkpoint, optimizes an embodied-manipulation objective, and produces 𝑊2 . The resulting sequence provides three linked transitions: 𝑊0 → 𝑊1 ,

𝑊1 → 𝑊2 ,

𝑊0 → 𝑊2 .

(14)

The transition 𝑊0 → 𝑊1 provides the first-stage reference. The decisive objective-shift test is 𝑊1 → 𝑊2 : it asks whether the same frame-adaptability requirement reappears after re-anchoring at an already RL-trained checkpoint and changing the RL objective. The cumulative transition 𝑊0 → 𝑊2 tests consistency across the complete two-stage sequence. We analyze both language and vision modules. 9

Unexplained-update ratio uh

ISO: An RLVR-Native Optimization Stack

Language Modules

Vision Modules

100

10 1

10 2

Layer

mlp.down_proj

Layer

10 2

self_attn.o_proj umix

Layer

attn.qkv uiso

Layer

mlp.gate_proj

u = 1 (no change explained)

Figure 5 | Both incoming frames must remain adaptable. Unexplained-update ratios 𝑢ℎ for 𝑊0 → 𝑊2 at 𝑟 = ⌊0.9𝑞⌋, for representative language and vision modules. The dashed line at 𝑢 = 1 corresponds to leaving the incoming checkpoint unchanged. Restrictions that freeze one or both incoming singular subspaces ( 𝑢mix , 𝑢𝐿 , 𝑢𝑅 )

leave a large fraction of the update unexplained at every layer, whereas fixing only the incoming spectrum (𝑢iso ) leaves a few percent. All transitions and truncation levels are reported in Appendix B.4.

Results: both frames must remain adaptable. For the cumulative transition 𝑊0 → 𝑊2 , Figure 5 shows that an optimal remix within both incoming subspaces leaves a median 87% of the checkpoint update unexplained. Retaining only the incoming output or input subspace still leaves 45% and 42%, respectively. These are optimistic residuals because the corresponding classes may freely refit their remaining core and spectrum. By contrast, retaining the incoming spectrum while adapting both frames leaves a median residual of only 1.8%. Thus, among the tested structural restrictions, neither incoming singular subspace can be frozen while retaining a low-residual endpoint description: the spectrum can remain fixed, but both frame variables must remain adaptable. The same qualitative conclusion holds for the key transition 𝑊1 → 𝑊2 , where the incoming checkpoint has already undergone RL and Stage II optimizes a distinct embodied-manipulation objective. Thus, the frame-adaptability requirement is not confined to the first RL stage. It reappears after an objective shift and re-anchoring at an RL-trained checkpoint. Results for all transitions and truncation levels are reported in Appendix B.4. Take-away 2: fix the spectrum, but keep both frames adaptable. Even permissive remix and one-sided reconstruction classes leave much of the checkpoint update unexplained. Thus, among the tested restrictions, the spectrum can remain fixed but both frame variables must remain adaptable, directly motivating ISO’s parameterization 𝑊 (𝑈, 𝑉 ) = 𝑈Σ0𝑉 ⊤ .

4. From Spectral Inheritance to Isospectral Optimization (ISO) Sections 2–3 identify complementary constraints for algorithm design. Section 2 shows that the base spectra can be reused while RLVR gains are retained and acquired. Section 3 then shows that both singular frames must remain adaptable. We use this separation as an inductive bias rather than as a claim that unconstrained RLVR follows an exact fixed-spectrum update law. Motivated by this evidence, we introduce Isospectral Optimization (ISO), an RLVR-native optimization stack that reuses the base spectra and exposes both singular frames as adaptable coordinates for offline expert composition and online RLVR training.

ISO: An RLVR-Native Optimization Stack

4.1. The ISO Principle: Reuse the Spectrum, Optimize the Frames Inherit the spectrum, optimize the frames. For a base matrix 𝑊0 = 𝑈0 Σ0𝑉0⊤ , ISO represents the weight matrix through the fixed-spectrum parameterization 𝑊 (𝑈, 𝑉 ) = 𝑈Σ0𝑉 ⊤ ,

𝑈 ∈ St( 𝑑out , 𝑞) ,

𝑉 ∈ St( 𝑑in , 𝑞) ,

𝑞 = min{ 𝑑out , 𝑑in } .

(15)

Equation 15 parameterizes the fixed-spectrum family F (𝑊0 ) defined in Section 2. Thus, every represented matrix shares the base spectrum Σ0 . In exact arithmetic, 𝑊 (𝑈, 𝑉 ) ∈ F (𝑊0 ). In implementation, this property holds up to floating-point error. This parameterization directly implements the separation established in the preceding sections: it reuses the base spectrum Σ0 while leaving both frame variables (𝑈, 𝑉 ) adaptable for optimization or composition. An RLVR-native optimization stack. ISO is a framework rather than a single optimizer. The same fixed-spectrum frame parameterization supports two complementary stages of the RLVR workflow. Offline, ISO-Merger consolidates shared-base RL specialists directly from their checkpoints, without post-merge data, additional rollouts, gradient updates, or an on-policy distillation stage. Online, ISO-Optimizer applies a conventional base optimizer, such as AdamW or Muon, to the frame variables (𝑈, 𝑉 ) during RLVR training. Both instantiations reuse the base spectrum and restore frame feasibility after composition or optimization. Proposition 4.1 (First-order spectrum preservation). Let 𝑊 = 𝑈Σ0𝑉 ⊤ ∈ F (𝑊0 ), where the singular values in Σ0 are positive and simple. For any differentiable curve 𝑊 ( 𝑡 ) ∈ F (𝑊0 ) with 𝑊 (0) = 𝑊 and tangent ¤ := 𝑊 we have

𝑑 𝑊 (𝑡) , 𝑑𝑡 𝑡 =0

¤ = 0. diag 𝑈 ⊤𝑊𝑉

(16)

By contrast, for an arbitrary perturbation 𝐻 , 𝑑 𝜎𝑘 (𝑊 + 𝜀𝐻 ) = 𝑢⊤ 𝑘 𝐻𝑣𝑘 , 𝑑𝜀 𝜀=0

𝑘 = 1, . . . , 𝑞.

(17)

For Stiefel tangent directions 𝜉𝑈 ∈ 𝑇𝑈 St( 𝑑out , 𝑞) , 𝜉𝑉 ∈ 𝑇𝑉 St( 𝑑in , 𝑞), differentiating Equation 15 gives the induced weight-space direction ¤ = 𝜉𝑈 Σ0𝑉 ⊤ + 𝑈Σ0 𝜉𝑉⊤ . 𝑊

(18)

Because 𝑈 ⊤ 𝜉𝑈 and 𝑉 ⊤ 𝜉𝑉 are skew-symmetric, Equation 18 satisfies Equation 16. Thus, feasible frame motion preserves the spectrum to first order, while the finite reconstruction 𝑈Σ0𝑉 ⊤ preserves it exactly. The proof is given in Appendix C.1. 4.2. ISO-Merger: Merging RL Experts Without Rollouts Data-free composition of shared-base RL experts. Training a single RLVR policy across heterogeneous domains can be expensive and unstable. A modular alternative is to train several specialists from a shared base and then consolidate their capabilities into a single policy. One common consolidation strategy is an additional on-policy distillation stage driven by online rollouts [16, 17]. ISO-Merger instead addresses the strictly checkpoint-only setting: it requires no post-merge data, rollout generation, gradient updates, or distillation. It pursues the same broad objective as on-policy distillation: recovering complementary specialist capabilities in a single policy. 11

ISO: An RLVR-Native Optimization Stack

We focus on specialists trained from the same base checkpoint, with each expert targeting a distinct domain or capability. Merging already-generalist policies trained on broad, substantially overlapping mixtures constitutes a different setting that we do not study here. Setup. Let {𝑊𝑖 } 𝑖𝐾=1 be the corresponding matrices from 𝐾 RL experts fine-tuned from the same base checkpoint 𝑊0 . We write 𝑊0 = 𝑈0 Σ0𝑉0⊤ , 𝑊𝑖 = 𝑈𝑖 Σ𝑖𝑉𝑖⊤ . Many data-free merging methods construct 𝑊★ = Comp(𝑊1 , . . . , 𝑊𝐾 ; 𝑊0 ) by combining Euclidean task vectors. ISO-Merger instead retains each expert’s learned frames (𝑈𝑖 , 𝑉𝑖 ) while reusing the shared base spectrum Σ0 . Equivalently, it represents expert 𝑖 by the fixed-spectrum checkpoint 𝑈𝑖 Σ0𝑉𝑖⊤ ∈ F (𝑊0 )

and constructs

𝑊★ = 𝑈★ Σ0𝑉★⊤ .

(19)

The resulting matrix therefore belongs to the fixed-spectrum family F (𝑊0 ) up to numerical precision. Expert directions in shared frame coordinates. For simple singular values, each singular pair has a joint sign ambiguity, ( 𝑢𝑘 , 𝑣𝑘 ) ↦→ (−𝑢𝑘 , − 𝑣𝑘 ) . We first align each expert’s singular pairs with the corresponding base pairs using the sign-canonicalization rule described in Appendix E. After alignment, we define the frame displacements Δ𝑈𝑖 := 𝑈𝑖 − 𝑈0 ,

Δ𝑉𝑖 := 𝑉𝑖 − 𝑉0 ,

(20)

and project them onto the Stiefel tangent spaces at the shared base: 𝜉𝑈,𝑖 = Π𝑈0 ( Δ𝑈𝑖 ) ,

𝜉𝑉,𝑖 = Π𝑉0 ( Δ𝑉𝑖 ) ,

(21)

where Π𝑈0 and Π𝑉0 denote the Stiefel tangent projections defined in Appendix E. We find empirically that trailing modes, the columns associated with small singular values, are less stable across experts and that retaining them degrades the merged model. For a keep ratio 𝜌keep ∈ (0, 1], define

𝑘keep := round 𝜌keep 𝑞 , 𝐷keep := Diag

𝑞 1{ 𝑘 ≤ 𝑘keep } 𝑘=1 ,

(22)

e𝑈,𝑖 = 𝜉𝑈,𝑖 𝐷keep , 𝜉

e𝑉,𝑖 = 𝜉𝑉,𝑖 𝐷keep . 𝜉

(23)

and mask the trailing columns:

Unless stated otherwise, we use 𝜌keep = 0.9. e𝑈,𝑖 and 𝜉 e𝑉,𝑖 need not individually satisfy the Stiefel Masking is a coordinate-selection operation: 𝜉 tangent constraints. We use them to represent each expert’s change in the dominant shared frame coordinates and impose feasibility only after aggregating the experts. Because all displacements are anchored at the same base factors (𝑈0 , 𝑉0 ), their linear combinations are well-defined in a common coordinate system. A unit-retention target. Different expert directions may overlap or interfere. To measure these interactions, we linearize the reconstruction map 𝑊 (𝑈, 𝑉 ) = 𝑈Σ0𝑉 ⊤ at (𝑈0 , 𝑉0 ) and associate expert 𝑖 with the local first-order effect proxy e𝑈,𝑖 Σ0𝑉 ⊤ + 𝑈0 Σ0 𝜉 e⊤ ∈ ℝ𝑑out × 𝑑in . 𝑔𝑖 := 𝜉 𝑉,𝑖 0

We then form the Gram matrix

Γ𝑖 𝑗 := ⟨𝑔𝑖 , 𝑔 𝑗 ⟩𝐹 ,

Γ ∈ ℝ𝐾 × 𝐾 .

(24)

(25) 12

ISO: An RLVR-Native Optimization Stack

For coefficients 𝑐 ∈ ℝ 𝐾 , let 𝑔 ( 𝑐) :=

𝐾 ∑︁

(26)

𝑐𝑖 𝑔𝑖 .

𝑖=1

For each nonzero expert proxy 𝑔𝑖 , define its self-retention in the merged proxy by ret𝑖 ( 𝑐) :=

⟨𝑔 ( 𝑐) , 𝑔 𝑖 ⟩𝐹 ( Γ𝑐) 𝑖 = , Γ𝑖𝑖 ∥ 𝑔𝑖 ∥ 2𝐹

𝑖 = 1, . . . , 𝐾.

(27)

The ideal target ret𝑖 ( 𝑐) = 1 for every expert yields Γ𝑐 = 𝑏,

𝑏 := diag( Γ) ∈ ℝ 𝐾 .

Because Γ may be ill-conditioned and the exact solution may require unstable coefficients, we use the ridge-stabilized system Γ + 𝜆 ridge 𝐼 𝐾 ¯ 𝑐 = 𝑏,

𝑐★ = clip (¯ 𝑐; 𝑐min , 𝑐max ) .

(28)

The clipping bounds limit extreme coefficients and excessive amplification. The complete procedure therefore targets, rather than guarantees, unit self-retention after ridge stabilization, clipping, tangent projection, and retraction. Project, retract, and reconstruct. After combining the masked expert directions, we project only the aggregate displacement back onto the Stiefel tangent spaces: 𝜉𝑈,★ = Π𝑈0

𝐾 ∑︁

! e𝑈,𝑖 𝑐★𝑖𝜉

𝑖=1

𝜉𝑉,★ = Π𝑉0

𝐾 ∑︁

(29)

! e𝑉,𝑖 𝑐★𝑖𝜉

𝑖=1

For a thin SVD we define the polar factor by

𝑋 = 𝑃 𝑋 𝑆𝑋 𝑄 ⊤ 𝑋, polar( 𝑋 ) := 𝑃 𝑋 𝑄 ⊤𝑋 .

(30)

When 𝑋 has full column rank, this is equivalently polar( 𝑋 ) = 𝑋 ( 𝑋 ⊤ 𝑋 ) −1/2 . For rank-deficient 𝑋 , the SVD expression selects one valid nearest Stiefel factor, which need not be unique. For ISO-Merger, the retraction arguments are in fact full column rank. For example, tangent feasibility gives ⊤ (𝑈0 + 𝜉𝑈,★) ⊤ (𝑈0 + 𝜉𝑈,★) = 𝐼 + 𝜉𝑈,★ 𝜉𝑈,★ ≻ 0, and analogously for 𝑉0 + 𝜉𝑉,★. We then reconstruct the merged factors and weight matrix as 𝑈★ = polar(𝑈0 + 𝜉𝑈,★) , 𝑉★ = polar(𝑉0 + 𝜉𝑉,★) ,

(31)

⊤

𝑊★ = 𝑈★ Σ0𝑉★ .

Because 𝑈★ and 𝑉★ have orthonormal columns, 𝑊★ shares the base singular values Σ0 up to floatingpoint error. We apply the same construction to two-dimensional embedding and unembedding matrices. One-dimensional parameters, such as normalization scales and biases, are composed using a simple average. ISO-Merger thus performs checkpoint-only composition in shared fixed-spectrum coordinates, requiring no post-merge data, rollouts, gradient updates, or distillation. Full algorithmic details are provided in Appendix E. 13

ISO: An RLVR-Native Optimization Stack

4.3. ISO-Optimizer: Online Fixed-Spectrum RLVR Training A fixed-spectrum transformation of a base optimizer. ISO-Optimizer does not replace the numerical update rule used by standard RLVR. Instead, it applies that rule to the singular-frame variables under a fixed base spectrum. Let Opt( 𝑋, 𝐺, 𝑠) denote one update of a base optimizer, such as AdamW or Muon, where 𝑋 is the optimized variable, 𝐺 is its gradient, and 𝑠 is the corresponding optimizer state. Standard weight-space training applies + (𝑊 + , 𝑠𝑊 ) = Opt (𝑊, 𝐺𝑊 , 𝑠𝑊 ) ,

𝐺𝑊 = ∇𝑊 L (𝑊 ) .

(32)

ISO[Opt] instead uses the fixed-spectrum parameterization 𝑊 (𝑈, 𝑉 ) = 𝑈Σ0𝑉 ⊤ ,

(33)

where Σ0 is inherited from the base matrix 𝑊0 = 𝑈0 Σ0𝑉0⊤ and remains fixed throughout training. The base optimizer supplies the update equations and state machinery, while ISO supplies the frame parameterization and the feasibility retraction. Factor gradients and tentative updates. weight matrix,

Given the gradient with respect to the reconstructed

𝐺𝑊 = ∇𝑊 L 𝑈Σ0𝑉 ⊤ ,

the Euclidean gradients with respect to the frame variables follow from the chain rule: ⊤ 𝐺𝑉 = 𝐺𝑊 𝑈Σ0 .

𝐺𝑈 = 𝐺𝑊 𝑉 Σ0 ,

(34)

A detailed derivation is provided in Appendix C.2. The factors Σ0 arise from the Jacobian of the reconstruction map: the same frame displacement produces a larger weight-space change for a mode with a larger singular value. The base optimizer applies its update rule independently to the two frame variables and their associated states: + ¯ 𝑠𝑈 ) = Opt (𝑈, 𝐺𝑈 , 𝑠𝑈 ) , (𝑈, (𝑉¯, 𝑠𝑉+ ) = Opt (𝑉, 𝐺𝑉 , 𝑠𝑉 ) .

(35)

The tentative factors 𝑈¯ and 𝑉¯ need not satisfy the Stiefel constraints. ISO restores feasibility using the polar retraction defined in Equation 30: ¯), 𝑈 + = polar(𝑈 𝑉 + = polar(𝑉¯) , +

(36)

+ ⊤

𝑊 = 𝑈 Σ0 ( 𝑉 ) .

In exact arithmetic, 𝑊 + ∈ F (𝑊0 ). In implementation, its singular values match Σ0 up to floating-point error. A generic optimizer transformation.

This construction defines Opt ↦−→ ISO[Opt] .

(37)

Using AdamW as the base update rule gives ISO-AdamW, while using Muon gives ISO-Muon. Both variants share the same fixed-spectrum frame parameterization and retraction, while inheriting the update equations and optimizer-state dynamics of their respective base rules. AdamW is our primary instantiation because it is the standard optimizer in the RLVR recipes studied here. ISO-Muon tests whether the same construction transfers across base update rules. Then, the central comparison is AdamW on 𝑊

versus

ISO[AdamW] on (𝑈, 𝑉 ) with fixed Σ0 .

(38) 14

ISO: An RLVR-Native Optimization Stack

Importantly, ISO[Opt] is not an Opt update on 𝑊 followed by projection onto F (𝑊0 ). The update rule and optimizer states are instantiated directly in the frame coordinates, changing the optimized variables, the effective factor-space geometry, and the feasible training trajectory. Although ISO introduces two factor variables, it does not enlarge the represented weight-space class. After every retraction, the reconstructed weight remains in the strictly more constrained fixedspectrum family F (𝑊0 ). Thus, the observed gains cannot be attributed to a larger feasible model class, although the factor-space parameterization and optimizer dynamics are also part of the method. Alternative spectrum-preserving optimizers formulated directly in weight space are also possible. Systematically comparing such formulations with factor-space ISO is left to future work. Weight decay. We set weight decay to zero for both the weight-space and ISO variants. Empirically, it provides no measurable benefit in our RLVR runs. At learning rates of order 10−6 , the standard coefficient 𝜆 = 10−2 induces only 𝜂𝜆 ≈ 10−8 relative shrinkage per step, below the BF16-visible update scale analyzed in prior work [1]. Implementation and overhead. The main additional structured operation in ISO-Optimizer is the polar retraction applied after each factor update. We implement it using an FP64 SVD-based polar computation for numerical stability. Numerical precision diagnostics are reported in Appendix G. Following a DION-style distributed implementation [22], we distribute the matrix retractions across multiple GPUs. In our RLVR profiling, the measured retraction cost corresponds to approximately 7% of the end-to-end step time, as reported in Section 5.2. In modern asynchronous RL systems, this non-dominant optimizer work is amenable to overlap with rollout generation, particularly in long-horizon and agentic settings where environment interaction can be substantially more expensive. Representing each two-dimensional weight through both 𝑈 and 𝑉 increases the nominal storage for trainable tensors and optimizer states relative to a single dense matrix. The fixed spectrum Σ0 requires no optimizer state. In our implementation, the factor tensors are fully sharded, and optimizer-state offloading can further reduce the per-device memory footprint. Given the observed optimization and training-step savings, these overheads are manageable in our setting and motivate further infrastructure optimization for both retraction scheduling and factor-state storage.

5. Experiments We evaluate ISO in two settings corresponding to its two algorithmic instantiations. First, we test whether ISO-Merger can compose RLVR experts without additional rollouts while preserving their specialist capabilities, pursuing an objective similar to that of on-policy distillation. Second, we test whether ISO-Optimizer improves RLVR convergence when applied to base optimizers such as AdamW and Muon. 5.1. ISO-Merger: Composing Heterogeneous RL Experts Without Rollouts Setup. We evaluate ISO-Merger in the setting it is designed for: composing multiple RLVR specialists from a shared base without post-merge data, rollouts, or distillation. We consider two expertcomposition settings. The first uses Qwen2.5-7B-Instruct [23] as the shared base and merges three RL experts specialized for coding [24], tool use [25], and long-context memory [26]. The second uses DeepSeek-R1-Distill-Qwen-1.5B as the shared base and merges two RL experts specialized for coding [32] and math [33]. We compare against representative training-free merging baselines: Task Arithmetic [27], TIES [28], TSV-Merge [29], RAM [30], and the more recent OrthoMerge-G-TIES [31]. All methods merge the same expert checkpoints and are evaluated directly after merging. Unless otherwise stated, generation results use average@16, and stochastic evalua15

ISO: An RLVR-Native Optimization Stack

Table 1 | Main results on Qwen2.5-7B-Instruct with 3 RL experts. Coding: pass accuracy (ACC) and unit-test accuracy (UT) on LiveBench (LB) and LiveCodeBench (LCB). Tool Use: Live and Non-Live averages following the BFCL v4 evaluation protocol. Memory: RULER HotpotQA and SQuAD at 32K–64K context lengths under the Recurrent setting. Best expert/merged result per column in bold. Shading indicates Experts and Ours . Coding

Tool Use

LB Method

LCB v2

Memory

Live

Non-Live

HotpotQA

SQuAD

ACC

32K

64K

32K

64K

Avg

Base [23] RLVR-Coder [24] RLVR-Tool [25] RLVR-Memory [26]

36.25 ± 0.86 37.42 ± 1.22 36.87 ± 1.79 37.32 ± 1.37

48.27 ± 1.44 49.42 ± 1.60 48.08 ± 0.82 50.05 ± 1.61

28.31 ± 0.51 30.17 ± 0.99 28.20 ± 1.95 27.41 ± 2.99

43.08 ± 0.82 44.24 ± 0.99 42.43 ± 1.90 41.70 ± 4.63

62.96 63.37 71.83 63.82

69.26 68.07 81.82 72.61

49.87 ± 1.45 50.21 ± 0.16 50.39 ± 0.68 78.60 ± 0.54

45.54 ± 0.58 45.61 ± 0.33 45.33 ± 0.26 77.95 ± 0.29

58.90 ± 1.07 60.51 ± 0.70 59.73 ± 1.18 79.98 ± 0.38

56.77 ± 0.61 56.84 ± 0.42 57.86 ± 0.52 78.52 ± 0.22

49.92 50.59 52.25 60.80

Task Arithmetic [27] TIES [28] TSV [29] RAM [30] OrthoMerge-G-TIES [31]

38.85 ± 0.70 52.18 ± 0.40 40.30 ± 2.18 52.04 ± 2.50 36.96 ± 0.86 49.23 ± 0.59 38.59 ± 0.79 50.38 ± 1.53 40.74 ± 2.00 53.22 ± 1.90

31.55 ± 0.18 31.46 ± 0.35 28.92 ± 1.79 31.31 ± 0.28 31.98 ± 0.84

47.06 ± 0.58 47.33 ± 0.30 43.37 ± 3.25 46.33 ± 0.30 47.48 ± 0.77

73.73 68.24 73.64 72.56 66.98

82.86 76.54 82.69 81.89 78.17

72.22 ± 0.22 76.42 ± 0.36 75.37 ± 0.29 76.17 ± 0.20 74.79 ± 0.49

70.49 ± 0.10 75.24 ± 0.14 74.10 ± 0.30 75.29 ± 0.49 74.20 ± 0.49

75.03 ± 0.29 78.27 ± 0.46 78.30 ± 0.17 78.42 ± 0.47 77.75 ± 0.12

74.95 ± 0.32 77.86 ± 0.59 77.26 ± 0.78 77.83 ± 0.46 76.32 ± 0.21

61.89 62.37 61.98 62.88 62.16

ISO-Merger

41.81 ± 2.00 52.73 ± 1.57

31.06 ± 1.88

45.69 ± 3.06 72.32

81.07

79.46 ± 0.78 76.77 ± 0.27 79.10 ± 0.14 78.01 ± 0.32 63.80

Table 2 | Main results on DeepSeek-R1-Distill-Qwen-1.5B with 2 RL experts. Coding: pass accuracy (ACC) and unit-test accuracy (UT) on LiveBench (LB) and LiveCodeBench (LCB). Math: ACC on AIME 2024, AIME 2025, AMC 2023, Minerva, and OlympiadBench. Avg is the average over all columns. Best expert/merged result per column in bold. Shading indicates Experts and Ours . Coding

Math

Minerva

OlympiadBench

ACC

Avg@32

Avg@8

Avg@4

Base [4] Archer2.0 [32] JustRL [33]

17.99 ± 0.29 26.66 ± 0.45 22.23 ± 0.23

26.24 ± 0.53 37.26 ± 0.55 31.85 ± 0.74

16.82 ± 0.28 26.90 ± 0.21 23.05 ± 0.21

20.85 ± 0.18 38.99 ± 0.37 30.18 ± 0.31

30.90 ± 1.15 42.05 ± 0.43 53.16 ± 0.44

23.40 ± 0.86 28.02 ± 0.44 36.84 ± 0.73

63.15 ± 0.38 73.44 ± 0.50 82.78 ± 0.51

27.33 ± 0.43 30.09 ± 0.23 34.71 ± 0.50

43.11 ± 0.26 49.99 ± 0.65 55.67 ± 0.42

29.98 39.27 41.16

Task Arithmetic [27] TIES [28] TSV [29] RAM [30] OrthoMerge-G-TIES [31]

26.22 ± 0.22 26.37 ± 0.73 26.40 ± 0.40 26.71 ± 0.29 26.46 ± 0.62

35.82 ± 0.11 36.75 ± 0.90 36.83 ± 0.15 36.99 ± 0.47 36.73 ± 0.37

26.70 ± 0.21 27.62 ± 0.13 27.61 ± 0.17 27.18 ± 0.63 27.74 ± 0.30

36.58 ± 0.32 39.45 ± 0.15 40.01 ± 0.26 38.83 ± 0.67 39.86 ± 0.64

50.83 ± 0.39 54.51 ± 0.72 53.16 ± 0.57 54.20 ± 0.55 54.55 ± 1.44

33.09 ± 1.16 35.76 ± 0.30 35.17 ± 0.72 35.80 ± 0.71 35.14 ± 0.87

80.82 ± 0.51 82.13 ± 0.14 80.97 ± 0.92 82.28 ± 0.38 81.88 ± 0.63

33.00 ± 0.34 33.76 ± 0.48 33.88 ± 0.46 34.10 ± 0.34 33.55 ± 0.42

54.31 ± 0.15 55.36 ± 0.15 54.68 ± 0.27 55.54 ± 0.25 55.47 ± 0.24

41.93 43.52 43.19 43.51 43.49

ISO-Merger

25.52 ± 0.08

36.42 ± 0.55

27.84 ± 0.13

41.84 ± 0.31

55.00 ± 0.52

37.81 ± 0.31

83.53 ± 1.02

34.77 ± 0.23

56.64 ± 0.05

44.38

Method

LCB v5

AIME 2024

AIME 2025

AMC 2023

Avg

tions are repeated over three independent runs with mean and standard deviation reported. Full benchmark protocols, decoding settings, baseline hyperparameters, and best@4/worst@4 results are provided in Appendix F. Results. Tables 1 and 2 show that ISO-Merger achieves the strongest overall data-free composition performance in both settings. On Qwen2.5-7B, ISO-Merger reaches an overall average of 63.80, compared with 62.88 for the strongest training-free baseline. On DeepSeek-R1-Distill-Qwen-1.5B, it reaches 44.38, compared with 43.52 for the strongest baseline. Appendix F shows that ISO-Merger essentially matches the strongest aggregate best@4 baseline under both backbones while improving worst@4 by 1.62 and 1.36 points, respectively. Thus, the mean improvements do not sacrifice upper-tail performance, and the worst@4 results suggest more consistent capability recovery across stochastic generations. Overall, ISO-Merger remains competitive with the original specialists while consolidating multiple capabilities into a single model. These results support the practical utility of spectral inheritance for shared-base expert composition: specialist capabilities can be combined in fixed-spectrum frame coordinates while reusing the shared base spectrum. 5.2. ISO-Optimizer: Optimizing in Fixed-Spectrum Coordinates Improves RLVR Setup and evaluation. We evaluate ISO-Optimizer by applying conventional base optimizers in fixed-spectrum frame coordinates rather than directly in weight space. All runs use Verl [9]. For mathematical reasoning, we train Qwen3-1.7B-Base and Qwen3-4B-Base [34] on DeepMath16

ISO: An RLVR-Native Optimization Stack

Table 3 | Math RLVR results. We report average@16 accuracy. Because evaluations remain noisy near convergence, for each run we select the checkpoint with the highest aggregate score among the final three evaluations and report all benchmark scores from that checkpoint. Model

Method

AIME24

AIME25

AMC23

Minerva

Olympiad

Avg.

Qwen3-1.7B-Base

Base AdamW Muon ISO-AdamW

3.75 13.96 16.25 16.04

1.25 12.92 8.33 11.04

23.04 43.45 43.60 44.50

18.49 32.17 32.24 33.11

18.43 39.25 38.06 39.01

12.99 28.35 27.70 28.74

Qwen3-4B-Base

Base AdamW Muon ISO-AdamW

6.88 25.21 25.42 27.29

5.00 20.42 19.58 23.13

25.45 63.55 63.63 65.74

20.97 45.40 46.51 45.15

22.43 53.87 56.02 55.99

16.15 41.69 42.23 43.46

0.45

acc=0.42

Accuracy (average@16)

0.40 0.35 0.30

2.2× fewer training steps

0.25 0.20 0.15

ISO-AdamW lr 7.5e-7 AdamW lr 3e-6 AdamW lr 2e-6

100

200

AdamW lr 1e-6 AdamW lr 7.5e-7 gain over best AdamW

Training step

300

400

Figure 6 | Aggregate math accuracy during RLVR training. ISO-AdamW uses a single learning rate of 7.5 × 10−7 , whereas the weight-space AdamW baselines are tuned over the sweep described in the text. Left (Qwen3-1.7B-Base): ISO-AdamW attains the highest final accuracy. Right (Qwen3-4B-Base): ISO-AdamW matches the final aggregate accuracy of the strongest AdamW run while reducing the number of training steps by a factor of approximately 2.2, and continues to improve thereafter. 103K [35] for 400 training steps with a global batch size of 256 and a mini-batch size of 256. Because the actor batch and PPO mini-batch have the same size and we use one PPO epoch, each actor update corresponds to a single training step. The 1.7B and 4B runs use 16 and 12 rollouts per prompt, respectively, together with online filtering of all-correct and all-incorrect prompt groups. ISO-AdamW uses a single learning rate of 7.5 × 10−7 for (𝑈, 𝑉 ).2 For the weight-space baselines, we sweep AdamW learning rates over {5 × 10−7 , 7.5 × 10−7 , 1 × 10−6 , 2 × 10−6 , 3 × 10−6 } and Muon learning rates over {2.5 × 10−5 , 5 × 10−5 , 7.5 × 10−5 , 1 × 10−4 } . We evaluate AIME 2024, AIME 2025, AMC 2023, Minerva, and OlympiadBench using 16 samples per problem. Evaluations are performed every 20 training Table 4 | Coding results on DS-1.5B. We steps (10 training steps on 8B) under identical decoding report average@8 LiveCodeBench accuracy settings. Full training and evaluation details are provided after 220 training steps. in Appendix H. Method

LCB v5

LCB v6

Avg.

Math results. Table 3 shows that ISO-AdamW achieves DS-1.5B 17.43 18.41 17.92 AdamW 24.01 26.24 25.13 the strongest aggregate score at both model scales. On ISO-AdamW 25.49 26.91 26.20 Qwen3-1.7B-Base, it reaches 28.74, compared with 28.35 for the best AdamW run and 27.70 for Muon. On Qwen3-4B-Base, it reaches 43.46, compared 2On Qwen3-1.7B-Base, we compare 7.5 × 10 −7 with 1 × 10 −6 and use the better-performing setting throughout the math experiments.

ISO: An RLVR-Native Optimization Stack

with 41.69 for AdamW and 42.23 for Muon.

Accuracy (average@8)

Figure 6 shows that these gains emerge early rather than only at the selected endpoint. At 1.7B, ISO-AdamW attains the highest final aggregate accuracy. At 4B, it reaches the strongest AdamW run’s final accuracy continue AdamW runs for 110 updates 0.28 to rule out under-training with approximately 2.2× fewer training steps and contin0.26 ues to improve thereafter. These gains are obtained with0.24 out enlarging the feasible weight-space model class: every 0.22 ISO iterate remains in the constrained fixed-spectrum family F (𝑊0 ). 0.20 0.18 Coding results. Beyond math reasoning, we train ISO-AdamW lr 1e-6 AdamW lr 2e-6 0.16 AdamW lr 5e-6 gain over best AdamW DS-1.5B on ArcherCodeR using a DAPO-style recipe AdamW lr 3e-6 0.14 and evaluate on LiveCodeBench v5 and v6. The ISO0 100 200 300 AdamW run is limited to 220 training steps, by which Training step point the validation curve has plateaued. Further training Figure 7 | Coding accuracy on DS-1.5B. would require repeated passes over the relatively small ISO-AdamW leads the tuned AdamW basecoding set and may exacerbate overfitting. Table 4 shows lines on LiveCodeBench. that ISO-AdamW improves over AdamW on both LiveCodeBench splits. Figure 7 shows the corresponding training curves: ISO-AdamW outperforms the strongest AdamW setting (5 × 10−6 ) at most evaluation checkpoints. To determine whether longer training closes the gap, we extend the two strongest AdamW runs to 330 steps. The 5 × 10−6 run peaks at 0.265 and then declines, while the 3 × 10−6 run ends at 0.256, a level that ISO-AdamW reaches by step 130. Neither extended baseline matches ISO-AdamW’s peak accuracy of 0.268 within its 220-step budget. Full details are provided in Appendix H. 0.45

acc=0.42

0.40

Accuracy (average@16)

ISO-Muon variant. Because ISO is agnostic to the base optimizer, we also instantiate ISO-Muon, which applies the Muon update in fixed-spectrum Stiefel coordinates. Figure 8 compares ISO-Muon, using a learning rate of 5 × 10−5 , with a Muon sweep over {5×10−5 , 7.5×10−5 , 10−4 } on Qwen3-4B-Base. We train all runs for 300 training steps. Muon with a learning rate of 2.5 × 10−5 performs strictly worse and is omitted from the figure for clarity. The qualitative pattern in Figure 6 persists: ISO-Muon matches the final accuracy of the strongest Muon run (0.422 at step 300) by step 220 and attains a higher final accuracy (0.428 versus 0.422). We omit ISO-Muon from Table 3 because it uses the shorter 300-step budget.

0.35 0.30

1.4× fewer training steps

0.25 0.20 0.15

ISO-Muon lr 5e-5 Muon lr 1e-4 Muon lr 7.5e-5

100

Muon lr 5e-5 gain over best Muon

150

200

Training step

250

300

Figure 8 | ISO-Muon versus Muon on Qwen3-4B-Base. ISO-Muon reaches the

Scaling to larger models. Beyond the 1.5B–4B mod- strongest Muon baseline’s final accuracy with els considered above, we test whether ISO’s benefits 220 rather than 300 training steps and finextend to a larger scale. We train Qwen3-8B-Base ishes with higher accuracy. on DeepMath-103K [35] using a global batch size of 256. We compare ISO-AdamW, with a learning rate of 7.5 × 10−7 , against weightspace AdamW using 2 × 10−6 , the strongest AdamW learning rate identified in the smaller-model sweeps. Neither method is further tuned at 8B. Training begins with an 8K maximum response length. At this scale, ISO-AdamW produces noticeably longer responses than AdamW, so we increase the cap to 16K for both runs after step 80 to reduce truncation. Evaluation uses a 16K cap throughout. As shown in Figure 9, ISO-AdamW reaches 0.509 after 210 training steps, whereas AdamW reaches 0.487 under the same training-step budget. To test whether AdamW is simply

ISO: An RLVR-Native Optimization Stack

Accuracy (average@16)

0.5

acc=0.495

continue AdamW for 60 updates to rule out under-training

0.4

2.7× fewer training steps

0.3

ISO-AdamW lr 7.5e-7 AdamW lr 2e-6 gain over AdamW

0.2 0

train: 8k max len

100

150

200

train: 16k max len

250

Training step

Figure 9 | Math accuracy on Qwen3-8B-Base. Shading marks the 8K and 16K max response-length phases in training. ISO-AdamW remains above AdamW and reaches its final accuracy earlier.

undertrained, we continue it for 60 additional training steps. Its accuracy improves only to 0.495 and then plateaus, whereas ISO-AdamW reaches the same level by step 100. This corresponds to a reduction in the training-step count by a factor of approximately 2.7. The persistent gap across evaluation checkpoints, obtained without additional tuning at 8B, suggests that the benefits of fixed-spectrum coordinates extend to the larger model scale. Retraction overhead. ISO introduces additional optimizer-update cost through the SVD-based polar retraction. Figure 10 profiles this cost on a representative Qwen3-4B-Base RLVR run with an 8K context window. Relative to AdamW, ISO increases the optimizer-update time by approximately 86 seconds per step, but this increase accounts for only about 7% of the end-to-end RL step time in this setting. The end-to-end runtime remains dominated by other parts, e.g., rollout generation, rather than by the optimizer update. The difference in Figure 10 | Optimizer-update and end-tototal step time also grows because ISO produces longer end training-step time for ISO and AdamW responses, thereby increasing rollout latency. This cost on Qwen3-4B-Base. profile differs from that of pre-training, where optimizer computation lies directly on the critical path of each step. RLVR includes a substantially longer generation phase, providing additional opportunities to potentially amortize the overhead or overlap actor-side computation with rollout generation, especially in modern asynchronous RL systems.

6. Related Work RLVR dynamics and optimization. Most advances in RLVR have focused on data and environments, learning objectives, or systems infrastructure. By contrast, the optimization layer that translates reward feedback into parameter-space motion remains comparatively underexplored and is typically inherited from pre-training, including the choice of optimizer and parameterization. Recent analyses have begun to distinguish RLVR from pre-training and supervised fine-tuning at both the policy and parameter levels. At the policy level, RLVR has been characterized through KL-proximal and conservative policyimprovement views. At the parameter level, its updates have been reported to be sparse, off-principal, and strongly shared across runs initialized from the same source model [18, 14, 15, 1]. Building on our prior observation of limited spectral drift in RLVR [1], we formalize and test spectral inheritance: 19

ISO: An RLVR-Native Optimization Stack

the base model’s singular-value spectra can remain functionally reusable while the associated input and output frames adapt. We then use this principle to guide RLVR optimization algorithm design. Matrix-geometric optimization and constrained parameterizations. A growing line of work treats matrix-valued parameters and updates as structured objects rather than as flattened Euclidean vectors. These methods act on different objects and impose different forms of structure. Muon orthogonalizes matrix-valued momentum updates, while DION develops scalable distributed approximations to such orthonormalized updates [11, 22]. Hyperball constrains the Frobenius norms of both weight matrices and optimizer updates, making the relative angular step size explicit rather than controlling it indirectly through weight decay [13]. The Modular Manifolds perspective advocates co-designing module-specific constraints and update norms, including Stiefel-constrained weights and a manifold version of Muon [36]. POET and POET-X preserve weight spectra through two-sided orthogonalequivalence reparameterizations, with an emphasis on stable and efficient LLM training [37, 38]. Related structure has also been used for parameter-efficient adaptation: Spectral Adapter modifies a leading spectral subspace, while StelLA learns orthonormal input and output subspaces for a low-rank adapter [39, 40]. An independent concurrent work Pion [41] also preserves weight spectra through left and right orthogonal-equivalence transformations. Pion is introduced as a spectrum-preserving optimizer for general LLM training, and further adapts its construction to RLVR, where its motivation explicitly draws on the spectral observation from our earlier work [1]. The overlap with ISO is therefore most direct in online RLVR optimization. The two approaches nevertheless differ in both construction and scope. Pion updates each weight matrix directly through a dedicated Lie-algebra update rule, together with its own scale control, momentum design, and approximate exponential map. ISO instead fixes the base spectrum, exposes the associated singular frames as optimization variables, and applies a chosen base optimizer, including AdamW or Muon, to those variables, followed by retraction. More fundamentally, Pion begins from spectrum preservation as an optimizer design principle for general LLM training, with its construction motivated by pre-training stability, scale control, and orthogonalequivalence geometry. ISO begins from an RLVR-specific empirical and functional characterization: we identify spectral inheritance, challenge raw near-isospectrality through dimension-aware calibration, and test base-spectrum reuse through functional interventions and sequential RL stages. We then promote this evidence to an RLVR-native optimization framework rather than a single optimizer. Its online instantiation transforms conventional base optimizers, while its offline instantiation provides a checkpoint-only method for composing shared-base RL experts. From matrix-geometric priors to an RLVR-native post-training stack. Taken together, these works demonstrate the value of matrix-geometric structure in optimization. Our contribution is not the first use of orthogonality, manifolds, or spectral constraints. Rather, ISO follows an evidence-to-design route tailored to RLVR. We begin by studying unconstrained RLVR and show, through dimensionaware calibration, that raw near-isospectrality alone is not discriminative. We then test whether the learned spectral changes are functionally necessary, whether the base spectrum can remain fixed throughout learning, and how the remaining checkpoint change is organized in the associated singular frames. The same learning pattern recurs across sequential RL stages with distinct objectives. Together, these analyses establish spectral inheritance: RLVR can reuse the source model’s weight spectra while acquiring new behavior through changes in the associated input and output singular frames. ISO promotes this empirically and functionally supported regularity from a descriptive observation to an explicit RLVR inductive bias. ISO uses this inductive bias to redesign the RLVR post-training stack around a common fixedspectrum principle. Rather than tying the idea to a particular optimizer or to one singular-frame reparameterization, ISO treats spectral inheritance as a reusable interface across different stages of

ISO: An RLVR-Native Optimization Stack

post-training. During online RLVR, it governs how reward-driven updates are represented and executed under the inherited spectra. After training, it provides a shared coordinate system for consolidating independently trained RL experts without additional rollouts or distillation. ISO-Optimizer and ISO-Merger are therefore not two unrelated algorithms, but two complementary realizations of the same stack-level design, spanning online policy learning and offline expert consolidation. The singular-frame construction studied here is one numerical realization of this principle rather than the definition of ISO itself. Alternative enforcement mechanisms and further applications within the same framework, including geometry-aware adapter initialization, adapter training, and adapter composition, remain open directions. In summary, ISO contributes an evidence-to-stack design: it identifies spectral inheritance from RLVR’s own dynamics, validates its functional relevance, and turns it into a common optimization principle spanning online policy learning and offline expert consolidation. Geometry-aware model merging. Most training-free merging methods combine Euclidean task vectors, with techniques such as Task Arithmetic, TIES, and TSV-Merge reducing interference through sign resolution or low-rank structure [27, 28, 29]. OrthoMerge instead merges orthogonal fine-tuning transformations in a Lie algebra and extends to generally fine-tuned models by separating orthogonal and residual components [31]. ISO-Merger is specialized to shared-base RL experts: it projects each expert to the geometry of the shared base spectrum, represents its frame displacement in a common base-anchored Stiefel tangent space, and performs checkpoint-only composition without post-merge prompts, rollouts, teacher queries, or optimization.

7. Conclusion We identify spectral inheritance as a recurring structure in RLVR: the base model’s weight spectra remain functionally reusable while new behavior is acquired through changes in the associated singular frames. Restoring the base spectra after training preserves most acquired performance, and keeping them fixed throughout training still supports strong reasoning and coding gains. Among the structural restrictions tested, simpler subspace-remixing and one-sided transformations leave substantially more of the checkpoint change unexplained, indicating that both frames must remain adaptable. This pattern also recurs across sequential RL stages with distinct objectives. Building on this separation, we introduce Isospectral Optimization (ISO). ISO-Merger composes shared-base RL experts without post-merge data, rollouts, gradient updates, or distillation, achieving the strongest aggregate performance among the compared data-free methods. ISO-Optimizer applies conventional base optimizers to the frame variables under fixed base spectra, improving aggregate accuracy and reaching matched accuracy in fewer training steps, including 2.7× fewer steps on Qwen3-8B-Base. Together, these results identify the spectral structure learned before RL as a reusable substrate for post-training: RLVR can inherit the spectrum and learn how it acts.

References [1] Hanqing Zhu, Zhenyu Zhang, Hanxian Huang, DiJia Su, Zechun Liu, Jiawei Zhao, Igor Fedorov, Hamed Pirsiavash, Zhizhou Sha, Jinwon Lee, et al. The path not taken: Rlvr provably learns off the principals. arXiv preprint arXiv:2511.08567, 2025. [2] xAI. Grok: Ai assistant, 2025. Accessed: 2025-09-24, continuously updated. [3] Scale AI. The next frontier of data training: Rl environments, February 2026.

ISO: An RLVR-Native Optimization Stack

[4] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. [5] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale, 2025. URL https://arxiv. org/abs/2503.14476, 1:2, 2025. [6] Jian Hu, Jason Klein Liu, Haotian Xu, and Wei Shen. Reinforce++: Stabilizing critic-free policy optimization with global advantage normalization. arXiv preprint arXiv:2501.03262, 2025. [7] Fahim Tajwar, Guanning Zeng, Yueer Zhou, Yuda Song, Daman Arora, Yiding Jiang, Jeff Schneider, Ruslan Salakhutdinov, Haiwen Feng, and Andrea Zanette. Maximum likelihood reinforcement learning. arXiv preprint arXiv:2602.02710, 2026. [8] Zilin Zhu, Chengxing Xie, Xin Lv, and slime Contributors. slime: An llm post-training framework for rl scaling. https://github.com/THUDM/slime, 2025. GitHub repository. Corresponding author: Xin Lv. [9] Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, pages 1279–1297, 2025. [10] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations, 2019. [11] Jingyuan Liu, Jianlin Su, Xingcheng Yao, Zhejun Jiang, Guokun Lai, Yulun Du, Yidao Qin, Weixin Xu, Enzhe Lu, Junjie Yan, et al. Muon is scalable for llm training. arXiv preprint arXiv:2502.16982, 2025. [12] Hanqing Zhu, Zhenyu Zhang, Wenyan Cong, Xi Liu, Sem Park, Vikas Chandra, Bo Long, David Z Pan, Zhangyang Wang, and Jinwon Lee. Apollo: Sgd-like memory, adamw-level performance. Proceedings of Machine Learning and Systems, 7, 2025. [13] Kaiyue Wen, David Hall, Tengyu Ma, and Percy Liang. Fantastic pretraining optimizers and where to find them. arXiv preprint arXiv:2509.02046, 2025. [14] Idan Shenfeld, Jyothish Pari, and Pulkit Agrawal. Rl’s razor: Why online reinforcement learning forgets less. arXiv preprint arXiv:2509.04259, 2025. [15] Sagnik Mukherjee, Lifan Yuan, Dilek Hakkani-Tur, and Hao Peng. Reinforcement learning finetunes small subnetworks in large language models. arXiv preprint arXiv:2505.11711, 2025. [16] Kevin Lu and Thinking Machines Lab. On-policy distillation. Thinking Machines Lab: Connectionism, 2025. https://thinkingmachines.ai/blog/on-policy-distillation. [17] Aohan Zeng, Xin Lv, Zhenyu Hou, Zhengxiao Du, Qinkai Zheng, Bin Chen, Da Yin, Chendi Ge, Chenghua Huang, Chengxing Xie, et al. Glm-5: from vibe coding to agentic engineering. arXiv preprint arXiv:2602.15763, 2026. [18] Fang Wu, Weihao Xuan, Ximing Lu, Mingjie Liu, Yi Dong, Zaid Harchaoui, and Yejin Choi. The invisible leash: Why rlvr may or may not escape its origin. arXiv preprint arXiv:2507.14843, 2025.

ISO: An RLVR-Native Optimization Stack

[19] Mingjie Liu, Shizhe Diao, Ximing Lu, Jian Hu, Xin Dong, Yejin Choi, Jan Kautz, and Yi Dong. Prorl: Prolonged reinforcement learning expands reasoning boundaries in large language models. arXiv preprint arXiv:2505.24864, 2025. [20] Jian Hu, Mingjie Liu, Ximing Lu, Fang Wu, Zaid Harchaoui, Shizhe Diao, Yejin Choi, Pavlo Molchanov, Jun Yang, Jan Kautz, et al. Brorl: Scaling reinforcement learning via broadened exploration. arXiv preprint arXiv:2510.01180, 2025. [21] Yifu Yuan, Haiqin Cui, Yaoting Huang, Yibin Chen, Fei Ni, Zibin Dong, Pengyi Li, Yan Zheng, and Jianye Hao. Embodied-r1: Reinforced embodied reasoning for general robotic manipulation. arXiv preprint arXiv:2508.13998, 2025. [22] Kwangjun Ahn, Byron Xu, Natalie Abreu, Ying Fan, Gagik Magakyan, Pratyusha Sharma, Zheng Zhan, and John Langford. Dion: Distributed orthonormalized updates. arXiv preprint arXiv:2504.05295, 2025. [23] Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2025. [24] Yinjie Wang, Ling Yang, Ye Tian, Ke Shen, and Mengdi Wang. Co-evolving llm coder and unit tester via reinforcement learning. arXiv preprint arXiv:2506.03136, 2025. [25] Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani-Tür, Gokhan Tur, and Heng Ji. Toolrl: Reward is all tool learning needs. arXiv preprint arXiv:2504.13958, 2025. [26] Hongli Yu, Tinghong Chen, Jiangtao Feng, Jiangjie Chen, Weinan Dai, Qiying Yu, Ya-Qin Zhang, Wei-Ying Ma, Jingjing Liu, Mingxuan Wang, et al. Memagent: Reshaping long-context llm with multi-conv rl-based memory agent. arXiv preprint arXiv:2507.02259, 2025. [27] Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089, 2022. [28] Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. Ties-merging: Resolving interference when merging models. Advances in neural information processing systems, 36:7093–7115, 2023. [29] Antonio Andrea Gargiulo, Donato Crisostomi, Maria Sofia Bucarelli, Simone Scardapane, Fabrizio Silvestri, and Emanuele Rodola. Task singular vectors: Reducing task interference in model merging. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 18695–18705, 2025. [30] Xiangchi Yuan, Dachuan Shi, Chunhui Zhang, Zheyuan Liu, Shenglong Yao, Soroush Vosoughi, and Wenke Lee. Behavior knowledge merge in reinforced agentic models, 2026. [31] Sihan Yang, Kexuan Shi, and Weiyang Liu. Orthogonal model merging. arXiv preprint arXiv:2602.05943, 2026.

ISO: An RLVR-Native Optimization Stack

[32] Jiakang Wang, Runze Liu, Lei Lin, Wenping Hu, Xiu Li, Fuzheng Zhang, Guorui Zhou, and Kun Gai. Aspo: Asymmetric importance sampling policy optimization. arXiv preprint arXiv:2510.06062, 2025. [33] Bingxiang He, Zekai Qu, Zeyuan Liu, Yinghao Chen, Yuxin Zuo, Cheng Qian, Kaiyan Zhang, Weize Chen, Chaojun Xiao, Ganqu Cui, et al. Justrl: Scaling a 1.5 b llm with a simple rl recipe. arXiv preprint arXiv:2512.16649, 2025. [34] Qwen Team. Qwen3 technical report, 2025. [35] Zhiwei He, Tian Liang, Jiahao Xu, Qiuzhi Liu, Xingyu Chen, Yue Wang, Linfeng Song, Dian Yu, Zhenwen Liang, Wenxuan Wang, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Deepmath-103k: A large-scale, challenging, decontaminated, and verifiable mathematical dataset for advancing reasoning. 2025. [36] Jeremy Bernstein. Modular manifolds. Thinking Machines Lab: Connectionism, 2025. https://thinkingmachines.ai/blog/modular-manifolds/. [37] Zeju Qiu, Simon Buchholz, Tim Xiao, Maximilian Dax, Bernhard Schölkopf, and Weiyang Liu. Reparameterized llm training via orthogonal equivalence transformation. Advances in Neural Information Processing Systems, 38:140775–140821, 2026. [38] Zeju Qiu, Lixin Liu, Adrian Weller, Han Shi, and Weiyang Liu. Poet-x: Memory-efficient llm training by scaling orthogonal transformation. arXiv preprint arXiv:2603.05500, 2026. [39] Fangzhao Zhang and Mert Pilanci. Spectral adapter: Fine-tuning in spectral space. arXiv preprint arXiv:2405.13952, 2024. [40] Zhizhong Li, Sina Sajadmanesh, Jingtao Li, and Lingjuan Lyu. Stella: Subspace learning in low-rank adaptation using stiefel manifold. Advances in Neural Information Processing Systems, 38:75066–75092, 2026. [41] Kexuan Shi, Hanxuan Li, Zeju Qiu, Yandong Wen, Simon Buchholz, and Weiyang Liu. Pion: A spectrum-preserving optimizer via orthogonal equivalence transformation. arXiv preprint arXiv:2605.12492, 2026. [42] John Schulman and Thinking Machines Lab. Lora without regret. Thinking Machines Lab: Connectionism, 2025. https://thinkingmachines.ai/blog/lora/. [43] Jiakang Wang, Runze Liu, Fuzheng Zhang, Xiu Li, and Guorui Zhou. Stabilizing knowledge, promoting reasoning: Dual-token constraints for rlvr. arXiv preprint arXiv:2507.15778, 2025. [44] Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024.

ISO: An RLVR-Native Optimization Stack

Appendix Outline The appendices are organized as follows. • Appendix A: Qualitative Index-Alignment Check. Provides rank-index singular-direction sanity checks for the spectrum-restoration intervention. • Appendix B: Fixed-Spectrum Theory and Frame-Adaptability Diagnostics. Establishes the fixed-spectrum distance and nearest representatives, derives the dimension-aware calibration, formalizes the reconstruction classes and unexplained-update ratios, and reports rank sensitivity. • Appendix C: First-Order Properties of the ISO Parameterization. Proves first-order spectrum preservation for feasible frame motion and derives the ISO factor gradients. • Appendix D: A Same-Base SFT–RLVR Case Study. Compares SFT and RLVR checkpoints derived from the same 14B backbone and reports an additional spectral-interpolation intervention. • Appendix E: ISO-Merger Details. Specifies sign canonicalization, tangent projection, mode masking, retention coefficients, retraction, parameter scope, and the complete merging algorithm. • Appendix F: Data-Free Merging Experimental Details. Describes the experts, baselines, hyperparameters, evaluation protocols, and additional distributional results for ISO-Merger. • Appendix G: Numerical Precision of the SVD-Based Retraction. Documents the SVD-based retraction, evaluates the accuracy–runtime trade-off of PyTorch SVD configurations, and motivates the FP64 GPU implementation. • Appendix H: Online RLVR Training Details. Gives the shared setup and the mathematical-reasoning and competitive-coding RLVR configurations, optimizer settings, training budgets, and evaluation protocols.

ISO: An RLVR-Native Optimization Stack

A. Qualitative Index-Alignment Check Why this check is needed. The main text avoids using individual singular-vector identities as formal evidence, since SVD bases are gauge-dependent under sign flips and rotations inside near-degenerate singular-value clusters. Nevertheless, because our spectrum-restoration intervention pairs singular values by rank index, it is useful to verify whether rank-index tracking is qualitatively stable in the RLVR runs we analyze. Rank-index singular-vector rotations. For each layer, we compute the principal angle between rankindex paired singular directions from the base model and the post-training checkpoint, after standard sign alignment. These plots should be interpreted only as qualitative sanity checks. The formal frameadaptability conclusions in Section 3 use projector-based reconstructions and unexplained-update ratios. Figures 11 and 12 provide a qualitative contrast between RLVR and SFT. Under RLVR, rankindex paired singular directions rotate mildly and smoothly, suggesting that the spectrum-restoration intervention is not dominated by large index-wise mismatch. Under SFT, many matched directions become nearly orthogonal, indicating that rank-index tracking is much less stable. We emphasize that these plots are not used as formal evidence for singular-subspace motion. The main diagnostic evidence is the gauge-invariant projector and reconstruction analysis in Section 3.

12.5 10.0 7.5 5.0 2.5 0.0

down_proj q_proj gate_proj up_proj k_proj v_proj o_proj 500 1000 1500 2000 2500 3000 3500 Singular value index

Degrees

Left singular vectors

15.0 12.5 10.0 7.5 5.0 2.5 0.0

Right singular vectors

500 1000 1500 2000 2500 3000 3500 Singular value index

Figure 11 | Qualitative singular-direction rotations under RL post-training. Mean principal angles between rank-index paired base and RL-trained singular directions remain small and smooth across attention and MLP layers. This suggests that rank-index tracking is qualitatively stable under RLVR in these runs. These plots are used only as a sanity check. Formal claims use projector-based diagnostics.

Right singular vectors

40 20 0

down_proj q_proj gate_proj up_proj k_proj v_proj o_proj 500 1000 1500 2000 2500 3000 3500 Singular value index

Degrees

Left singular vectors

40 20 0

500 1000 1500 2000 2500 3000 3500 Singular value index

Figure 12 | Qualitative singular-direction rotations under SFT. Mean angles between rank-index paired base and SFT singular directions increase toward near-orthogonality for many ranks. In highdimensional spaces, unrelated directions are typically nearly orthogonal, so this indicates that simple rank-index tracking becomes much less informative under SFT. 26

ISO: An RLVR-Native Optimization Stack

B. Fixed-Spectrum Theory and Frame-Adaptability Diagnostics Notation.

Let 𝑊 ∈ ℝ𝑑out × 𝑑in and 𝑞 = min{ 𝑑out , 𝑑in }. We write 𝜎 (𝑊 ) = 𝜎1 (𝑊 ) , . . . , 𝜎𝑞 (𝑊 )

for all singular values in nonincreasing order, including zeros with multiplicity. A thin SVD is 𝑊 = 𝑈Σ𝑉 ⊤ ,

𝑈 ∈ St( 𝑑out , 𝑞) ,

𝑉 ∈ St( 𝑑in , 𝑞) ,

Σ = Diag( 𝜎 (𝑊 )) .

Here Diag( 𝑎) constructs a diagonal matrix from a vector, whereas diag( 𝐴) extracts the diagonal of a matrix. B.1. The Fixed-Spectrum Family and Nearest Representatives Proposition B.1 (Two-sided characterization of the fixed-spectrum family). For any source matrix 𝑊0 , F (𝑊0 ) := 𝑍 ∈ ℝ𝑑out × 𝑑in : 𝜎 ( 𝑍 ) = 𝜎 (𝑊0 ) (39) = 𝑄 𝐿𝑊0 𝑄 ⊤𝑅 : 𝑄 𝐿 ∈ O( 𝑑out ) , 𝑄 𝑅 ∈ O( 𝑑in ) . Proof. Orthogonal multiplication preserves singular values, proving one inclusion. For the reverse inclusion, write 𝑊0 = 𝑈0 Σ0𝑉0⊤ , 𝑍 = 𝑈𝑍 Σ0𝑉𝑍⊤ . Extend 𝑈0 , 𝑈𝑍 and 𝑉0 , 𝑉𝑍 to complete orthogonal bases and choose 𝑄 𝐿 , 𝑄 𝑅 mapping the source bases to the target bases. Then 𝑄 𝐿𝑊0 𝑄 ⊤𝑅 = 𝑍 . □ Proposition B.2 (Exact distance to the fixed-spectrum family). For 𝑊, 𝑊0 ∈ ℝ𝑑out × 𝑑in , dist𝐹 𝑊, F (𝑊0 ) = ∥ 𝜎 (𝑊 ) − 𝜎 (𝑊0 ) ∥ 2 .

(40)

If 𝑊 = 𝑈Σ𝑉 ⊤ and Σ0 = Diag( 𝜎 (𝑊0 )), then 𝑈Σ0𝑉 ⊤ ∈ arg min ∥𝑊 − 𝑍 ∥ 𝐹 .

(41)

𝑍 ∈ F (𝑊0 )

Proof. Every 𝑍 ∈ F (𝑊0 ) has Frobenius norm ∥𝑊0 ∥ 𝐹 , so ∥𝑊 − 𝑍 ∥ 2𝐹 = ∥𝑊 ∥ 2𝐹 + ∥𝑊0 ∥ 2𝐹 − 2⟨𝑊, 𝑍 ⟩𝐹 . By von Neumann’s trace inequality, ⟨𝑊, 𝑍 ⟩𝐹 ≤

𝑞 ∑︁

𝜎𝑘 (𝑊 ) 𝜎𝑘 (𝑊0 ) .

𝑘=1

The bound is attained by 𝑍 = 𝑈Σ0𝑉 ⊤ . Substitution gives min ∥𝑊 − 𝑍 ∥ 2𝐹 =

𝑍 ∈ F (𝑊0 )

𝑞 ∑︁

𝜎𝑘 (𝑊 ) − 𝜎𝑘 (𝑊0 ) .

𝑘=1

□

ISO: An RLVR-Native Optimization Stack

Remark B.3 (Gauge and nonuniqueness of a rebased representative). The distance to the fixedspectrum family and the set of nearest points are basis-independent, but a particular representative 𝑈Σ0𝑉 ⊤ need not be unique. Suppose a block C of the target spectrum is exactly repeated. Replacing 𝑈 C ← 𝑈 C 𝑅,

𝑉 C ← 𝑉 C 𝑅,

𝑅 ∈ O(|C|)

leaves 𝑊 unchanged but produces the rebased block 𝑈 C 𝑅Σ0, C 𝑅⊤𝑉 C⊤ .

Every such representative is a nearest point in F (𝑊0 ). It is identical for all 𝑅 when Σ0, C is proportional to the identity. More generally, with 𝜎¯ C denoting the block mean, 𝑅Σ0, C 𝑅⊤ − Σ0, C 𝐹 ≤ 2 Σ0, C − 𝜎 ¯C 𝐼 𝐹 .

(42)

Thus the ambiguity is small when Σ0, C is nearly flat. Near-degenerate but distinct singular values do not create exact gauge freedom, but can make the numerical singular vectors ill-conditioned. Our rebasing experiment evaluates the representative selected by the numerical SVD. Appendix A reports a qualitative stability check. Corollary B.4 (Finite-horizon spectral drift). Let 𝑊0 , . . . , 𝑊𝑇 be a matrix trajectory and 𝐸𝑡 = 𝑊𝑡+1 − 𝑊𝑡 . Then 𝑡 −1 ∑︁ ∥ 𝜎 (𝑊𝑡 ) − 𝜎 (𝑊0 ) ∥ 2 = dist𝐹 𝑊𝑡 , F (𝑊0 ) ≤ ∥𝑊𝑡 − 𝑊0 ∥ 𝐹 ≤ ∥ 𝐸𝜏 ∥ 𝐹 . (43) 𝜏=0

B.2. Spectral-Distance Metrics and Dimension-Aware Calibration For matrix ℓ, let Δ𝑊 ( ℓ ) = 𝑊1( ℓ ) − 𝑊0( ℓ ) . We use (ℓ)

𝛿Σ

(ℓ)

𝜌Σ

dist𝐹 𝑊1( ℓ ) , F (𝑊0( ℓ ) ) ∥𝑊0( ℓ ) ∥ 𝐹

(44)

dist𝐹 𝑊1( ℓ ) , F (𝑊0( ℓ ) ) ∥ Δ𝑊 ( ℓ ) ∥ 𝐹

Proposition B.5 (First-order spectrum-changing subspace and isotropic calibration). Assume 𝑊0 is full rank with simple singular values and write 𝑊0 = 𝑈0 Σ0𝑉0⊤ . The first-order spectrum-changing subspace at 𝑊0 , equivalently the normal space of its fixed-spectrum family, is 𝑁𝑊0 F (𝑊0 ) = 𝑈0 Diag( 𝑎)𝑉0⊤ : 𝑎 ∈ ℝ𝑞 . (45) The orthogonal projection of a perturbation 𝐻 onto this spectrum-changing subspace is Πspec ( 𝐻 ) = 𝑈0 Diag diag(𝑈0⊤ 𝐻𝑉0 ) 𝑉0⊤ . If vec( 𝐻 )/∥ 𝐻 ∥ 𝐹 is isotropic, then

" 𝔼

∥ Πspec ( 𝐻 ) ∥ 2𝐹 ∥ 𝐻 ∥2 𝐹

Consequently, 𝜅spec =

# =

𝑞 . 𝑑out 𝑑in

⊤ 2 𝑑out 𝑑in ∥ diag(𝑈0 𝐻𝑉0 ) ∥ 2 𝑞 ∥ 𝐻 ∥ 2𝐹

(46)

(47)

(48)

satisfies 𝔼[ 𝜅spec ] = 1. Thus, one is the isotropic dimensional reference under this normalization. 28

ISO: An RLVR-Native Optimization Stack

Proof. For a simple full-rank spectrum, fixing the 𝑞 singular values imposes 𝑞 independent first-order constraints. The matrices {𝑢0,𝑘 𝑣0⊤,𝑘 }𝑞𝑘=1 form an orthonormal basis for this spectrum-changing subspace, which gives Equation (46). An isotropic direction places expected squared energy in a fixed subspace in proportion to its dimension. The spectrum-changing and ambient dimensions are 𝑞 and 𝑑out 𝑑in , respectively. □ B.3. Reconstruction Classes and Unexplained-Update Ratios For 𝑟 < 𝑞, write (𝑟 )

𝑊𝑡

= 𝑈𝑡( 𝑟 ) Σ𝑡( 𝑟 ) 𝑉𝑡( 𝑟 )

⊤

(𝑟 )

𝑃𝑡

= 𝑈𝑡( 𝑟 ) 𝑈𝑡( 𝑟 )

⊤

(𝑟 )

= 𝑉𝑡( 𝑟 ) 𝑉𝑡( 𝑟 )

𝑄𝑡

⊤

We call a transition 𝑊 𝑗 → 𝑊𝑖 admissible at rank 𝑟 when 𝜎𝑟 (𝑊𝑡 ) > 𝜎𝑟+1 (𝑊𝑡 ) ,

𝑡 ∈ { 𝑖, 𝑗 },

so that the retained projectors are uniquely defined. Let 𝐴 = 𝑊𝑖( 𝑟 ) . Define n o ⊤ Cmix = 𝑈 𝑗( 𝑟 ) 𝐶 𝑉 𝑗( 𝑟 ) : 𝐶 ∈ ℝ𝑟 ×𝑟 , n o C𝐿 = 𝑈 𝑗( 𝑟 ) 𝐵 : 𝐵 ∈ ℝ𝑟 × 𝑑in , o n ⊤ C𝑅 = 𝐵 𝑉 𝑗( 𝑟 ) : 𝐵 ∈ ℝ𝑑out ×𝑟 , o n F𝑗( 𝑟 ) = 𝑈Σ (𝑗 𝑟 ) 𝑉 ⊤ : 𝑈 ∈ St( 𝑑out , 𝑟 ) , 𝑉 ∈ St( 𝑑in , 𝑟 ) .

(49)

Proposition B.6 (Optimal checkpoint reconstructions). The Frobenius-optimal reconstructions are (𝑟 )

(𝑟 )

b𝐿 = 𝑃 𝐴, 𝑊 𝑗

bmix = 𝑃 𝐴𝑄 , 𝑊 𝑗 𝑗

(𝑟 )

biso = 𝑈 Σ 𝑊 𝑗 𝑖

b𝑅 = 𝐴𝑄 , 𝑊 𝑗

(𝑟 ) ⊤

𝑉𝑖

(50) .

The remix reconstruction can equivalently be written as (𝑟 ) (𝑟 ) ⊤ (𝑟 ) (𝑟 ) bmix = 𝑈 ( 𝑟 ) 𝐵 ( 𝑟 ) 𝑉 ( 𝑟 ) ⊤ , 𝑊𝑖 𝑉 𝑗 . 𝐵𝑖 𝑗 = 𝑈 𝑗 𝑊 𝑗 𝑖𝑗 𝑗

(51)

The core 𝐵𝑖(𝑗𝑟 ) is unconstrained and may rotate, mix, rescale, and change the represented spectrum. Thus, Cmix fixes only the incoming input and output spans, not the individual frames or spectrum. Moreover, biso ∈ arg min ∥ 𝐴 − 𝑍 ∥ 𝐹 , 𝑊

(52)

(𝑟)

𝑍 ∈ F𝑗

and the corresponding normalized residuals are 𝑒ℎ =

bℎ ∥ 𝐹 ∥𝐴 − 𝑊 , ∥ 𝐴∥𝐹

ℎ ∈ {mix, 𝐿, 𝑅, iso},

(53)

while the displacement from the unchanged incoming checkpoint, used as a scale reference, is ∥𝑊𝑖( 𝑟 ) − 𝑊 𝑗( 𝑟 ) ∥ 𝐹 𝑒drift :=

∥𝑊𝑖( 𝑟 ) ∥ 𝐹

(54)

ISO: An RLVR-Native Optimization Stack

This reference is not an additional reconstruction hypothesis. The isospectral residual has the closed form 𝑒iso =

∥ Σ𝑖( 𝑟 ) − Σ (𝑗 𝑟 ) ∥ 𝐹 ∥ Σ𝑖( 𝑟 ) ∥ 𝐹

(55)

The corresponding unexplained-update ratios are 𝑒ℎ 𝑢ℎ :=

𝑒drift

bℎ ∥ 𝐹 ∥𝑊𝑖( 𝑟 ) − 𝑊 ∥𝑊𝑖( 𝑟 ) − 𝑊 𝑗( 𝑟 ) ∥ 𝐹

ℎ ∈ {mix, 𝐿, 𝑅, iso} .

(56)

If 𝑒drift > 0, then 0 ≤ 𝑢ℎ ≤ 1,

ℎ ∈ {mix, 𝐿, 𝑅, iso} .

Proof. The first three expressions are orthogonal projections onto Cmix , C𝐿 , and C𝑅 . In particular, h i (𝑟 ) (𝑟 ) (𝑟 ) (𝑟 ) (𝑟 ) ⊤ (𝑟 ) (𝑟 ) (𝑟 ) ⊤ 𝑃 𝑗 𝑊𝑖 𝑄 𝑗 = 𝑈 𝑗 𝑈𝑗 𝑊𝑖 𝑉 𝑗 𝑉𝑗 , which gives Equation (51). The isospectral expression follows by applying Proposition B.2 to the rank-𝑟 matrices 𝑊𝑖( 𝑟 ) and 𝑊 𝑗( 𝑟 ) . bℎ gives Finally, 𝑊 𝑗( 𝑟 ) belongs to all four reconstruction classes. Therefore, the optimality of 𝑊 bℎ ∥ 𝐹 ≤ ∥𝑊 ( 𝑟 ) − 𝑊 ( 𝑟 ) ∥ 𝐹 , ∥𝑊𝑖( 𝑟 ) − 𝑊 𝑖 𝑗 which proves 0 ≤ 𝑢ℎ ≤ 1. □ Remark B.7 (Interpretation of the reconstruction test). The reconstruction classes have different dimensions, so we do not interpret their residual ordering as a complexity-normalized model-selection result. The purpose of the test is parameterization sufficiency: whether freezing the corresponding incoming span still permits a low-residual endpoint description. Because Cmix , C𝐿 , and C𝑅 are more permissive than the associated fixed-spectrum restrictions, their residuals are optimistic lower bounds on the error caused by freezing that frame structure. Remark B.8 (Gauge and interpretation). The projectors 𝑃𝑡( 𝑟 ) , 𝑄 𝑡( 𝑟 ) and all four residual values are biso may not be invariant to basis changes inside the retained subspaces. If Σ𝑖( 𝑟 ) has repeated blocks, 𝑊 the unique nearest point, but 𝑒iso remains unique. The remix class contains the internal fixed-spectrum slice n o (𝑟 ) (𝑟 ) (𝑟 ) ⊤ 𝑈 𝑗 𝑅𝑈 Σ 𝑗 𝑅𝑉⊤ 𝑉 𝑗 : 𝑅𝑈 , 𝑅𝑉 ∈ O( 𝑟 ) . Hence, failure of the more permissive remix class also rules out an explanation based solely on internal fixed-spectrum rotations. Remark B.9 (Why 𝑟 < 𝑞). At full thin rank, at least one of 𝑃𝑡( 𝑟 ) , 𝑄 𝑡( 𝑟 ) is the identity. For square matrices both are. The remix and one-sided tests would therefore degenerate. We use 𝑟 < 𝑞 so that both retained subspaces are proper and informative.

ISO: An RLVR-Native Optimization Stack

B.4. Rank Sensitivity of the Frame-Adaptability Test Rank sensitivity.

For 𝛼 ∈ {0.2, 0.4, 0.6, 0.8, 0.9}, we use the integer truncation rank 𝑟𝛼 = max{1, ⌊ 𝛼𝑞⌋} .

(57)

At each 𝛼, a matrix–transition pair is included only when 𝑟𝛼 is admissible at both endpoints. Inadmissible pairs are omitted at that value of 𝛼. For each included pair, we compute 𝑢ℎ =

𝑒ℎ , 𝑒drift

W0 W1

0.6

0.8

10 2 100

0.2

0.4

0.6

0.8

10 2 100

10 1

10 2

100

0.2

0.4

0.6

0.8

10 1

0.4

0.6

0.8

0.2

0.4

0.6

0.8

100

0.2

0.4

0.6

0.8

0.4

0.6

0.8

r/q (truncation fraction)

umix

10 2

0.8

0.2

0.4

0.6

0.8

100

0.2

0.4

0.6

0.8

0.2

0.4

0.6

0.8

10 1

10 1 0.2

100

0.6

10 1

10 1 10 2

0.2

0.4

0.2

0.4

0.6

0.8

10 2

r/q (truncation fraction)

uiso

L16 mlp.down_proj

100

0.2

L0 mlp.gate_proj

Unexplained-update ratio uh = eh/edrift

Language Modules

0.4

10 1

L35 mlp.up_proj

Vision Modules

0.2

W0 W2

100

10 1

10 2 100

W1 W2

100

10 1

(58)

L17 self_attn.o_proj

100

ℎ ∈ {mix, 𝐿, 𝑅, iso} .

r/q (truncation fraction)

u = 1 (no change explained)

Figure 13 | Rank sensitivity of the unexplained-update ratio. Unexplained-update ratio 𝑢ℎ versus the truncation fraction 𝑟 /𝑞 for representative language and vision modules and all three transitions. The dashed line 𝑢 = 1 corresponds to retaining the unmodified incoming checkpoint. The ordering in Figure 5 is stable across ranks: 𝑢iso stays below 2% throughout the sweep, while the subspace-retaining alternatives leave tens of percent unexplained at every rank. At 𝑟 /𝑞 = 0.2, the median across the twelve displayed matrix–transition panels is 𝑢mix = 0.99. The gap is already pronounced at small 𝑟 : restricting the reconstruction to the dominant incoming input and output spans explains little of the observed rank-𝑟 checkpoint change, whereas retaining the incoming spectrum and adapting both frames remains low-residual.

C. First-Order Properties of the ISO Parameterization C.1. Proof of Proposition 4.1 Proof. We prove the two claims in Proposition 4.1. 31

ISO: An RLVR-Native Optimization Stack

First, consider a differentiable curve 𝑊 ( 𝜀) ∈ F (𝑊0 ) with 𝑊 (0) = 𝑊 = 𝑈Σ0𝑉 ⊤ . Since all 𝑞 singular values are positive and simple, we may locally choose differentiable Stiefel representatives 𝑈 ( 𝜀) and 𝑉 ( 𝜀) such that 𝑊 ( 𝜀) = 𝑈 ( 𝜀) Σ0𝑉 ( 𝜀) ⊤ ,

𝑈 (0) = 𝑈,

𝑉 (0) = 𝑉.

(59)

Let 𝑈¤ =

𝑑 , 𝑈 ( 𝜀) 𝑑𝜀 𝜀=0

𝑉¤ =

𝑑 , 𝑉 ( 𝜀) 𝑑𝜀 𝜀=0

¤ = 𝑊

𝑑 . 𝑊 ( 𝜀) 𝑑𝜀 𝜀=0

(60)

Differentiating the fixed-spectrum representation gives ¤ = 𝑈Σ ¤ 0𝑉 ⊤ + 𝑈Σ0𝑉¤ ⊤ . 𝑊

(61)

Multiplying on the left by 𝑈 ⊤ and on the right by 𝑉 yields ¤ = 𝑈 ⊤𝑈¤ Σ0 + Σ0 𝑉¤ ⊤𝑉. 𝑈 ⊤𝑊𝑉

(62)

Because 𝑈 ( 𝜀) and 𝑉 ( 𝜀) remain on the Stiefel manifold, differentiating 𝑈 ( 𝜀) ⊤𝑈 ( 𝜀) = 𝐼 and 𝑉 ( 𝜀) ⊤𝑉 ( 𝜀) = 𝐼 at 𝜀 = 0 gives 𝑈 ⊤𝑈¤ + 𝑈¤ ⊤𝑈 = 0,

Thus 𝑈 ⊤𝑈¤ and 𝑉¤ ⊤𝑉 are skew-symmetric. Since Σ diagonal. Therefore

𝑉 ⊤𝑉¤ + 𝑉¤ ⊤𝑉 = 0.

is diagonal, both 𝑈 ⊤𝑈¤ Σ

(63)

¤ ⊤𝑉 have zero 0 and Σ0𝑉

¤ ) = 0. diag(𝑈 ⊤𝑊𝑉

(64)

Second, let 𝐻 be an arbitrary perturbation and define 𝑊 ( 𝜀) = 𝑊 + 𝜀𝐻.

(65)

For each singular value, choose differentiable singular vectors 𝑢𝑘 ( 𝜀) and 𝑣𝑘 ( 𝜀) with singular value 𝜎𝑘 ( 𝜀). Then Differentiating at 𝜀 = 0 gives

𝜎𝑘 ( 𝜀) = 𝑢𝑘 ( 𝜀) ⊤𝑊 ( 𝜀) 𝑣𝑘 ( 𝜀) .

(66)

⊤ ⊤ ¤𝑘 . 𝜎′𝑘 (0) = 𝑢¤ ⊤ 𝑘 𝑊 𝑣𝑘 + 𝑢𝑘 𝐻𝑣𝑘 + 𝑢𝑘 𝑊 𝑣

(67)

Using 𝑊 𝑣𝑘 = 𝜎𝑘 𝑢𝑘 and 𝑢⊤ 𝑊 = 𝜎𝑘 𝑣𝑘⊤ , this becomes 𝑘 ⊤ ⊤ ¤𝑘 . 𝜎′𝑘 (0) = 𝜎𝑘 𝑢¤ ⊤ 𝑘 𝑢𝑘 + 𝑢𝑘 𝐻𝑣𝑘 + 𝜎𝑘 𝑣𝑘 𝑣

(68)

Since 𝑢𝑘 ( 𝜀) and 𝑣𝑘 ( 𝜀) remain unit vectors, differentiating 𝑢𝑘 ( 𝜀) ⊤ 𝑢𝑘 ( 𝜀) = 1 and 𝑣𝑘 ( 𝜀) ⊤ 𝑣𝑘 ( 𝜀) = 1 gives 𝑢¤ ⊤ 𝑘 𝑢𝑘 = 0,

𝑣𝑘⊤ 𝑣¤ 𝑘 = 0.

(69)

Hence 𝑑 𝜎𝑘 (𝑊 + 𝜀𝐻 ) = 𝑢⊤ 𝑘 𝐻𝑣𝑘 . 𝑑𝜀 𝜀=0

(70)

Collecting this identity for 𝑘 = 1, . . . , 𝑞 shows that diag(𝑈 ⊤ 𝐻𝑉 ) is exactly the vector of first-order singular-value changes. This proves the proposition. □ C.2. Factor-Gradient Derivation Let L (𝑊 ) be the RLVR loss, let 𝐺𝑊 = ∇𝑊 L (𝑊 ), and use the fixed-spectrum parameterization 𝑊 (𝑈, 𝑉 ) = 𝑈Σ0𝑉 ⊤ ∈ F (𝑊0 ) .

(71)

For infinitesimal Stiefel-factor changes 𝜉𝑈 ∈ 𝑇𝑈 St( 𝑑out , 𝑞) and 𝜉𝑉 ∈ 𝑇𝑉 St( 𝑑in , 𝑞), the induced weight motion is ¤ = 𝜉𝑈 Σ0𝑉 ⊤ + 𝑈Σ0 𝜉𝑉⊤ . 𝑊

(72) 32

ISO: An RLVR-Native Optimization Stack

The first-order change in loss is ¤ ] = ⟨𝐺𝑊 , 𝑊 ¤ ⟩𝐹 𝑑 L [𝑊 ⊤ = ⟨𝐺𝑊 𝑉 Σ0 , 𝜉𝑈 ⟩𝐹 + ⟨𝐺𝑊 𝑈Σ0 , 𝜉𝑉 ⟩𝐹 .

(73)

Therefore the raw factor gradients used by ISO are 𝐺𝑈 = 𝐺𝑊 𝑉 Σ0 ,

⊤ 𝐺𝑉 = 𝐺𝑊 𝑈Σ0 .

(74)

These are the Euclidean gradients in the ambient factor coordinates. ISO does not explicitly construct a weight-space projected gradient. Instead, the fixed-spectrum parameterization and polar retraction restrict every represented iterate to F (𝑊0 ) up to numerical precision.

D. A Same-Base SFT–RLVR Case Study The main-text case study in Section 2.1 compares an SFT transition and an RLVR transition initialized from different base checkpoints. To control for this difference, we repeat the analysis using two public 14B checkpoints derived from the same Qwen2.5-14B base: ( DeepSeek-R1-Distill-Qwen-14B (distilled SFT), Qwen2.5-14B −→ Qwen-2.5-14B-SimpleRL-Zoo (RLVR). Both endpoints are measured relative to the shared base checkpoint. We restrict the comparison to the seven transformer projection matrices whose parameter names and shapes match the shared Qwen2.514B backbone, avoiding differences introduced by checkpoint-specific tokenizer or configuration changes. This comparison therefore controls for the starting backbone weights, while the training data, objectives, and optimization procedures remain intentionally different. Figure 14 summarizes the results. Raw spectral proximity. The two endpoints differ sharply in their spectral displacement from the shared base. At the scale of Figure 14b, the per-rank changes of the RLVR endpoint are visually near zero. Its base-normalized spectral distance is approximately 𝛿Σ ≈ 6 × 10 −6

across layers and projection types. By contrast, the distilled-SFT endpoint substantially changes the spectrum: 𝛿Σ is approximately 10−2 for most projection types and reaches approximately 0.4 for the attention output projection. Dimension-aware calibration. Raw proximity alone does not establish a preferentially spectrumpreserving update direction, so we additionally report the dimension-normalized spectrum-changing energy 𝜅spec . Across projection types, RLVR yields mean values between 1.4 and 3.2, with an overall mean of 1.9. These values remain order-one relative to the isotropic dimensional reference, with the largest deviations concentrated in the earliest layers. The distilled-SFT endpoint yields mean 𝜅spec values between 117 and 4810, with an overall mean of 873 and the largest concentration in the attention output projection. The same-base comparison therefore reproduces the main-text contrast: SFT concentrates its displacement in spectrum-changing coordinates by roughly two to three orders of magnitude, whereas RLVR exhibits no comparable concentration. This difference cannot be explained by the two transitions starting from different backbone checkpoints.

ISO: An RLVR-Native Optimization Stack

(b) Per-rank spectral change Δσi /σi 1

2.0

Δσi /σi (%)

Singular value σi

(a) Singular value spectra

1.5 1.0

Shared base SFT end RL end

0.5 1

Rank index i

256

1024

10−2 mean over matrices per matrix (q/k/v/o/gate/up/down)

10−5 0

SFT

Layer index

κspec (log scale)

δΣ (log scale)

layer in (a,b)

10−1

10−4

−1 −2

10−3

Rank index i

256

1024

(d) Dimension-normalized spectrum-changing energy κspec 103 102 mean over matrices per matrix (q/k/v/o/gate/up/down)

101

isotropic reference = 1

100 0

Layer index

Figure 14 | Same-base spectral dynamics. The distilled-SFT and RLVR endpoints are both compared with their shared Qwen2.5-14B base. (a) Singular-value profiles for v_proj in layer 24. (b) Rank-wise relative spectral change: the RLVR curve remains near zero at the plotted scale, whereas distilled SFT systematically reshapes the spectrum. (c) Layer-wise base-normalized spectral distance 𝛿Σ , shown per projection matrix (thin) and averaged across projection types (thick). The RLVR endpoint remains several orders of magnitude closer to the fixed-spectrum family of the shared base. (d) Dimension-normalized spectrum-changing energy 𝜅spec : RLVR remains order-one relative to the isotropic dimensional reference, whereas distilled SFT lies roughly two to three orders of magnitude above it.

D.1. Reverse Spectral Substitution To complement the main-text spectrum-restoration experiment, we reverse the intervention by keeping the SFT singular frames fixed while interpolating the spectrum from the SFT endpoint toward the RLVR endpoint. As shown in Figure 15, performance remains nearly unchanged along this path, indicating that substituting the RLVR spectrum alone does not transfer the acquired behavior into the SFT frames.

ISO: An RLVR-Native Optimization Stack

Figure 15 | Task performance is insensitive to spectral interpolation in the SFT singular frames. e ( 𝛼) = We hold the SFT left and right singular frames fixed and interpolate only the spectrum, 𝑊 ⊤ 𝑈SFT [(1 − 𝛼) ΣSFT + 𝛼ΣRL ] 𝑉SFT , for 𝛼 ∈ [0, 1]. Performance remains nearly unchanged as an arbitrary proportion of the RL spectrum is substituted into the SFT frames. Hollow circles and diamonds show the original SFT and RL checkpoints, respectively. This complements Figure 4(a), which performs the converse intervention by restoring the SFT/base spectrum in the RL-trained frames.

E. ISO-Merger Details E.1. Setup For each 2D weight matrix, let

𝑊0 = 𝑈0 Σ0𝑉0⊤

(75)

be the SVD of the shared base model, and let 𝑊𝑖 = 𝑈𝑖 Σ𝑖𝑉𝑖⊤ ,

𝑖 = 1, . . . , 𝐾

(76)

be the corresponding SVDs of 𝐾 RL experts fine-tuned from the same base. ISO-Merger uses the base spectrum Σ0 as the shared spectrum and merges expert-specific motion through the Stiefel factors (𝑈𝑖 , 𝑉𝑖 ). E.2. Sign canonicalization The SVD determines each singular-vector pair only up to a joint sign: ( 𝑢𝑘 , 𝑣𝑘 ) and (−𝑢𝑘 , − 𝑣𝑘 ) represent the same matrix. Numerical routines choose these signs arbitrarily, so an unaligned displacement 𝑈𝑖 − 𝑈0 can contain spurious columns of norm ∼2 that reflect bookkeeping rather than learned motion. We canonicalize the gauge column-wise against the base:

𝑠𝑖,𝑘 := sign ⟨𝑢0,𝑘 , 𝑢𝑖,𝑘 ⟩ , 𝑢𝑖,𝑘 ← 𝑠𝑖,𝑘 𝑢𝑖,𝑘 ,

sign(0) := 1,

𝑣𝑖,𝑘 ← 𝑠𝑖,𝑘 𝑣𝑖,𝑘 .

(77) (78)

ISO: An RLVR-Native Optimization Stack

The same sign is applied to both frames, so the alignment is a valid gauge transformation that leaves 𝑊𝑖 unchanged. Exactly repeated singular values additionally admit a joint in-block rotation 𝑈𝑐 → 𝑈𝑐 𝑄 , 𝑉𝑐 → 𝑉𝑐 𝑄 . Generic weight matrices have simple spectra, so we do not canonicalize this freedom explicitly. E.3. Stiefel tangent projection For an anchor 𝑈0 ∈ St( 𝑚, 𝑞), the tangent space is 𝑇𝑈0 St( 𝑚, 𝑞) = {𝜉 ∈ ℝ𝑚 × 𝑞 : 𝑈0⊤ 𝜉 + 𝜉⊤𝑈0 = 0} .

(79)

For expert 𝑖, define the frame displacements Δ𝑈𝑖 := 𝑈𝑖 − 𝑈0 ,

Δ𝑉𝑖 := 𝑉𝑖 − 𝑉0 .

(80)

ISO-Merger uses their orthogonal projections onto the corresponding tangent spaces: 𝜉𝑈,𝑖 = Π𝑈0 ( Δ𝑈𝑖 ) = Δ𝑈𝑖 − 𝑈0 sym(𝑈0⊤ Δ𝑈𝑖 ) ,

sym( 𝐴) =

1 ( 𝐴 + 𝐴⊤ ) . 2

(81)

The right-frame tangent 𝜉𝑉,𝑖 = Π𝑉0 ( Δ𝑉𝑖 ) is computed analogously. E.4. Top-𝑘keep singular-mode masking Let 𝜌keep ∈ (0, 1] and define 𝑘keep := ⌊ 𝜌keep 𝑞⌉ ,

𝐷keep := Diag

𝑞 1{ 𝑘 ≤ 𝑘keep } 𝑘=1 .

(82)

Thus, 𝐷keep keeps the leading 𝑘keep columns. Empirically, the trailing modes (columns associated with small singular values) are estimated noisily and disagree across experts, and retaining them degrades the merged model. We therefore mask the trailing columns of each displacement, e𝑈,𝑖 = 𝜉𝑈,𝑖 𝐷keep , 𝜉

e𝑉,𝑖 = 𝜉𝑉,𝑖 𝐷keep . 𝜉

(83)

Masking is a coordinate-selection operation and need not preserve the Stiefel tangent constraints. We use the masked displacements only to define local first-order effect proxies and impose feasibility after aggregation. We use 𝜌keep = 0.9 unless otherwise stated. E.5. Retention Coefficients with a Unit-Retention Target For expert 𝑖, define the local first-order effect proxy e⊤ . e𝑈,𝑖 Σ0𝑉 ⊤ + 𝑈0 Σ0 𝜉 𝑔𝑖 = 𝜉 𝑉,𝑖 0

We then form the Gram matrix For 𝑐 ∈ ℝ 𝐾 , define 𝑔 ( 𝑐) := proxy by

Γ𝑖 𝑗 := ⟨𝑔𝑖 , 𝑔 𝑗 ⟩𝐹 ,

Γ ∈ ℝ𝐾 × 𝐾 .

(84)

(85)

Í𝐾

𝑖=1 𝑐 𝑖 𝑔 𝑖 . For each nonzero proxy 𝑔 𝑖 , define its self-retention in the merged

ret𝑖 ( 𝑐) :=

⟨𝑔 ( 𝑐) , 𝑔 𝑖 ⟩𝐹 ( Γ𝑐) 𝑖 = . Γ𝑖𝑖 ∥ 𝑔𝑖 ∥ 2𝐹

(86)

For any zero-norm proxy 𝑔𝑖 , we omit the corresponding row and column from the Gram system and set its coefficient to zero. The ideal target ret𝑖 ( 𝑐) = 1 gives Γ𝑐 = 𝑏,

𝑏 := diag( Γ) ∈ ℝ 𝐾 .

(87) 36

ISO: An RLVR-Native Optimization Stack

We solve the ridge-stabilized system

( Γ + 𝜆 ridge 𝐼 𝐾 )¯𝑐 = 𝑏,

(88)

and clip the coefficients to prevent sign reversal or over-amplification of any expert: 𝑐★ = clip(¯ 𝑐, 𝑐min , 𝑐max ) ,

[ 𝑐min , 𝑐max ] = [0, 1.5] .

(89)

We use 𝜆 ridge = 10−12 as a small numerical stabilizer. The procedure targets, rather than guarantees, unit self-retention after ridge stabilization, clipping, aggregate tangent projection, and retraction. E.6. Retraction and reconstruction We combine the masked displacements and project the result onto the tangent space at the anchor: 𝜉𝑈,★ = Π𝑈0

𝐾 ∑︁

𝜉𝑉,★ = Π𝑉0

e𝑈,𝑖 , 𝑐★𝑖𝜉

𝐾 ∑︁

𝑖=1

For a thin SVD

e𝑉,𝑖 . 𝑐★𝑖𝜉

(90)

𝑖=1

𝑋 = 𝑃 𝑋 𝑆𝑋 𝑄 ⊤ 𝑋,

we define the polar factor by

polar( 𝑋 ) := 𝑃 𝑋 𝑄 ⊤𝑋 .

(91)

with the analogous identity for 𝑉0 + 𝜉𝑉,★. Thus the ISO-Merger retraction arguments are full column rank. The final merged factors and matrix are 𝑈★ = polar(𝑈0 + 𝜉𝑈,★) ,

𝑉★ = polar(𝑉0 + 𝜉𝑉,★) ,

𝑊★ = 𝑈★ Σ0𝑉★⊤ ,

(92)

which carries the base spectrum Σ0 up to numerical precision. E.7. Merged parameter scope All per-layer 2D projection matrices and the embedding/unembedding matrices are merged with the construction above. One-dimensional parameters (normalization scales, attention biases) are merged with a standard task-vector average, 𝐾

𝑤★ = 𝑤0 +

1 ∑︁ 𝐾

( 𝑤𝑖 − 𝑤0 ) .

(93)

𝑖=1

E.8. Full ISO-Merger algorithm The full procedure is summarized in Algorithm 1.

ISO: An RLVR-Native Optimization Stack

Algorithm 1 ISO-Merger Require: Base model 𝜃0 , expert models {𝜃𝑖 } 𝑖𝐾=1 , keep ratio 𝜌keep (0.9), ridge 𝜆 ridge (10−12 ), clip range [ 𝑐min , 𝑐max ] ([0, 1.5]) Ensure: Merged model 𝜃★ 1: for each 2D weight matrix 𝑊0 (per-layer projections and embedding/unembedding) do 2: Compute 𝑊0 = 𝑈0 Σ0𝑉0⊤ with 𝑞 singular values 3: 𝑘keep ← round( 𝜌keep 𝑞) 𝑞 4: 𝐷keep ← Diag((1{ 𝑘 ≤ 𝑘keep }) 𝑘=1 ) 5: for each expert 𝑖 do 6: Compute 𝑊𝑖 = 𝑈𝑖 Σ𝑖𝑉𝑖⊤ 7: for each singular mode 𝑘 do ⊲ joint sign canonicalization (App. E.2) 8: 𝑠𝑖,𝑘 ← sign(⟨𝑢0,𝑘 , 𝑢𝑖,𝑘 ⟩) 9: 𝑢𝑖,𝑘 ← 𝑠𝑖,𝑘 𝑢𝑖,𝑘 , 𝑣𝑖,𝑘 ← 𝑠𝑖,𝑘 𝑣𝑖,𝑘 10: end for 11: Δ𝑈𝑖 ← 𝑈𝑖 − 𝑈0 , Δ𝑉𝑖 ← 𝑉𝑖 − 𝑉0 12: 𝜉𝑈,𝑖 ← Π𝑈0 ( Δ𝑈𝑖 ), 𝜉𝑉,𝑖 ← Π𝑉0 ( Δ𝑉𝑖 ) e𝑈,𝑖 ← 𝜉𝑈,𝑖 𝐷keep , 𝜉 e𝑉,𝑖 ← 𝜉𝑉,𝑖 𝐷keep 13: 𝜉 ⊲ mask trailing modes (App. E.4) ⊤ ⊤ e e 14: 𝑔𝑖 ← 𝜉𝑈,𝑖 Σ0𝑉0 + 𝑈0 Σ0 𝜉𝑉,𝑖 15: end for 16: I ← { 𝑖 : ∥ 𝑔𝑖 ∥ 𝐹 > 0} 17: Set 𝑐★𝑖 ← 0 for 𝑖 ∉ I 18: Γ𝑖 𝑗 ← ⟨𝑔𝑖 , 𝑔 𝑗 ⟩𝐹 for 𝑖, 𝑗 ∈ I 19: 𝑏 ← diag( Γ) 20: Solve ( Γ + 𝜆 ridge 𝐼 | I | )¯𝑐 I = 𝑏 21: 𝑐★I ← clip(¯ 𝑐 I , 𝑐min , 𝑐max ) ⊲ entrywise Í 𝐾 ★e Í 𝐾 ★e 22: 𝜉𝑈,★ ← Π𝑈0 𝑖=1 𝑐𝑖 𝜉𝑈,𝑖 , 𝜉𝑉,★ ← Π𝑉0 𝑖=1 𝑐𝑖 𝜉𝑉,𝑖 ⊲ project and retract 23: 𝑈★ ← polar(𝑈0 + 𝜉𝑈,★), 𝑉★ ← polar(𝑉0 + 𝜉𝑉,★) 24: 𝑊★ ← 𝑈★ Σ0𝑉★⊤ 25: end for 1 Í𝐾 26: Merge 1D parameters by task-vector average: 𝑤★ = 𝑤0 + 𝐾 𝑖=1 ( 𝑤𝑖 − 𝑤0 ) 27: return 𝜃★

F. Data-Free Merging Experimental Details Backbones and experts. For the Qwen2.5 setting, we use Qwen2.5-7B-Instruct as the shared base and merge three RLVR experts: CURE for coding [24], ToolRL for tool use [25], and MemAgent for long-context memory [26]. For the DeepSeek-R1-Distill setting, we use DeepSeek-R1-Distill-Qwen-1.5B as the shared base and merge Archer2.0 for coding [32] and JustRL for math [33]. Merging baselines. We compare against five data-free baselines: Task Arithmetic [27], TIES [28], TSV-Merge [29], RAM [30], and OrthoMerge-G-TIES [31]. Task Arithmetic linearly combines task vectors. The first four methods operate on Euclidean task-vector representations, while OrthoMergeG-TIES additionally introduces a geometry-aware orthogonal component and is described separately below. Let 𝜃0 denote the shared base model and let {𝜃𝑖 } 𝑖𝐾=1 be 𝐾 expert models fine-tuned from this same initialization. The task vector of expert 𝑖 is 𝜏𝑖 := 𝜃𝑖 − 𝜃0 .

(94)

ISO: An RLVR-Native Optimization Stack

A data-free merging method constructs a merged update 𝜏merge without using training data and outputs 𝜃merge = 𝜃0 + 𝜏merge . (95) For a matrix-shaped parameter in layer ℓ, we use the layer-wise notation (ℓ)

Δ𝑊𝑖

:= 𝑊𝑖( ℓ ) − 𝑊0( ℓ ) ,

𝑊 ( ℓ ) ∈ ℝ𝑚 ℓ × 𝑛 ℓ .

(96)

Task Arithmetic merges experts by linearly combining their task vectors and adding the resulting update back to the base model: 𝜏TA = 𝜆

𝐾 ∑︁

𝜏𝑖 ,

𝜃TA = 𝜃0 + 𝜏TA .

(97)

𝑖=1

Task Arithmetic is computationally cheap, requiring only vector additions and a global scaling coefficient. However, it does not explicitly address sign conflicts, redundant updates, layer-wise geometry, or sparse task-specific signals. As a result, interference and signal dilution can occur when expert updates overlap or when important updates are distributed over disjoint parameter regions. TIES-Merging follows three steps: Trim, Elect Sign, and Merge. It first removes small-magnitude entries from each task vector, then elects a consensus sign for every coordinate, and finally averages only the updates whose signs agree with the elected sign. Let 𝜌TIES ∈ (0, 1] be the retained density. The trimmed update for expert 𝑖 is 𝑚𝑖 = TopKMask(| 𝜏𝑖 | , 𝜌TIES ) ,

e 𝜏𝑖 = 𝑚 𝑖 ⊙ 𝜏𝑖 ,

(98)

where 𝑚𝑖𝑝 = 1 indicates that coordinate 𝑝 is among the largest 𝜌TIES fraction of entries of | 𝜏𝑖 |. The elected sign at coordinate 𝑝 is ! 𝐾 ∑︁ 𝑝 𝛾 𝑝 = sign e 𝜏𝑖 . (99) 𝑖=1

TIES then keeps only the experts aligned with this sign: 𝑝 𝑝 𝐴𝑝 = 𝑖 : e 𝜏𝑖 ≠ 0, sign(e 𝜏𝑖 ) = 𝛾 𝑝 .

(100)

The merged coordinate is 1 Í  𝑝    𝜏𝑖 , 𝑖∈ 𝐴𝑝 e 𝜏TIES = | 𝐴 𝑝 |   0,  𝑝

| 𝐴 𝑝 | > 0,

(101)

| 𝐴 𝑝 | = 0,

and the final model is 𝜃TIES = 𝜃0 + 𝜆𝜏TIES .

(102)

TIES remains an element-wise method, but it is more robust than naive averaging when many small updates are irrelevant or when experts update the same coordinate in opposite directions. Its main limitation is that it does not explicitly model matrix-level subspace geometry. TSV-Merge, short for Task Singular Vectors Merge, operates on layer-wise task matrices rather than fully flattened vectors. For each 2D weight matrix, TSV decomposes each expert update with SVD, keeps the leading singular components, orthogonalizes the concatenated singular-vector bases, and reconstructs the merged matrix.

ISO: An RLVR-Native Optimization Stack

For layer ℓ, compute

(ℓ)

Δ𝑊𝑖

= 𝑈𝑖 Σ𝑖𝑉𝑖⊤ .

After truncating to rank 𝑘ℓ , (ℓ)

Δ𝑊𝑖

(103) ⊤

≈ 𝑈𝑖( 𝑘 ) Σ𝑖( 𝑘 ) 𝑉𝑖( 𝑘 ) .

(104)

The truncated bases and spectra are concatenated as (𝑘)

(𝑘)

𝑈 = [𝑈1 | · · · |𝑈 𝐾 ] ,

(𝑘)

Σ = blockdiag( Σ1 , . . . , Σ 𝐾 ) .

𝑉 = [𝑉1 | · · · |𝑉𝐾 ] ,

(105)

TSV reduces singular-vector interference by orthogonalizing 𝑈 and 𝑉 . In the whitening form, 𝑈⊥ = 𝑈 (𝑈 ⊤𝑈 ) −1/2 ,

𝑉⊥ = 𝑉 (𝑉 ⊤𝑉 ) −1/2 .

(106)

A numerically stable implementation can use the orthogonal Procrustes form. If 𝑈 = 𝑃𝑈 𝐷𝑈 𝑄𝑈⊤ and 𝑉 = 𝑃𝑉 𝐷𝑉 𝑄𝑉⊤ , then 𝑈⊥ = 𝑃𝑈 𝑄𝑈⊤ , 𝑉⊥ = 𝑃𝑉 𝑄𝑉⊤ . (107) The merged task matrix is reconstructed as b Δ ( ℓ ) = 𝑈⊥ Σ𝑉⊥⊤ ,

(ℓ)

𝑊TSV = 𝑊0

+ 𝛼b Δ (ℓ) .

(108)

For non-matrix parameters, such as biases or normalization vectors, TSV typically falls back to Task Arithmetic. TSV is a spectral, subspace-level method. By decorrelating singular-vector bases, it can reduce interference that is invisible to element-wise rules. This comes at a higher computational cost because SVDs are required for matrix-shaped layers. The rank 𝑘ℓ is either treated as a hyperparameter or chosen through an automatic rank-reduction rule. RAM is designed for sparse RL updates, where naive averaging can dilute coordinates that are important for only one expert. For each coordinate 𝑝, RAM identifies the experts with non-negligible updates: 𝑝 𝐴 𝑝 = 𝑖 ∈ {1, . . . , 𝐾 } : | 𝜏𝑖 | > 𝜖act , 𝑐 𝑝 = | 𝐴 𝑝 |, (109) where 𝜖act is an active-update threshold. The merged coordinate is

𝑝

𝜏RAM =

  0,     𝑝  

𝜏𝑗 ,

1 ∑︁ 𝑝   𝜏𝑖 ,    𝑐𝑝  𝑖∈ 𝐴𝑝 

𝑐 𝑝 = 0, 𝑐 𝑝 = 1 and 𝐴 𝑝 = { 𝑗 },

(110)

𝑐 𝑝 ≥ 2.

Thus, inactive coordinates are discarded, task-unique coordinates are preserved without being divided by 𝐾 , and shared coordinates are averaged only over the experts that actively update them. The final model is 𝜃RAM = 𝜃0 + 𝜏RAM . (111) RAM is element-wise like TIES, but its selection criterion is different. Instead of trimming by global magnitude density and resolving signs, RAM separates inactive, unique, and shared update regions. This makes it suitable when RL-trained experts store task-specific behavior in sparse and partially non-overlapping parameter subsets. OrthoMerge [31] is a geometry-preserving merging framework that factors each expert update into an implicit orthogonal transformation of the base weights and a Euclidean residual, merges the 40

ISO: An RLVR-Native Optimization Stack

orthogonal components on the rotation manifold, and merges the residuals with a standard elementwise rule. The released implementation provides six variants: two Procrustes-target constructions (C, which fits the rotation only to rows whose update conflicts with the mean task vector, and G, which fits it to the full expert weights) combined with three residual backends (Task Arithmetic, TIES, TSV). We report the G-TIES variant for two reasons. First, TIES is a strong residual backend in both of our merging settings: among the element-wise baselines it attains the best or near-best overall average on both backbones (Tables 1 and 2), so we pair OrthoMerge with the classical method that is already competitive in our comparison. Second, the G construction fits the Procrustes rotation to the full expert weights, so the extracted transformation represents the entire expert update. The C construction instead fits it only to the rows that conflict with the mean task vector. Since ISO-Merger likewise composes each expert’s full update, represented as motion of its singular frames under the fixed base spectrum, G-TIES is the directly comparable variant. For each 2D weight matrix (excluding layer norms), OrthoMerge first solves a one-sided orthogonal Procrustes problem per expert, (ℓ)

𝑅𝑖

(ℓ) ⊤

= arg min 𝑊0( ℓ ) 𝑅 − 𝑊𝑖( ℓ ) 𝐹 = 𝑈¯𝑉¯⊤ ,

𝑊0

𝑅 ∈O( 𝑛ℓ )

(ℓ)

𝑊𝑖

= 𝑈¯𝑆¯𝑉¯⊤ ,

(112)

so that the rotation acts only on the input (column) space of the layer. Each rotation is mapped to a skew-symmetric matrix by the inverse Cayley transform, (ℓ)

𝐴𝑖

= 𝑅 𝑖( ℓ ) + 𝐼

−1

(ℓ)

𝑅𝑖

−𝐼 ,

(ℓ)

𝐴𝑖

⊤

= − 𝐴𝑖( ℓ ) ,

(113)

and the 𝐾 skew matrices are aggregated with a magnitude-corrected mean that averages direction and intensity separately, Í𝐾 Í ( ℓ) 𝐴 (ℓ) (ℓ) 𝐴mrg = 𝐾1 𝑖𝐾=1 𝐴𝑖 𝐹 · Í 𝐾𝑖=1 (𝑖ℓ ) , (114) 𝑖=1 𝐴 𝑖

𝐹

which prevents the norm shrinkage that a plain average of nearly orthogonal directions would induce. The merged rotation is recovered with the forward Cayley map and applied to the base weights, (ℓ) ( ℓ ) −1 (ℓ) (ℓ) (ℓ) (ℓ) 𝑅mrg = 𝐼 − 𝐴mrg 𝐼 + 𝐴mrg , 𝑊rot = 𝑊0 𝑅mrg . (115) The part of each expert update not captured by its own rotation is treated as a Euclidean residual, (ℓ)

𝛿𝑖

= 𝑊𝑖( ℓ ) − 𝑊0( ℓ ) 𝑅 𝑖( ℓ ) ,

(116)

(for non-matrix parameters 𝛿𝑖 = 𝜏𝑖 ), and the residuals are merged with the TIES rule of trimming, sign election, and sign-consistent averaging described above. The final model is (ℓ) (ℓ) 𝑊OM = 𝑊rot + 𝜆 𝜏TIES {𝛿𝑖 } . (117) OrthoMerge is the geometry-aware baseline closest to ISO-Merger, but the two methods act in different coordinates. OrthoMerge applies a one-sided orthogonal transformation on the input side of each matrix: 𝑊0( ℓ ) 𝑅 preserves the left singular directions and singular values of 𝑊0( ℓ ) , so any output-side frame change must be represented through its Euclidean residual path. ISO-Merger instead composes two-sided frame changes under the shared base spectrum, adapting both 𝑈 and 𝑉 in fixed-spectrum Stiefel coordinates. OrthoMerge’s rotation class is contained in the more permissive output-subspace-retaining class C𝐿 analyzed in Section 3. Even this larger class leaves a substantial unexplained-update ratio, whereas retaining the incoming spectrum while adapting both frames gives a low-residual description. This motivates ISO-Merger’s two-sided frame parameterization. 41

ISO: An RLVR-Native Optimization Stack

Baseline hyperparameters. We follow the official implementations for all baselines. For TIES, we use 𝜆 = 1.0 and 𝜌TIES = 0.2. For TSV-Merge, we use 𝛼 = 1.0 with automatic rank reduction. For RAM, we use 𝜖 = 10−5 . For ISO-Merger, all SVD operations are performed in FP64. For OrthoMerge-G-TIES, we use the official implementation with its default hyperparameters: FP32 SVD-based Procrustes, 20% retained density and 𝜆 = 1.0 for the TIES residual stage. Evaluation protocol. For the Qwen2.5 setting, we evaluate coding on LiveBench and LiveCodeBench, tool use on BFCL, and long-context memory on RULER HotpotQA and SQuAD at 32K–64K context lengths. For the DeepSeek-R1-Distill setting, we evaluate coding on LiveBench and LiveCodeBench, and math on AIME24, AIME25, AMC23, Minerva, and OlympiadBench. Unless otherwise specified, we generate 16 rollouts per prompt and report mean and standard deviation over three independent runs. Decoding settings. For DeepSeek-R1-Distill math and coding evaluations, we use temperature 0.6 and top- 𝑝 = 0.95. For Qwen2.5 LiveBench and LiveCodeBench, we use temperature 1.0. For BFCL, we use single-sample near-greedy decoding with temperature 0.001, following the official evaluation protocol. For RULER long-context evaluation, we use temperature 0.7. Inference is conducted with vLLM on NVIDIA H100 GPUs. Additional distributional metrics. Among 𝑛 sampled responses, let 𝑐 denote the number with binary correctness equal to one. For 𝑛 ≥ 𝑘, we report 𝑐 𝑛− 𝑐 𝑘 𝑘 𝑘 = . , worst@ best@𝑘 = 1 − 𝑛 𝑘

𝑛 𝑘

In addition to average@16, we report best@4 and worst@4 to characterize the upper and lower tails of stochastic decoding performance. • best@k=pass@k, the probability that at least one of 𝑘 randomly drawn samples solves the problem (best-case behaviour over 𝑘 draws). • worst@k= 𝑘𝑐 𝑛𝑘 , the probability that all 𝑘 randomly drawn samples solve the problem (consistency / worst-case reading). For the Qwen2.5 setting, unit-test accuracy of LB/LCB coding benchmark is the fraction of unit tests passed, a real value in [0, 1] rather than a binary outcome. BFCL is evaluated with singlesample, near-greedy decoding with temperature 0.001, following the official leaderboard protocol. We therefore report best@4 and worst@4 only for the remaining benchmarks with binary per-sample correctness. Tables 5, 6, 7, and 8 show that ISO-Merger achieves the highest upper and lower tails under both backbones. In particular, for worst@4, which is substantially more challenging, ISO-Merger reaches an overall average of 54.85, compared with 53.23 for the strongest training-free baseline. On DeepSeek-R1-Distill-Qwen-1.5B, ISO-Merger reaches a total average of 34.98, improving over the best baseline at 33.62. For best@4, ISO-Merger remains competitive with the original specialists while composing multiple skills into a single model. These results further demonstrate that different RL experts initialized from the same base model can be combined in fixed-spectrum Stiefel coordinates.

ISO: An RLVR-Native Optimization Stack

Table 5 | Main results on Qwen2.5-7B-Instruct with 3 RL experts under Best@4. Coding: pass accuracy (ACC) on LiveBench (LB) and LiveCodeBench (LCB). Memory: RULER HotpotQA and SQuAD at 32K–64K context lengths under the Recurrent setting. Best expert/merged result per column in bold. Shading indicates Experts and Ours . Coding

Memory

LCB v2

ACC

32K

64K

32K

64K

Avg

Base RLVR-Coder RLVR-Tool RLVR-Memory

48.35 ± 1.46 48.97 ± 1.20 51.54 ± 2.17 49.74 ± 2.61

39.93 ± 0.64 41.13 ± 1.50 39.91 ± 1.66 37.49 ± 3.49

68.98 ± 1.44 70.03 ± 0.33 69.31 ± 0.61 86.21 ± 0.26

66.54 ± 1.51 66.33 ± 0.79 67.71 ± 1.53 86.38 ± 0.90

79.62 ± 1.23 81.90 ± 0.59 81.12 ± 0.80 87.26 ± 0.32

77.20 ± 0.30 77.60 ± 0.42 78.74 ± 0.65 88.54 ± 0.39

63.44 64.33 64.72 72.60

Task Arithmetic TIES TSV RAM OrthoMerge-G-TIES [31]

51.99 ± 1.78 50.04 ± 2.18 48.82 ± 0.29 48.73 ± 0.54 52.20 ± 2.00

42.46 ± 0.28 42.34 ± 1.42 39.04 ± 2.44 41.06 ± 0.25 42.52± 1.77

81.65 ± 0.60 85.80 ± 0.18 84.75 ± 0.37 85.91 ± 0.52 84.38 ± 0.37

81.37 ± 0.37 84.32 ± 0.04 83.54 ± 0.37 84.17 ± 0.61 83.47 ± 0.40

84.13 ± 0.44 87.58 ± 0.31 85.84 ± 0.36 88.05 ± 0.37 86.39 ± 0.35 87.72 ± 0.44 86.62 ± 0.43 88.39 ± 0.57 86.06 ± 0.26 87.66 ± 0.41

71.53 72.73 71.71 72.48 72.72

Ours

50.39 ± 1.98

40.68 ± 1.99 86.05 ± 0.78 84.92 ± 0.37 86.98 ± 0.21 88.23 ± 0.38 72.88

Method

HotpotQA

SQuAD

Table 6 | Main results on Qwen2.5-7B-Instruct with 3 RL experts under Worst@4. Coding: pass accuracy (ACC) on LiveBench (LB) and LiveCodeBench (LCB). Memory: RULER HotpotQA and SQuAD at 32K–64K context lengths under the Recurrent setting. Best expert/merged result per column in bold. Shading indicates Experts and Ours . Coding

Memory

LCB v2

ACC

32K

64K

32K

64K

Avg

Base RLVR-Coder RLVR-Tool RLVR-Memory

24.44 ± 0.38 25.84 ± 1.57 21.87 ± 0.84 24.26 ± 0.76

16.99 ± 0.11 19.96 ± 0.41 16.78 ± 1.66 17.28 ± 2.64

30.76 ± 0.95 30.37 ± 0.84 30.61 ± 1.23 70.22 ± 0.73

24.34 ± 0.24 24.72 ± 0.80 23.48 ± 1.03 68.40 ± 0.14

37.71 ± 1.31 38.49 ± 1.38 37.83 ± 1.69 69.66 ± 0.80

35.55 ± 0.92 35.89 ± 0.65 35.89 ± 1.27 67.09 ± 0.51

28.30 29.21 27.74 52.82

Task Arithmetic TIES TSV RAM OrthoMerge-G-TIES [31]

26.24 ± 2.29 30.48 ± 2.58 26.12 ± 1.78 27.41 ± 1.87 28.71 ± 2.11

21.46 ± 0.52 21.41 ± 0.66 19.39 ± 1.36 21.72 ± 0.57 21.65 ± 0.66

61.86 ± 0.16 65.91 ± 0.74 65.33 ± 0.99 65.30 ± 0.34 64.88 ± 1.15

58.54 ± 0.90 64.89 ± 0.89 65.12 ± 0.20 69.79 ± 0.59 63.43 ± 0.55 69.36 ± 0.90 65.82 ± 0.90 69.57 ± 0.43 64.80 ± 0.59 68.31 ± 0.38

61.44 ± 0.56 66.66 ± 0.46 65.32 ± 0.98 65.75 ± 0.61 63.62 ± 0.54

49.07 53.23 51.49 52.59 52.00

Ours

33.50 ± 2.28 21.80 ± 1.61 71.78 ± 0.96 66.12 ± 0.31 69.75 ± 0.29

Method

HotpotQA

SQuAD

66.13 ± 0.52 54.85

ISO: An RLVR-Native Optimization Stack

Table 7 | Main results on DeepSeek-R1-Distill-Qwen-1.5B with 2 RL experts under Best@4. Coding ACC on LB and LCB v5. Math: ACC on AIME24, AIME25, AMC23, Minerva, and OlympiadBench. Best expert/merged result per column in bold. Shading indicates Experts and Ours . Coding

Method Base [4] Archer2.0 [32] JustRL [33]

Math

LCB v5

AIME24

AIME25

AMC23

Minerva

Olympiad

Total

ACC

Avg

29.68 ± 0.17 25.96 ± 0.42 54.49 ± 1.34 33.34 ± 0.99 80.49 ± 0.80 41.67 ± 2.00 56.05 ± 0.56 45.95 36.11 ± 0.45 36.75 ± 0.27 64.74 ± 0.72 39.88 ± 1.12 86.49 ± 0.12 43.38 ± 0.52 61.19 ± 0.32 52.65 31.67 ± 0.42 32.63 ± 0.62 71.62 ± 0.16 49.25 ± 1.32 90.83 ± 0.65 45.22 ± 1.08 64.49 ± 0.91 55.10

Task Arithmetic [27] 36.10 ± 0.45 37.18 ± 0.35 72.34 ± 0.54 TIES [28] 34.31 ± 0.69 37.66 ± 0.31 73.40 ± 0.42 TSV [29] 35.84 ± 0.34 37.67 ± 0.39 72.41 ± 1.07 RAM [30] 35.34 ± 0.87 36.96 ± 1.04 73.06 ± 0.62 OrthoMerge-G-TIES [31] 35.18 ± 0.72 37.47 ± 0.50 73.95 ± 1.16 Ours

47.12 ± 2.32 47.37 ± 0.77 46.94 ± 1.18 49.50 ± 1.81 47.13 ± 0.96

90.51 ± 0.17 89.62 ± 0.38 89.93 ± 1.10 90.36 ± 0.63 89.82 ± 0.51

44.61 ± 0.46 45.10 ± 0.87 45.10 ± 0.92 44.98 ± 0.76 44.36 ± 0.56

64.20 ± 0.56 64.35 ± 0.35 64.94 ± 0.42 64.59 ± 0.55 64.74 ± 0.68

56.01 55.97 56.12 56.40 56.09

34.04 ± 0.56 36.87 ± 0.09 72.69 ± 0.34 50.43 ± 0.71 91.31 ± 0.61 44.73 ± 0.87 64.99 ± 0.56 56.44

Table 8 | Main results on DeepSeek-R1-Distill-Qwen-1.5B with 2 RL experts under Worst@4. Coding ACC on LB and LCB v5. Math: ACC on AIME24, AIME25, AMC23, Minerva, and OlympiadBench. Best expert/merged result per column in bold. Shading indicates Experts and Ours . Coding

Method Base Archer2.0 JustRL

LCB v5

AIME24

AIME25

AMC23

Minerva

Olympiad

Total

ACC

Avg

8.38 ± 0.38 8.95 ± 0.47 12.42 ± 0.92 12.96 ± 0.56 46.02 ± 0.64 14.95 ± 0.76 30.17 ± 0.73 19.12 15.95 ± 0.80 17.39 ± 0.24 19.80 ± 0.63 17.36 ± 0.36 57.83 ± 1.03 18.75 ± 1.08 38.22 ± 0.60 26.47 14.02 ± 0.22 14.42 ± 0.14 33.61 ± 0.93 27.25 ± 1.15 72.89 ± 0.65 23.90 ± 0.60 46.32 ± 0.14 33.20

Task Arithmetic 15.82 ± 0.26 TIES 17.76 ± 0.87 TSV 15.91 ± 0.52 RAM 17.41 ± 1.11 OrthoMerge-G-TIES [31] 16.45 ± 0.67 Ours

Math

16.67 ± 0.50 18.22 ± 0.27 17.46 ± 0.16 17.84 ± 0.25 18.27 ± 0.19

28.49 ± 0.56 33.40 ± 1.45 32.33 ± 0.38 33.90 ± 0.86 33.21 ± 1.99

20.99 ± 0.47 25.21 ± 0.22 24.69 ± 1.48 24.10 ± 0.14 24.86 ± 0.60

68.35 ± 0.85 71.85 ± 0.97 68.36 ± 1.44 71.46 ± 0.29 70.85 ± 0.80

21.94 ± 0.92 23.28 ± 1.25 24.02 ± 0.87 25.12 ± 0.35 23.77 ± 0.21

43.41 ± 0.64 45.23 ± 0.07 42.86 ± 0.81 45.48 ± 0.12 45.28 ± 0.60

30.81 33.57 32.23 33.62 33.24

16.18 ± 0.18 18.95 ± 0.45 36.10 ± 1.01 27.41 ± 0.83 73.23 ± 1.81 25.12 ± 1.21 47.85 ± 0.12 34.98

ISO: An RLVR-Native Optimization Stack

G. Numerical Precision of the SVD-Based Retraction In exact arithmetic, the ISO reconstruction 𝑊 + = 𝑈 + Σ0 ( 𝑉 + ) ⊤

has singular values diag( Σ0 ) whenever 𝑈 + and 𝑉 + have orthonormal columns. In practice, however, the polar retraction is computed using a finite-precision SVD, so numerical errors in the decomposition can weaken this invariant. We therefore compare four PyTorch SVD configurations: FP32 and FP64 on CPU and GPU. As a representative sanity check, we extract the q_proj matrix from layer 10 of Qwen3-1.7B-Base, 𝑊0 ∈ ℝ2048×2048 ,

compute

𝑊0 = 𝑈Σ𝑉 ⊤ ,

and reconstruct

𝑊rec = 𝑈Σ𝑉 ⊤ .

We report the mean squared spectral reconstruction error 𝜖𝜎 :=

1 𝑞

∥ 𝜎 (𝑊rec ) − 𝜎 (𝑊0 ) ∥ 22 ,

(118)

together with the wall-clock decomposition time. Table 9 shows that FP32 GPU SVD introduces substantially larger numerical error than FP64 GPU SVD: 7.7899 × 10−4 versus 2.8924 × 10−8 in this representative matrix. FP64 reduces the error by approximately 2.7 × 104 while increasing the measured runtime only from 0.894 to 0.923 seconds. We therefore use FP64 GPU SVD for the polar retraction throughout ISO. Table 9 | Runtime and mean squared spectral reconstruction error under different PyTorch SVD precision and device configurations. Precision

Device

Mean squared spectral error

Time (s)

FP32 FP64 FP32 FP64

CPU CPU GPU GPU

3.6633e-7 1.1126e-8 7.7899e-4 2.8924e-8

2.115 3.699 0.894 0.923

H. Online RLVR Training Details The ISO parameterization, factor gradients, and retraction are described in Section 4.3. This appendix reports the training and evaluation configurations used in the online RLVR experiments. Common setup. All experiments are implemented using Verl [9] and run on NVIDIA A100 80 GB GPUs. We use the DAPO algorithm [5] and hold all non-optimizer settings fixed between the weight-space and ISO variants. We use asymmetric policy-ratio clipping with ( 𝜖low , 𝜖high ) = (0.2, 0.28) .

(119) 45

ISO: An RLVR-Native Optimization Stack

The KL coefficient is set to

𝛽KL = 10 −3 ,

with the KL term applied as a loss-shaping term rather than as part of the rollout reward. We enable online dynamic filtering by removing prompt groups whose sampled rollouts are either all correct or all incorrect. Unless stated otherwise, training rollouts use temperature 1.0 and top- 𝑝 = 1.0. We set weight decay to zero for both the weight-space and ISO variants. At the learning rates used here, the standard coefficient 𝜆 = 10−2 falls below the BF16-visible update scale and provides no measurable benefit in our RLVR runs [1]. Mathematical reasoning. We train Qwen3-1.7B-Base, Qwen3-4B-Base, and Qwen3-8B-Base [34] on DeepMath-103K [35], following prior RLVR post-training recipes [42]. For Qwen3-1.7B-Base and Qwen3-4B-Base, we use a global prompt batch size of 256 and train for 400 training steps, corresponding to approximately one nominal pass over the prompt set before online filtering. The 1.7B runs sample 16 rollouts per prompt, whereas the 4B runs sample 12. Both use a maximum prompt length of 1,024 tokens and a maximum response length of 8,192 tokens. ISO-AdamW uses a learning rate of 7.5 × 10−7 for the frame variables (𝑈, 𝑉 ), followed by polar retraction after every tentative factor update.3 For weight-space AdamW, we sweep 5 × 10−7 , 7.5 × 10−7 , 1 × 10−6 , 2 × 10−6 , 3 × 10−6 . For the weight-space Muon baselines, we sweep 2.5 × 10−5 , 5 × 10−5 , 7.5 × 10−5 , 1 × 10−4 . The ISO-Muon experiment is conducted on Qwen3-4B-Base for 300 training steps using a learning rate of 5 × 10−5 . For Qwen3-8B-Base, we retain the global prompt batch size of 256 with a rollout size of 12. We compare ISO-AdamW with learning rate 7.5 × 10−7 against weight-space AdamW with learning rate 2 × 10−6 , the strongest AdamW setting identified in the smaller-model sweeps. Neither method is further tuned at 8B. Both runs are trained for 210 training steps. To test whether the AdamW baseline is undertrained, we continue it for an additional 60 training steps. The 8B runs begin with a maximum response length of 8,192 tokens. Because ISO-AdamW produces longer responses at this scale, we increase the maximum response length to 16,384 tokens for both methods after step 80. Evaluation uses a 16,384-token cap throughout. We evaluate mathematical reasoning on AIME 2024, AIME 2025, AMC 2023, Minerva, and OlympiadBench. For each problem, we sample 16 responses using temperature 1.0 and top- 𝑝 = 0.8. The maximum evaluation length is 8,192 tokens for the 1.7B and 4B models and 16,384 tokens for the 8B model. For Table 3, we select the checkpoint with the highest aggregate score among the final three evaluations and report all benchmark scores from that checkpoint. Competitive coding. For competitive coding, we train DeepSeek-R1-Distill-Qwen-1.5B [4] on the ArcherCodeR training split [43], which contains 6,753 problems. We use a global prompt batch size of 128 and sample 16 rollouts per problem. The maximum prompt length is 2,048 tokens, and the maximum response length is 32,768 tokens. 3On Qwen3-1.7B-Base, we compare learning rates 7.5 × 10 −7 and 1 × 10 −6 . The former performs better, so we use it

throughout the math experiments.

ISO: An RLVR-Native Optimization Stack

ISO-AdamW uses a learning rate of 1 × 10−6 for (𝑈, 𝑉 ), followed by polar retraction after each factor update. For the weight-space AdamW baseline, we sweep 1 × 10−6 , 2 × 10−6 , 3 × 10−6 , 5 × 10−6 , where 1 × 10−6 is the learning rate used by the original Archer-style recipe. We train the primary runs for 220 training steps. At a prompt batch size of 128, this budget corresponds to approximately 220 × 128 ≈ 4.17 6753 nominal passes over the prompt set before accounting for online filtering. Because the coding dataset is relatively small, longer training can enter a multi-pass overfitting regime. To test whether the primary budget truncates the weight-space baseline prematurely, we continue the two strongest AdamW runs, with learning rates 3 × 10−6 and 5 × 10−6 , to 330 training steps. For evaluation, we report results on LiveCodeBench [44] v5 and v6. We sample 8 responses per problem using temperature 0.8 and top- 𝑝 = 1.0, and report average accuracy under the same evaluation protocol for all methods.

Record · ID 386854 · SHA-256 e2d7f51ffe3a72c0
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.