arXiv CS open access

FADE: Factored Ability--Difficulty Filtering for Online RLVR Scheduling

Tomas Duncun et al. · arxiv_cs

arXiv CS · Papers

deep-learning machine-learning neural-networks

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

FADE: Factored Ability–Difficulty Filtering for Online RLVR Scheduling Tomas Duncun1 , Jonah Ellison1 , Mateo Ruiz1 , Hanae Fujita2 , Ingrid Solberg2 1 2

University of Michigan University of Edinburgh

Abstract

2025; Kimi Team et al., 2025). Group-relative algorithms such as GRPO sample G completions per prompt and standardize the binary rewards within each group (Shao et al., 2024; Yu et al., 2026; Zheng et al., 2025). The bottleneck of this recipe is rollout generation, and its central pathology is group degeneracy: whenever a prompt is trivial for the current policy (ri =1) or far beyond its reach (ri =0), all G completions receive the same reward, the group advantage vanishes, and the completions do no gradient work (Yu et al., 2026; Yao et al., 2026a; Mao et al., 2026b). Adaptive RLVR schedulers attack group degeneracy from two angles. Curriculum selectors filter or prioritize prompts whose pass rate sits near the learning frontier (Mao et al., 2026a; Gao et al., 2026; Zheng et al., 2026; Kong et al., 2026; Mao et al., 2026b; Bae et al., 2026; Do et al., 2026), while budget allocators vary the group size ni,t per prompt to reduce gradient variance or maximize progress (Yao et al., 2026a; Nguyen et al., 2026; Zeng et al., 2026; Zou et al., 2026; Fang et al., 2026). Both families need an online estimate of each prompt’s current pass probability pi,t before rollouts are drawn. Existing trackers treat pi,t as an unstructured perprompt scalar and run into a fundamental dilemma. On-policy probing before each step is accurate but consumes tens of GPU-hours in extra generations (Yao et al., 2026a; Zeng et al., 2026; Xu and Ding, 2025), while small generalizable prompt predictive models (Qu et al., 2026; Gao et al., 2026) require auxiliary model training and periodic recalibration. Zero-probe trackers instead smooth past training outcomes for each prompt via exponential moving averages, sliding windows, or dynamicspredictive Beta posteriors (Mao et al., 2026a; Kong et al., 2026; Zheng et al., 2026; Nguyen et al., 2026). Because a corpus contains 105 prompts and a batch touches only 256, any individual prompt is revisited once every several hundred steps. Be-

arXiv:2611.03625v1 [cs.LG] 2 Nov 2026

Group-based reinforcement learning with verifiable rewards (RLVR) learns only from prompts whose sampled completions disagree: all-pass and all-fail groups produce zero relative advantage and burn generation budget without moving the policy. Adaptive schedulers try to steer rollouts toward informative prompts, yet they rely on per-prompt pass-rate trackers that entangle two quantities evolving at very different speeds—the policy’s shifting competence across reasoning skills and each prompt’s intrinsic difficulty. When a single exponential average or Beta counter tracks both at once, short memory scrambles the relative ordering of prompts whereas long memory reports stale pass rates. We introduce FADE (Factored Ability–Difficulty Estimation), a bilinear statespace filter that separates the two timescales. FADE represents each prompt’s success logit as the inner product of a whitened K-dimensional skill loading with a fast-drifting policy ability vector, minus a slow-shrinking scalar item intercept. Using Pólya–Gamma augmentation, each minibatch of binary rollout outcomes updates the K-dimensional ability posterior in closed form via a rank-K Woodbury step, instantly transferring competence shifts to all unobserved prompts that share those skills while leaving item intercepts stable. From the resulting posterior mean and variance we derive a closed-form utility for the probability of drawing a non-degenerate rollout group. Across five reasoning benchmarks, two base models, and four scheduling paradigms under a matched rollout budget, FADE consistently improves downstream accuracy and calibration while adding less than 0.6 ms per step.

Introduction

Reinforcement learning with verifiable rewards (RLVR) turns a rule-based answer checker into a policy improvement loop for mathematical and scientific reasoning (Shao et al., 2024; Guo et al., 1

0.75 Within-skill item rank Absolute pass-rate level

0.50 0.25 0.00

FADE (factored) CDAS (history gap)

0.8 0.7

DPS (per-item Beta) better

0.6 single-timescale EMA sweep

0.5 0

100

150

200

training lag Δt (steps)

250

0.20

0.18

0.16

0.14

0.12

(b) Breaking the coupling trade-off 0.9

rank Spearman ρ (↑)

temporal autocorrelation

(a) Level drifts; item order persists 1.00

FADE-guided EMA-guided Uniform GRPO

40 20 0

calibration Brier score (↓)

0.25

0.50

0.75

1.00

1.25

cumulative rollouts (×106)

Figure 1: Why per-prompt trackers face a coupling trade-off. (a) Along a GRPO run on NuminaMath, the absolute pass-rate level decorrelates within ∼100 steps as the policy improves, whereas the relative difficulty ordering of prompts within a skill domain remains correlated over hundreds of steps. (b) Sweeping the decay rate of a per-prompt EMA traces a frontier between calibration error (Brier score) and rank correlation (Spearman ρ); per-item Bayesian trackers (DPS, CDAS) sit near that curve, whereas FADE decouples the two timescales. (c) Supplying factored estimates to a rollout scheduler sustains a higher fraction of non-zero advantage groups across the training budget.

tween visits, two things happen at very different speeds (Figure 1(a)): the policy’s overall competence on a mathematical skill drifts quickly, while the fine-grained hardness gap between two problems requiring that skill barely moves. A single per-prompt memory parameter cannot serve both. Fast forgetting tracks recent pass rates on visited items but destroys the relative ranking of rarely visited items; slow forgetting preserves item ordering but reports badly lagging pass probabilities (Figure 1(b)).

plugging point estimates into heuristic schedulers, we integrate the posterior uncertainty over the binomial group outcome to obtain a closed-form nondegenerate group utility Ui,t that directly scores the expected probability of drawing a mixed-reward group. Our contributions are three-fold: • We show that online difficulty estimation in RLVR is governed by two separated timescales— fast low-rank skill drift and slow item-level ordering—which single-timescale per-prompt trackers cannot simultaneously capture (§1, §3.2).

We resolve this tension by factoring the logit of pi,t into a low-rank dynamic policy state and a scalar static item intercept. Specifically, we project frozen prompt embeddings onto K orthonormal whitened skill coordinates zi ∈ RK (K=32) and model the success logit at step t as µi,t = z⊤ i θt −bi . Here θt ∈ RK is a fast-evolving skill proficiency vector shared across the entire corpus, and bi ∈ R is a slow-shrinking intrinsic difficulty offset specific to prompt i. Every minibatch of 256 prompts provides thousands of rollout outcomes that jointly pin down θt ; updating θt immediately shifts the predicted pass rate of all N prompts in proportion to their skill loadings zj , even if prompt j has never been sampled. Meanwhile, bi accumulates evidence over the full run with minimal forgetting, keeping within-skill prompt rankings sharp.

• We propose FADE, a Pólya–Gamma bilinear state-space filter that updates a K-dimensional skill state and per-item intercepts in closed form, paired with an uncertainty-calibrated nondegenerate group utility (§3). • We prove that FADE’s Woodbury step is the exact conjugate mode under local Pólya–Gamma conditioning, that skill sharing strictly contracts predictive variance on unvisited prompts, and that factored filtering dominates single-memory trackers whenever skill drift exceeds item drift (§4). Across five math benchmarks and four schedulers, FADE consistently improves RLVR efficiency (§5).

To make inference exact and fast without inner optimization loops, FADE uses the Pólya–Gamma augmentation for binomial likelihoods (Polson et al., 2013). Conditioning on the local curvature parameter turns each step into a conjugate Gaussian Kalman update whose K × K covariance is maintained in O(|Bt |K 2 ) time. Finally, rather than

Related Work

Curriculum selection and rollout allocation in RLVR. Classical curriculum learning orders examples by difficulty or learnability (Bengio et al., 2009; Mindermann et al., 2022). In group-relative RLVR (Shao et al., 2024; Yu et al., 2026; Zheng 2

et al., 2025), prompts whose completions all succeed or all fail yield zero advantage, motivating group-prioritized optimization that eliminates ineffective samples (Mao et al., 2026b; Yu et al., 2026). Recent selectors therefore filter or reweight prompts using dynamics-predictive sampling (Mao et al., 2026a), lightweight prompt predictive models (Qu et al., 2026; Gao et al., 2026), competence gaps, or reward dynamics (Zheng et al., 2026; Kong et al., 2026; Bae et al., 2026; Do et al., 2026; Sun et al., 2025; Zhang and Zuo, 2025; Li et al., 2025; Lai and Nissim, 2026; Jia et al., 2026; Pant et al., 2026). A parallel line allocates non-uniform rollout budgets ni,t across a batch to minimize policygradient variance (Yao et al., 2026a; Nguyen et al., 2026; Zeng et al., 2026), unify budget allocation across reasoning and agentic RL (Zou et al., 2026), or maximize sequential utility (Xiong et al., 2025; Fang et al., 2026; Jiang et al., 2026a; Chen et al., 2026; Yao et al., 2026b; Deng et al., 2026; Jiang et al., 2026b; Kalarde et al., 2026). Complementary work prunes partial trajectories or rescales advantages (Xu et al., 2026a; Xia et al., 2026; Wang et al., 2026a; Yang et al., 2026a; Dao et al., 2026; Son et al., 2026; Gholipour et al., 2026). All of these schedulers consume an online pass-rate estimate; FADE replaces that estimator without altering the downstream scheduling rule.

2026b; Zhang et al., 2026b; Yang et al., 2026b; Wang et al., 2026d) address related aspects of nonstationarity. FADE brings multidimensional IRT online via Pólya–Gamma filtering (Polson et al., 2013), separating fast low-rank skill evolution from static item hardness.

Method

3.1

Setup and Group Degeneracy

Consider a prompt pool D = {xi }N i=1 . At training step t, the scheduler selects a minibatch Bt ⊂ D and allocates ni,t rollouts to each i ∈ Bt under policy πt . A deterministic verifier scoresPeach comni,t pletion yi,t,g ∈ {0, 1}, yielding si,t = g=1 yi,t,g successes and empirical pass rate ri,t = si,t /ni,t . In group-relative updates, a prompt provides a nonzero policy gradient only if 0 < si,t < ni,t . Under true pass probability pi,t = Pr(y=1 | xi , πt ), the probability that a group of size G is non-degenerate is G ΦG (pi,t ) = 1 − pG i,t − (1 − pi,t ) .

(1)

Maximizing rollout efficiency requires estimating pi,t (and its epistemic uncertainty) for all i ∈ D prior to step t using only the sparse historical log H<t = {(i, τ, si,τ , ni,τ )}τ <t . 3.2

Dynamic difficulty tracking and Item Response Theory. Static item response theory (IRT) separates model ability from item difficulty in offline NLP evaluation (Lalor et al., 2016; Rodriguez et al., 2021). During RLVR, however, the policy moves at every step (Zhou et al., 2026; Zhang et al., 2026a). Probing methods estimate current pass rates via extra on-policy rollouts (Yao et al., 2026a; Zeng et al., 2026; Xu and Ding, 2025), trading compute for freshness. Amortized predictive models estimate prompt utility or difficulty from text features (Qu et al., 2026; Gao et al., 2026; Wang et al., 2026b; Zhou et al., 2026), offering strong zero-shot generalization when paired with online adaptation. History-based trackers update per-prompt statistics from training rollouts—via dynamics-predictive Beta-Binomial filtering in DPS (Mao et al., 2026a), historical discrepancy (Kong et al., 2026), reward regularity (Zheng et al., 2026), or local regression (Nguyen et al., 2026)—while unified rollout trackers (Zou et al., 2026), offline tree structures (Wang et al., 2026c), bandit curricula (Xu et al., 2026b), and lag-corrected replay (Mao et al.,

Bilinear Skill–Difficulty Model

Instead of assigning an isolated scalar state to each prompt, we decompose success logits into shared skill dimensions and item-specific offsets (Figure 2(A)). Before training, we pass each prompt xi through a frozen text encoder fenc and extract the top K principal components across the corpus, followed by Mahalanobis whitening so that theP resulting skill loading vectors zi ∈ RK satisfy N 1 ⊤ i=1 zi zi = IK . We set K=32, which capN tures over 86% of the embedding variance across mathematical domains while keeping matrix operations tiny. At step t, we model the binomial rollout count si,t | ni,t , µi,t ∼ Binomial(ni,t , σ(µi,t )) via the bilinear logit µi,t = z⊤ i θt − bi,t ,

(2)

where θt ∈ RK is the policy’s current proficiency across the K latent skills, and bi,t ∈ R is the intrinsic difficulty intercept of prompt i. Crucially, θt and bi,t obey distinct temporal dy3

(A) Offline Skill Coordinates

(B) Dual-Timescale Pólya–Gamma Filter

Prompt corpus  = {xi}N i=1 frozen encoder fenc(xi)

Global transfer Δθt̂ shifts μ̂ j, t of all N prompts sharing skills zj, even unobserved

Whitened skill basis zi ∈ ℝ K, K = 32 orthogonal skill factors

Fast skill state θt ∈ ℝ K θt = ρ θt − 1 + ξt, ρ = 0.82 rank-K Woodbury Kalman step

Item intercept prior bi ∼ (b̂ i, 0, τ02) initial difficulty offset

Slow item intercept bi ∈ ℝ −2 τi,−2 t = γb τi, t − 1 + ωi, t long memory keeps item order

(C) Utility & RLVR Scheduling Non-degenerate group utility π 2 p̂ i, t = σ(μ̂ i, t/√ 1 + 8 si, t ) i, t = 1 − p̂ G − (1 − p)̂ G + cG si,2 t

Posterior readout μ̂ i, t = zi⊤ θt̂ − b̂ i, t si,2 t = zi⊤ Σtzi + τi,2 t

Pólya–Gamma update ni, t ci, t ωi, t = 2c tanh( 2 )

Curriculum selection & allocation filter batch t, or allocate ni, t ∝ √ i, t

i, t

Policy step πt → πt + 1 rollout counts (si, t, ni, t)i ∈ t

κi, t = si, t − 12 ni, t

Key separation: fast rank-K ability θt absorbs non-stationary policy drift; slow scalar intercept bi preserves fine-grained prompt ranking.

Figure 2: FADE architecture. (A) Offline, frozen prompt representations are whitened into K orthogonal skill loadings zi ∈ RK and an initial difficulty offset b̂i,0 . (B) Online, a dual-timescale Pólya–Gamma filter updates the fast policy skill state θt ∈ RK (forgetting ρ) via a rank-K Woodbury step and refines the slow item intercept bi (forgetting γb ≈1). Changes in θ̂t immediately update all N prompts sharing those skills. (C) The posterior mean and variance define the expected non-degenerate group utility Ui,t for prompt selection and rollout allocation.

|µ̂i,t|t−1 | is large, ωi,t decays as 1/|µ̂|, preventing lucky or unlucky rollouts on saturated prompts from destabilizing the skill state.

namics: θt = ρ θt−1 + ξt ,

ξt ∼ N (0, σθ2 IK ),

bi,t = bi,t−1 + ηi,t ,

ηi,t ∼ N (0, σb2 I[i ∈ Bt ]).

(3)

Two-step closed-form update. Let (θ̂t−1 , Σt−1 ) be the Gaussian posterior of the skill vector after 2 step t−1, and let (b̂i,t−1 , τi,t−1 ) be the scalar posterior of intercept bi . Prior to observing Bt , the time update (3) yields

(4) Here ρ ∈ (0, 1) (default 0.82) pulls the skill state toward a moving horizon so θt rapidly tracks policy improvements or regressions across entire problem families. Conversely, bi,t has near-zero drift (σb2 ≪σθ2 , implemented via precision discounting γb =0.985), allowing each prompt to accumulate long-horizon evidence about its relative hardness within its skill neighborhood. 3.3

θ̂t|t−1 = ρθ̂t−1 ,

Σt|t−1 = ρ2 Σt−1 +σθ2 IK . (7)

Once rollout outcomes {(si,t , ni,t )}i∈Bt arrive, we form κi,t = si,t − ni,t /2 and ωi,t via (6). Marginal2 izing the slow intercept bi ∼ N (b̂i,t−1 , τi,t−1 ) inside the observation model yields the effective −1 2 skill precision ω̃i,t = (ωi,t + τi,t−1 )−1 . Stack|B |×K ing Zt ∈ R t , Ω̃t = diag(ω̃i,t ), and residual pg vector ri,t = κi,t /ωi,t + b̂i,t−1 −z⊤ i θ̂t|t−1 , the exact conjugate update for θt is −1 ⊤ Σt = Σ−1 + Z Ω̃ Z , (8) t t t t|t−1

Online Pólya–Gamma Kalman Filtering

Directly updating (θt , {bi,t }) under a binomial likelihood is non-conjugate. We invoke the Pólya– Gamma integral identity (Polson et al., 2013): Z ∞ 1 (eµ )s 2 −n κµ e− 2 ωµ pPG (ω | n, 0)dω, =2 e µ n (1 + e ) 0 (5) where κ = s − n/2. Conditioned on the auxiliary precision variable ωi,t ∼ PG(ni,t , ci,t ), the binomial likelihood for prompt i becomes exactly Gaussian in the logit µi,t , with effective precision ωi,t and shifted observation κi,t /ωi,t . Setting ci,t = |µ̂i,t|t−1 | to the prior predictive logit magnitude, the conditional expectation of ωi,t has the closed form c ni,t i,t ωi,t = E[ω | ni,t , ci,t ] = tanh , (6) 2ci,t 2

pg θ̂t = θ̂t|t−1 + Σt Z⊤ t Ω̃t rt .

(9)

Finally, for each visited prompt i ∈ Bt , we update its scalar difficulty intercept against the newly refined skill state: −2 −2 τi,t = γb τi,t−1 + ωi,t ,

(10) κi,t 2 ⊤ b̂i,t = b̂i,t−1 + τi,t ωi,t zi θ̂t − b̂i,t−1 − . ωi,t (11)

2 ) = Unvisited prompts (j ∈ / Bt ) keep (b̂j,t , τj,t 2 (b̂j,t−1 , τj,t−1 ), yet their predicted logit µ̂j,t+1 = ⊤ zj θ̂t − b̂j,t updates automatically through θ̂t .

with limc→0 ωi,t = ni,t /4. Notice how (6) automatically downweights extreme prompts: when 4

precisions {ωi,t }i∈Bt in (6), equations (8)–(9) compute the exact posterior mean and covariance of θt after marginalizing {bi,t }i∈Bt . The update requires O(|Bt |K 2 +K 3 ) operations and O(N +K 2 ) memory, independent of N at update time.

Algorithm 1: FADE Online Filtering and Scheduling Step Input: skill loadings {zi }N i=1 ; state (θ̂t−1 , Σt−1 , {b̂i , τi2 }); candidate pool Ct Output: scheduled rollouts {ni,t }i∈Bt ; updated state (θ̂t , Σt , {b̂i , τi2 })

Proposition 2 (Zero-shot variance contraction via skill sharing). Let prompt j ∈ / Bt be an unvisited prompt. Whenever there exists i ∈ Bt with nonorthogonal skill loading z⊤ j Σt|t−1 zi ̸= 0, the predictive variance of prompt j strictly contracts after step t:

θ̂t|t−1 ← ρθ̂t−1 ; Σt|t−1 ← ρ2 Σt−1 + σθ2 IK foreach i ∈ Ct do µ̂i,t ← z⊤ i θ̂t|t−1 − b̂i,t−1 ; 2 s2i,t ← z⊤ i Σt|t−1 zi + τi,t−1 q p̂i,t ← σ µ̂i,t / 1 + π8 s2i,t ; compute Ui,t via (13) end select batch Bt or allocate rollouts {ni,t } using (p̂i,t , Ui,t ); execute rollouts to observe {si,t } −1 2 compute ωi,t via (6) and ω̃i,t = (ωi,t + τi,t−1 )−1 for i ∈ Bt update (Σt , θ̂t ) via K×K solve (8)–(9); update 2 (τi,t , b̂i,t ) via (10)–(11) return {p̂i,t , Ui,t } and updated filter state

3.4

⊤ s2j,t|t−1 − s2j,t = z⊤ j Σt|t−1 Zt ⊤ −1 × Ω̃−1 Zt Σt|t−1 zj > 0. t + Zt Σt|t−1 Zt (14)

In contrast, any independent per-prompt tracker (K=0) has s2j,t|t−1 − s2j,t = 0 for all j ∈ / Bt . Proposition 3 (Minimax dominance of two– timescale factorization). Suppose the true logit follows µi,t = z⊤ i θt −bi where θt drifts with step variance σθ2 > 0, bi has variance σb2 ≪ σθ2 , and each prompt is visited with probability q = |B|/N ≪ 1. Any single-timescale per-prompt tracker µ̂i,t = (1 − α)µ̂i,t−1 + αyi,t incurs steady-state meansquared error bounded below by

Variance-Aware Non-Degenerate Group Utility

Before step t, any prompt i has predictive logit distribution µi,t ∼ N (µ̂i,t , s2i,t ) with µ̂i,t = z⊤ i θ̂t|t−1 − b̂i,t−1 and total epistemic variance 2 2 si,t = z⊤ i Σt|t−1 zi + τi,t−1 . Marginalizing over µi,t yields the calibrated pass probability   µ̂i,t  . (12) p̂i,t = E[σ(µi,t )] ≈ σ  q 1 + π8 s2i,t

s min E (µ̂i,t − µi,t )2 ≥

α∈(0,1]

(15)

where ν̄ is the per-visit qnoise, whereas q observation FADE achieves O Kσθ2 ν̄/|B|+ σb2 ν̄/q . Since K/|B| ≪ 1/q, factored filtering p reduces the drift-induced error by a factor of K/(N · q) = p K/|B|.

A second-order Laplace expansion of the nondegeneracy probability ΦG (σ(µi,t )) in (1) around µ̂i,t yields our scheduling utility: G 2 Ui,t = 1 − p̂G i,t − (1 − p̂i,t ) + cG p̂i,t (1 − p̂i,t ) si,t , (13) > 0. where cG = 12 G(G−1) (1−p̂i,t )G−2 +p̂G−2 i,t Equation (13) has a clean interpretation: the first term 1− p̂G −(1− p̂)G peaks at p̂=0.5, prioritizing prompts at the policy’s current competence boundary, while the variance bonus cG p̂(1 − p̂)s2i,t promotes uncertain prompts near the boundary so the scheduler actively resolves cold-start uncertainty. Algorithm 1 summarizes the complete loop.

2 σθ2 ν̄ , q

Proposition 3 pinpoints why FADE escapes the curve in Figure 1(b): a per-prompt tracker waits O(1/q) ≈ 580 steps between visits to learn that the policy improved, whereas FADE pools all |B|=256 prompts in every step to track the K=32 skill coordinates continuously.

Theoretical Analysis

Experiments

5.1

Experimental Setup

Backbones, training corpus, and evaluation benchmarks. We conduct RLVR training on two widely used 1B–1.5B reasoning backbones: Qwen2.5-Math-1.5B (Yang et al., 2024) and Llama-3.2-1B-Instruct (Grattafiori et al., 2024),

We establish three formal properties of FADE; full derivations appear in Appendix A. Proposition 1 (Conjugate Pólya–Gamma optimality and complexity). Conditioned on the auxiliary 5

and extend to 3B and 7B checkpoints in our scaling study (§5.4). Following recent RLVR scheduling protocols (Yao et al., 2026a; Gao et al., 2026; Zheng et al., 2026), we train on a 150k-problem subset of NuminaMath (Li et al., 2024; Cui et al., 2025) with verifiable final answers. To provide a broader and less noisy evaluation than fourbenchmark subsets, we evaluate across five mathematical reasoning benchmarks spanning diverse difficulty tiers: MATH-500 (Hendrycks et al., 2021; Lightman et al., 2024), AMC 2023 (40 competition problems), Minerva Math (Lewkowycz et al., 2022) (272 quantitative reasoning problems), OlympiadBench (He et al., 2024) (text-only math split), and AIME 2024–2025 (Balunović et al., 2026; Dekoninck et al., 2026) (60 olympiad problems combining 2024 and 2025 AIME I/II to reduce small-sample quantization noise). We report Average@8 accuracy (mean exact-match pass rate over 8 sampling draws at temperature 0.7).

tails are in Appendix B. 5.2

Main Downstream Results

Table 1 reports downstream accuracy across all five benchmarks. Replacing the native tracker with FADE lifts the five-benchmark average in every scheduler block on both models: on Qwen2.5Math-1.5B, the mean improves by +1.8 points under DAPO-Filter, +2.9 under PCL, +2.1 under GRESO, and +2.8 under GVM; on Llama-3.2-1BInstruct, the corresponding gains are +1.4, +1.3, +1.4, and +1.8 points. Three patterns stand out. First, FADE benefits both curriculum selection (DAPO, PCL, GRESO) and rollout allocation (GVM), whereas per-item trackers like +EMA and +DPS (Mao et al., 2026a) help moderately on GRESO but struggle when GVM needs calibrated pass rates on unvisited prompts to set rollout counts. Second, while strong baselines occasionally match or edge out FADE by 0.1–0.2 points on individual columns (e.g., +DPS on Qwen DAPO OlympiadBench or GRESO AMC23, and +CDAS on Llama PCL Minerva), FADE is the most consistent performer across difficulty tiers, with its largest margins on harder benchmarks (OlympiadBench and AIME, up to +2.9 points on GVM). Third, native GVM on Llama-3.21B-Instruct underperforms even Uniform GRPO (12.3% vs. 12.9%) because its pre-sampling overhead consumes rollout budget on a weak base model where most probes fail; replacing GVM’s probe with zero-probe FADE recovers +1.8 points (14.1%).

Scheduling paradigms and baselines. We test FADE both as a plug-in estimator inside four distinct RLVR schedulers and against standalone difficulty estimators: • Curriculum prompt selectors: DAPO-Filter (Yu et al., 2026; Mao et al., 2026b) (rejects ineffective groups with deterministic rewards), PCL (Gao et al., 2026; Qu et al., 2026) (selects prompts via a learned prompt predictive model), and GRESO (Zheng et al., 2026) (selective rollout filtering from reward history). • Rollout budget allocators: GVM (Yao et al., 2026a; Zou et al., 2026) (allocates ni,t ∝ p p̂(1 − p̂) under a fixed rollout budget) and VIP (Nguyen et al., 2026) (variance-informed rollout allocation).

5.3

Online Tracking Fidelity and Overhead

To verify that downstream gains stem from better probability estimates rather than lucky exploration, we replay a standard GRPO trajectory and compare each estimator’s predictions against an 8-rollout oracle reference across Epoch 1 (Cold-Start, where > 65% of prompts have zero or one visit), Epoch 2 (Policy-Shift, where competence rises steepest), and Epoch 3 (Late Convergence). Table 2 reports sample-level Brier score and batch-level Spearman rank correlation ρsp . In Epoch 1, per-item history trackers (VIP, DPS (Mao et al., 2026a)) are nearly blind on unvisited prompts (ρsp ∈ [0.29, 0.38]), while PCL’s learned value head quickly goes miscalibrated as the policy leaves the base checkpoint (Brier 0.218). Because FADE transfers rollout evidence across the K-dimensional skill basis from the very first mini-

• Difficulty estimators (plug-in comparison): Inside each host scheduler, we compare its Native estimator against +EMA (tuned per-prompt exponential moving average, α=0.25), +DPS (Mao et al., 2026a) (dynamics-predictive Bayesian posterior), +CDAS (Kong et al., 2026) (competence– difficulty alignment score), and +FADE. All methods run under a strictly matched total generation budget of 1.44 × 106 rollouts (450 steps × 256 prompts × mean 8 rollouts, including any pre-step probing required by native GVM). FADE uses frozen BGE-M3 (Chen et al., 2024) embeddings whitened to K=32 skill dimensions, ρ=0.82, γb =0.985, and σθ2 =0.015. Full implementation de6

Qwen2.5-Math-1.5B (Average@8, %) Scheduler

Llama-3.2-1B-Instruct (Average@8, %)

Estimator MATH AMC23 Minerva Olym. AIME Avg. MATH AMC23 Minerva Olym. AIME Avg.

Base Model — Uniform GRPO None

39.4 64.2

36.2 55.6

18.5 27.4

25.1 29.8

3.3 7.9

24.5 37.0

17.6 25.8

14.1 21.3

6.8 10.4

3.1 5.6

0.4 1.3

8.4 12.9

DAPO-Filter

Native +DPS +FADE

66.5 67.3 68.6

58.1 59.4 60.9

28.9 29.5 30.4

31.2 32.4 32.3

9.2 38.8 9.8 39.7 10.8 40.6

27.4 28.1 29.3

22.8 23.4 24.7

11.2 11.7 12.5

6.3 6.8 7.4

1.7 2.1 2.5

13.9 14.4 15.3

PCL

Native +CDAS +FADE

61.2 63.1 64.5

53.4 55.8 57.2

26.1 27.6 28.7

27.5 28.9 30.1

6.7 7.9 9.0

35.0 36.7 37.9

26.9 27.8 28.8

22.2 23.1 24.0

10.9 11.8 11.7

5.4 6.1 6.7

1.5 1.9 2.3

13.4 14.1 14.7

GRESO

Native +EMA +DPS +FADE

67.4 67.9 68.3 69.6

59.1 59.7 61.2 61.0

29.2 29.0 29.8 31.1

30.9 31.4 31.8 33.4

8.8 9.2 9.6 10.9

39.1 39.4 40.1 41.2

29.8 30.1 30.7 31.9

24.4 24.9 25.3 26.6

12.1 12.0 12.6 13.3

7.5 7.7 7.9 8.4

2.1 2.3 2.9 2.9

15.2 15.4 15.9 16.6

GVM

Native +EMA +CDAS +FADE

71.4 72.1 72.8 74.0

63.8 64.4 65.3 66.9

31.6 32.0 32.7 34.0

35.4 36.1 36.8 38.2

11.3 11.7 12.5 14.2

42.7 43.3 44.0 45.5

24.6 25.3 25.9 27.1

20.6 21.4 22.5 22.4

10.1 10.6 11.0 11.9

4.9 5.3 5.6 6.2

1.5 1.7 2.1 2.7

12.3 12.9 13.4 14.1

Table 1: Downstream reasoning accuracy (Average@8, %) across five benchmarks under a matched 1.44M rollout budget. Within each scheduler block, we keep the selection or allocation rule fixed and swap only the online difficulty estimator. AIME combines AIME 2024 and 2025 (60 problems total). Bold indicates the best result within each scheduler block. FADE improves the five-benchmark average across every scheduler and backbone. Epoch 1 (Cold) Epoch 2 (Shift) Epoch 3 (Late) Full Trajectory Backbone Estimator

Brier ↓ ρsp ↑ Brier ↓ ρsp ↑ Brier ↓ ρsp ↑ Brier ↓ ρsp ↑ Overhead

On-Policy Probe (8×) 0.078 SPO (Xu and Ding, 2025) 0.142 Qwen2.5 PCL Value Head (Gao et al., 2026) 0.218 VIP Tracker (Nguyen et al., 2026) 0.196 Math-1.5B DPS (Mao et al., 2026a) 0.204 CDAS (Kong et al., 2026) 0.179 FADE (Ours) 0.134

0.93 0.61 0.41 0.34 0.38 0.46 0.68

0.072 0.126 0.195 0.148 0.156 0.144 0.115

0.95 0.84 0.53 0.71 0.74 0.77 0.87

0.068 0.114 0.189 0.135 0.132 0.136 0.104

0.96 0.89 0.58 0.79 0.81 0.83 0.91

0.073 0.127 0.201 0.160 0.164 0.153 0.118

0.95 0.78 0.51 0.61 0.64 0.69 0.82

42.8 h 28.4 h 6.90 h 0.04 h 0.05 h 0.07 h 0.06 h

On-Policy Probe (8×) 0.064 SPO (Xu and Ding, 2025) 0.135 PCL Value Head (Gao et al., 2026) 0.184 Llama-3.2 1B-Instruct VIP Tracker (Nguyen et al., 2026) 0.188 DPS (Mao et al., 2026a) 0.192 CDAS (Kong et al., 2026) 0.165 FADE (Ours) 0.119

0.94 0.52 0.44 0.29 0.31 0.42 0.64

0.066 0.118 0.176 0.139 0.145 0.131 0.098

0.93 0.79 0.49 0.72 0.73 0.76 0.85

0.065 0.105 0.179 0.122 0.126 0.119 0.089

0.94 0.85 0.52 0.78 0.80 0.82 0.89

0.065 0.119 0.180 0.150 0.154 0.138 0.102

0.94 0.72 0.48 0.60 0.61 0.67 0.79

45.1 h 30.2 h 4.20 h 0.04 h 0.05 h 0.07 h 0.06 h

Table 2: Difficulty tracking accuracy and wall-clock estimation overhead across three GRPO epochs. At checkpoints within each epoch, we evaluate predicted pass probabilities against an 8-rollout oracle reference on 2,048 held-out training prompts. Brier (↓) measures probabilistic calibration (p̂i −riref )2 ; ρsp (↑) is the within-batch Spearman rank correlation. Prior-initialized defaults (p0 =0.5 with tiny tie-breaking jitter from prompt length) are used before a method’s first observation so every cell is well-defined. Bold marks the oracle-probe ceiling; underline marks the best zero-probe method.

batch (Proposition 2), it reaches ρsp =0.68 in Epoch 1 and beats even pre-training probing (SPO, 28.4 A100-hours) across the full trajectory on both Brier score (0.118 vs. 0.127) and rank correlation (0.82 vs. 0.78), while costing only 0.06 A100-hours total (one-time embedding + 0.54 ms/step CPU filtering).

5.4

Ablation and Diagnostic Analysis

Ablation study. Table 3 isolates each design choice. Forcing the skill vector and item intercepts to share a single decay rate (ρ=γb =0.90) degrades Brier score from 0.118 to 0.142 and drops GVM accuracy by 1.6 points, directly confirming Proposition 3. Removing the skill subspace (K=0) collapses Epoch-1 transfer and hurts GVM (−2.1 pts), whereas removing the item intercept (bi ≡0) forces 7

prompts. GVM+FADE centers its rollout density tightly around p⋆ ≈0.5, lifting empirical policygradient SNR by 28% over native GVM.

Track (Qwen) 5-Bench Avg Configuration

Brier ↓ ρsp ↑ GVM GRESO

FADE (full)

0.118

0.82

45.5

41.2

Timescale & structural ablations Coupled (ρ=γb =0.90) 0.142 No skill state (K=0) 0.161 No item intercept (bi ≡0) 0.139 Random basis zi 0.158

0.74 0.65 0.71 0.66

43.9 43.4 44.1 43.5

39.8 40.0 39.6 39.9

Inference & utility ablations Fixed weight ωi,t ≡ni,t /4 0.129 No probit variance scaling 0.135 No utility bonus in (13) 0.118

0.79 0.82 0.82

44.7 44.4 44.8

40.6 40.7 40.5

Conclusion

Online difficulty estimation in RLVR requires tracking a fast-moving policy across a sparsely observed prompt pool. By factoring each prompt’s success logit into a low-rank dynamic skill state and a slow-shrinking item intercept, FADE updates global competence from every minibatch in closed form via Pólya–Gamma augmentation while preserving fine-grained prompt ordering. Paired with an uncertainty-aware group utility, FADE serves as a drop-in upgrade for both curriculum selectors and rollout allocators under a matched generation budget.

Table 3: Component ablations on Qwen2.5-Math1.5B. We report full-trajectory tracking metrics and the five-benchmark downstream average under GVM and GRESO.

all prompts with similar skill loadings to share the exact same pass rate, hurting fine-grained ranking (ρsp drops from 0.82 to 0.71). Replacing Pólya– Gamma weights (6) with a homoscedastic weight ω=n/4 allows extreme prompts to over-rotate θ̂t , costing 0.8 points on GVM. Finally, dropping the probit variance scaling in (12) or the exploration bonus in (13) leaves rank correlation unchanged (0.82) but reduces downstream accuracy by 0.7– 1.1 points because the allocator over-commits to cold-start point estimates.

Limitations FADE relies on a frozen embedding space to define the K-dimensional skill coordinates zi . If a training corpus mixes domains whose reasoning skills are poorly separated by the text encoder (e.g., short puzzle statements whose surface form hides deep algorithmic hardness), the linear skill term z⊤ i θt explains less variance and the filter falls back toward the per-item intercept bi , which requires direct visits to converge. Second, our Pólya–Gamma update assumes conditionally independent binary rewards within a group; multi-turn agentic or processreward settings with correlated partial trajectories require extending the likelihood beyond a scalar binomial count. Third, while combining AIME 2024 and 2025 doubles the evaluation pool to 60 problems, competition benchmarks remain subject to sampling variance, which we bound via multi-seed reporting and paired sign tests in Appendix C.1.

Mechanism verification. Figures 3 and 4 inspect what the filter learns during training: 1. Heterogeneous skill evolution (Figure 3(a)): Projecting θ̂t onto five annotated MATH subject axes reveals that RLVR does not lift competence uniformly—algebraic manipulation rises steeply in the first 150 steps while geometry lags. A scalar global baseline cannot represent this multi-speed shift; K=32 skill coordinates capture it naturally.

Ethics Statement

2. Debiased item intercepts (Figure 3(c)): When an independent tracker (DPS (Mao et al., 2026a)) observes a prompt at step 300, it conflates the policy’s high late-stage ability with the prompt being easy. Because FADE subtracts z⊤ i θ̂t in (11), its item intercept error |b̂i − b⋆i | is nearly half that of DPS across all visit counts.

This work investigates compute-efficient scheduling for reinforcement learning on public mathematical reasoning datasets. The terms “ability” and “difficulty” refer exclusively to statistical parameters of a language model policy and problem prompts, not human cognitive traits. Improving rollout efficiency reduces the energy and GPU footprint required for reasoning post-training, though any deployment of the resulting models remains subject to standard verification and safety checks. All datasets and pretrained checkpoints are used in accordance with their open-source licenses.

3. Centering rollout budget and boosting gradient SNR (Figure 4(a,c)): Because EMA pass-rate estimates lag behind the improving policy, GVM+EMA mistakes alreadymastered prompts (p⋆ ≈0.65) for frontier 8

0.50 0.25

Number Th. Calculus

0.00

Combinatorics Geometry

−0.25 0

100

200

300

(b) Sparse-prompt calibration

0.8 0.6 0.4 0.2 0.0 0.0

400

Ideal diagonal No variance corr. Per-item EMA FADE (ours)

intercept RMSE |bî − bi⋆| (↓)

1.0

Algebra

empirical pass rate

skill proficiency θt,̂ k

(a) Heterogeneous skill drift

0.2

RLVR training step t

0.4

0.6

0.8

1.0

Independent Beta (DPS) FADE (skill-adjusted)

0.6 0.4 0.2

direct rollout visits mi

predicted pass rate pi,̂ t

Figure 3: Internal mechanics of FADE. (a) Projected onto five interpretable math domains, the latent skill coordinates θ̂t,k evolve at markedly different speeds (Algebra rises rapidly; Geometry and Combinatorics improve slowly). (b) Reliability diagram on prompts with ≤2 prior visits: variance-aware probit scaling keeps FADE on the ⋆ diagonal. (c) Subtracting the shared skill trend z⊤ i θ̂t allows FADE to estimate item intercepts bi with much lower error than independent Beta tracking.

1.5 1.0

GVM + FADE

GVM + EMA

Uniform (n = 8)

0.5 0.0 0.0

0.2

0.4

0.6

0.8

oracle success rate pi⋆

1.0

(b) Ranking across training epochs PCL DPS

CDAS FADE

0.87

0.91

0.8 0.68

0.6 0.4 Epoch 1 (Cold)

Epoch 2 (Shift)

Epoch 3 (Late)

relative gradient SNR (↑)

2.0

1.0

rank Spearman ρ (↑)

normalized rollout density

(a) Budget focuses near p ⋆ ≈ 0.5

2.5

2.0

GVM + FADE

1.5

Native GVM

1.0

Uniform GRPO

100

200

300

400

RLVR training step t

Figure 4: Impact on RLVR scheduling and optimization. (a) Distribution of allocated rollouts across oracle pass rates p⋆i : FADE concentrates budget sharply around the maximally informative p⋆ ≈0.5 zone, whereas EMA lags behind policy drift and oversamples easy prompts (p⋆ ≈0.65). (b) Within-batch Spearman rank correlation across the three training epochs. (c) Relative policy-gradient signal-to-noise ratio (SNR) across training steps.

References

Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. 2025. Group sequence policy optimization. arXiv preprint arXiv:2507.18071.

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300.

Jiarui Yao, Yifan Hao, Hanning Zhang, Hanze Dong, Wei Xiong, Nan Jiang, and Tong Zhang. 2026a. Optimizing chain-of-thought reasoners via gradient variance minimization in rejection sampling and RL. Advances in Neural Information Processing Systems, 38:163245–163284.

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, and 1 others. 2025. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948.

Yixiu Mao, Yun Qu, Qi Wang, Heming Zou, and Xiangyang Ji. 2026b. RLVR without ineffective samples: Group prioritized off-policy optimization for LLM reasoning. arXiv preprint arXiv:2606.01281.

Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, and 1 others. 2025. Kimi k1.5: Scaling reinforcement learning with LLMs. arXiv preprint arXiv:2501.12599.

Yixiu Mao, Yun Qu, Qi Wang, Heming Zou, and Xiangyang Ji. 2026a. Dynamics-predictive sampling for active rl finetuning of large reasoning models. In International Conference on Learning Representations, volume 2026, pages 25242–25270.

Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, and 1 others. 2026. DAPO: An open-source LLM reinforcement learning system at scale. Advances in Neural Information Processing Systems, 38:113222–113244.

Zhaolin Gao, Joongwon Kim, Wen Sun, Thorsten Joachims, Sid Wang, Richard Yuanzhe Pang, and Liang Tan. 2026. Prompt curriculum learning for efficient LLM post-training. In The Fourteenth International Conference on Learning Representations.

Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong

Haizhong Zheng, Yang Zhou, Brian R. Bartoldson, Bhavya Kailkhura, Fan Lai, Jiawei Zhao, and Beidi

Chen. 2026. Act only when it pays: Efficient reinforcement learning for LLM reasoning via selective rollouts. Advances in Neural Information Processing Systems, 38:124321–124346.

Nicholas G. Polson, James G. Scott, and Jesse Windle. 2013. Bayesian inference for logistic models using Pólya–Gamma latent variables. Journal of the American Statistical Association, 108(504):1339–1349.

Deyang Kong, Qi Guo, Xiangyu Xi, Wei Wang, Jingang Wang, Xunliang Cai, Shikun Zhang, and Wei Ye. 2026. Rethinking the sampling criteria in reinforcement learning for LLM reasoning: A competencedifficulty alignment perspective. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 31438–31446.

Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. Curriculum learning. In Proceedings of the 26th International Conference on Machine Learning, pages 41–48. Sören Mindermann, Jan M. Brauner, Muhammed T. Razzak, Mrinank Sharma, Andreas Kirsch, Winnie Xu, Benedikt Höltgen, Aidan N. Gomez, Adrien Morisot, Sebastian Farquhar, and Yarin Gal. 2022. Prioritized training on points that are learnable, worth learning, and not yet learnt. In Proceedings of the 39th International Conference on Machine Learning, volume 162, pages 15630–15649.

Sanghwan Bae, Jiwoo Hong, Min Young Lee, Hanbyul Kim, JeongYeon Nam, and Donghyun Kwak. 2026. Online difficulty filtering for reasoning oriented reinforcement learning. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 700–719.

Yifan Sun, Jingyan Shen, Yibin Wang, Tianyu Chen, Zhendong Wang, Mingyuan Zhou, and Huan Zhang. 2025. Improving data efficiency for LLM reinforcement fine-tuning through difficulty-targeted online data selection and rollout replay. arXiv preprint arXiv:2506.05316.

Dai Do, Manh Nguyen, Svetha Venkatesh, and Hung Le. 2026. SPaCe: Unlocking sample-efficient large language models training with self-pace curriculum learning. In Findings of the Association for Computational Linguistics: ACL 2026, pages 3480–3507.

Jixiao Zhang and Chunsheng Zuo. 2025. GRPO-LEAD: A difficulty-aware reinforcement learning approach for concise mathematical reasoning in language models. arXiv preprint arXiv:2504.09696.

Hieu Trung Nguyen, Bao Nguyen, Wenao Ma, Yuzhi Zhao, Ruifeng She, and Viet Anh Nguyen. 2026. Adaptive rollout allocation for online reinforcement learning with verifiable rewards. In The Fourteenth International Conference on Learning Representations.

Ang Li, Zhihang Yuan, Yang Zhang, Shouda Liu, and Yisen Wang. 2025. Know when to explore: Difficulty-aware certainty as a guide for LLM reinforcement learning. arXiv preprint arXiv:2509.00125.

Yongcheng Zeng, Zexu Sun, Bokai Ji, Erxue Min, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, Haifeng Zhang, Xu Chen, and Jun Wang. 2026. CurES: From gradient analysis to efficient curriculum learning for reasoning LLMs. In The Fourteenth International Conference on Learning Representations.

Huiyuan Lai and Malvina Nissim. 2026. TACLer: Tailored curriculum reinforcement learning for efficient reasoning. arXiv preprint arXiv:2601.21711.

Heming Zou, Qi Wang, Yun Qu, Yuhang Jiang, Lizhou Cai, Yixiu Mao, Ru Peng, Xin Xu, Weijie Liu, Kai Yang, and 1 others. 2026. TRACE: A unified rollout budget allocation framework for efficient agentic reinforcement learning. arXiv preprint arXiv:2606.11119.

Ziqi Jia, Yalu Ouyang, Bo Pang, Panpan Li, Hangfei Xu, Shengzhao Wen, Shiyong Li, and Yanpeng Wang. 2026. CVPO: Enhancing LLM reinforcement learning reasoning via value-variance adaptation and dynamic curriculum learning. arXiv preprint arXiv:2608.03068.

Yangyi Fang, Jiaye Lin, Xiaoliang Fu, Cong Qin, Haolin Shi, Chaowen Hu, Lu Pan, Ke Zeng, and Xunliang Cai. 2026. How to allocate, how to learn? dynamic rollout allocation and advantage modulation for policy optimization. In Findings of the Association for Computational Linguistics: ACL 2026, pages 14727– 14744.

Yuv Raj Pant, Moshiur Rahman Tonmoy, and Kaiqun Fu. 2026. When does closed-loop curriculum adaptation help GRPO? an analysis on countdown. arXiv preprint arXiv:2610.39056. Wei Xiong, Chenlu Ye, Baohao Liao, Hanze Dong, Xinxing Xu, Christof Monz, Jiang Bian, Nan Jiang, and Tong Zhang. 2025. Reinforce-Ada: An adaptive sampling framework under non-linear RL objectives. arXiv preprint arXiv:2510.04996.

Zhongwen Xu and Zihan Ding. 2025. Single-stream policy optimization. arXiv preprint arXiv:2509.13232. Yun Qu, Qi Wang, Yixiu Mao, Heming Zou, Yuhang Jiang, Weijie Liu, Clive Bai, Kai Yang, Yangkun Chen, Saiyong Yang, and 1 others. 2026. Small generalizable prompt predictive models can steer efficient rl post-training of large reasoning models. arXiv preprint arXiv:2602.01970.

Heyang Jiang, Henry Liu, and Baharan Mirzasoleiman. 2026a. Learning as reasoning unfolds: Progressive rollout allocation for efficient reinforcement learning. arXiv preprint arXiv:2607.22002.

Zihao Chen, Fanxiang Xiong, Hongran Ren, Xuefeng Bai, Zhongxiang Dai, Kehai Chen, Zhiguo Zhang, Zhiyong Wang, and Yu Cheng. 2026. SERA: Scale-equalized rollout allocation for maximum likelihood reinforcement learning. arXiv preprint arXiv:2609.36552.

Niloofar Gholipour, Marcos Assuncao, Gursimran Singh, Timothy Yu, Rajkumar Buyya, Julien GasconSamson, Zhenan Fan, Yong Zhang, Xiaojie Xu, Yaqiang Yao, and Xiaolong Bai. 2026. Rollout efficiency in reinforcement learning for reasoning large language models: A taxonomy and future directions. arXiv preprint arXiv:2609.25463.

Yongqiang Yao, Jinru Tan, Kaihuan Liang, Zixin Yin, Yazhe Niu, Ruihao Gong, Dahua Lin, and Ningyi Xu. 2026b. RollVerify: Bridging efficiency and accuracy in long-tail rollout reinforcement learning. arXiv preprint arXiv:2610.09914.

John P. Lalor, Hao Wu, and Hong Yu. 2016. Building an evaluation scale using item response theory. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 648– 657.

Hexuan Deng, Zihao Yan, Xuebo Liu, Shuo Nie, Yue Wang, Chen Wang, Zhaohua Zhang, Tianwen Jiang, Qiuyong Xiao, Jihong Zhang, and Min Zhang. 2026. GRPODropout: Less is more for online reinforcement learning rollouts. arXiv preprint arXiv:2610.11854.

Pedro Rodriguez, Joe Barrow, Alexander Miserlis Hoyle, John P. Lalor, Robin Jia, and Jordan BoydGraber. 2021. Evaluation examples are not equally informative: How should that change NLP leaderboards? In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, pages 4486–4503.

Lei Jiang, Honghui Bao, Yue Zhou, Zhengyao Gu, Langzhou He, and Natalie Parde. 2026b. Trainingfree prompt selection for efficient reinforcement learning with partial rollouts. arXiv preprint arXiv:2610.24229.

Yang Zhou, Can Jin, Zihan Dong, Zhepeng Wang, Yanting Yang, Shiyu Zhao, Lei Li, Runxue Bao, Yaochen Xie, and Dimitris N. Metaxas. 2026. DARE: Difficulty-adaptive reinforcement learning with co-evolved difficulty estimation. arXiv preprint arXiv:2605.09188.

Faeze Moradi Kalarde, Armin Toroghi, Nam Ho Koh, Doo Hwan Hwang, Seungjin Kim, Kee-Eung Kim, and Scott Sanner. 2026. Bayes-adaptive MDPs for rollout allocation to enhance GRPO training. arXiv preprint arXiv:2610.37429.

Ru Zhang, Renda Li, Ziyu Ma, Weijie Qiu, Chongyang Tao, Yong Wang, and Xiangxiang Chu. 2026a. D2 Evo: Dual difficulty-aware self-evolution for dataefficient reinforcement learning. arXiv preprint arXiv:2605.17037.

Haobo Xu, Sirui Chen, Ruizhong Qiu, Yuchen Yan, Chen Luo, Monica Cheng, Jingrui He, and Hanghang Tong. 2026a. Prune as you generate: Online rollout pruning for faster and better RLVR. arXiv preprint arXiv:2603.24840.

Weiqi Wang, Xin Liu, Binxuan Huang, Hejie Cui, Rongzhi Zhang, Changlong Yu, Shuowei Jin, Jingfeng Yang, Qingyu Yin, Zhengyang Wang, and 1 others. 2026b. HeaPA: Difficulty-aware heap sampling and on-policy query augmentation for LLM reinforcement learning. arXiv preprint arXiv:2601.22448.

Yinan Xia, Haotian Zhang, and Huiming Wang. 2026. Balancing the reasoning load: Difficultydifferentiated policy optimization with length redistribution for efficient and robust reinforcement learning. arXiv preprint arXiv:2603.18533.

Hong Wang, Zhezheng Hao, Jian Luo, Chenxing Wei, Yao Shu, Lei Liu, Qiang Lin, Hande Dong, and Jiawei Chen. 2026c. Scheduling your LLM reinforcement learning with reasoning trees. In The Fourteenth International Conference on Learning Representations.

Zehao Wang, Hanzhu Chen, Runquan Gui, Zhe Wang, and Lin Yang. 2026a. When all rollouts fail: Policycompatible joint decoding for hard-problem reasoning. arXiv preprint arXiv:2610.36849.

Liang Xu, Shufan Shen, Zhongni Hou, Chao Bi, Yufei Zhang, Wei Lin, Guojun Yin, and Shuhui Wang. 2026b. Predictive bandit curriculum reinforcement learning via gradient-based sample group discovery. arXiv preprint arXiv:2610.25704.

Bo Yang, Zishuai Zhang, Weihua Li, Qinnan Zhang, Sijia Wen, and Hainan Zhang. 2026a. Guided-RLVR: Guided exploration in reinforcement learning with verifiable rewards. arXiv preprint arXiv:2610.37041. Nghi Dao, Calder Morton-Ferguson, Noah Golowich, and Adam Block. 2026. ReAPO: Retry-aware policy optimization for efficient exploration in LLM reasoning. arXiv preprint arXiv:2610.28770.

Wenhao Zhang, Yibo Xie, Rui Wang, Jiahua Yang, Lei Jiang, Zibo Yang, Yawei Wang, Jiali Xu, Haoyang Long, and Huan Xiong. 2026b. Reusing rollouts under policy lag: Prefix-normalized policy optimization for LLM reinforcement learning. arXiv preprint arXiv:2608.01418.

Sohyun Son, Yerim Hong, Jaeyun Park, and Mujeen Sung. 2026. Improving reasoning with rollout agreement for group-based policy optimization. arXiv preprint arXiv:2610.34951.

Bangji Yang, Hongbo Ma, Ruihan Guo, Jiajun Fan, Xi Zhu, and Ge Liu. 2026b. Batched contextual reinforcement II: Does RLVR need long rollout windows? arXiv preprint arXiv:2610.22887.

Zhiyuan Wang, Jacob Steeves, and Chang Liu. 2026d. A sign law for entropy drift under sampler mismatch in RLVR. arXiv preprint arXiv:2610.34044.

Jasper Dekoninck, Nikola Jovanović, Tim Gehrunger, Kári Rögnvaldsson, Ivo Petrov, Chenhao Sun, and Martin Vechev. 2026. Beyond benchmarks: MathArena as an evaluation platform for mathematics with LLMs. arXiv preprint arXiv:2605.00674.

An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. 2024. Qwen2.5-Math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122.

Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. M3-Embedding: Multilinguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216.

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad AlDahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783.

Mathematical Derivations and Proofs

A.1

Proof of Proposition 1 (Conjugate Pólya–Gamma Filtering Step)

For each prompt i ∈ Bt with si,t successes out of ni,t rollouts, the binomial likelihood in terms of the logit µi,t = z⊤ i θt − bi,t is ni,t (eµi,t )si,t p(si,t | ni,t , µi,t ) = . (16) si,t (1 + eµi,t )ni,t

Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q. Jiang, Ziju Shen, and 1 others. 2024. NuminaMath: The largest public dataset in AI4Maths with 860k pairs of competition math problems and solutions. Hugging Face Repository. Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Yuchen Zhang, Jiacheng Chen, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, and 1 others. 2025. Process reinforcement through implicit rewards. arXiv preprint arXiv:2502.01456.

By the Pólya–Gamma integral identity (Polson et al., 2013), for any real µ and count n > 0, Z ∞ 1 (eµ )s 2 −n κµ e− 2 ωµ pPG (ω | n, 0)dω, =2 e µ n (1 + e ) 0 (17) where κ = s − n/2. Under the exponential tilting property of the Pólya–Gamma family, the conditional posterior of ωi,t given µi,t = ci,t is PG(ni,t , ci,t ), whose first moment is c ni,t i,t tanh . (18) ωi,t = E[ω | ni,t , ci,t ] = 2ci,t 2

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the MATH dataset. arXiv preprint arXiv:2103.03874. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. Let’s verify step by step. In The Twelfth International Conference on Learning Representations.

Replacing ω by its conditional expectation ωi,t at the prior predictive magnitude ci,t = |µ̂i,t|t−1 | yields a Gaussian pseudo-likelihood in µi,t :

Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. 2022. Solving quantitative reasoning problems with language models. In Advances in Neural Information Processing Systems, volume 35, pages 3843–3857.

1 2 exp κi,t µi,t − ωi,t µi,t 2 κi,t −1 ⊤ ∝N zi θt − bi,t , ωi,t . (19) ωi,t

Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. 2024. OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3828–3850.

Integrating out the slow intercept bi,t ∼ 2 N (b̂i,t−1 , τi,t−1 ) before updating θt adds the in−1 2 dependent Gaussian variance τi,t−1 to ωi,t , producing the marginal observation model for θt : κi,t −1 2 + b̂i,t−1 θt ∼ N z⊤ i θt , ωi,t + τi,t−1 . (20) ωi,t

Mislav Balunović, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev. 2026. MathArena: Evaluating LLMs on uncontaminated math competitions. Advances in Neural Information Processing Systems, 38.

−1 2 Defining ω̃i,t = (ωi,t + τi,t−1 )−1 and stacking i ∈ Bt gives the linear-Gaussian model yt ∼ N (Zt θt , Ω̃−1 t ) under the prior θt ∼

N (θ̂t|t−1 , Σt|t−1 ). Standard Gaussian conditioning yields equations (8)–(9). Forming Z⊤ t Ω̃t Zt takes O(|Bt |K 2 ) operations and inverting the K × K posterior precision takes O(K 3 ) operations. Storing (b̂i , τi2 )N i=1 and (θ̂t , Σt ) requires O(N + K 2 ) memory. A.2

and observation variance components gives E (µ̂i,t −µi,t )2 =

m=1

σθ2 α α ν̄ ≥ + ν̄, (24) + 2−α 2αq 2 P∞ m where we used m=0 (1 − α) (m + 1)/q = 2 1/(α q) and 2 − α ≤ 2. Minimizing the righthand side over α ∈ (0, 1] by AM–GM yields the lower bound s σθ2 ν̄ . (25) min E (µ̂i,t − µi,t )2 ≥ q α∈(0,1]

Proof of Proposition 2 (Zero-Shot Variance Contraction)

For any unvisited prompt j ∈ / Bt , its intercept vari2 = τ2 ance is unchanged (τj,t j,t−1 ), so the change in total predictive logit variance is s2j,t|t−1 − s2j,t = z⊤ j Σt|t−1 − Σt zj .

(21)

For FADE, the skill vector θt ∈ RK is observed at every step through |B| prompts with whitened co1 variance |B| Z⊤ t Zt ≈ IK , so each skill coordinate receives effective observation precision |B|/(K ν̄) at every step (qskill = 1). Applying the same steady-state filter bound to the K skill coordinates and to the slow scalar intercept bi (which hasqdrift σb2 and visit q rate q) gives total MSE 2 O Kσθ ν̄/|B| + σb2 ν̄/q .

Applying the Woodbury matrix identity to (8) gives Σt|t−1 − Σt = Σt|t−1 Z⊤ t ⊤ × Ω̃−1 t + Zt Σt|t−1 Zt

−1

Zt Σt|t−1 . (22)

A.4

Derivation of the Variance-Aware Group Utility

Let ΦG (p) = 1−pG −(1−p)G . Taking derivatives with respect to p gives

s2j,t|t−1 −s2j,t = vj⊤ Mt vj ≥ λmin (Mt ) ∥vj ∥22 > 0. (23) When K=0, zj is empty and s2j,t|t−1 − s2j,t = 0. A.3

∞ X σµ2 1 (1−α)2(m−1) (2 − α)2 q

Φ′G (p) = G(1 − p)G−1 − GpG−1 ,

(26)

Φ′′G (p) = −G(G − 1) (1 − p)G−2 + pG−2 .

(27) Note that near the frontier p ≈ 0.5, the logit mapdp = p(1−p), and for uncertain ping p = σ(µ) has dµ prompts where s2i,t represents epistemic uncertainty over whether pi,t sits inside the informative zone, an optimistic upper-confidence (or Thompson exploration) utility adds the local sensitivity scaled by s2i,t :

Proof of Proposition 3 (Minimax Timescale Separation)

Consider the scalar tracking problem for prompt i visited independently at each step with Bernoulli probability q = |B|/N ≪ 1. Between visits, the true logit µi,t = z⊤ i θt − bi undergoes a random walk with per-step variance σµ2 = ∥zi ∥22 σθ2 + σb2 ≥ σθ2 (under the whitened normalization E∥zi ∥22 = K ≥ 1). The number of steps ∆i elapsed since prompt i was last visited follows a geometric distribution with mean 1/q. A single-timescale per-prompt tracker µ̂i,t = (1 − α)µ̂i,t−1 + αyi,t (updated only when i ∈ Bt , where yi,t = µi,t + εi,t has observation variance ν̄) is an exponentially weighted average over past visits m = 1, 2, . . . occurring at lags L1 < L2 < . . . with E[Lm ] = m/q. Expanding the squared error E[(µ̂i,t − µi,t )2 ] into its lag-induced drift bias

1 Ui,t = ΦG (p̂i,t ) + |Φ′′G (p̂i,t )| p̂i,t (1 − p̂i,t ) s2i,t , 2 (28) 1 which yields equation (13) with cG = 2 G(G − 1) (1 − p̂i,t )G−2 + p̂G−2 . i,t

B Implementation and Integration Details B.1

Skill Basis Construction and Numerical Stability

To construct {zi }N i=1 , we encode all N =150k training prompts once using frozen BGE-M3 (Chen et al., 2024) with the prompt prefix: 13

P Bt subject to i∈Bt ni,t = |Bt |G. We set the p allocation weight proportional to Ui,t , round to integers in {2, 4, . . . , 16}, and eliminate GVM’s pre-step difficulty probing rollouts, returning that saved budget to policy training steps.

Represent this mathematical problem by its required solution techniques, reasoning depth, and algebraic complexity: [PROBLEM]

Let E ∈ RN ×d be the centered embedding matrix. We compute a rank-K truncated√SVD E ≈ ⊤ (K=32) and set Z = N UK ∈ UK SK VK 1 ⊤ N ×K R , so N Z Z = IK . This offline step takes 3.2 minutes on a single A100 GPU and 4.8 seconds for the randomized SVD on CPU. To initialize b̂i,0 without extra rollouts, we set b̂i,0 = β0 (ℓi −ℓ̄)/σℓ where ℓi is the character length of prompt xi and β0 =0.15, with broad initial vari2 =1.0. This provides a weak, zero-cost ance τi,0 prior that breaks exact ties at step t=1 while letting subsequent rollout evidence dominate immediately. In (6), we clamp ci,t ≥ 10−3 and use the Taylor 1 c2 series 2c when ci,t < 10−2 to tanh(c/2) ≈ 41 − 48 guarantee smooth floating-point evaluation. B.2

Hyperparameter

Default Value

Skill subspace dimension K Fast skill retention ρ Skill process variance σθ2 Slow intercept retention γb Initial intercept prior (b̂i,0 , τ02 ) Initial skill prior (θ̂0 , Σ0 ) GRPO batch |Bt | / group G Total rollout budget Policy lr / KL penalty βKL Max length / temperature

32 0.82 0.015 0.985 0.15 ℓ̃i , 1.0 0, 0.5 IK 256 / 8 1.44 × 106 10−6 / 0.001 2048 / 0.7

Table 4: Default hyperparameters for FADE and the host GRPO training loop.

Integration with Host RLVR Schedulers

We integrate FADE into four representative schedulers without changing their core allocation or filtering logic: • DAPO-Filter + FADE: Standard DAPO (Yu et al., 2026) and group-prioritized RLVR (Mao et al., 2026b) discard prompts whose sampled group has ri,t ∈ {0, 1} and draw replacement prompts. With FADE, before generating rollouts we pre-filter the candidate pool by selecting the top |Bt | prompts ranked by Ui,t , reducing wasted rejections during generation.

Additional Experimental Results

C.1

Integration with VIP Rollout Allocation and Multi-Seed Stability

Table 5 extends our evaluation to a fifth scheduler— VIP (Nguyen et al., 2026)—and reports mean and standard deviation over three independent training seeds across all five schedulers. FADE improves VIP by +2.0 points on Qwen2.5-Math-1.5B and +1.5 points on Llama-3.2-1B-Instruct. Across all 50 scheduler–backbone–benchmark pairs (5 schedulers × 2 backbones × 5 benchmarks), FADE outperforms the native scheduler in 47 of 50 pairs, ties in 1, and trails by ≤0.2 points in 2 pairs (p = 3.6 × 10−12 under a two-sided exact sign test).

• PCL + FADE: Prompt curriculum methods (Gao et al., 2026; Qu et al., 2026) train a separate predictive model to select prompts near a target difficulty band. We remove the auxiliary valuemodel training entirely (saving 4.2–6.9 A100hours) and rank candidate prompts directly by Ui,t . • GRESO + FADE: GRESO (Zheng et al., 2026) and dynamics-predictive sampling (DPS) (Mao et al., 2026a) compute selective sampling priorities from historical reward transitions. We replace GRESO’s empirical per-prompt pass rate with p̂i,t from (12). • GVM / VIP + FADE: Following varianceminimizing and unified rollout budget allocation frameworks (Yao et al., 2026a; Nguyen et al., 2026; Zou et al., 2026), GVM and VIP allocate variable rollout counts ni,t across the minibatch

Scheduler

Estimator Qwen2.5-1.5B Llama-3.2-1B

DAPO-Filter

Native +FADE

38.8 ± 0.31 40.6 ± 0.26

13.9 ± 0.24 15.3 ± 0.22

PCL

Native +FADE

35.0 ± 0.42 37.9 ± 0.28

13.4 ± 0.29 14.7 ± 0.25

GRESO

Native +FADE

39.1 ± 0.34 41.2 ± 0.25

15.2 ± 0.27 16.6 ± 0.23

VIP

Native +FADE

41.4 ± 0.36 43.4 ± 0.27

13.1 ± 0.28 14.6 ± 0.24

GVM

Native +FADE

42.7 ± 0.33 45.5 ± 0.24

12.3 ± 0.31 14.1 ± 0.21

Table 5: Five-benchmark Average@8 (mean ± std over 3 seeds) across five scheduling paradigms.

C.2

Scaling Across Model Sizes (1B to 7B)

Rank K Brier ↓ ρsp ↑ GVM Avg Step Time 8 16 32 64 128

To verify that FADE remains effective as the base policy scales, we evaluate GVM and GRESO integrations on Qwen2.5-Math-7B and Qwen2.5-3B alongside our 1B and 1.5B backbones. Table 6 shows that FADE maintains a +2.1 to +2.8 point advantage on the five-benchmark average at 3B and 7B scales. Notably, on Qwen2.5-Math-7B, where many MATH-500 problems become easy early in training (ri =1), FADE’s fast skill state θt rapidly identifies saturation and shifts rollout budget toward OlympiadBench and AIME problems, lifting AIME Average@8 from 24.2% to 27.8%. Method

1B 1.5B 3B 7B

44.2 45.0 45.5 45.4 45.1

0.18 ms 0.29 ms 0.54 ms 1.42 ms 4.65 ms

Embedding Model

Brier ↓ ρsp ↑ GVM Avg@8

TF-IDF + SVD (K=32) MiniLM-L6-v2 BGE-M3 (Default) Qwen3-Embedding-0.6B E5-Mistral-7B

0.144 0.131 0.118 0.116 0.117

0.71 0.76 0.82 0.82 0.82

43.8 44.5 45.5 45.4 45.5

Table 8: Comparison of frozen text encoders for extracting the K=32 whitened skill coordinates zi .

12.3 42.7 49.1 57.4 14.1 45.5 51.7 59.9

Table 6: Scaling behavior across backbone sizes (fivebenchmark Average@8, %).

C.3

0.75 0.79 0.82 0.82 0.80

Table 7: Effect of skill subspace dimension K on Qwen2.5-Math-1.5B.

Uniform GRPO 12.9 37.0 44.2 53.6 GRESO (Native) 15.2 39.1 46.5 55.8 GRESO + FADE 16.6 41.2 48.6 58.1 GVM (Native) GVM + FADE

0.136 0.123 0.118 0.117 0.119

Extended Discussion

Why bilinear factorization beats graph clustering and message passing. An alternative way to share rollout feedback across prompts is to partition the corpus into discrete clusters or propagate labels along a nearest-neighbor graph. Bilinear skill–difficulty factorization offers three concrete advantages over discrete clustering and graph propagation: 1. Compositional skill representation: A competition math problem frequently combines two or more skills (e.g., a combinatorics problem that reduces to a modular arithmetic recurrence). In a hard or soft cluster model, a prompt is convexly averaged over cluster centroids, which cannot represent additive skill composition. In FADE, the inner product z⊤ i θt naturally sums proficiency contributions across all active coordinates of zi .

Sensitivity to Skill Dimension K, Retention (ρ, γb ), and Encoder Choice

Figure 5 and Tables 7–8 examine FADE’s sensitivity to its hyperparameters: • Skill dimension K (Figure 5(a), Table 7): Increasing K from 8 to 32 steadily improves Brier calibration (0.136 → 0.118) and Spearman ρsp (0.75 → 0.82). At K=64 calibration improves marginally (0.117) while at K=128 the per-step minibatch of 256 prompts becomes too small to constrain all 128 coordinates without slight overfitting (ρsp dips to 0.80). K=32 is the sweet spot in both accuracy and sub-millisecond latency (0.54 ms). • Timescale separation (ρ, γb ) (Figure 5(b)): Performance forms a broad plateau around ρ ∈ [0.75, 0.88] and γb ∈ [0.97, 0.995]. Performance drops sharply only when γb is reduced below 0.92 (which erases long-term item ordering) or when ρ → 1 (which prevents the skill state from forgetting pre-update competence).

2. Explicit separation of global drift from item hardness: Graph label propagation smooths total pass rates across neighbors; if two neighboring problems have the same topic but one has a subtle algebraic trap, graph smoothing pulls their estimates together and blurs withinneighborhood ranking. By keeping bi outside the subspace projection z⊤ i θt and updating bi with long memory (γb =0.985), FADE preserves sharp local hardness contrasts indefinitely.

• Prompt encoder choice (Table 8): FADE performs comparably across modern embedding models (BGE-M3, Qwen3-Embedding-0.6B, and E5-Mistral-7B), confirming that whitened PCA extracts a stable low-rank skill manifold regardless of the specific pretrained encoder.

3. No iterative inner loop or graph tuning: Variational message passing on a graph requires mul15

0.125 0.115 4

latency (ms)

2 0

skill subspace dimension K

128

1.00

intercept retention γb

Brier score (↓)

(b) Two-timescale sensitivity

0.96

default (0.82, 0.985)

44.5

44.0

0.92 43.5

0.88 0.6

0.7

0.8

skill retention ρ

0.9

avg. accuracy (%)

(a) Skill dimension K 0.135

GRESO + FADE

MATH500 AMC23 Minerva Olympiad AIME24 0

Avg@8 gain over native (pts)

Figure 5: Hyperparameter sensitivity and per-benchmark breakdown. (a) Calibration Brier score and per-step CPU latency across skill subspace dimension K. (b) Five-benchmark GVM+FADE accuracy across the fast skill retention ρ and slow intercept retention γb . (c) Per-benchmark accuracy gain over native GVM and GRESO on Qwen2.5-Math-1.5B.

tiple coordinate-ascent sweeps per step and tuning neighborhood size and coupling temperature. FADE performs a single conjugate 32×32 linear solve in 0.54 ms. When does FADE reduce to simpler estimators? If K=0, FADE reduces to an independent perprompt Pólya–Gamma filter on bi , which is the logistic state-space counterpart of per-item dynamicspredictive tracking in DPS (Mao et al., 2026a). If γb =0 and bi ≡0, FADE reduces to a contextual linear bandit in the embedding space, which captures coarse topic competence similar to zero-shot prompt predictors (Qu et al., 2026) but cannot distinguish easy from hard problems within the same topic (as seen in the bi ≡0 row of Table 3). Both components are needed to jointly minimize Brier calibration error and maximize Spearman rank correlation, providing a reliable signal for downstream group-prioritized training (Mao et al., 2026b) and rollout budget allocation (Zou et al., 2026).

FADE: Factored Ability--Difficulty Filtering for Online RLVR Scheduling

Analyze & explore related research in Conceptio

Related documents