arXiv CS open access

Learning to Difference: Adaptive Reversible Differencing (AdaRDiff) for Time Series Forecasting

2026 · arxiv_cs

arXiv CS · Papers · 2026

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

Learning to Difference: Adaptive Reversible Differencing (AdaRDiff) for Time Series Forecasting Morad Laglil1,2 , Younes Hlal1 , Marouane El Hadari1 , Emilie Devijver1 , Eric Gaussier1

arXiv:2608.28134v1 [cs.LG] 28 Aug 2026

Univ. Grenoble Alpes, CNRS, Grenoble INP, LIG, 38000 Grenoble, France 2 Savoye, 21600 Longvic, France {morad.laglil, emilie.devijver, eric.gaussier}@univ-grenoble-alpes.fr, younes.hlal@etu.univ-grenoble-alpes.fr, marouane.el-hadari@grenoble-inp.org

Abstract Reliable long-horizon time series forecasting is an important yet difficult problem. Trends and seasonality introduce complex temporal structure that challenges learning-based forecasting models. Differencing, which subtracts nearby past values to remove such structure, is the classical remedy, but its reliance on hand-picked orders and periods has kept it largely absent from recent deep architectures. We propose Adaptive Reversible Differencing (AdaRDiff), a generalized differencing approach that uses learnable weights to simplify the series through weighted differencing with previous time instants. This yields stabilized residuals on which forecasting is performed, after which the removed components are restored autoregressively to reconstruct the forecast, capturing trend and seasonality jointly through a single operator. This reconstruction admits a closed-form convolutional expression, which parallelizes on GPU and yields up to 33.7× speedup over the naive recurrence. We furthermore rely on a two-phase training schedule that separates temporal structure discovery from reconstruction learning, as suggested by a theoretical analysis of the gradient when using a linear forecasting model. AdaRDiff attains state-of-the-art forecast accuracy across eight benchmarks spanning electricity, weather, traffic, and energy, at negligible parameter cost. Furthermore, it is designed as a plug-and-play module: integrating AdaRDiff improves eight diverse backbones, from linear models to Transformers, in the large majority of cases, by up to 25.9% with a linear backbone and 18.3% with iTransformer.

Introduction

Time series forecasting is fundamental to many real-world applications, such as traffic flow prediction, product sales forecasting, and energy consumption management, where accurate predictions support proactive decision-making. Forecasting over long horizons, commonly referred to as Longterm Time Series Forecasting (LTSF) (Zhou et al. 2021), is particularly valuable and has accordingly drawn sustained attention. In all these settings, forecasting quality depends on how well a model extracts and exploits the temporal patterns present in the historical window (Lin et al. 2024b). Among these patterns, trend and seasonality dominate: real-world series combine long-term drifts and recurring periodic cycles into most of their structured variation (Cleveland, Cleveland, and Terpenning 1990), and these components are precisely what makes forecasting difficult for

learning-based models. Zhang and Qi (Zhang and Qi 2005) showed empirically that feedforward networks struggle to capture trend and seasonal patterns from raw series, whereas detrending and deseasonalizing the data beforehand substantially reduces forecasting error. The classical tool for this is differencing: replacing each observation by its difference with a previous value removes low-order trends (1st -order differencing) or periodic components (seasonal differencing), leaving a simpler, more regular residual for the model to predict (Hyndman and Athanasopoulos 2021). Isolating such a residual is a powerful lever, and one that architectural sophistication alone may not supply. Such preprocessing is fundamental to classical statistical models, including ARIMA and SARIMA (Box and Jenkins 1976), which have guided time series analysis for decades. Despite its proven effectiveness in traditional time series analysis, this principle has been largely overlooked in modern deep learning–based forecasting models. We believe this is due to a key limitation of classical differencing: the scheme is rigid and must be specified by hand, including the orders, seasonal periods, and their coefficients, which demands data exploration and domain expertise and does not fit end-to-end training. AdaRDiff instead learns the scheme directly from data, discovering the relevant lags and their weights rather than requiring them to be fixed in advance. Recent deep architectures have begun to reintroduce related preprocessing through time series decomposition, as in DLinear (Zeng et al. 2023) and CycleNet (Lin et al. 2024a), but these methods target a fixed form of structure and do not adaptively capture the range of trends and periodicities encountered across datasets. This motivates an adaptive, data-driven form of differencing that captures multiple seasonalities and trends jointly, and can be paired with any forecasting model. We propose here such a framework, Adaptive Reversible Differencing (AdaRDiff). AdaRDiff is model-agnostic: rather than a standalone forecaster, it is a differencing module that wraps around a forecasting backbone, simplifying the series before the backbone predicts and reconstructing the forecast afterward. It generalizes classical differencing by replacing the hand-set coefficients with learnable, channel-wise differencing weights: for each time step, it forms a weighted combination of past observations within a fixed window and subtracts it from the current value, so that the model learns a differencing scheme tailored to the data. The backbone then

operates on the resulting simplified residual. As AdaRDiff is reversible, the forecast is then reconstructed autoregressively, applying the learned weights step by step to the backbone’s output so that each predicted value builds on the previously reconstructed ones, recovering the forecast in the original space and restoring the trend and seasonal structure that differencing had removed. The learned differencing weights are moreover interpretable, as their magnitudes at each lag reveal the dominant periods and dependencies the model has identified. AdaRDiff lets even a lightweight Linear or MLP backbone reach state-of-the-art accuracy across benchmarks spanning multiple domains. Our main contributions are as follows: • We introduce AdaRDiff, a plug-and-play differencing module whose learnable weights capture trend and seasonality jointly and reversibly, recovering predictions in the original space while training end-to-end with any forecasting backbone. • We theoretically analyze the training dynamics of AdaRDiff with a linear backbone through the gradient with respect to its parameters, revealing a tension between the differencing and reconstruction pathways that motivates a two-phase training schedule separating temporal structure discovery from reconstruction learning. • We evaluate AdaRDiff through extensive experiments on eight benchmark datasets and eight forecasting backbones, demonstrating its effectiveness.

Related Work

LTSF has a long statistical history, from exponential smoothing to the Box–Jenkins ARIMA/SARIMA family (Box and Jenkins 1976), built on well-established statistical principles. We build on these but compare against recent deeplearning approaches, grouped into lightweight linear/MLP and Transformer-based models. Orthogonal to the choice of forecaster, reversible instance normalization (RevIN) (Kim et al. 2021) has become a standard component of modern deep forecasting architectures: it normalizes each input window and restores its statistics at the output, mitigating distribution shift between training and test data. AdaRDiff is fully compatible with RevIN and employs it as an optional component. Linear and MLP models. Lightweight linear- and MLPbased models often match or surpass far larger architectures on LTSF at a fraction of their cost. DLinear (Zeng et al. 2023) decomposes the series into trend and seasonal parts by moving average, and then forecasts each part with a linear model, while TimeMixer (Wang et al. 2024) mixes multi-scale components through MLP blocks. A second line exploits periodicity and low-rank structure to simplify the series and shrink the model: CycleNet (Lin et al. 2024a) learns recurrent cycles and subtracts them before forecasting the residual; SparseTSF (Lin et al. 2024b) decouples periodicity from trend by downsampling into cross-period subsequences, forecasting with under a thousand parameters; TimeBase (Huang et al. 2025) extracts core basis components and forecasts at the segment level rather than point by point; and MixLinear

(Ma, Luo, and Sha 2026) combines segment-based trend extraction in the time domain with low-rank spectral filtering in the frequency domain, reaching competitive accuracy with roughly a hundred parameters. However those methods capture only a single fixed form of the input structure in isolation. In contrast, AdaRDiff, learns a single differencing operator that jointly captures trend and multiple seasonalities. Transformer-based models. The Transformer (Vaswani et al. 2017), effective at modeling long-range dependencies, has been widely adapted to LTSF. Informer (Zhou et al. 2021) reduces attention complexity for long sequences, while FEDformer (Zhou et al. 2022) combines seasonal–trend decomposition with frequency-domain attention, iTransformer (Liu et al. 2024) treats each series as a token to model multivariate correlations, and PatchTST (Nie et al. 2023) segments each series into subseries-level patches and applies attention under channel independence. TQNet (Lin et al. 2025) uses periodically shifted learnable queries in a single-layer attention module to capture global inter-variable correlations. These models learn temporal or cross-variate structure implicitly within attention. AdaRDiff instead handles per-series temporal structure explicitly, in a model-agnostic differencing module that can wrap a Transformer as readily as a linear model. A separate line of work develops time series foundation models (Ansari et al. 2024; Das et al. 2024), pretrained on large corpora for zero- and few-shot transfer. We do not compare against them: their pretraining corpora often overlap with standard benchmarks, which precludes a fair comparison with fully-supervised methods, a data-leakage issue documented in prior work (Laglil et al. 2026).

AdaRDiff: Adaptive Reversible Differencing

Problem setup. Given a look-back window of L observations across C channels, the task is to forecast the next H steps. Our method processes each channel independently, so we present the univariate case (C = 1) throughout: the input is x = (x1 , . . . , xL ), the ground-truth target window xf = (xL+1 , . . . , xL+H ), and the forecast x̂f = (x̂L+1 , . . . , x̂L+H ).

3.1

Adaptive Differencing and Reconstruction

AdaRDiff is a model-agnostic, reversible framework for time series forecasting. It operates in three stages, summarized in Figure 1: (i) a learnable, channel-wise differencing operator transforms the input into residuals; (ii) a forecasting backbone maps past residuals to predicted future residuals; and (iii) an autoregressive reconstruction recovers the forecast in the original space. Channel-Wise Differencing. Given x = (x1 , . . . , xL ) and a window size P , we learn differencing weights δ = (δ1 , . . . , δP ) ∈ RP and form the residuals z = (z1 , . . . , zL−1 ) ∈ RL−1 by zt−1 = xt −

P X j=1

δj xt−j ,

t ∈ {2, . . . , L},

(1)

In practice, the series is left-padded by repeating x1 a total of P −1 times, i.e. xt := x1 for t ≤ 1, so that differencing applies from t = 2. Since channels may exhibit heterogeneous dynamics, a distinct δ is learned per channel. This subsumes classical schemes: δ1 =1 gives first-order differencing, δ1 =2, δ2 =−1 second-order, and δℓ =1 seasonal differencing at lag ℓ (all other coefficients zero); combining lags lets a single operator remove an affine trend and a periodic component at once. Backbone. The backbone fθ maps past residuals to predicted future residuals: ẑ = fθ (z) ∈ RH .

(2)

We primarily use a linear model (ẑ = Wb z + b, Wb ∈ RH×(L−1) ) and a two-layer MLP but more complex backbones are evaluated in the ablation study (Section 4.3). Autoregressive Reconstruction. Given the predicted future residuals ẑ ∈ RH , the forecast in the original space is recovered by inverting (1) through a linear recurrence: x̂L+1−j := xL+1−j for j = 1, . . . , P , and the forecast is recursively defined by x̂L+k = ẑk +

P X

δj x̂L+k−j ,

k = 1, . . . , H.

(3)

j=1

Computing x̂f directly through the recursive application of Eq. (3) prevents parallelization across the horizon. We thus recast it as a convolution, amenable to parallelization. Let c = (c1 , . . . , cH ) collect the initial-condition terms, ck = PP j=k δj xL+k−j for k ≤ P and ck = 0 otherwise, and let h be the linear recurrence sequence (Elaydi 2005) based on the same recurrence as the reconstruction, with hn = 0 for n < 0, h0 = 1, and hn =

P X

n ≥ 1.

δj hn−j ,

(4)

j=1

Training. The full model, with parameters (δ, θ), is trained end-to-end, the target objective being the forecast loss Lforecast = 21 ∥x̂f − xf ∥2 ,

(6)

Since the reconstruction (5) depends polynomially on δ, this objective is non-convex and we therefore optimize it by stochastic gradient descent.

3.2

Gradient Analysis with Linear Backbone

We study the training dynamics of AdaRDiff through the gradient with respect to its parameters, taking the linear backbone as the analytically tractable case. Throughout, L := Lforecast . Backbone gradient. backbone gradient is

With e := x̂f − xf , the linear-

∇Wb L = (Th⊤ e) z⊤ ,

(Th⊤ e)t =

H X

hk−t ek ,

(7)

k=t

Proposition 1. The forecast (3) satisfies k−1 X x̂L+k = h ∗ (ẑ + c) k = hm (ẑk−m + ck−m ). (5) m=0

In matrix notation, Eq. (5) can be written as x̂f = Th (ẑ + c), where Th ∈ RH×H is the lower-triangular Toeplitz matrix with (Th )n,k = hn−k . Both terms can be parallelized on GPUs: h is obtained through repeated squaring (Knuth 1997) of the companion-matrix1 of the recurrence in Eq. (4), requiring O(log H) sequential steps—against O(H) of the naive recurrence—and the Toeplitz product is realized as a one-dimensional convolution across horizon and batch, as detailed in Appendix B.  δ1 ··· δ 1

Figure 1: Overview of AdaRDiff. (i) A learnable differencing operator transforms the input into residuals. (ii) A forecasting backbone predicts future residuals. (iii) The forecast is reconstructed autoregressively in the original space.

P −1 δP

The companion matrix 



.. . .

so the effective error at position t is a backward convolution of e with h. Without reconstruction (Th = I) this collapses to e z⊤ , where each position sees only its own error; reconstruction couples positions through h, whose growth is governed by the spectral radius ρ of the companion matrix. Proposition 2 (Growth of hn and Error Propagation). For (hn ) as in Eq. (4)2 : 1. If ρ < 1: ∀ρ′ ∈ (ρ, 1), ∃C > 0, |hn | ≤ Cρ′ n , and |(Th⊤ e)t | = O(∥e∥); 2. If ρ > 1, supn |hn | = ∞. The coupling is thus localized when ρ < 1: each position receives feedback dominated by nearby errors, enabling position-specific learning, and the effective error is bounded independently of the horizon. This uniform control is lost as ρ exceeds 1, in which case h is no longer bounded. 2 The case ρ = 1 occurs on a measure-zero set of parameters and is omitted here for readability; it is treated in Appendix A.

Differencing Gradient. The gradient of L in δj decomposes into three terms, one per pathway through which δ acts on the forecast: j H−1 X X ∂hn ∂L (j) = ϕev (n) − p⊤ x2:L + (Th⊤ e)t xL+t−j , | {z } ∂δj ∂δ j t=1 n=0 | {z } II: differencing | {z } I: reconstruction

where ϕev (n) =

III: boundary

k=n+1 ek vk−n with vm (j) L−1

(8) = ẑm + cm ,

p⊤ = (Th⊤ e)⊤ Wb , and x2:L ∈ R is the lag-j shift of (j) 3 x2:L , i.e. (x2:L )s = xs+1−j . As the third term of Eq. (8) arises only because the first P forecast steps are reconstructed from the last observed context values. It is supported on at most P of the H horizon positions, a vanishing fraction for long horizons; we therefore focus on Terms I and II of Eq. (8). Term I is the only term involving the sensitivity of h to δ, whose behavior is again governed by ρ. Proposition 3 (Growth of ∂hn /∂δj and Term I). For (j) j ∈ {1, . . . , P }, the sensitivity gn := ∂hn /∂δj is the self(j) (j) convolution gn = (h ∗ h(j) )n with hn := hn−j , and 2 satisfies: (j)

1. If ρ < 1: ∀ρ′ ∈ (ρ, 1), ∃Cg > 0, ∀n ≥ 0, |gn | ≤ Cg ρ′ n , and |Term I| = O(∥e∥ ∥v∥); (j) 2. If ρ > 1, supn |gn | = ∞. Term I is therefore bounded independently of the horizon only in the stable regime (ρ < 1). Since nothing in the objective prevents δ from leaving the stable regime, we introduce an optional reparameterization that makes ρ explicitly controllable, as stated in Theorem 1. Theorem 1. If ∥δ∥1 < 1 in Eq. (4), then ρ < 1.

Proposition 4 (Autocovariance coupling). Let (xt ) be zero-mean weakly stationary with autocovariance γ(k) = E[xt xt−k ]. Then, for any fixed parameters (δ, Wb , b), the differencing term of Eq. (8) has expectation L+H−2 X (j) Ex p⊤ x2:L = Φ(m) γ(m + j), (10) m=1−L

where the weights Φ(m) depend on the parameters but not on the lag j. There is a fundamental tension between Terms I and II: Term II encourages δ to match the data autocorrelation structure, potentially implying large ∥δ∥1 , whereas Term I enforces stable reconstruction dynamics, requiring smaller ∥δ∥1 . The learned weights δ are therefore a compromise which depends on the relative magnitudes of these two contributions.

3.3

Two-Phase Training

We address the tension between Terms I and II with a twophase training scheme that separates temporal structure discovery from reconstruction learning. Phase 1: residual loss. We first train the differencing operator and the backbone in residual space, bypassing reconstruction entirely: the backbone output ẑ is compared directly against the differenced target, minimizing P X 2 1 Lres = 2 ∥ẑ − zf ∥ , zf,k = xL+k − δj xL+k−j , j=1

(11) where zf applies the differencing operator to the target window. Because Th is absent from the forward pass, Term I vanishes and, with ep := ẑ − zf , the gradient reduces to ∂Lres (j) (j) = e⊤ p xf − Wb x2:L ∂δj

Concretely, we ℓ1 -normalize an unconstrained δ̃ ∈ R and rescale by a gain α, δjeff =

α δ̃j , ∥δ̃∥1

(9)

so that ∥δ eff ∥1 = |α|: direction and magnitude are decoupled, and α directly controls proximity to the stability condition. Fixing α < 1 would enforce ρ < 1 but imposes the same magnitude on every channel, whereas channels may require different amounts of differencing; we therefore learn α per channel. Note that we consider the reparametrization as an optional procedure, which may not be needed on all datasets and forecasting configurations. Term II, from the differencing pathway, does not involve (j) the sensitivity gn . As stated in Proposition 4, its expected value is a linear functional of the input autocovariance in which the lag j enters only through data autocovariance, so the learning signal for δ is also shaped by the data’s secondorder temporal structure—trend and periodicity—a property specific to this pathway. 3 We adopt the left-padding convention of Eq. (1) (xt := x1 for t ≤ 1), so that all shifted indices are well-defined.

(12)

(j)

with xf ∈ RH the lag-j shift of the target window, (j) (j) (xf )k = xL+k−j . This gradient carries no sensitivity gn and is bounded by O (∥Wb ∥ ∥x∥ + ∥xf ∥)∥ep ∥ , independently of ρ, so Phase 1 fits the data structure free of the reconstruction’s conditioning. Phase 2: forecast loss. We then optimize the full objective Lforecast = 21 ∥x̂f − xf ∥2 , reintroducing Th and Term I. Initialized from the Phase 1 solution, δ already fits the data structure, so Term I refines the reconstruction rather than discovering structure under an ill-conditioned gradient. This schedule improves accuracy by a substantial margin over single-phase training with Lforecast (Section 4.3).

4 4.1

Experiments

Experimental Setup

Datasets. To evaluate our proposed method, we conduct experiments on several widely adopted benchmark datasets: the ETT collection (ETTh1, ETTh2, ETTm1, ETTm2) (Zhou et al. 2021), Weather, Traffic, Electricity, and Solar-Energy (Lai et al. 2018). Further dataset details are in Appendix C.

AdaRDiff-Linear (Ours) MSE MAE

AdaRDiff-MLP (Ours) MSE MAE

MixLinear (2026) MSE MAE

ETTh1

96 192 336 720

0.360 0.402 0.426 0.423

0.390 0.419 0.436 0.450

0.389 0.409 0.449 0.487

0.412 0.434 0.446 0.485

0.351 0.395 0.411 0.423

ETTh2

96 192 336 720

0.268 0.330 0.352 0.378

0.333 0.374 0.399 0.424

0.272 0.330 0.352 0.398

0.340 0.378 0.404 0.445

ETTm1

96 192 336 720

0.286 0.326 0.356 0.410

0.336 0.360 0.378 0.408

0.291 0.327 0.362 0.421

ETTm2

96 192 336 720

0.159 0.216 0.268 0.350

0.250 0.289 0.327 0.380

Traffic Electricity Weather

96 192 336 720

0.140 0.181 0.234 0.302

96 192 336 720

TimeBase (2025) MSE MAE

TQNet (2025) MSE MAE

SparseTSF (2024) MSE MAE

TimeMixer (2024) MSE MAE

iTransformer (2024) MSE MAE

CycleNet (2024) MSE MAE

DLinear (2023) MSE MAE

PatchTST (2023) MSE MAE

TimesNet (2023) MSE MAE

FEDformer (2022) MSE MAE

0.383 0.423 0.434 0.456

0.349 0.387 0.408 0.439

0.384 0.410 0.418 0.446

0.377 0.424 0.450 0.521

0.404 0.437 0.454 0.517

0.362 0.404 0.435 0.426

0.389 0.412 0.428 0.448

0.410 0.448 0.482 0.475

0.441 0.465 0.490 0.500

0.389 0.424 0.456 0.545

0.421 0.446 0.469 0.532

0.379 0.416 0.447 0.477

0.403 0.425 0.445 0.483

0.378 0.415 0.449 0.507

0.402 0.425 0.449 0.517

0.377 0.413 0.436 0.455

0.408 0.431 0.446 0.475

0.437 0.456 0.494 0.632

0.454 0.469 0.494 0.578

0.485 0.481 0.522 0.604

0.500 0.498 0.521 0.575

0.283 0.337 0.356 0.380

0.340 0.376 0.395 0.423

0.292 0.339 0.358 0.400

0.345 0.387 0.410 0.448

0.282 0.354 0.387 0.415

0.350 0.393 0.419 0.447

0.294 0.340 0.360 0.383

0.346 0.377 0.398 0.425

0.315 0.383 0.385 0.432

0.380 0.415 0.438 0.471

0.305 0.405 0.411 0.448

0.361 0.421 0.436 0.470

0.271 0.332 0.362 0.415

0.337 0.380 0.408 0.449

0.294 0.412 0.471 0.740

0.360 0.437 0.478 0.609

0.276 0.342 0.364 0.395

0.339 0.385 0.405 0.434

0.349 0.500 0.445 0.438

0.403 0.488 0.465 0.465

0.401 0.425 0.427 0.462

0.451 0.464 0.471 0.493

0.345 0.363 0.384 0.416

0.333 0.353 0.391 0.439

0.375 0.382 0.407 0.435

0.311 0.338 0.364 0.413

0.351 0.371 0.386 0.414

0.298 0.347 0.373 0.437

0.352 0.381 0.395 0.429

0.314 0.348 0.368 0.419

0.359 0.376 0.386 0.413

0.332 0.355 0.386 0.452

0.384 0.398 0.416 0.457

0.315 0.349 0.381 0.437

0.369 0.388 0.409 0.439

0.307 0.337 0.364 0.410

0.353 0.371 0.387 0.411

0.314 0.347 0.367 0.415

0.350 0.381 0.387 0.415

0.298 0.335 0.366 0.420

0.352 0.373 0.394 0.421

0.359 0.368 0.429 0.477

0.391 0.398 0.438 0.474

0.406 0.450 0.436 0.462

0.441 0.477 0.466 0.479

0.160 0.214 0.265 0.345

0.250 0.291 0.327 0.381

0.170 0.223 0.275 0.360

0.261 0.298 0.332 0.384

0.162 0.218 0.270 0.352

0.256 0.293 0.328 0.380

0.167 0.220 0.283 0.370

0.257 0.295 0.340 0.391

0.167 0.219 0.271 0.353

0.259 0.297 0.330 0.380

0.192 0.253 0.307 0.380

0.285 0.329 0.362 0.412

0.179 0.239 0.309 0.387

0.274 0.314 0.356 0.407

0.159 0.214 0.268 0.353

0.249 0.289 0.326 0.384

0.163 0.223 0.291 0.407

0.257 0.304 0.355 0.433

0.165 0.219 0.268 0.352

0.260 0.298 0.333 0.386

0.200 0.274 0.340 0.384

0.288 0.337 0.382 0.407

0.339 0.397 0.449 0.451

0.406 0.452 0.491 0.499

0.192 0.234 0.276 0.342

0.141 0.183 0.234 0.303

0.201 0.245 0.288 0.339

0.172 0.212 0.257 0.324

0.227 0.260 0.295 0.344

0.146 0.185 0.236 0.309

0.198 0.241 0.281 0.331

0.162 0.200 0.251 0.318

0.216 0.249 0.288 0.339

0.174 0.216 0.260 0.325

0.231 0.267 0.299 0.345

0.163 0.201 0.258 0.329

0.223 0.254 0.300 0.348

0.159 0.203 0.253 0.317

0.212 0.252 0.291 0.337

0.164 0.209 0.242 0.338

0.220 0.258 0.283 0.333

0.174 0.215 0.262 0.319

0.242 0.277 0.319 0.359

0.149 0.193 0.240 0.312

0.199 0.243 0.281 0.334

0.176 0.219 0.277 0.344

0.234 0.270 0.311 0.356

0.289 0.340 0.370 0.420

0.342 0.394 0.408 0.421

0.131 0.146 0.161 0.196

0.225 0.241 0.259 0.295

0.128 0.146 0.161 0.196

0.225 0.244 0.261 0.296

0.139 0.154 0.171 0.209

0.233 0.248 0.265 0.298

0.139 0.153 0.169 0.207

0.231 0.245 0.262 0.294

0.138 0.150 0.184 0.207

0.237 0.246 0.277 0.296

0.139 0.155 0.171 0.208

0.239 0.250 0.265 0.300

0.142 0.164 0.171 0.209

0.247 0.275 0.260 0.313

0.135 0.155 0.169 0.204

0.233 0.253 0.267 0.301

0.128 0.143 0.159 0.197

0.223 0.237 0.254 0.287

0.141 0.155 0.170 0.209

0.244 0.258 0.275 0.309

0.141 0.156 0.172 0.208

0.240 0.256 0.267 0.299

0.202 0.218 0.232 0.299

0.308 0.322 0.332 0.375

0.226 0.220 0.224 0.271

0.341 0.336 0.337 0.378

96 192 336 720

0.387 0.397 0.412 0.449

0.270 0.273 0.280 0.298

0.367 0.382 0.396 0.434

0.265 0.272 0.279 0.298

0.390 0.404 0.416 0.452

0.268 0.276 0.283 0.283

0.394 0.403 0.417 0.456

0.267 0.271 0.278 0.298

0.381 0.396 0.397 0.439

0.268 0.277 0.277 0.299

0.389 0.399 0.417 0.449

0.268 0.272 0.279 0.299

0.396 0.404 0.419 0.458

0.294 0.295 0.302 0.309

0.374 0.393 0.409 0.450

0.273 0.283 0.292 0.314

0.381 0.394 0.406 0.441

0.266 0.273 0.279 0.300

0.396 0.404 0.417 0.457

0.272 0.275 0.283 0.310

0.363 0.382 0.399 0.432

0.250 0.258 0.268 0.289

0.605 0.627 0.631 0.700

0.325 0.340 0.349 0.371

0.664 0.613 0.612 0.664

0.431 0.382 0.379 0.410

96 192 336 720

0.189 0.212 0.229 0.237

0.258 0.277 0.290 0.296

0.162 0.178 0.187 0.195

0.217 0.232 0.248 0.251

0.339 0.345 0.431 0.241

0.465 0.445 0.470 0.308

0.192 0.213 0.222 0.235

0.239 0.252 0.261 0.264

0.184 0.201 0.220 0.236

0.246 0.253 0.263 0.282

0.205 0.215 0.213 0.232

0.241 0.265 0.276 0.272

0.232 0.238 0.234 0.273

0.271 0.293 0.301 0.319

0.217 0.208 0.238 0.270

0.255 0.257 0.309 0.319

0.194 0.205 0.218 0.239

0.255 0.251 0.257 0.278

0.200 0.239 0.231 0.237

0.234 0.294 0.298 0.277

0.205 0.227 0.225 0.249

0.239 0.280 0.290 0.291

0.243 0.253 0.222 0.254

0.284 0.311 0.287 0.298

0.231 0.257 0.260 0.260

0.254 0.301 0.319 0.290

1st Count

Models

Solar

Metrics

Table 1: Forecasting comparison with lookback L = 720 and prediction length H ∈ {96, 192, 336, 720}. Baseline results are taken from the TimeBase (Huang et al. 2025) paper where available, and otherwise reproduced using the official code of each baseline. The best results are shown in red, second best in blue. Dataset

Two-Phase

Single-Phase

Impr. (%)

ETTh1 ETTh2 ETTm1 ETTm2 Weather Electricity Traffic Solar

0.404 0.342 0.350 0.255 0.221 0.162 0.434 0.253

0.458 0.350 0.361 0.262 0.223 0.217 0.503 0.256

+11.9 +2.2 +3.0 +2.8 +0.9 +25.6 +13.7 +1.2

Overall

+7.7

Table 2: Two-phase versus single-phase training, test MSE averaged over horizons H ∈ {96, 192, 336, 720} at lookback L = 336. Impr. is the relative MSE reduction of twophase over single-phase; positive favors two-phase. Better value in each pair in red. Baselines. We compare AdaRDiff against 11 state-ofthe-art long-term forecasting baselines, grouped as (i) MLP/Linear-based methods: DLinear (Zeng et al. 2023), SparseTSF (Lin et al. 2024b), TimeMixer (Wang et al. 2024), CycleNet (Lin et al. 2024a), TQNet (Lin et al. 2025), TimeBase (Huang et al. 2025), and MixLinear (Ma, Luo, and Sha 2026); (ii) Transformer-based methods: FEDformer (Zhou et al. 2022), iTransformer (Liu et al. 2024), and PatchTST (Nie et al. 2023); and (iii) CNN-based methods: TimesNet (Wu et al. 2023). Hyperparameters and Implementation Details. Owing to its minimal design, AdaRDiff’s only method-specific hyperparameter is the window size P . On the smaller ETT datasets it is tuned over {4, 24, 96}, a grid spanning the dominant daily period of both the hourly ETTh variants (24) and the 15-minute ETTm variants (96), together with a short window (4) capturing only short-range dependence; on the other datasets P is set to the dominant period (Appendix C).

Three further choices are selected per dataset on the validation set: the initialization of δ, which is zero (δ = 0), uniform (δj = 1/P ), or first-order (δ1 = 1, else 0); whether to apply RevIN (Kim et al. 2021) for distribution shift; and whether to apply the ℓ1 -reparameterization of Eq. (9). In the main results we evaluate linear and MLP backbones, with all hyperparameters tuned on the validation set. Models are implemented in PyTorch (Paszke et al. 2019) and trained with Adam (Kingma and Ba 2014) for 30 epochs in Phase 1 and 10 in Phase 2, with early stopping. We report MSE and MAE, and run all experiments on a single NVIDIA H100 (80 GB). Full details are in Appendix B.

4.2

Main Results

Table 1 reports the full forecasting comparison.4 Across the eight benchmarks, the AdaRDiff-Linear backbone attains the best results more often than the other methods, on both MSE and MAE (on 15 and 11 cases respectively), with the MLP backbone being second on MSE (14); together the two variants dominate the count of top-ranked scores by a wide margin over every competing method. The gains are largest on Solar-Energy, where the MLP backbone lowers MSE by 11– 16% over the strongest baseline across all horizons. Photovoltaic output alternates zero-valued nights with gradual daytime rises and falls (Lin et al. 2024a), so consecutive values are highly predictable from the immediate past: short-range structure rather than a seasonal cycle. AdaRDiff suits this, its differencing weights concentrating at the smallest lags rather than the daily period (Figure 4), acting as a low-order differencing. On the high-dimensional Traffic dataset (862 channels), AdaRDiff matches the strong PatchTST baseline 4 Results for shorter look-back windows (L ∈ {96, 336, 512}) are reported in Appendix C.

Models ETTh1

Original +AdaRDiff

Linear DLinear

MLP

0.441 0.404

0.428 0.425

0.425 0.409

MSE

MixLinear SparseTSF CycleNet PatchTST iTransformer 0.401 0.400

0.420 0.402

0.417 0.408

0.419 0.413

0.438 0.427

MAE

ETTh1

0.395

0.2800

0.4100 0.390

8.4%

3.8%

0.6%

0.4%

4.4%

2.0%

1.3%

2.4%

25.9%

24.4%

0.371 0.345 7.1%

0.355 0.344 3.2%

0.350 0.338 3.6%

0.351 0.355

0.351 0.347 1.1%

-1.1%

0.396 0.361

MSE

0.470 0.355

0.4025

0.380

0.180 0.2725 0.2700

0.175

0.4000 0.375

8.8%

0.2675

0.3975 0.170

0.370

ETTm1

0.2775 0.2750

0.4050

Original +AdaRDiff

0.362 0.350

0.363 0.353

0.357 0.349

0.379 0.356

0.374 0.356

0.355 0.350

0.351 0.342

0.365 0.367

Promotion

3.3%

2.8%

2.1%

6.2%

5.0%

1.6%

2.5%

-0.4%

Original +AdaRDiff

0.293 0.255

0.274 0.256

0.268 0.253

0.267 0.258

0.266 0.258

0.252 0.256

0.258 0.258

0.281 0.270

0.158

Promotion

13.1%

6.6%

5.9%

3.4%

3.1%

-1.5%

-0.2%

3.8%

0.154

Original +AdaRDiff

0.245 0.221

0.243 0.237

0.230 0.225

0.235 0.230

0.2650

0.3950 0.3925

0.365

144

168

0.2625

0.165

200

Window size P

144

168

200

Window size P

Weather

0.160

MAE

Promotion

0.462 0.342

MAE

ETTh2

Original +AdaRDiff

0.185

0.4075 0.385

MSE

Promotion

ETTm2

0.190

Electricity

0.142

0.240

0.257 0.226

Promotion

9.9%

8.1%

6.5%

10.5%

11.8%

2.5%

2.3%

1.8%

Original Electricity +AdaRDiff

0.177 0.162

0.168 0.165

0.165 0.162

0.176 0.168

0.173 0.167

0.160 0.159

0.169 0.163

0.203 0.166

Promotion

8.4%

1.9%

1.6%

4.6%

3.4%

1.0%

3.4%

18.3%

Original +AdaRDiff

0.450 0.434

0.436 0.435

0.408 0.406

0.441 0.439

0.438 0.437

0.423 0.421

0.413 0.422

0.528 0.438

Promotion

3.6%

0.3%

0.4%

0.0%

0.5%

-2.2%

17.1%

Original +AdaRDiff

0.258 0.253

0.253 0.262

0.217 0.191

0.280 0.269

0.278 0.266

0.261 0.249

0.214 0.200

0.238 0.207

Promotion

2.0%

-3.5%

11.7%

3.7%

4.3%

4.4%

6.5%

13.1%

0.218

0.150

Solar

Table 3: Effect of AdaRDiff on each backbone (MSE), averaged over horizons at L = 336. Promotion is the relative MSE reduction (red = regression). at a fraction (ca. 6.6%) of its parameter count, and clearly outperforms all other methods. We attribute this to the strong temporal-lag structure of Traffic (Lin et al. 2024a): by conditioning each prediction on recent lagged values, AdaRDiff’s recursive reconstruction directly exploits the short-range dependencies that dominate this dataset. Notably, all of these results are obtained with a lightweight linear or MLP backbone, indicating that the learnable differencing operator, rather than backbone capacity, drives the improvement.

4.3

Ablation Study and Model Analysis

Two-phase training. Table 2 compares the two-phase schedule with single-phase training under identical settings. Two-phase training lowers test MSE on all eight datasets, by 7.7% on average, with the largest gains on the highestdimensional datasets: Electricity (+25.6%, 321 channels) and Traffic (+13.7%, 862 channels). Since a distinct δ is learned per channel, these are the settings in which singlephase optimization must resolve the tension between the two pathways across hundreds of independent δ, exactly where separating the two phases helps most. AdaRDiff as a Plug-and-Play Module. Table 3 evaluates AdaRDiff as a plug-and-play module across eight backbones spanning linear models, MLPs, and Transformers. AdaRDiff improves MSE in the large majority of configurations (56 of 64 backbone–dataset pairs), and no regression exceeds 3.5%. Three patterns emerge. First, the gains are largest on backbones that lack any dedicated mechanism for temporal structure. The plain linear model and MLP improve by up to 25.9% (linear, ETTh2), and a simple linear backbone equipped with AdaRDiff matches or surpasses much larger Transformer baselines—evidence that the improvement originates in the adaptive differencing and reconstruc-

0.137

0.216

0.136

0.146

0.214

0.135

0.144

0.212

0.236

0.138

0.148

0.142

Traffic

0.238

0.139 0.220

0.152

MSE

0.257 0.230

0.222

0.234

0.232

0.134 0.230

0.133

144

Window size P

168

200

MAE

0.236 0.221

0.140

MAE

Weather

0.245 0.225

0.141

0.224

0.156

MSE

ETTm2

144

168

200

Window size P

Figure 2: Performance of the AdaRDiff/Linear model with varied P . The forecast horizon is set as 96. The dashed line marks the window size minimizing MSE. tion rather than in backbone capacity. Second, and more revealing, AdaRDiff also improves backbones that already model temporal structure explicitly. DLinear removes trend by moving average, SparseTSF and CycleNet model fixed periodic patterns, and MixLinear separates trend from low-rank spectral components; each captures a single form of structure in isolation. In contrast, AdaRDiff models trend and seasonality jointly through one learnable differencing operator, and therefore remains complementary even to backbones with built-in decomposition, improving SparseTSF and MixLinear by up to 11.8% and 10.5% respectively on Weather. Third, the benefit is most pronounced on iTransformer, which models cross-variate correlations rather than per-series temporal dynamics. Here AdaRDiff’s per-channel differencing supplies the temporal component the backbone lacks, yielding large gains on the high-dimensional Electricity (+18.3%) and Traffic (+17.1%) datasets. That AdaRDiff strengthens even these specialized backbones establishes it as an effective and broadly portable forecasting component. Effect of the window size P . Figure 2 reports MSE against the differencing window P on four datasets of differing periods. The best P broadly reflects the temporal structure of each dataset: ETTh1, with a daily period (24 steps), is optimal at P = 24, while the longer-period, high-dimensional datasets Weather and Electricity (periods of 144 and 168) favor larger windows. On ETTm2, MSE changes little between P = 24 and its dominant period P = 96, showing that AdaRDiff is robust to this choice. Overall, performance tracks the dominant period while remaining stable to moderate variations in P. Efficiency analysis. Figure 3(a) reports the number of parameters, the compute (MACs per sample), the peak GPU memory (in MB) and the average epoch time (in seconds) on Electricity. As a plug-and-play module, AdaRDiff adds only C ×P parameters (572K with a linear backbone, comparable to CycleNet and far below PatchTST at 8.7M), and its epoch time (37.6 s) matches the lightweight linear models. The higher MACs and memory come from the reconstruction, which builds the sequence h in O(C P 3 log H); the

Baselines

12.7M

FITS

133.6K

AdaRDiff (Ours)

125 MB 497 MB

572.3K

CycleNet

31.3 s

42.7M 23.38G

573K

DLinear

166.4M

1.04M

333M

iTransformer

5.47M

TimeMixer

5.58M

PatchTST

33.2 s

158 MB

41.1 s

1.79G

828 MB 30.30G

8.69M

FEDformer

37.6 s

147 MB

65.6 s

1.38 GB

537 s

14.17G

22.14M

18.03 GB

10.48G

33.7×

43.1 s

1.13 GB

Reconstruction speedup (×)

AdaRDiff/Linear (Ours) SparseTSF

827 s

2.36 GB

558 s

26.5×

25 20 13.5×

15 10 5

7.3× 3.9× naive 1.0×

104

103

105

107

106

107

108

109

Parameters

1011

1010

102

MACs

103

104

Peak GPU Memory (MB)

200

400

600

800

Epoch Time (s)

192

336

720

1000

Forecast horizon H

(a) Compute, time, and memory

(b) Reconstruction speedup

Figure 3: Efficiency of AdaRDiff/Linear. (a) Number of parameters, compute (MACs per sample), peak GPU memory, and epoch time across models on Electricity (forecast length and look-back window both set to 720; epoch time and memory measured at batch size 12). (b) Inference speedup of the closed-form convolutional reconstruction over the naive sequential reconstruction as a function of the forecast horizon H.

Previous hour

0.10 k

0.3

ETTh1

0.15

Daily period (24)

0.05 0.00

Daily period (24)

0.1 0.0

0.25

100

lag k

125

150

175

200

Traffic

Weekly period (168)

0.20 k

Electricity

Weekly period (168)

0.2

100

lag k

125

150

175

200

Solar Daily period (144)

0.2

Daily period (24)

0.05

0.3

0.15 0.10

0.1 0.0

0.00 0

100 lag k

125

150

175

200

0.0125

100

lag k

125

150

175

200

Synthetic: xt = at + b + t

0.0100 k

No periodic pattern

0.0075 0.0050 0.0025 0.0000 0

100

lag k

125

150

175

200

Figure 4: Learned differencing weights δk across lags for AdaRDiff with look-back L = 336, forecast horizon H = 96, window size extended to P = 200 and δ initialized to zero). Curves show the mean over channels, with shaded regions denoting ± one standard deviation.

cubic dependence on P dominates on long-period datasets (P =168 on Electricity). Since accuracy remains stable to moderate variations in P (Figure 2), reducing P saves compute at negligible accuracy loss: on Electricity, P =48 vs. 168 cuts the reconstruction cost ∼ 43× for only a 2.9% accuracy loss. Crucially, although the raw MAC count is high, our reconstruction replaces the sequential recurrence with a parallel formulation of O(log H) which runs efficiently on GPU. This explains why the epoch time is low despite the peak memory usage and compute values. Figure 3(b) shows that our reconstruction is up to 33.7× faster than the naive recurrence, the gain growing with the horizon. AdaRDiff thus attains its accuracy gains at a compute cost which is partly controllable through P , and a runtime competitive with the lightest baselines.

Visualization and Interpretation of the Learned Differencing Weights. AdaRDiff learns the differencing weights δ directly from the training data. Figure 4 shows the learned δk across lags. On ETTh1, Electricity, and Traffic the weights exhibit clear seasonal structure: sharp peaks at the daily period (24) on ETTh1, and at both the daily (24) and weekly (168) periods on Electricity and Traffic, aligning with the known period lengths of each dataset (Appendix C). On ETTh1 the coefficient at lag 1 is also large and decays quickly, indicating strong short-range dependence on the immediate past. On a synthetic series with a linear trend and no periodicity (xt = at + b + εt ), by contrast, the weights are near-uniform (δk ≈ 1/P = 0.005) with no periodic peaks; applied as a differencing operator, such weights subtract the moving average of the recent past, removing the linear trend—leaving a stable residual—while averaging out the noise. The model thus produces distinct weight patterns for distinct temporal structures, adapting its differencing to each series without human hard-coding and providing an interpretable account of its behavior.

Conclusion

We introduced AdaRDiff, a model-agnostic module that revisits classical differencing through a learnable, reversible operator. We provided both a theoretical analysis of how AdaRDiff behaves with a linear backbone, as well as an experimental validation including different backbones. Across eight benchmarks and eight backbones, AdaRDiff attains state-of-the-art accuracy, yields consistent plug-and-play gains, and produces interpretable differencing weights that recover temporal structures. More broadly, our results show that classical forecasting tools offer principled inductive biases that make deep models more robust and interpretable when adapted to end-to-end learning. A compelling next step is to bring this perspective to time series foundation models: by handling trend and seasonality explicitly, a learnable differencing operator addresses the dominant temporal structure directly, which could let these models reach strong accuracy with far fewer parameters while exposing, through the learned weights, the temporal patterns they otherwise capture only implicitly.

References Ansari, A. F.; Stella, L.; Turkmen, C.; Zhang, X.; Mercado, P.; Shen, H.; Shchur, O.; Rangapuram, S. S.; Pineda Arango, S.; Kapoor, S.; Zschiegner, J.; Maddix, D. C.; Mahoney, M. W.; Torkkola, K.; Gordon Wilson, A.; Bohlke-Schneider, M.; and Wang, Y. 2024. Chronos: Learning the Language of Time Series. Transactions on Machine Learning Research. Box, G.; and Jenkins, G. 1976. Time Series Analysis: Forecasting and Control. Holden-Day series in time series analysis and digital processing. Holden-Day. ISBN 9780816211043. Cleveland, R. B.; Cleveland, W. S.; and Terpenning, I. 1990. STL: A seasonal-trend decomposition procedure based on loess. Journal of Official Statistics, 6: 3–73. Das, A.; Kong, W.; Sen, R.; and Zhou, Y. 2024. A decoderonly foundation model for time-series forecasting. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org. Elaydi, S. 2005. An Introduction to Difference Equations. Undergraduate Texts in Mathematics. New York, NY: Springer, 3 edition. ISBN 978-0-387-27602-1. Fei, J.; Yi, K.; Fan, W.; Zhang, Q.; and Niu, Z. 2025. Amplifier: bringing attention to neglected low-energy components in time series forecasting. In Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth Symposium on Educational Advances in Artificial Intelligence, AAAI’25/IAAI’25/EAAI’25. AAAI Press. ISBN 978-157735-897-8. Huang, Q.; Zhou, Z.; Yang, K.; Yi, Z.; Wang, X.; and Wang, Y. 2025. TimeBase: The Power of Minimalism in Efficient Long-term Time Series Forecasting. In Singh, A.; Fazel, M.; Hsu, D.; Lacoste-Julien, S.; Berkenkamp, F.; Maharaj, T.; Wagstaff, K.; and Zhu, J., eds., Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, 26227–26246. PMLR. Hyndman, R. J.; and Athanasopoulos, G. 2021. Forecasting: Principles and Practice. Australia: OTexts, 3rd edition. Kim, T.; Kim, J.; Tae, Y.; Park, C.; Choi, J.-H.; and Choo, J. 2021. Reversible instance normalization for accurate timeseries forecasting against distribution shift. In International Conference on Learning Representations. Kingma, D. P.; and Ba, J. 2014. Adam: A Method for Stochastic Optimization. CoRR, abs/1412.6980. Knuth, D. E. 1997. The Art of Computer Programming, Volume 2: Seminumerical Algorithms. Addison-Wesley, 3rd edition. Laglil, M.; Pracca, B.; Devijver, E.; and Gaussier, E. 2026. Foundation Models and Fine-Tuning: Toward a New Generation of Models for Time Series Forecasting. arXiv:2607.23146. Lai, G.; Chang, W.-C.; Yang, Y.; and Liu, H. 2018. Modeling Long- and Short-Term Temporal Patterns with Deep Neural

Networks. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR ’18, 95–104. New York, NY, USA: Association for Computing Machinery. ISBN 9781450356572. Lin, S.; Chen, H.; Wu, H.; Qiu, C.; and Lin, W. 2025. Temporal Query Network for Efficient Multivariate Time Series Forecasting. In Forty-second International Conference on Machine Learning. Lin, S.; Lin, W.; HU, X.; Wu, W.; Mo, R.; and Zhong, H. 2024a. CycleNet: Enhancing Time Series Forecasting through Modeling Periodic Patterns. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. Lin, S.; Lin, W.; Wu, W.; Chen, H.; and Yang, J. 2024b. SparseTSF: modeling long-term time series forecasting with 1k parameters. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org. Liu, Y.; Hu, T.; Zhang, H.; Wu, H.; Wang, S.; Ma, L.; and Long, M. 2024. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting. In The Twelfth International Conference on Learning Representations. Liu, Y.; Wu, H.; Wang, J.; and Long, M. 2022. Nonstationary Transformers: Exploring the Stationarity in Time Series Forecasting. In Oh, A. H.; Agarwal, A.; Belgrave, D.; and Cho, K., eds., Advances in Neural Information Processing Systems. Ma, A.; Luo, D.; and Sha, M. 2026. MixLinear: Extreme Low-Resource Multivariate Time Series Forecasting with 0.1K Parameters. In International Conference on Learning Representations (ICLR). Nie, Y.; H. Nguyen, N.; Sinthong, P.; and Kalagnanam, J. 2023. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. In International Conference on Learning Representations. Niu, Y.; Deng, J.; and Tong, Y. 2026. PhaseFormer: From Patches to Phases for Efficient and Effective Time Series Forecasting. In International Conference on Learning Representations (ICLR). Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; Desmaison, A.; Köpf, A.; Yang, E.; DeVito, Z.; Raison, M.; Tejani, A.; Chilamkurthy, S.; Steiner, B.; Fang, L.; Bai, J.; and Chintala, S. 2019. PyTorch: An Imperative Style, HighPerformance Deep Learning Library. In Advances in Neural Information Processing Systems (NeurIPS), 8026–8037. Qiu, X.; Hu, J.; Zhou, L.; Wu, X.; Du, J.; Zhang, B.; Guo, C.; Zhou, A.; Jensen, C. S.; Sheng, Z.; and Yang, B. 2024. TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods. Proc. VLDB Endow., 17(9): 2363–2377. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, Ł.; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30. Wang, S.; Wu, H.; Shi, X.; Hu, T.; Luo, H.; Ma, L.; Zhang, J. Y.; and ZHOU, J. 2024. TimeMixer: Decomposable Mul-

tiscale Mixing for Time Series Forecasting. In International Conference on Learning Representations (ICLR). Wu, H.; Hu, T.; Liu, Y.; Zhou, H.; Wang, J.; and Long, M. 2023. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis. In International Conference on Learning Representations. Zeng, A.; Chen, M.; Zhang, L.; and Xu, Q. 2023. Are Transformers Effective for Time Series Forecasting? In Proceedings of the AAAI Conference on Artificial Intelligence. Zhang, P.; and Qi, M. 2005. Neural network forecasting for seasonal and trend time series. European Journal of Operational Research, 160: 501–514. Zhang, Y.; and Yan, J. 2023. Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multivariate Time Series Forecasting. In The Eleventh International Conference on Learning Representations. Zhou, H.; Zhang, S.; Peng, J.; Zhang, S.; Li, J.; Xiong, H.; and Zhang, W. 2021. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting. In ThirtyFifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, February 2-9, 2021, 11106–11115. Virtual Event: AAAI Press. Zhou, T.; Ma, Z.; Wen, Q.; Wang, X.; Sun, L.; and Jin, R. 2022. FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting. In Chaudhuri, K.; Jegelka, S.; Song, L.; Szepesvari, C.; Niu, G.; and Sabato, S., eds., Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, 27268–27286. PMLR.

A A.1

Theoretical Proofs

Proof of Proposition 1

Write vk := ẑk + ck , so that the identity to prove, Eq. (5), Pk−1 reads x̂L+k = m=0 hm vk−m . Throughout we use the conventions δj = 0 for j > P and hn = 0 for n < 0, and argue by strong induction on k. Base case (k = 1). Since L ≥ P , every term x̂L+1−j in the reconstruction (3) is an observed value xL+1−j . Hence x̂L+1 = ẑ1 +

P X

δj xL+1−j = ẑ1 + c1 = v1 = h0 v1 ,

j=1

using c1 = k = 1.

j=1 δj xL+1−j and h0 = 1, which is Eq. (5) at

Inductive step. Fix k ≥ 2 and assume Eq. (5) for all indices 1, . . . , k − 1. In (3), the term x̂L+k−j is a previously reconstructed forecast when k −j ≥ 1 and an observed value xL+k−j when k − j ≤ 0. Splitting the sum at j = k, the PP observed part is j=k δj xL+k−j = ck , so x̂L+k = vk +

k−1 X j=1

δj x̂L+k−j .

Each remaining index satisfies k − j ∈ {1, . . . , k − 1}, so the Pk−j−1 induction hypothesis gives x̂L+k−j = hi vk−j−i . i=0 Substituting and reindexing by m = i + j (so m runs over 1, . . . , k − 1 and, for each m, j over 1, . . . , m), k−1 X

δj

k−j−1 X

j=1

hi vk−j−i =

k−1 m X X

δj hm−j vk−m

m=1 j=1

i=0

k−1 X

hm vk−m ,

m=1

the inner sum equaling hm by the recurrence defining h. With h0 = 1, x̂L+k = vk +

k−1 X

hm vk−m =

m=1

k−1 X

hm vk−m ,

m=0

which is Eq. (5) at index k and completes the induction.

A.2

Proof of Proposition 2

Setup. The sequence (hn ) with h0 = 1, hn = 0 for n < 0, PP and hn = j=1 δj hn−j (n ≥ 1) is the fundamental solution of a P th-order constant-coefficient linear PPrecurrence, with characteristic polynomial χ(λ) = λP − j=1 δj λP −j . Let λ1 , . . . , λr be its distinct roots, m1 , . . . , mr their multiplicities, and ρ = maxi |λi |; these are the eigenvalues of the companion matrix of δ. By the general solution of such recurrences (Elaydi 2005, Cor. 2.24), hn =

r X

pi (n) λin ,

deg pi ≤ mi − 1.

(13)

i=1

H X k=t

hk−t ek ≤

|hm | max |ek | ≤ Ch ∥e∥,

m≥0

uniformly in H; thus |(Th⊤ e)t | = O(∥e∥). (ii) Case ρ = 1, unit-modulus roots simple. Roots with |λi | < 1 contribute vanishing, hence bounded, terms. A root with |λi | = 1 is simple, so mi = 1 and pi is a constant. Summing the finitely many terms of (13) gives C > 0 with |hn | ≤ C for all n. (iii) Case ρ > 1 (or ρ = 1 with a repeated unit-modulus root). Split the roots by modulus, D = {i : |λi | = ρ}, S = {i : |λi | < ρ}, and write hn = hdom +hsub n n accordingly. ⋆ Let m = maxi∈D mi and D = {i ∈ D : mi = m}. We show supn |hn | = ∞ in five steps; the crux (Step 1) is that the specific initialization h0 = 1, hn = 0 (n < 0) forces the dominant modes to be present. P Step 1 (every mode is present). Let H(x) := n≥0 hn xn . Multiplying the recurrence of hn by xn and summing over

P n ≥ 1, the left side is n≥1 hn xn = H(x)−h0 = H(x)−1, while on the right, exchanging the finite sum over j with the sum over n and substituting ℓ = n − j, X X hn−j xn = xj hℓ xℓ = xj H(x), n≥1

ℓ≥1−j

where the last equality holding because hℓ = 0 for ℓ < 0. PP Hence H(x) − 1 = H(x) j=1 δj xj , i.e. H(x)D(x) = 1 PP with D(x) := 1 − j=1 δj xj , so H(x) = 1/D(x). The numerator is the constant 1 exactly because h0 = 1 and hℓ = 0 for ℓ P < 0; another initialization would leave bound−1 ary terms xj ℓ=1−j hℓ xℓ and a nonconstant numerator. As D(x) = xP χ(1/x), the roots of D are the 1/λi with multiplicities mi (λi ̸= 0 since δQ P ̸= 0), and D(0) = 1 fixes the normalization, so H(x) = P − λi x)−mi . In its partiali (1P mi Ai,t (1 − λi x)−t , we fraction expansion H(x) = i t=1 extract the top coefficient Ai,mi by multiplying through by (1 − λi x)mi . On the product form this cancels the i-th factor, Y (1 − λi x)mi H(x) = (1 − λi′ x)−mi′ , i′ ̸=i

while on the expansion it gives (1 − λi x)mi H(x) =

mi X

Ai,t (1 − λi x)mi −t

t=1

+ (1 − λi x)mi

XX i′ ̸=i

Ai′ ,t (1 − λi′ x)−t .

Evaluating at x = 1/λi , every term with mi −t > 0 vanishes and the remainder vanishes through its factor (1−λi x)mi (its poles 1/λi′ being finite there, as λi′ ̸= λi ), so only t = mi survives: Y Ai,mi = (1 − λi′ /λi )−mi′ ̸= 0, i′ ̸=i

a finite product of nonzero factors. Finally, P using the negative-binomial expansion (1 − n n n+t−1 λi x)−t = n≥0 n+t−1 λ x , whose coefficient i t−1 t−1 is a polynomial in n of degree t − 1, and matching the n coefficient P of x across the two forms of H(x) (so that hn = i pi (n)λin ) gives mi X n+t−1 pi (n) = Ai,t , t−1 t=1 of degree exactly mi − 1: the t = mi term alone attains that degree, with leading coefficient ai = Ai,mi /(mi − 1)! ̸= 0, and the lower-t terms (degree < mi − 1) cannot cancel it. Thus d := maxi∈D deg pi = m − 1 ≥ 0, and ai ̸= 0 for every i ∈ D⋆ . Step 2 (a lemma on unit-modulus sums). Lemma 1. If µ1 , . . . , µs are distinct with P |µr | = 1 and α1 , . . . , αs are not all zero, then g(n) = r αr µrn satisfies |g(n)| ≥ ϵ on an infinite set, for some ϵ > 0.

P Proof. The Cesàro average N1 n≤N |g(n)|2 tends to P 2 2 ′ r |αr | > 0: diagonal terms give |αr | , while P for r n̸= r , 1 ω = µr µr′ has |ω| = 1, ω ̸= 1, so | N n≤N ω | ≤ 2/(N |1 − ω|) → 0. If lim supn |g(n)| = 0 then |g(n)| → 0 and the average vanishes, a contradiction; the claim follows. (The bound holds only along an infinite set: g may vanish at individual n, e.g. g(n) = 2 cos θn.) Step 3 (subdominant part negligible). If S = ∅ this is trivial. Else let ρsub = maxi∈S |λi | < ρ. As every pi has P −1 n ρsub , so with n−d ≤ 1 degree ≤ P − 1, |hsub n | ≤ Bn (n ≥ 1), n |hsub n | (14) ≤ B nP −1 ρsub /ρ → 0, d n n ρ since nβ q n → 0 for any q ∈ (0, 1), β ≥ 0. Step 4 (dominant rate). With µi = λi /ρ (distinct, unimodular), X hdom n = g(n) + η(n), g(n) = ai µin , d n n ρ ⋆ i∈D

where η collects the degree-≤ d−1 remainders of the D⋆ modes and all of D \ D⋆ (so η ≡ 0 if d = 0). Each such term is a polynomial of degree ≤ d − 1 times a unimodular µni , hence O(n−1 ) after dividing by nd ; so η(n) → 0. By Step 1 the ai (i ∈ D⋆ ) are nonzero, so Lemma 1 applies to g. Step 5 (conclusion). Take ϵ > 0 and infinite N0 with |g| ≥ ϵ there. Choose N1 with |η(n)| ≤ ϵ/4 (n ≥ N1 ), |/(nd ρn ) ≥ 3ϵ/4 on N0 beyond N1 ; and N2 giving |hdom n sub with |hn |/(nd ρn ) ≤ ϵ/4 (n ≥ N2 ) by (14). On N = {n ∈ N0 : n ≥ max(N1 , N2 )}, |hn | 3ϵ ϵ ϵ ≥ − = , d n n ρ 4 4 2 so |hn | ≥ 2ϵ nd ρn on N . Since ρ ≥ 1 and d ≥ 1 whenever ρ = 1 (a repeated unit root has m ≥ 2), the bound diverges along N ; hence supn |hn | = ∞.

A.3

Proof of Proposition 3

Convolution identity. Since h0 = 1 does not depend on δ, PP (j) we have g0 = 0, and differentiating hn = k=1 δk hn−k with respect to δj gives, for n ≥ 1, gn(j) = hn−j +

P X

(j)

δk gn−k ,

gn(j) = 0 (n < 0),

k=1

so g (j) obeys the same recurrence as h, forced by hn−j . P (j) Let G (j) (x) := n≥0 gn xn . Multiplying by xn and sum(j)

(j) (j) ming over n ≥ 1, the left side P is G (x)n− g0 j= G (x), while the shift identities = x H(x) and n≥1 hn−j x P (j) n k (j) (x) (both by the substitution used n≥1 gn−k x = x G in Step 1, the negative-index terms vanishing) give

G (j) (x) = xj H(x) + G (j) (x)

P X k=1

i.e.

G (j) (x)D(x) = xj H(x).

δ k xk ,

A.5

Dividing by D and substituting 1/D = H, xj H(x) G (j) (x) = = xj H(x)2 , D(x) (j)

(15)

(j)

i.e. gn = (h ∗ h(j) )n with hn = hn−j , since H2 generates h ∗ h and multiplication by xj shifts by j. Each item now follows from the corresponding bound on h, using n X |gn(j) | ≤ |hi | |hn−i−j |. (16) i=0

Throughout we use |ϕev (n)| = (Cauchy–Schwarz).

k>n ek vk−n

≤ ∥e∥ ∥v∥

(i) Case ρ < 1. Fix ρ′ ∈ (ρ, 1) and pick ρ′′ ∈ (ρ, ρ′ ). By Proposition 2(1), |hn | ≤ Cρ′′ n for some C > 0, so (j) (16) gives |gn | ≤ C 2 (n + 1)ρ′′ n−j ≤ Cg ρ′ n , since (n + P (j) 1)(ρ′′ /ρ′ )n → 0. Hence n≥0 |gn | ≤ Cg /(1 − ρ′ ) =: Cg′ < ∞ and |Term I| =

H−1 X

gn(j) ϕev (n) ≤ Cg′ ∥e∥ ∥v∥,

n=0

independently of H.

H−1 X

(n + 1) ∥e∥ ∥v∥ = O H 2 ∥e∥ ∥v∥ .

n=0

(iii) Case ρ > 1, or ρ = 1 with Qr a repeated unit-modulus root. By (15), G (j) (x) = xj i=1 (1−λi x)−2mi : the poles are those of H, at 1/λi , with multiplicities doubled to 2mi . The numerator xj vanishes only at x = 0, and λi ̸= 0, so no pole is cancelled. The argument of Proposition 2(3) therefore applies verbatim to g (j) with mi replaced by 2mi : every mode is present with nonzero dominant leading coefficients, and (j) Lemma 1 yields c > 0 and an infinite set N with |gn | ≥ 2m−1 n cn ρ on N , where m is the maximal multiplicity of a root of modulus ρ. This bound diverges along N in both sub-cases: if ρ > 1 the exponential factor diverges, while if ρ = 1 a repeated unit root gives m ≥ 2, hence 2m − 1 ≥ 3 (j) and c n2m−1 → ∞. Therefore supn |gn | = ∞.

A.4

Proof of Theorem 1

Let λ be any root of the characteristic polynomial χ(λ) = PP PP λP − j=1 δj λP −j , so that λP = j=1 δj λP −j . Suppose, for contradiction, that |λ| ≥ 1. Taking moduli and applying the triangle inequality, |λ|P ≤

P X j=1

Setting. We work at the population level under the datagenerating process, taking X := (x1−P , . . . , xL+H )⊤ , so every lagged value required by the differencing operator and by the shifts x(j) is an observation of (xt ) and E[xu xv ] = γ(u − v) holds throughout. Step 1: everything is affine in X. With a linear backbone, z = AX (from (1)), c = CX and xf = SX for matrices determined by δ. By (2) and Proposition 1, x̂f = Th (Wb AX + b + CX), so e = M X + b′ with M := Th (Wb A + C) − S and b′ := Th b, both independent of X. Hence p⊤ = (Th⊤ e)⊤ Wb = X⊤ N + q⊤ ,

N := M ⊤ Th Wb , (17) whose entries Nv,s are indexed by the coordinates v ∈ {1 − P, . . . , L + H} of X and s ∈ {1, . . . , L − 1} of z. (j)

Step 2: reduction to second moments. Since (x2:L )s = xs+1−j , and s + 1 − j ≥ 2 − P lies in the range of X, we (j) may write x2:L = Ej X with (Ej )s,u = 1{u = s + 1 − j}. By (17), (j)

(ii) Case ρ = 1, unit-modulus roots simple. By Propo(j) sition 2(ii), |hn | ≤ C, so (16) gives |gn | ≤ C 2 (n + 1) = O(n) and |Term I| ≤ C 2

Proof of Proposition 4

|δj | |λ|P −j ≤ |λ|P −1

P X

|δj | = |λ|P −1 ∥δ∥1 ,

j=1

where the second inequality uses |λ|P −j ≤ |λ|P −1 for j ≥ 1, valid since |λ| ≥ 1. As |λ| ≥ 1 implies |λ|P −1 > 0, dividing by |λ|P −1 gives |λ| ≤ ∥δ∥1 < 1, contradicting |λ| ≥ 1. Hence every root satisfies |λ| < 1, and since ρ is the largest root modulus, ρ < 1.

p⊤ x2:L = X⊤ N Ej X + q⊤ Ej X. Taking expectations, the linear term vanishes as E[X] = 0, and E[X⊤ BX] = tr(BΓ) for deterministic B gives (j) E[p⊤ x2:L ] = tr(N Ej Γ), with Γu,v = E[xu xv ] = γ(u − v). Step 3: extracting the lag. Expanding the trace entrywise and using (Ej )s,u = 1{u = s + 1 − j} to collapse the sum over u: tr(N Ej Γ) =

L−1 X

L+H X

Nv,s Γs+1−j, v

s=1 v=1−P

Nv,s γ(s + 1 − j − v).

s,v

Setting m := v − s − 1, so that s + 1 − j − v = −(m + j), and using γ(−k) = γ(k), (j) E p⊤ x2:L =

L+H−2 X

Φ(m) γ(m + j),

(18)

m=1−P −L

P where Φ(m) := v−s−1=m Nv,s , the endpoints of the range being attained at (v, s) = (1 − P, L − 1) and (L + H, 1). The weights Φ(m) are sums of entries of N , hence depend on (δ, Wb , b) but not on j: the lag enters only through the argument of γ.

B B.1

More details of AdaRDiff

Implementation Details

Overview. AdaRDiff wraps a forecasting backbone in two operations that share the same learnable weights δ ∈ RC×P : a causal differencing pass on the input, and a reconstruction pass on the backbone output (Algorithm 1). Both are implemented as depthwise conv1d over the channel dimension, so the module adds C×P parameters and no sequential work

Algorithm 1: AdaRDiff forward pass C×L

Require: context x ∈ R , phase ∈ {1, 2}; target xf ∈ RC×H 1: initialize differencing weights δ, gain α ← 1, and backbone parameters θ 2: if reparam then δ ← α δ/∥δ∥1 {per channel, Eq. (9)} 3: if RevIN then x ← (x−µ)/σ, with µ, σ the per-channel mean and standard deviation of x 4: z ← x[:, 1:] − conv1d(x, δ) {differencing, Eq. (1)} 5: ẑ ← fθ (z) {predicted residuals} 6: if phase = 1 then 7: if RevIN then xf ← (xf − µ)/σ {same statistics as x} 8: zf ← xf − conv1d([ x[:, −P :], xf ], δ) {difference the target} 9: return (ẑ, zf ) {loss ∥ẑ − zf ∥2 } 10: else 11: h ← Sequence-h(δ, H) {Alg. 2} 12: c ← conv1d(x[:, −P :], δ) {initial conditions} 13: x̂f ← conv1d(ẑ + c, h) {reconstruction, Prop. 1} 14: if RevIN then x̂f ← σx̂f + µ 15: return x̂f {loss ∥x̂f − xf ∥2 } 16: end if

beyond the O(log H) recurrence of Algorithm 2. Training proceeds in two phases (Section 3.3): Phase 1 supervises the backbone in residual space, bypassing reconstruction entirely; Phase 2 optimizes the forecast loss through the full pipeline. Reconstruction components. Reconstruction needs three ingredients: The sequence h. Stacking P consecutive terms into the state sn = (hn , . . . , hn−P +1 )⊤ turns the recurrence (4) into a one-step linear map sn+1 = F sn , where F is the companion matrix of δ; hence sn = F n s0 with s0 = (1, 0, . . . , 0)⊤ , and hn = (sn )1 . Computing s0 , . . . , sH−1 one at a time costs H sequential steps. Repeated squaring reduces this to ⌈log2 H⌉: the key identity is that F ℓ advances any state by ℓ positions, F ℓ sn = sn+ℓ . So if we have already computed the first ℓ states s0 , . . . , sℓ−1 and the matrix G = F ℓ , applying G to all of them at once yields the next ℓ states sℓ , . . . , s2ℓ−1 ; squaring G then gives F 2ℓ for the following round. Each round doubles the number of known states, so ⌈log2 H⌉ rounds cover the whole horizon, and within a round the single matrix product G [s0 · · · sℓ−1 ] produces all new states in parallel (Algorithm 2). P The initial conditions c. The terms ck = j≥k δj xL+k−j are a single convolution of the last P context values with δ, zero-padded so that taps reaching past the context contribute nothing. PThe forecast. With v = ẑ + c, the reconstruction x̂L+k = m<k hm vk−m is the causal convolution of v with h, i.e. the Toeplitz product Th v. Options. Two components are optional hyperparameters. RevIN normalizes each channel before differencing and inverts the transform on the forecast, handling distribution

Algorithm 2: Sequence-h(δ, H) by repeated squaring 1: F ← companion matrix of δ 2: S ← [ s0 ] with s0 = (1, 0, . . . , 0)⊤ ;

G ← F {S holds s0 , . . . , s|S|−1 ; G = F |S| } 3: while |S| < H do 4: S ← [ S, G S ] {G sn = sn+|S| : append the next |S| states} 5: G ← G2 {F ℓ → F 2ℓ for the next round} 6: end while 7: return hn = (sn )1 for n = 0, . . . , H − 1

shift. The reparameterization of line 2 constrains ∥δ∥1 = |α| with α learned per channel.

Supplementary Details of the Experiments

C.1

Experiment Settings

Datasets. We evaluate on widely adopted LTSF benchmarks: the ETT series (ETTh1, ETTh2, ETTm1, ETTm2) (Zhou et al. 2021), Weather, Traffic, Electricity, and Solar-Energy (Lai et al. 2018). Dataset splitting and normalization follow prior work (Liu et al. 2024; Lin et al. 2024a): the ETT datasets use a 6 : 2 : 2 train/validation/test split, and the remaining datasets a 7 : 1 : 2 split. Table 4 summarizes each dataset. All exhibit periodic structure—daily, and additionally weekly for Electricity and Traffic. Combined with the sampling frequency, this determines the dominant period of each dataset, e.g. 24 steps for the hourly ETTh variants and 168 for Electricity. These periods can be inferred from an autocorrelation (ACF) analysis, following the procedure of CycleNet (Lin et al. 2024a), and we use them to guide the choice of window size P . Hyperparameter search. All hyperparameters are selected per dataset on the validation set, by grid search over the following ranges. The window size P is tuned as described in the main text ({4, 24, 96} on the ETT datasets, the dominant period elsewhere). The initialization of the differencing weights δ is chosen among zero (δ = 0), uniform (δj = 1/P ), and first-order (δ1 = 1, else 0); RevIN (Kim et al. 2021) and the ℓ1 -reparameterization of Eq. (9) are each toggled on or off. Optimization hyperparameters are swept over initial learning rates {1×10−3 , 2×10−3 , 5×10−3 , 1× 10−2 } and batch sizes {32, 64, 256}. For the MLP backbone, the hidden dimension is additionally selected from {128, 512, 1024}.

C.2

More experimental results

Comparison under the unified TFB evaluation. Table 5 reports a complementary comparison under the unified evaluation protocol of the Time Series Forecasting Benchmark (TFB) (Qiu et al. 2024), which standardizes datasets, splits, and metrics across methods, with each method is given its best result over look-back lengths {96, 336, 512} rather than a fixed look-back window. This setting favors strong baselines, since each is tuned to its most advantageous context length; we nonetheless evaluate AdaRDiff over the same lengths

Dataset

ETTh1 ETTh2

SolarETTm1 Electricity ETTm2 Energy

Timesteps Channels Frequency

17,420 7 1h

69,680 7 15 min

Periodicity

Daily

Dominant Period

26,304 321 1h Daily, Weekly

52,560 137 10 min

168

144

Daily

Traffic

Weather

17,544 862 1h Daily, Weekly

52,696 21 10 min

168

144

Daily

Table 4: Dataset information. The dominant period is given in timesteps; for datasets with both daily and weekly cycles it is the longer (weekly) period.

and report its best result for a fair comparison. Even so, AdaRDiff attains the best accuracy on the large majority of dataset–horizon pairs: its linear and MLP backbones together take the top MSE on 29 of 35 settings and the top MAE on 22, dominating the first-place count over every competing method, including the recent PhaseFormer (Niu, Deng, and Tong 2026), MixLinear (Ma, Luo, and Sha 2026), and Amplifier (Fei et al. 2025). On dataset-level average MSE, AdaRDiff improves over the strongest baseline on six of the seven datasets, with the largest margin on ETTh2 (0.331 against 0.347 for MixLinear), followed by Weather (0.215 against 0.222) and ETTm2 (0.249 against 0.256). The sole exception is Traffic, where PatchTST edges ahead (0.397 against 0.399). These results show that AdaRDiff performs consistently across look-back lengths and evaluation setups, remaining state-of-the-art with only a lightweight linear or MLP backbone. Effect of the look-back length. Figure 5 reports AdaRDiffLinear’s test MSE as the look-back length L varies over {48, 96, 192, 336, 512, 720} at each forecast horizon. Several consistent trends emerge. First, accuracy improves almost monotonically with L: a longer history gives the differencing operator more context to identify the relevant lags, and the shortest window (L = 48) is the weakest on every dataset. Second, these gains saturate quickly at short horizons: beyond L ≈ 336 the H = 96 curves are nearly flat, and on ETTh1 the longer L = 512 window is no better than L = 336. Third, and more notably, the benefit of a longer look-back grows with the forecast horizon: extending L from 48 to 720 reduces MSE far more at H = 720 than at H = 96 (e.g. a drop of 0.089 versus 0.035 on ETTh1, and 0.081 versus 0.031 on ETTm2), since forecasting further ahead requires more history to anchor the trend and seasonal structure. Datasets with longer periodic structure, such as Electricity and Solar-Energy, continue to benefit from windows up to L = 720, consistent with their weekly and daily cycles. Overall, AdaRDiff extracts most of its short-horizon accuracy from a moderate look-back while making effective use of longer histories precisely where they matter most—at long horizons. Qualitative forecasts. Figure 6 shows AdaRDiff-Linear’s forecasts (horizon H = 96) against the ground truth on representative channels from six datasets. The forecasts follow the dominant temporal structure of each series: the sharp re-

curring spikes of Traffic, the daily cycles of Electricity and the ETT variants, and the day–night plateaus of Solar-Energy, where AdaRDiff correctly holds the low nighttime level and rises on the daytime ramp. Errors appear mainly at the peaks of the most irregular channels (e.g. Traffic channel 0), where the amplitude of individual spikes is occasionally underestimated, but the phase and period of the pattern are consistently recovered. These examples illustrate how the learned differencing operator reconstructs both periodic and trending behavior directly in the original series space.

ETTh1

ETTh2

Test MSE

0.50 0.45

ETTm1

0.6

0.5

0.4

0.3

0.40 0.3 48

192

336

512

720

0.2

0.3 48

192

336

512

Look-back length L

Weather

Electricity

720

192

336

512

720

Look-back length L

192

336

512

720

Look-back length L

Solar 0.5

0.4

Test MSE

ETTm2 0.4

0.25 0.4

0.3

0.20 0.3

0.2

0.15 48

192

336

512

720

0.2 48

Look-back length L

192

336

512

720

Look-back length L

192

336

512

720

Look-back length L

Prediction horizon H = 96

H = 192

H = 336

H = 720

Figure 5: Effect of the look-back length L on AdaRDiff-Linear’s test MSE, for each dataset and forecast horizon H ∈ {96, 192, 336, 720}.

ETTh1 channel 0

ETTh1 channel 1 2

value

0 2

168

Electricity channel 64

0.0

1.5 168

value

Traffic channel 0

168

Traffic channel 689

4 2 0

0 168

168

Solar channel 27

Solar channel 54

1.0

2.0 168

0.5

1.5

0.5

1.0

Electricity channel 320

1.0

0.5

ETTm2 channel 0

0.0

value

ETTm1 channel 0

1 168

t (0 = forecast origin)

observed past

168

ground truth

0 t (0 = forecast origin)

AdarDiff forecast

Figure 6: Qualitative forecasts of AdaRDiff-Linear (horizon H = 96) on representative channels from six datasets. Each panel shows the observed past (blue), the ground-truth future (black), and the AdaRDiff forecast (orange dashed); the shaded region marks the forecast horizon.

AdaRDiff-Linear (2026) MSE MAE

AdaRDiff-MLP (2026) MSE MAE

PhaseFormer (2026) MSE MAE

MixLinear (2026) MSE MAE

Amplifier (2025) MSE MAE

iTransformer (2024) MSE MAE

TimeMixer (2024) MSE MAE

Crossformer (2023) MSE MAE

DLinear (2023) MSE MAE

PatchTST (2023) MSE MAE

TimesNet (2023) MSE MAE

stationary (2022) MSE MAE

ETTh1

96 192 336 720 Avg

0.355 0.393 0.421 0.417 0.397

0.386 0.411 0.426 0.444 0.417

0.360 0.409 0.428 0.454 0.413

0.391 0.423 0.435 0.460 0.427

0.386 0.412 0.442 0.414 0.413

0.397 0.420 0.431 0.440 0.422

0.365 0.397 0.418 0.416 0.399

0.387 0.406 0.421 0.437 0.413

0.376 0.414 0.442 0.480 0.428

0.393 0.420 0.446 0.479 0.434

0.386 0.424 0.449 0.495 0.439

0.405 0.440 0.460 0.487 0.448

0.372 0.413 0.438 0.486 0.427

0.401 0.430 0.450 0.484 0.441

0.411 0.409 0.433 0.501 0.439

0.435 0.438 0.457 0.514 0.461

0.379 0.408 0.440 0.471 0.424

0.403 0.419 0.440 0.493 0.439

0.377 0.409 0.431 0.457 0.419

0.397 0.425 0.444 0.477 0.436

0.389 0.440 0.523 0.521 0.468

0.412 0.443 0.487 0.495 0.459

0.591 0.615 0.632 0.828 0.666

0.524 0.540 0.551 0.658 0.568

ETTh2

96 192 336 720 Avg

0.267 0.325 0.351 0.380 0.331

0.333 0.373 0.396 0.423 0.381

0.268 0.329 0.358 0.388 0.336

0.337 0.375 0.403 0.431 0.387

0.284 0.347 0.374 0.413 0.354

0.350 0.387 0.408 0.439 0.396

0.291 0.344 0.365 0.389 0.347

0.343 0.380 0.400 0.423 0.387

0.291 0.355 0.384 0.422 0.363

0.342 0.400 0.420 0.451 0.403

0.297 0.372 0.388 0.424 0.370

0.348 0.403 0.417 0.444 0.403

0.281 0.349 0.366 0.401 0.349

0.351 0.387 0.413 0.436 0.397

0.728 0.723 0.740 1.386 0.894

0.603 0.607 0.628 0.882 0.680

0.300 0.387 0.490 0.704 0.470

0.364 0.423 0.487 0.597 0.468

0.274 0.348 0.377 0.406 0.351

0.337 0.384 0.416 0.441 0.395

0.334 0.404 0.389 0.434 0.390

0.370 0.413 0.435 0.448 0.416

0.347 0.379 0.358 0.422 0.377

0.387 0.418 0.413 0.457 0.419

ETTm1

96 192 336 720 Avg

0.285 0.325 0.359 0.415 0.346

0.336 0.359 0.380 0.412 0.372

0.289 0.327 0.361 0.426 0.351

0.342 0.368 0.382 0.418 0.378

0.296 0.335 0.364 0.417 0.353

0.349 0.375 0.389 0.419 0.383

0.332 0.346 0.389 0.430 0.374

0.361 0.370 0.392 0.418 0.385

0.293 0.329 0.365 0.429 0.354

0.347 0.367 0.387 0.422 0.381

0.300 0.341 0.374 0.429 0.361

0.353 0.380 0.396 0.430 0.390

0.293 0.335 0.368 0.426 0.355

0.345 0.372 0.386 0.417 0.380

0.314 0.374 0.413 0.753 0.464

0.367 0.410 0.432 0.613 0.455

0.300 0.336 0.367 0.419 0.356

0.345 0.366 0.386 0.416 0.378

0.289 0.329 0.362 0.416 0.349

0.343 0.368 0.390 0.423 0.381

0.340 0.392 0.423 0.475 0.407

0.378 0.404 0.426 0.453 0.415

0.415 0.494 0.577 0.636 0.530

0.410 0.451 0.490 0.535 0.472

ETTm2

96 192 336 720 Avg

0.161 0.217 0.270 0.359 0.252

0.250 0.288 0.326 0.382 0.311

0.161 0.214 0.266 0.353 0.248

0.251 0.288 0.326 0.382 0.312

0.173 0.247 0.292 0.382 0.273

0.259 0.309 0.337 0.391 0.324

0.183 0.247 0.290 0.383 0.276

0.271 0.316 0.338 0.394 0.330

0.168 0.227 0.276 0.364 0.259

0.258 0.298 0.334 0.394 0.321

0.175 0.242 0.282 0.375 0.268

0.266 0.312 0.337 0.394 0.327

0.165 0.225 0.277 0.360 0.257

0.256 0.298 0.332 0.387 0.318

0.296 0.369 0.588 0.750 0.501

0.391 0.416 0.600 0.612 0.505

0.164 0.224 0.277 0.371 0.259

0.255 0.304 0.337 0.401 0.324

0.165 0.221 0.276 0.362 0.256

0.255 0.293 0.327 0.381 0.314

0.189 0.254 0.313 0.413 0.292

0.265 0.310 0.345 0.402 0.331

0.210 0.338 0.432 0.554 0.384

0.294 0.373 0.416 0.476 0.390

Weather

96 192 336 720 Avg

0.141 0.183 0.235 0.306 0.216

0.193 0.234 0.291 0.345 0.266

0.141 0.185 0.232 0.303 0.215

0.198 0.236 0.285 0.336 0.264

0.150 0.196 0.247 0.358 0.238

0.202 0.246 0.284 0.349 0.270

0.182 0.251 0.305 0.373 0.278

0.235 0.283 0.320 0.364 0.300

0.147 0.188 0.239 0.316 0.222

0.199 0.238 0.276 0.328 0.260

0.157 0.200 0.252 0.320 0.232

0.207 0.248 0.287 0.336 0.270

0.147 0.192 0.247 0.318 0.226

0.198 0.243 0.284 0.330 0.264

0.143 0.195 0.254 0.335 0.232

0.210 0.261 0.319 0.385 0.294

0.170 0.216 0.258 0.323 0.242

0.230 0.273 0.307 0.362 0.293

0.150 0.191 0.242 0.312 0.224

0.200 0.239 0.279 0.330 0.262

0.168 0.219 0.278 0.353 0.255

0.214 0.262 0.302 0.351 0.282

0.188 0.240 0.322 0.396 0.286

0.242 0.290 0.328 0.378 0.309

Electricity

96 192 336 720 Avg

0.132 0.148 0.162 0.197 0.160

0.228 0.240 0.260 0.296 0.256

0.129 0.147 0.162 0.199 0.159

0.223 0.240 0.261 0.299 0.256

0.137 0.152 0.166 0.204 0.165

0.228 0.241 0.256 0.291 0.254

0.220 0.216 0.230 0.274 0.235

0.291 0.291 0.307 0.340 0.307

0.132 0.149 0.165 0.203 0.162

0.227 0.241 0.258 0.292 0.255

0.134 0.154 0.169 0.194 0.163

0.230 0.250 0.265 0.288 0.258

0.153 0.168 0.189 0.228 0.184

0.256 0.269 0.291 0.320 0.284

0.134 0.146 0.165 0.237 0.171

0.231 0.243 0.264 0.314 0.263

0.140 0.154 0.169 0.204 0.167

0.237 0.251 0.268 0.301 0.264

0.143 0.158 0.168 0.214 0.171

0.247 0.260 0.267 0.307 0.270

0.169 0.180 0.204 0.205 0.189

0.271 0.280 0.304 0.304 0.290

0.171 0.180 0.204 0.221 0.194

0.274 0.283 0.305 0.319 0.295

Traffic

96 192 336 720 Avg

0.396 0.408 0.417 0.453 0.419

0.276 0.279 0.283 0.302 0.285

0.372 0.389 0.398 0.437 0.399

0.266 0.274 0.279 0.301 0.280

0.394 0.380 0.429 0.440 0.411

0.260 0.250 0.278 0.284 0.268

0.707 0.648 0.668 0.727 0.688

0.433 0.401 0.407 0.431 0.418

0.396 0.413 0.421 0.456 0.421

0.278 0.285 0.291 0.307 0.290

0.363 0.384 0.396 0.445 0.397

0.265 0.273 0.277 0.308 0.281

0.369 0.400 0.407 0.461 0.409

0.257 0.272 0.272 0.316 0.279

0.526 0.503 0.505 0.552 0.521

0.288 0.263 0.276 0.301 0.282

0.395 0.407 0.417 0.454 0.418

0.275 0.280 0.286 0.308 0.287

0.370 0.386 0.396 0.435 0.397

0.262 0.269 0.275 0.295 0.275

0.595 0.613 0.626 0.635 0.617

0.312 0.322 0.332 0.340 0.327

0.603 0.611 0.628 0.646 0.622

0.330 0.338 0.342 0.350 0.340

1st Count

Models Metrics

Table 5: Multivariate forecasting comparison across seven datasets with forecast horizons τ ∈ {96, 192, 336, 720}, under the TFB protocol (Qiu et al. 2024). Baseline results are taken from TFB, each reported at its best look-back length over {96, 336, 512}; AdaRDiff is evaluated identically. Best result in red, second best blue; our method in dark red.

Learning to Difference: Adaptive Reversible Differencing (AdaRDiff) for Time Series Forecasting

Analyze & explore related research in Conceptio

Related documents