CLaST: Context-aware Contrastive VAE for Probabilistic Time Series Forecasting

arXiv:2608.20025v1 [cs.LG] 20 Aug 2026

CLaST: Context-aware Contrastive VAE for Probabilistic Time Series Forecasting 1st Alexander Marusov

2nd Dmitry Anikin

3rd Vitaliy Pozdnyakov

Applied AI Institute Moscow, Russia

Applied AI Institute Moscow, Russia

Applied AI Institute Moscow, Russia

4th Aleksandr Yugay

5th Petr Sokerin

6th Ilya Kuleshov

Applied AI Institute Moscow, Russia

Applied AI Institute Moscow, Russia

Applied AI Institute Moscow, Russia

7th Alexey Zaytsev Applied AI Institute Moscow, Russia

Abstract—Probabilistic forecasting models are widely used for time series forecasting in domains such as energy systems, finance, medicine, and transportation. In recent years, deep generative models have shown strong results on probabilistic forecasting, yet many conventional approaches struggle to capture internal temporal dependencies, leading to latent representations with limited expressive power. To address this limitation, we propose CLaST, a VAE framework for probabilistic multivariate time series forecasting. Unlike existing generative models, CLaST learns embeddings that preserve contextual similarity between observations through our contrastive loss function. Experiments across nine widely adopted benchmarks demonstrate that CLaST consistently surpasses strong baseline methods. In short-term forecasting tasks, our approach achieves improvements of up to 16.4% in CRPS and 14.4% in NMAE over the second-best method. Furthermore, in long-term prediction CLaST attains superior overall performance, exceeding the second-best method by up to 48.6% and 25.1% in CRPS and NMAE, respectively. Index Terms—probabilistic time series forecasting, similarity learning, generative models, contrastive learning

I. I NTRODUCTION Time series analysis presents a unique set of challenges: temporal dependencies, non-stationarity, multimodality, missing data, and the need to quantify predictive uncertainty. A central task in this domain is time-series forecasting, in which models are required to infer future dynamics from imperfect and often volatile historical data. In a wide range of applications, leveraging deep generative models has consistently improved predictive performance. Recurrent (DeepAR [1]), attention-based (PatchTST [2]), diffusion (TSDiff-Cond [3]), and flow-based (TFM [4]) approaches offer strong predictive performance. VAE-based methods like K 2 VAE [5] and LaST [6] further structure latents using dynamical systems or trend/seasonal decomposition. However, learning informative latent representations remains a significant challenge. While estimating mutual information (MI) between input observations and their representa-

tions offers a promising approach to enhance embedding informativeness, existing neural MI estimators—such as MINE [7], CLUB [8], and STUB [6]—are computationally intensive, prone to high variance, and exhibit slow convergence in highdimensional settings [9]–[11]. Such instability undermines the optimization process and ultimately restricts the quality of the learned representations. Furthermore, current approaches often fail to capture latent temporal dependencies embedded within the contextual structure of sequential data. Here contextual similarity reflects how closely two observations align in terms of their semantics. For instance, July 1 and July 2 temperature readings share a summer heat contextual regime, whereas July 1 and January 1 belong to distinct seasonal regimes. Crucially, this notion diverges from conventional statistical correlation: variables may co-vary due to shared confounders yet remain contextually distinct. For example, electricity demand and ice cream sales often peak simultaneously in summer, but they describe fundamentally different physical processes and thus occupy separate contextual domains. To address these limitations, we propose CLaST, a variational autoencoder framework for probabilistic time series forecasting that explicitly preserves contextual structure. To adequately capture contextual similarity, we consider a class of time series in which the covariance between observations depends solely on the temporal lag δ (i.e., cov(xt , xt+δ ) = f (δ)), while a time-varying mean maintains overall nonstationarity. Although classic statistical literature has implicitly addressed such processes, to the best of our knowledge, they lack a unified nomenclature. We therefore introduce the term Lag-Invariant Non-stationary Time Series (LINTS) process. Unlike strict stationarity assumptions that frequently limit applicability to real-world data, the LINTS formulation retains realistic temporal dynamics through its evolving mean while imposing only a single, interpretable constraint

on the dependence structure. Under the LINTS process our approach leverages similarity matrices—a cornerstone of contrastive self-supervised learning [12]—to model the degree of closeness between observations in time. Specifically, we constrain the latent similarity matrices to mirror the structural patterns inherent in the input sequences. Extending the LaST architecture, we replace conventional mutual information (MI) estimators, which are prone to instability, with a contrastivebased penalty that preserves contextual similarity between objects. This formulation simultaneously enhances training stability and facilitates a more robust capture of underlying similarity dependencies. The main contributions of this work are as follows: • CLaST: Context-aware Contrastive VAE for Probabilistic Time Series Forecasting. To enhance probabilistic forecasting performance, we introduce a variational autoencoder (VAE) framework grounded in a novel objective function that explicitly accounts for the contextual similarity between samples. Our method ensures that the learned embeddings preserve meaningful sequential dependencies while effectively capturing intrinsic temporal relationships. • Theoretical properties of our loss function. We derive the proposed objective function analytically under the LINTS process. Notably, Theorem III.3 establishes the theoretical optimality of our loss function within the specified class of similarity matrices. By formalizing lag-structured priors within the optimization objective, the loss injects structural knowledge into the learning pipeline. Given sufficiently high-capacity encoders, this formulation compels the model to extract representations that preserve the lag-aware information of the input time series. • Empirical validation. Extensive experiments on both short- and long-term forecasting across nine established datasets from diverse domains, including Electricity, ETT, and Weather, demonstrate that CLaST generalizes effectively across heterogeneous time-series modalities. For the short-term setting it consistently outperforms strong baselines, achieving up to 16.4% relative improvement in CRPS and 14.4% in NMAE. Regarding the long-term predictions our approach surpasses second-best result up to 48.6% and 25.1% in CRPS and NMAE correspondingly. We provide the code implementation of the proposed method:https://anonymous.4open.science/r/CLaST3407/README.md. II. R ELATED W ORK a) Classic and deep learning methods.: Classic approaches (ARIMA [13], VAR [14], SVR [15], exponential smoothing [16]) struggle with high-dimensional, nonlinear dynamics [17]. Deep learning models address this: RNNs like LSTM [18] enable probabilistic forecasting (e.g., DeepAR [1]), while Transformers like PatchTST [2] capture long-range dependencies via patch-based attention. Conversely, linear models like DLinear [19] remain strong base-

lines by decomposing series into trend and seasonal components. b) Variational Autoencoders.: VAEs [20] offer probabilistic forecasting but often yield unstructured latents. Structured variants address this: LaST [6] factorizes latents into trend/seasonal components with MI regularization, while K 2 VAE [5] combines VAEs with Koopman operators and Kalman filtering to explicitly model temporal dynamics and improve long-horizon accuracy. c) Ordinary Differential Equations.: Continuous-time models (Neural ODE [21], Latent ODE [22], GRU-ODE [23]) handle irregular sampling naturally but incur high computational costs and lack latent regularization. Flow-based alternatives like Trajectory Flow Matching (TFM) [4] efficiently learn conditional trajectory derivatives via continuous normalizing flows without stochastic simulation. DeNOTS [24] introduces a negative feedback mechanism to stabilize vector fields, thereby enabling the modeling of long-term dependencies. d) Diffusion models.: Diffusion [25] generates probabilistic forecasts by reversing a noise process conditioned on past observations. While effective for multi-horizon forecasting [26] and imputation [27], they are computationally intensive and lack latent interpretability. TSDiff-Cond [3] improves scalability by integrating S4 layers with convolutional channel mixing in an SSSD-based architecture. Following the literature review, we chose LaST [6], K 2 VAE [5], DeNOTS [24], DeepAR [1], TFM [4], TSDiff [3], and PatchTST [2] as baseline models due to their strong reported performance across their respective categories. III. M ETHOD To present our proposed method, we first start with the problem statement. Then we outline the overall pipeline of the CLaST approach, followed by a detailed description of its key components. The notation appears throughout the text as needed for clarity. For a complete list of notation used throughout the paper, we refer the reader to the online documentation available on our GitHub repository. A. Problem Statement. We study probabilistic forecasting for multivariate time series. Let xt ∈ RF denote the observations of F variables at time t. Given a historical window xt−N +1:t of length N , the task is to predict the future sequence xt+1:t+L over a horizon L. In the probabilistic setting, the model outputs a predictive distribution p(xt+1:t+L | xt−N +1:t ), which captures both future values and their uncertainty. B. CLaST We now introduce CLaST (Contrastive LaST), a generative– contrastive model for probabilistic time series forecasting. CLaST builds upon the LaST architecture [6] while fundamentally revising its latent regularization strategy. To address data non-stationarity, LaST employs dedicated seasonal and trend encoders and decoders, providing informative representations for each component. Thus, LaST

separately produces latent representations for trend Zt and seasonality Zs , along with a predictor module that uses the Discrete Fourier Transform (DFT) for seasonal forecasting. A central component of LaST is its loss function. The evidence lower bound (ELBO) term LELBO ensures that the learned trend and seasonal embeddings are sufficiently informative to both reconstruct the input time series and enable accurate predictions. The authors employ a slightly modified MINE-based mutual information estimator to maximize the amount of information preserved from the original input X in the trend and seasonal representations, Zt and Zs . The corresponding objectives are denoted as IMINE (X, Zt ) and IMINE (X, Zs ), respectively. Meanwhile, the term ISTUB (Zs , Zt ) promotes disentanglement between the trend and seasonal representations, encouraging them to capture complementary, non-overlapping information. The final loss function for LaST is defined in eq. 1.

Our approach for constructing the ground truth similarity matrix G from time series is described in Section III-E. We start from a correlation matrix R, which captures the statistical dependencies in the data. Since correlations do not directly reflect similarity relationships, we introduce a mapping that transforms the correlation space into a similarity space, yielding the matrix G with elements gij presented in eq. (6). In Section III-F, we derive a closed-form expression for b by formulating and solving the estimated similarity matrix G a corresponding optimization problem. In Theorem III.3 we b We prove that this closeddefine the elements gbij (fθ ) of G. form solution is the unique global minimizer. Substituting the ground truth matrix G (eq. (6)) and the b estimated matrix G(Z) (Thm. III.3) into the equation (3) yields the final loss function (2). D. Theoretical Background

LLaST = LELBO + IMINE (X, Zs ) + IMINE (X, Zt ) − ISTUB (Zs , Zt ).

(1)

However, as discussed earlier, MI estimations could be unstable in practice. We replace the MI-based objective with a contextual similarity alignment loss that encourages latent representations to preserve the contextual structure of the input time series. The central idea is to align the estimated similarity b matrix G(Z) with the data-induced similarity matrix G = G(X), which is called ground truth similarity matrix. Here, b G(Z) is computed from the trend and seasonal representations Zt and Zs , while G serves as the similarity matrix. Although the similarity matrices estimated from trend emb t ), and seasonal embeddings, G(Z b s ), both apbeddings, G(Z proximate the same ground-truth matrix G, the corresponding representations remain separable in the embedding space, as illustrated in our experiments1 . This separation arises because the LaST framework explicitly models trend and seasonal components as distinct components. All other parts include the LaST’s core architecture, in particular, separate trend and seasonal latents, and the DFTbased predictor would remain the same. The full CLaST objective with three terms is then given by b t ) 2 + G − G(Z b s ) 2 . (2) LCLaST = LELBO + G − G(Z F F Here, 2

b G − G(Z) = F

X

2 gij − gbij (fθ ) .

(3)

i,j

Detailed definitions and discussions of these estimated and ground truth similarity matrices are postponed to Section III-D. C. Method Overview In Section III-D, we formally define the ground-truth G b that underpin our loss and estimated similarity matrices G function. 1 https://anonymous.4open.science/r/CLaST-3407/figs/trend seasonal.png

To explain our method, we consider a multivariate time series consisting of N observations {xi }N i=1 , where each xi ∈ RF represents the F -dimensional observation at time step i. The time series can thus be represented as a matrix X ∈ RN ×F . To utilize a loss function based on pairwise comparisons, we introduce two complementary matrices: a ground truth similarity matrix and an estimated similarity matrix that are core concepts in contrastive self-supervised learning [12]. The ground truth similarity matrix G = {gij }N i,j=1 ∈ N ×N R encodes prior knowledge about the contextual structure of the time series. Each entry gij represents the desired level of similarity closeness between observations collected at time moments i ∈ [1; N ] and j ∈ [1; N ]. This matrix, therefore, defines the target relational structure that the learned representations are expected to preserve. b The estimated similarity matrix G(Z) = {ĝij }N i,j=1 ∈ N ×N R , on the other hand, captures the similarities inferred b by the model itself. The matrix G(Z) is computed from N the embedding matrix Z = {zi }i=1 ∈ RN ×h , where each representation zi = fθ (xi ) ∈ Rh is obtained by applying an b encoder function fθ to the input signal xi . As a result, G reflects how close the samples are in the learned representation space. b with the ground truth matrix G, we By comparing G obtain a natural objective for training the encoder. Minimizing the discrepancy between these two matrices encourages the learned embeddings to preserve the contextual relationships specified by the prior knowledge encoded in G: X (gij − gbij (fθ ))2 → min . (4) i,j

θ

We use objective function 4 as our penalty in eq. (3) for trend and seasonal components. Defining this penalty requires specifying the ground truth matrix G (Section III-E) and the b (Section III-F). estimated matrix G

E. Ground Truth Similarity Matrix G 1) Correlation As A Contextual Proxy: To define the matrix G, we will rely on statistical correlations between observations. In particular, we consider a multivariate time series {xi }N i=1 modeled as a Lag-Invariant Non-stationary Time Series (LINTS). In this framework, the inter-observation covariance depends exclusively on the temporal lag: cov(xi , xi+δ ) = f (δ). While the dependence kernel is shift-invariant, the process itself remains non-stationary due to its time-varying mean. The sample correlation matrix R is constructed as follows. 1) Centering and normalizing. For each time point i ∈ PF (j) [1; N ], subtract a location estimate xi = F1 j=1 xi to obtain the centered vector, then normalize it to unit Euclidean length: xi − xi x̃i = . ∥xi − xi ∥2 This yields observations x̃i that lie on the unit sphere in RF . 2) Computing pairwise similarities. The correlation mae is obtained as the Gram matrix of X e = trix R ⊤ ⊤ [x̃1 , . . . , x̃N ]: e =X eX e ⊤ ∈ RN ×N . R 3) Symmetric Toeplitz structure. For each lag δ ∈ {0, . . . , N − 1} we compute the median empirical correlation rδ = median {e rij : |i − j| = δ} . Replacing each entry reij by r|i−j| yields a symmetric Toeplitz matrix R whose (i, j)-entry depends only on the time lag |i − j|: rij = r|i−j| ,

i, j = 1, . . . , N.

Thus the entire correlation structure is summarized by the N numbers r0 , r1 , . . . , rN −1 , with r0 = 1 since Rii = 1. The Toeplitz property reflects that covariance depends only on lag, e ij = R e ji follows from the definition of while the symmetry R correlation. We now define a matrix G derived from the correlation matrix R. Each entry gij ∈ [0, 1] quantifies the degree to which observations collected at time points i and j play similar roles within the underlying dynamical process. Because the correlations rij lie in [−1; 1], we need a mapping m : [−1; 1] → [0; 1] that converts correlation values into probabilities of similarity. Following [12], we set the main diagonal of G to zero, as these entries correspond to the trivial self-similarity of each sample. We seek a principled mapping from correlations to similarity scores. Theorem III.1 is derived using classical Bayesian decision theory under Gaussian class-conditional assumptions. We show that for LINTS-process, the Bayes-optimal posterior mapping from the observed lag-τ correlation to contextual similarity is logistic (sigmoid). For simplicity, Theorem III.1 is stated for the univariate case.

Theorem III.1 (Optimal mapping from feature to similarity space). Let {xi }N i=1 be a LINTS-process: covariance depends only on the fixed lag δ ≥ 0, so cov(xi , xi+δ ) = f (δ). Let Yδ ∈ {0, 1} be a similarity indicator, and let ρy ∈ (−1, 1) denote the population lag-δ correlation associated with Yδ = y. Assume: (A1)

there exists continuously differentiable function g ∈ C 1 ((−1; 1)) such that zδ := g(rδ ) satisfies   (zδ − µy )2 1 , exp − p(zδ | Yδ = y) = √ 2σ 2 2πσ 2 µy := g(ρy ), σ 2 independent of y;

(A2)

ρ0 ̸= ρ1 .

Then, in the small-correlation regime |rδ | ≪ 1, P (Yδ = 1 | rδ ) = σ(αrδ + β),

(5)

for some α, β ∈ R, where σ(x) = (1 + e−x )−1 . Remark III.2 (Example of a valid transformation). A classical choice satisfying Assumption A1 is the Fisher z-transform g(r) =

1+r 1 log , 2 1−r

for which, under mild mixing conditions, g(rδ ) is asymptotically Gaussian with variance independent of the true correlation. This motivates the modeling assumption in (A1). The proof of the Theorem III.1 can be found in the Appendix A. 2) Target contextual Similarity: The intuition underlying the connection between the data space and the similarity space, formalized in Theorem III.1, is illustrated in Figure 1. Accordingly, the entries of the true dependency matrix G are defined using a sigmoid function as follows: ( smin + (1 − smin ) σ(αr|i−j| ), i ̸= j, gij = (6) 0, i = j, where smin ∈ (0, 0.5) specifies a lower bound on contextual affinity and α ∈ [1, 10] controls the sensitivity of dependency to variations in correlation. We introduce smin to reflect the assumption that even maximally negatively correlated vectors still share structural information and should not have zero contextual similarity. Our ablation studies F demonstrate that the sigmoid function provides the optimal mapping from correlation space to similarity space, a result validated both theoretically and empirically. b F. Estimated Similarity Matrix G Now we need to derive an expression of the estimated simib To do this, we consider the optimization problarity matrix G. b ∈ RN ×N lem of learning an estimated similarity matrix G N from a symmetric distance matrix D = {dij }i,j=1 ∈ RN ×N , where dii = 0. Here dij = d(zi , zj ) denotes the distance (e.g. euclidean) between representations zi and zj at time moments

Feature Space

Similarity Space Similarity 1.0

𝑥𝑡 (Reference Vector)

𝜃𝑡+1 ≈ 90° 𝜃𝑡+2 ≈ 180°

Similarity (𝑥𝑡 , 𝑥𝑡+1 )

𝑥𝑡+1 (Near Orthogonal)

𝑥𝑡+2 (Almost Opposite)

Similarity (𝑥𝑡 , 𝑥𝑡+2 )

𝑆𝑚𝑖𝑛 -1.0

0.0

1.0

𝐜𝐨𝐬 𝜽

Fig. 1: Connection between the original feature space and the contextual similarity space. In the feature space, xt is the observation at the current time step, and xt+1 and xt+2 are the inputs at the subsequent time steps. In our formulation, the correlation coefficient r between normalized observations corresponds to the cosine of the angle between them (r = cos θ). Correlations are mapped to contextual similarity scores via a sigmoid with a lower bound smin , yielding a smooth notion of contextual proximity.

b is assumed to be Toeplitz, i.e., its entries i and j. The matrix G depend only on the temporal lag |i − j|. Following [12], we cast the problem of estimating the b as a Laplacian estimation task in graph similarity matrix G b by solving signal processing [28], [29]. We therefore obtain G an optimization problem specifically tailored to time-series data, restricting the search space to G, the set of symmetric, non-negative Toeplitz matrices with a zero main diagonal. b + Rlog (G) b → min , Tr(DG) b G∈G

(7)

b = τ P ĝij ln(ĝij ) − 1 is a logarithmic where Rlog (G) i̸=j regularizer. Theorem III.3. (Necessary and sufficient conditions). Let D be the matrix of pairwise distances, and the regularizer b = τ P ĝij (ln(ĝij ) − 1). The matrix G b with Rlog (G) i̸=j elements   ϕ|i−j| 1{i̸=j} , gbij = exp − τ P with ϕk = N 1−k |i−j|=k dij is the unique global minimizer for the optimization problem (7). The detailed derivation and proof of Theorem III.3 are given in Appendix B. IV. E XPERIMENTAL R ESULTS A. Experiments Setup a) Datasets: We evaluate our models on nine publicly available datasets that span several domains: ETT (ETTh1, ETTh2, ETTm1, ETTm2) [30], Traffic [31], Electricity2 , 2 https://archive.ics.uci.edu/dataset/321/electricityloaddiagrams20112014, accessed in January 2026.

Weather3 . Detailed information can be found in the Appendix C. b) Metrics.: We evaluate the forecasts using two commonly used metrics: the Continuous Ranked Probability Score (CRPS) and the Normalized Mean Absolute Error (NMAE) that correspond to the quality of uncertainty estimation and forecasting correspondingly. The detailed descriptions of the metrics are in the Appendix D. c) Baselines.: We compare our approach against several generative models, including modern K 2 VAE [5], LaST [6], TSDiff [3], and TFM [4]. Additionally, we include PatchTST [32], a point-wise forecasting model, for reference. d) Backbone.: To choose an appropriate backbone encoder, we considered three types of backbone architectures: DLinear, MLP and PatchTST. The hyperparameters for the selected backbones are detailed in Appendix E. For both short- and long-term forecasting tasks, we consistently employ PatchTST as the backbone encoder. B. Main Results a) Short-term forecasting (N = 96, L = 24): For short-term prediction, all experiments are performed with a forecasting horizon of L = 24 and an context length of N = 96. As shown in Table I, CLaST delivers the best performance across several datasets, highlighting its effectiveness for probabilistic forecasting. It consistently outperforms the secondbest method on datasets ETTh1, ETTh2, ETTm1, ETTm2, and Weather, achieving relative improvements of 0.9–16.4% in CRPS and 4.4–14.4% in NMAE. Our comparative analysis reveals that when PatchTST is trained within the CLaST 3 https://www.bgc-jena.mpg.de/wetter/, accessed in January 2026.

TABLE I: Comparison of short-term probabilistic time series forecasting performance (N = 96, L = 24) across various real-world datasets. Lower CRPS and NMAE scores indicate superior forecasting accuracy. Reported results show the mean values and standard errors, computed from three independent runs involving model retraining and evaluation. Boldface highlights the best-performing method, while underlining denotes the second-best result. Dataset

Metric

CLaST (ours)

LaST

K 2 VAE

DeepAR

TFM

TSDiff

PatchTST

DeNOTS

Electricity

CRPS NMAE

0.089 ± 0.000 0.108 ± 0.000

0.144 ± 0.021 0.166 ± 0.022

0.082 ± 0.000 0.107 ± 0.000

0.117 ± 0.001 0.151 ± 0.002

0.150 ± 0.011 0.201 ± 0.014

0.115 ± 0.006 0.150 ± 0.008

0.116 ± 0.002 0.116 ± 0.002

0.201 ± 0.002 0.274 ± 0.001

Traffic

CRPS NMAE

0.203 ± 0.001 0.241 ± 0.000

0.309 ± 0.011 0.360 ± 0.010

0.164 ± 0.001 0.211 ± 0.002

0.207 ± 0.001 0.264 ± 0.002

0.321 ± 0.013 0.426 ± 0.022

0.208 ± 0.004 0.253 ± 0.004

0.256 ± 0.001 0.256 ± 0.001

0.386 ± 0.019 0.500 ± 0.032

ETTh1

CRPS NMAE

0.220 ± 0.002 0.267 ± 0.003

0.254 ± 0.004 0.289 ± 0.005

0.222 ± 0.002 0.283 ± 0.006

0.384 ± 0.071 0.498 ± 0.088

0.540 ± 0.018 0.623 ± 0.026

0.373 ± 0.041 0.492 ± 0.037

0.294 ± 0.006 0.294 ± 0.006

0.503 ± 0.008 0.645 ± 0.017

ETTh2

CRPS NMAE

0.316 ± 0.004 0.371 ± 0.004

0.460 ± 0.058 0.546 ± 0.077

0.372 ± 0.005 0.405 ± 0.005

1.168 ± 0.091 1.485 ± 0.138

1.738 ± 0.451 1.892 ± 0.384

0.989 ± 0.213 1.281 ± 0.245

0.508 ± 0.055 0.508 ± 0.055

1.920 ± 0.398 2.207 ± 0.421

ETTm1

CRPS NMAE

0.180 ± 0.004 0.218 ± 0.004

0.217 ± 0.004 0.257 ± 0.008

0.186 ± 0.005 0.228 ± 0.003

0.359 ± 0.008 0.473 ± 0.005

0.556 ± 0.038 0.634 ± 0.038

0.299 ± 0.036 0.396 ± 0.059

0.247 ± 0.005 0.247 ± 0.005

0.561 ± 0.048 0.721 ± 0.050

ETTm2

CRPS NMAE

0.229 ± 0.000 0.271 ± 0.002

0.274 ± 0.004 0.325 ± 0.007

0.314 ± 0.027 0.316 ± 0.013

0.999 ± 0.182 1.273 ± 0.201

1.784 ± 0.121 1.904 ± 0.151

0.468 ± 0.066 0.630 ± 0.087

0.369 ± 0.022 0.369 ± 0.022

1.591 ± 0.889 1.815 ± 0.906

Weather

CRPS NMAE

0.296 ± 0.011 0.345 ± 0.016

0.538 ± 0.057 0.676 ± 0.090

0.683 ± 0.011 0.491 ± 0.014

0.360 ± 0.025 0.425 ± 0.017

1.008 ± 0.322 1.206 ± 0.403

0.327 ± 0.060 0.412 ± 0.093

0.403 ± 0.035 0.403 ± 0.035

0.434 ± 0.022 0.562 ± 0.030

TABLE II: Comparison of long-term probabilistic time series forecasting performance (N = 96, L = 720) across various real-world datasets. Lower CRPS and NMAE scores indicate superior forecasting accuracy. Reported results show the mean values and standard errors, computed from three independent runs involving model retraining and evaluation. Boldface highlights the best-performing method, while underlining denotes the second-best result. Dataset

Metric

CLaST (ours)

LaST

K 2 VAE

DeepAR

TFM

TSDiff

PatchTST

DeNOTS

Electricity

CRPS NMAE

0.126 ± 0.002 0.146 ± 0.002

0.146 ± 0.001 0.169 ± 0.001

0.115 ± 0.006 0.140 ± 0.004

0.163 ± 0.009 0.204 ± 0.011

0.210 ± 0.017 0.292 ± 0.022

0.126 ± 0.003 0.165 ± 0.004

0.156 ± 0.002 0.156 ± 0.002

0.186 ± 0.009 0.186 ± 0.009

Traffic

CRPS NMAE

0.247 ± 0.001 0.288 ± 0.001

0.301 ± 0.003 0.343 ± 0.002

0.189 ± 0.002 0.243 ± 0.000

0.220 ± 0.003 0.286 ± 0.006

0.473 ± 0.018 0.637 ± 0.023

0.297 ± 0.027 0.368 ± 0.033

0.276 ± 0.002 0.276 ± 0.002

0.414 ± 0.143 0.419 ± 0.151

ETTh1

CRPS NMAE

0.342 ± 0.005 0.379 ± 0.004

0.573 ± 0.009 0.614 ± 0.010

0.334 ± 0.008 0.428 ± 0.012

0.690 ± 0.211 0.830 ± 0.240

0.590 ± 0.060 0.839 ± 0.088

0.575 ± 0.059 0.701 ± 0.054

0.511 ± 0.001 0.511 ± 0.001

0.555 ± 0.014 0.750 ± 0.019

ETTh2

CRPS NMAE

0.544 ± 0.001 0.588 ± 0.002

2.531 ± 0.054 2.602 ± 0.038

1.000 ± 0.235 0.769 ± 0.083

2.332 ± 0.013 2.611 ± 0.053

2.495 ± 0.688 3.204 ± 0.992

2.254 ± 0.106 2.488 ± 0.104

1.541 ± 0.022 1.541 ± 0.022

2.082 ± 0.136 2.356 ± 0.104

ETTm1

CRPS NMAE

0.309 ± 0.000 0.341 ± 0.001

0.421 ± 0.013 0.487 ± 0.015

0.266 ± 0.024 0.328 ± 0.039

0.466 ± 0.073 0.623 ± 0.069

0.592 ± 0.010 0.855 ± 0.025

0.483 ± 0.020 0.629 ± 0.025

0.404 ± 0.029 0.404 ± 0.029

0.576 ± 0.009 0.614 ± 0.010

ETTm2

CRPS NMAE

0.508 ± 0.001 0.534 ± 0.001

1.871 ± 0.241 2.022 ± 0.251

0.989 ± 0.275 0.713 ± 0.105

2.064 ± 0.646 2.388 ± 0.633

2.899 ± 0.782 3.783 ± 0.975

1.642 ± 0.450 1.915 ± 0.430

1.240 ± 0.152 1.240 ± 0.152

1.931 ± 0.016 2.008 ± 0.036

Weather

CRPS NMAE

0.447 ± 0.001 0.482 ± 0.001

0.672 ± 0.028 0.835 ± 0.043

0.661 ± 0.007 0.606 ± 0.008

0.465 ± 0.064 0.555 ± 0.067

1.344 ± 0.246 1.542 ± 0.422

0.375 ± 0.030 0.465 ± 0.033

0.685 ± 0.034 0.685 ± 0.034

0.803 ± 0.071 0.865 ± 0.084

generative framework, it demonstrably surpasses its pointwisetrained counterpart across every benchmark dataset, as measured by the NMAE. The observed performance gains range from a minimum improvement of 5.86% on the Traffic dataset to a substantial maximum of 26.97% on ETTh2, underscoring the significant advantage conferred by our CLaST generative learning paradigm. b) Long-term forecasting (N = 96, L = 720): As presented in Table II, CLaST achieves superior CRPS and NMAE performance on the demanding ETTh2 and ETTm2 datasets, yielding reductions of up to 48.6% in CRPS on ETTh2 and 25.1% in NMAE on ETTm2. Integrating PatchTST into the CLaST framework yields substantial NMAE improvements across most benchmarks, with relative gains ranging from 6.41% on Electricity to 61.84% on ETTh2. The exceptionally high dimensionality of the Traffic (862 channels) and Electricity (321 channels) datasets renders longterm forecasting particularly challenging. Consequently, we

extended our evaluation to two additional energy-domain datasets, Solar and ERCOT [33]. Due to the substantial computational resources already expended on the main Table II for long-term experiments running architectures like diffusion-based (TSDiff), attention-based (PatchTST), flowbased (TFM), and ODE-based (e.g. DeNOTS) was not feasible within our remaining budget. We therefore restrict this comparison to the two most relevant VAE-based baselines: K 2 V AE, the strongest empirical competitor on the main benchmarks, and LaST, the foundational architecture upon which CLaST is built. As demonstrated in Table III, CLaST consistently attains the best long-term forecasting results, outperforming the second-best model by margins of up to 12.9% in CRPS and 11% in NMAE. As shown in Figure 3 our CLasT effectively models both trend and seasonal components. c) Ablation studies: To verify that the observed performance gains stem from our regularization strategy rather than the choice of encoder, we conduct additional ablation

TABLE III: Comparison of long-term probabilistic time series forecasting performance on Solar and ERCOT datasets (N = 96, L = 720). Lower CRPS and NMAE scores indicate superior forecasting accuracy. Reported results show the mean values and standard errors, computed from three independent runs involving model retraining and evaluation. Boldface highlights the best-performing method, while underlining denotes the second-best result.

construction. In practice, this overhead is modest: LaST trains ∼28% faster per epoch than CLaST 6 , yet CLaST delivers substantially better performance. As we mentioned earlier in our main results with PatchTST backbone, CLaST improves CRPS/NMAE by up to 46.8%/45.6% Using the FeedNet, gains reach 46.17%/52.37 This marginal computational cost is strongly justified by the consistent, backbone-agnostic improvements in predictive quality.

Dataset

Metric

CLaST (ours)

K2 VAE

LaST

VI. C ONCLUSION

ERCOT

CRPS NMAE

0.080 ± 0.001 0.090 ± 0.002

0.084 ± 0.001 0.103 ± 0.002

0.095 ± 0.004 0.106 ± 0.005

Solar

CRPS NMAE

0.529 ± 0.010 0.573 ± 0.016

0.696 ± 0.020 0.637 ± 0.006

0.607 ± 0.015 0.644 ± 0.011

This paper presented CLaST, a Context-aware VAE framework for probabilistic time series forecasting. Standard generative objectives often fail to encourage informative latent representations, particularly in sequential settings. CLaST mitigates this issue by introducing a natural theoretically-grounded loss function that accounts for contextual similarity between observations. It transfers temporal structure from the input space to the latent space using theoretically grounded loss functions, ensuring that the embeddings preserve contextual structure of dependent data. Across a range of benchmarks covering diverse domains such as Electricity, ETT, and Weather, the proposed method demonstrates consistent and meaningful gains over strong baselines. These improvements reflect CLaST’s ability to capture intrinsic temporal relationships while maintaining the flexibility of latent-variable models. In short-term forecasting tasks, our approach achieves improvements of up to 16.4% in CRPS and 14.4% in NMAE. Furthermore, for the longterm forecasting CLaST attains superior overall performance, exceeding the second-best method by up to 48.6% and 25.1% in CRPS and NMAE, respectively, indicating enhanced uncertainty quantification and reduced forecasting error without undermining prior modeling advances.

experiments on three datasets (ETTh1, ETTh2, Weather) using different backbone architectures. With PatchTST as the backbone, CLaST improves both CRPS and NMAE by up to 46.8% and 45.6%, respectively 4 . When using the original FeedNet backbone from LaST, the gains reach 46.17% (CRPS) and 52.37% (NMAE) 5 . We conducted a detailed ablation study to understand the model analysis. In particular to examine the effect of hyperparameter choices, we present a sensitivity analysis in Appendix G. Appendix G1 studies the effect of the latent space dimensionality while Appendix G2 analyzes the impact of the context length. Additionally, we study how well temporal structure is preserved in the latent space. Our results in Appendix H indicate that the learned embeddings retain temporal dependencies. The implementation details are provided in the Appendix E. V. D ISCUSSION How the proposed loss enhances forecasting. Accurate probabilistic forecasting requires capturing diverse temporal dynamics and quantifying uncertainty. When latent representations collapse, decoders produce over-smoothed predictions and degraded CRPS. Our loss prevents this by structuring the latent space to preserve lag-decay correlations, keeping temporally distinct inputs separable and enabling sharper, better-calibrated forecasts (Fig. 2, Fig. 3). The resulting gains in CRPS and NMAE directly reflect this principled temporal regularisation. Theoretical limitations. CLaST assumes inter-observation covariance depends solely on temporal lag, not absolute time. This aligns with many real-world domains where dependency structures remain stable despite non-stationary means: climatological series with long-term trends, financial indicators under trend-stationary models [34], and industrial sensor data with invariant physical dynamics [35]. Practical limitations. The contextual similarity alignment loss incurs O(N 2 ) complexity due to full similarity-matrix 4 https://anonymous.4open.science/r/CLaST-3407/tables/last clast patchtst.

md

R EFERENCES [1] D. Salinas, V. Flunkert, J. Gasthaus, and T. Januschowski, “Deepar: Probabilistic forecasting with autoregressive recurrent networks,” International journal of forecasting, vol. 36, no. 3, pp. 1181–1191, 2020. [2] Y. Nie, N. H. Nguyen, P. Sinthong, and J. Kalagnanam, “A time series is worth 64 words: Long-term forecasting with transformers,” in International Conference on Learning Representations, 2023. [3] M. Kollovieh, A. F. Ansari, M. Bohlke-Schneider, J. Zschiegner, H. Wang, and Y. B. Wang, “Predict, refine, synthesize: Self-guiding diffusion models for probabilistic time series forecasting,” Advances in Neural Information Processing Systems, vol. 36, pp. 28 341–28 364, 2023. [4] X. N. Zhang, Y. Pu, Y. Kawamura, A. Loza, Y. Bengio, D. Shung, and A. Tong, “Trajectory flow matching with applications to clinical time series modelling,” Advances in Neural Information Processing Systems, vol. 37, pp. 107 198–107 224, 2024. [5] X. Wu, X. Qiu, H. Gao, J. Hu, B. Yang, and C. Guo, “K2vae: A koopman-kalman enhanced variational autoencoder for probabilistic time series forecasting,” arXiv preprint arXiv:2505.23017, 2025. [6] Z. Wang, X. Xu, W. Zhang, G. Trajcevski, T. Zhong, and F. Zhou, “Learning latent seasonal-trend representations for time series forecasting,” Advances in Neural Information Processing Systems, vol. 35, pp. 38 775–38 787, 2022. [7] M. I. Belghazi, A. Baratin, S. Rajeshwar, S. Ozair, Y. Bengio, A. Courville, and D. Hjelm, “Mutual information neural estimation,” in International conference on machine learning. PMLR, 2018, pp. 531–540.

5 https://anonymous.4open.science/r/CLaST-3407/tables/last clast feednet.

md

6 https://anonymous.4open.science/r/CLaST-3407/tables/time.md

[8] P. Cheng, W. Hao, S. Dai, J. Liu, Z. Gan, and L. Carin, “CLUB: A contrastive log-ratio upper bound of mutual information,” in Proceedings of the 37th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, H. D. III and A. Singh, Eds., vol. 119. PMLR, 13–18 Jul 2020, pp. 1779–1788. [Online]. Available: https://proceedings.mlr.press/v119/cheng20b.html [9] D. Shin and H. Kim, “An experimental study of neural estimators of the mutual information between random vectors modeling power spectrum features,” EURASIP Journal on Advances in Signal Processing, vol. 2024, 01 2024. [10] Q. Guo, J. Chen, D. Wang, Y. Yang, X. Deng, J. Huang, L. Carin, F. Li, and C. Tao, “Tight mutual information estimation with contrastive fenchel-legendre optimization,” Advances in Neural Information Processing Systems, vol. 35, pp. 28 319–28 334, 2022. [11] C. Chan, A. Al-Bashabsheh, H. P. Huang, M. Lim, D. S. H. Tam, and C. Zhao, “Neural entropic estimation: A faster path to mutual information estimation,” arXiv preprint arXiv:1905.12957, 2019. [12] R. Balestriero and Y. LeCun, “Contrastive and non-contrastive selfsupervised learning recover global and local spectral embedding methods,” in Advances in Neural Information Processing Systems (NeurIPS 2022), 2022, pp. 26 671–26 685. [13] G. E. Box and D. A. Pierce, “Distribution of residual autocorrelations in autoregressive-integrated moving average time series models,” Journal of the American statistical Association, vol. 65, no. 332, pp. 1509–1526, 1970. [14] B. Biller and B. L. Nelson, “Modeling and generating multivariate timeseries input processes using a vector autoregressive technique,” ACM Transactions on Modeling and Computer Simulation (TOMACS), vol. 13, no. 3, pp. 211–237, 2003. [15] L.-J. Cao and F. E. H. Tay, “Support vector machine with adaptive parameters in financial time series forecasting,” IEEE Transactions on neural networks, vol. 14, no. 6, pp. 1506–1518, 2003. [16] P. R. Winters, “Forecasting sales by exponentially weighted moving averages,” Management science, vol. 6, no. 3, pp. 324–342, 1960. [17] M. Jin, H. Y. Koh, Q. Wen, D. Zambon, C. Alippi, G. I. Webb, I. King, and S. Pan, “A survey on graph neural networks for time series: Forecasting, classification, imputation, and anomaly detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. [18] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997. [19] A. Zeng, M. Chen, L. Zhang, and Q. Xu, “Are transformers effective for time series forecasting?” arXiv preprint arXiv:2205.13504, 2022. [20] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv e-prints, pp. arXiv–1312, 2013. [21] R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, “Neural ordinary differential equations,” Advances in neural information processing systems, vol. 31, 2018. [22] Y. Rubanova, R. T. Chen, and D. K. Duvenaud, “Latent ordinary differential equations for irregularly-sampled time series,” Advances in neural information processing systems, vol. 32, 2019. [23] E. De Brouwer, J. Simm, A. Arany, and Y. Moreau, “Gru-ode-bayes: Continuous modeling of sporadically-observed time series,” Advances in neural information processing systems, vol. 32, 2019. [24] I. Kuleshov, E. Romanenkova, V. A. Zhuzhel, G. Boeva, E. Vorsin, and A. Zaytsev, “DeNOTS: Stable deep neural ODEs for time series,” in The Fourteenth International Conference on Learning Representations (ICLR), 2026. [Online]. Available: https://openreview.net/forum?id= SFoDJZ1sSk [25] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840– 6851, 2020. [26] K. Rasul, C. Seward, I. Schuster, and R. Vollgraf, “Autoregressive denoising diffusion models for multivariate probabilistic time series forecasting,” in International conference on machine learning. PMLR, 2021, pp. 8857–8868. [27] Y. Tashiro, J. Song, Y. Song, and S. Ermon, “Csdi: Conditional scorebased diffusion models for probabilistic time series imputation,” Advances in neural information processing systems, vol. 34, pp. 24 804– 24 816, 2021. [28] X. Dong, D. Thanou, P. Frossard, and P. Vandergheynst, “Learning laplacian matrix in smooth graph signal representations,” IEEE Transactions on Signal Processing, vol. 64, no. 23, pp. 6160–6173, 2016. [29] V. Kalofolias, “How to learn a graph from smooth signals,” in Artificial intelligence and statistics. PMLR, 2016, pp. 920–929.

[30] H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long sequence time-series forecasting,” in The Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Conference, vol. 35, no. 12. AAAI Press, 2021, pp. 11 106–11 115. [31] G. Lai, W.-C. Chang, Y. Yang, and H. Liu, “Modeling long-and short-term temporal patterns with deep neural networks,” in The 41st international ACM SIGIR conference on research & development in information retrieval, 2018, pp. 95–104. [32] Y. Nie, “A time series is worth 64words: Long-term forecasting with transformers,” arXiv preprint arXiv:2211.14730, 2022. [33] O. Shchur, A. F. Ansari, C. Turkmen, L. Stella, N. Erickson, P. GuerronQuintana, M. Bohlke-Schneider, and Y. Wang, “fev-bench: A realistic benchmark for time series forecasting,” Boston College Department of Economics, Tech. Rep., 2025. [34] J. D. Hamilton, Time series analysis. Princeton university press, 2020. [35] C. K. Williams and C. E. Rasmussen, Gaussian processes for machine learning. MIT press Cambridge, MA, 2006, vol. 2, no. 3. [36] J. Zhang, X. Wen, Z. Zhang, S. Zheng, J. Li, and J. Bian, “Probts: Benchmarking point and distributional forecasting across diverse prediction horizons,” Advances in Neural Information Processing Systems, vol. 37, pp. 48 045–48 082, 2024. [37] H. Zhou, S. Zhang, J. Peng, S. Zhang, J. Li, H. Xiong, and W. Zhang, “Informer: Beyond efficient transformer for long sequence time-series forecasting,” in Proceedings of the AAAI conference on artificial intelligence, vol. 35, no. 12, 2021, pp. 11 106–11 115. [38] H. Wu, J. Xu, J. Wang, and M. Long, “Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting,” Advances in neural information processing systems, vol. 34, pp. 22 419– 22 430, 2021. [39] M. Zamo and P. Naveau, “Estimation of the continuous ranked probability score with limited information and applications to ensemble weather forecasts,” Mathematical Geosciences, vol. 50, no. 2, pp. 209–234, 2018. [40] T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A nextgeneration hyperparameter optimization framework,” in Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2019, pp. 2623–2631.

A PPENDIX Below, we provide the complete mathematical derivations, lemma proofs, and reductions used to obtain the final closedform solution. A. Optimal mapping from feature to similarity space Proof. By Assumptions A1–A2, zδ := g(rδ ) is Gaussian with class-conditional means µy = g(ρy ) (µ0 ̸= µ1 ) and common variance σ 2 . The log-likelihood ratio is linear: Λ(zδ ) = log

p(zδ | Yδ = 1) = αzδ + β ′ , p(zδ | Yδ = 0)

α :=

µ1 − µ0 . σ2

By Bayes’ rule, the posterior is P (Yδ = 1 | zδ ) = σ(αzδ + β) with β := β ′ + log(π1 /π0 ). For small rδ , a first-order Taylor expansion g(rδ ) = g(0) + g ′ (0)rδ + o(rδ ) yields P (Yδ = 1 | rδ ) = σ(α̃rδ + β̃) + o(rδ ), α̃ := αg ′ (0),

β̃ := β + αg(0),

completing the proof. B. Necessary and sufficient conditions Proof. Since G consists of symmetric, non-negative Toeplitz b ∈ G is parameterized matrices with zero diagonal, any G by gb = (b g1 , . . . , gbN −1 ) with gbk ≥ 0, where gbij = gb|i−j| .

Defining the lag-averaged distance ϕk := N 1−k problem (7) reduces to min J (b g) = 2

g bk >0

N −1 X

P

|i−j|=k dij ,

  (N − k) ϕk gbk + τ gbk (ln gbk − 1) .

(8)

k=1

J is separable and strictly convex on {b gk > 0} because each term has second derivative 2τ (N − k)/b gk > 0. Setting ∂J /∂b gk = 0 yields the unique stationary point gbk⋆ = exp(−ϕk /τ ),

k = 1, . . . , N − 1,

which, by strict convexity, is the unique global minimizer b is therefore the of (8). The corresponding Toeplitz matrix G unique solution to (7). C. Datasets Description We evaluate on nine standard multivariate time-series benchmarks: ETTh1/2 and ETTm1/2 (hourly/15-min transformer measurements, 2016–2018), Electricity (hourly consumption for 321 users, 2012–2014), Weather (10-min meteorological records, 2020), Traffic (hourly road occupancy across 862 sensors, 2015–2016), Solar (hourly energy consumption and weather data with solar irradiance and meteorological variables), and ERCOT (hourly electricity load across 8 U.S. regions, 2004–2021) [5], [33], [36]–[38]. These datasets span diverse temporal resolutions, dimensionalities, and seasonal patterns. To encode temporal context, we append 11 auxiliary features: sinusoidal embeddings (cos(2πt), sin(2πt)) for year, quarter, month, week, and day cycles, plus a normalized intersample interval (ti − ti−1 )/∆t. All series are chronologically partitioned into 70% training, 10% validation, and 20% testing splits.

DLinear: We modify the original architecture [19] to project trend/seasonal components into latent space (dim 64, or 128 for Electricity/Traffic) using a running-mean window of 25. MLP: A two-layer ReLU MLP with hidden dimension tuned over {64, 128, 256} and fixed latent output dimension 64. PatchTST: Separate PatchTST encoders [2] model trend and seasonal components, each followed by linear heads for latent mean/variance estimation. Patch length and stride are fixed to 16 and 8; latent dimension is 64 (128 for Electricity/Traffic). Remaining hyperparameters are optimized via Optuna7 or follow the original implementation8 . F. Mapping functions from correlation to similarity space We evaluate the impact of different mappings from the correlation space to the similarity space. Specifically, we compare the theoretically motivated sigmoid mapping with alternative choices, including ReLU, absolute value, and a parabolic transformation. As shown in the Table IV, the sigmoid mapping consistently yields the best performance across both datasets (Electricity and Traffic) according to both CRPS and NMAE. TABLE IV: Impact of mapping functions from correlation to similarity space on probabilistic forecasting performance for Electricity and Traffic datasets. Lower CRPS and NMAE indicate better performance. Results are reported as mean ± standard error . Dataset

Metric

Sigmoid

ReLU

Absolute

Parabola

Electricity

CRPS NMAE

0.107 ± 0.000 0.132 ± 0.000

0.111 ± 0.000 0.137 ± 0.000

0.111 ± 0.001 0.140 ± 0.001

0.111 ± 0.001 0.139 ± 0.001

Traffic

CRPS NMAE

0.270 ± 0.004 0.323 ± 0.009

0.274 ± 0.001 0.332 ± 0.001

0.278 ± 0.003 0.338 ± 0.004

0.285 ± 0.012 0.345 ± 0.013

D. Evaluation Metrics

G. Sensitivity Analysis

We report forecast performance using two standard metrics: Normalized Mean Absolute Error (NMAE) for point accuracy and Continuous Ranked Probability Score (CRPS) for probabilistic calibration. NMAE is computed as N PF 1 X i=1 |xni − x̂ni | NMAE(x, x̂) = , PF n N n=1 i=1 |xi |

1) Sensitivity to Latent Space Dimensionality: The dimensionality of the latent space is selected based on validation set optimization. However, it is important to assess the model’s sensitivity to this parameter, as a stable model should not exhibit significant performance degradation when using comparable dimensionalities. Our empirical results 9 demonstrate that CLaST achieves consistent forecasting performance across latent dimensions of 32, 64, 128, and 256. 2) Backbone Sensitivity to Context Length: All encoders are tested with different context lengths of 24, 48, 96, 192, and 384 time steps. The impact of the encoder choice and the context length is provided in our supplementary materials 10 . The Transformer-based CLaST model generally performs best, with the MLP backbone typically ranking second. However, for some long-horizon forecasts–particularly on the Traffic and ETTh1 datasets–the MLP model can outperform the

where x̂ denotes the median of 100 sampled trajectories. CRPS measures the discrepancy between the predictive CDF F (z) and the observation x: Z
2 CRPS = F (z) − I{x ≤ z} dz, R

approximated via Monte Carlo sampling [39]. For both metrics, lower values indicate better performance. E. Implementation Details We tune CLaST hyperparameters on the validation set via Optuna [40], searching α ∈ [1, 10], smin ∈ (0, 0.5), and τ ∈ (0.01, 10). We evaluate three encoder backbones:

7 https://anonymous.4open.science/r/CLaST-3407/tables/patchtst backbone.

md 8 https://github.com/yuqinie98/PatchTST 9 https://anonymous.4open.science/r/CLaST-3407/tables/dim abl.png 10 https://anonymous.4open.science/r/CLaST-3407/tables/backbones.png

CLaST 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

1.0

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0

23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0

LaST 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

1.0

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

Fig. 2: Average cosine similarity between embeddings of input data across time steps for LaST and CLaST on the Weather dataset with context length and horizon N = L = 24.

Transformer. On the other hand, for the Weather dataset, the Transformer model achieves better results than the MLP in certain long-horizon settings, while underperforming in others. H. Temporal Structure in Learned Embeddings To better understand the effect of the proposed training objective on the learned representations, we analyze the temporal structure of the resulting embedding space. Figure 2 presents the average pairwise cosine similarity of input embeddings across different temporal positions for LaST and CLaST. Both approaches exhibit a diagonal structure in the resulting similarity matrices, indicating that embeddings corresponding to temporally adjacent timestamps are more similar, while similarity decreases with increasing temporal distance. However, LaST shows uniformly high average cosine similarity across the matrix, suggesting overly smooth representations with limited discriminability between distant timestamps. In contrast, CLaST yields lower and more heterogeneous similarity values, leading to a more pronounced diagonal pattern and improved preservation of temporal structure, due to the contrastive regularization and temporal bias imposed by the training loss.

0.0 0.2 0.4 0.6

ground truth seasonal median seasonal 90% C.I. trend median trend 90% C.I.

0.8 1.0 100

200

300

400

500

600

700

800

600

700

800

(a) ETTm2. 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0

100

200

300

400

500

(b) Electricity.

Fig. 3: CLaST decomposed forecasts (N = 96, L = 720). Each figure shows ground truth, seasonal, and trend with 90% confidence intervals from 100 samples.

Related documents

• CLaST: Context-aware Contrastive VAE for Probabilistic Time Series Forecasting#1077936
• Parameter-Efficient Adaptation of Pretrained Language Models for Time-Series Forecasting#1109368
• In-Context Time Series Classification with Random Convolutional Features#386866
• Self-Gating Attention for Efficient Time Series Forecasting#332528
• TimEE: End-to-end Time Series Classification via In-Context Learning#349635