Trio: Learning Time-Series Forecasting with Temporal-Spatial-Sample Attention and Structural Causal Priors Tao Chen1 , Yexu Zhou1 , Zhi Gong1 , Hengwei He1 , Hongda Li1 , Zhewei Chen1 , Dongjing Wang2 , Xin Zhang2 , Decheng Liu3 , Chunlei Peng3 , Zheng Chen1 , Wenyue Ding1 1
2
SUPCON Technology Co., Ltd. Hangzhou, China, Hangzhou Dianzi University, HangZhou, 310018, China, 3 Xidian University, Xi’an 710071, Shaanxi, China.
arXiv:2606.07291v1 [cs.LG] 5 Jun 2026
Abstract Multivariate time-series forecasting requires models to reason over temporal dynamics, cross-variable dependencies, and historical input-output correspondences. Recent PriorData Fitted Networks (PFNs) suggest that synthetic tasks can be useful for learning transferable inference behavior. However, directly transferring this paradigm to time-series forecasting remains difficult, since temporal order, dynamic lags, and recurring historical patterns are not naturally captured by ordinary tabular priors. Motivated by this observation, we propose Trio, a sample-aware time-series forecasting architecture based on Temporal-Spatial-Sample attention. Temporal attention captures within-window dynamics, spatial attention models inter-variable dependencies, and sample attention retrieves relevant historical lookback-future pairs to guide the current prediction. Rather than claiming a fully general PFN-style forecaster, our goal is to study how historical input-output examples can be explicitly organized and reused within a forecasting model. We further introduce a Time-Series Structural Causal Model (TS-SCM) generator to create structured synthetic forecasting tasks with dynamic lags, cross-variable interactions, noise, feedback, and distributional drift. Experiments on synthetic, industrial, and public benchmarks show that the proposed architecture improves forecasting performance. Exploratory zero-shot experiments further suggest that TS-SCM-generated tasks may provide useful structural priors, while fully general PFN-style timeseries forecasting remains an open problem.
Introduction Multivariate time-series forecasting (Jung and Kim 2026; Zhou et al. 2024; Hu et al. 2025)is a fundamental task in many real-world scenarios, such as energy systems, industrial monitoring, traffic prediction, and environmental analysis. Recent Transformer-based models have achieved strong performance by improving temporal modeling, patchwise representation, and cross-variable dependency learning. However, most existing forecasting models are still trained in a dataset-specific supervised manner, where the model must be optimized separately for each target dataset. This limits their ability to learn a more general forecasting mechanism that can transfer across different time-series tasks. Recently, Prior-Data Fitted Networks (PFNs), represented by TabPFN (Hollmann et al. 2022, 2025; Grinsztajn et al.
(a) Short Input Period Time Series Prediction
(b) Long Input Period Time Series Prediction
(c) Context example Time Series Prediction
Figure 1: Three representative input paradigms: (a) short historical input; (b) long contextual input without distinguishing input–output relationships; and (c) our approach, which leverages long historical context while explicitly modeling input–output correspondences.
2026), have shown a promising alternative paradigm in tabular learning. Instead of training from scratch on each dataset, PFN-style models learn an inference strategy from largescale synthetic tasks generated by a prior distribution, and then transfer this learned strategy to unseen tasks. This idea has also motivated recent attempts (Hoo et al. 2025; Cai, Sun, and Zhong 2025) in time-series forecasting. However, directly extending tabular PFN models to time series is insufficient, because tabular data do not explicitly contain temporal order, dynamic lagged dependencies, or historical input-output correspondences. Time-series forecasting requires models to reason jointly over temporal dynamics, cross-variable structures, and sample-level historical patterns. Motivated by PFN-style learning, we study a more modest question: how can a forecasting model explicitly reuse historical input-output examples within a long time-series context? To this end, we propose a sample-aware forecasting architecture based on three-dimensional attention. The model decomposes multivariate time-series modeling into temporal, spatial, and sample dimensions. Temporal atten-
tion captures within-window dynamic patterns, spatial attention models dependencies among variables, and sample attention retrieves relevant historical lookback-future pairs to guide the current prediction. In this way, the model does not treat long historical context as a flat sequence, but uses historical segments as structured examples that encode how past inputs evolve into future outcomes. In addition to the architecture, we introduce a TimeSeries Structural Causal Model (TS-SCM) generator to explore whether structured synthetic tasks can serve as useful forecasting priors. Unlike simple synthetic generators that mainly produce smooth curves or weakly coupled variables, TS-SCM constructs time series through dynamic lagged dependencies, cross-variable interactions, latent factors, noise processes, error accumulation, and distributional drift. These mechanisms better reflect the structural complexity of realworld time series and provide a richer prior for learning transferable forecasting behavior. Based on the proposed architecture and TS-SCM generator, we conduct experiments on synthetic, industrial, and public benchmark datasets. The results demonstrate that the three-dimensional attention architecture effectively improves multivariate forecasting performance. These mechanisms are intended to expose the model to structural patterns commonly observed in real-world time series, such as delayed effects, cross-variable propagation, and non-stationary transitions. Our main contributions are summarized as follows: • We propose Trio, a sample-aware time-series forecasting architecture that decomposes multivariate forecasting into temporal, spatial, and sample-level attention. The model uses long histories as structured input-output examples rather than undifferentiated context. • We develop TS-SCM, a forecasting-oriented synthetic task generator that simulates multivariate dynamical systems with heterogeneous node mechanisms, edge-level dynamic lags, delayed propagation, noise, feedback, and drift. • We conduct experiments on synthetic delayeddependency tasks, industrial datasets, and public benchmarks to validate the proposed architecture. We also include exploratory zero-shot experiments to assess whether TS-SCM-generated tasks provide useful transfer cues.
Model Architecture Given a multivariate time series X ∈ RB×L×V , where B, L, and V denote the batch size, input length, and number of variables, respectively, the goal is to predict a future sequence Ŷ ∈ RB×T ×V . Instead of treating the entire historical context as a single continuous sequence, we reorganize it into a set of historical input–output pairs and one current query window. Let Lx denote the length of each lookback segment and T denote the prediction length. The last Lx time steps are used as the current input: Curx = XL−Lx +1:L ∈ RB×Lx ×V .
(1)
The preceding context is divided into S non-overlapping historical blocks, each containing a lookback segment and its following future segment: (s) (His(s) x , Hisy ),
s = 1, . . . , S,
(2)
where B×Lx ×V His(s) , x ∈R
B×T ×V His(s) . y ∈R (s)
(3)
(s)
Here, each historical pair (Hisx , Hisy ) provides an explicit input–output example that describes how a previous lookback pattern evolves into its future outcome. The model uses these historical examples as conditional evidence for predicting the future of Curx . Strict temporal ordering. To avoid information leakage, all historical lookback-future pairs are strictly placed before the current query window. Let Lx be the lookback length, T be the prediction length, and S be the number of historical pairs. Given an input context of length L = S(Lx + T ) + Lx , we define the s-th historical pair as His(s) x = Xas :as +Lx −1 ,
His(s) y = Xas +Lx :as +Lx +T −1 , (4)
where as = (s − 1)(Lx + T ) + 1,
s = 1, . . . , S.
(5)
The current input window is defined as Curx = XS(Lx +T )+1:S(Lx +T )+Lx .
(6)
Therefore, for every historical pair, max index(His(s) y ) < min index(Curx ), which ensures that historical future prompts are past observations relative to the current window and never contain the target future of Curx .
Patch Tokenization and Historical Prompt Construction We first convert both the current and historical sequences into patch-level representations. For each variable, a patch embedding layer maps the lookback segment into Px input tokens: Ex = PatchEmbedx (·) ∈ RPx ×D ,
(7)
where D is the hidden dimension. The current input is embedded as Cur = PatchEmbedx (Curx ) ∈ RB×V ×Px ×D .
(8)
Similarly, the historical lookback segments are embedded as S B×V ×S×Px ×D Hisx = PatchEmbedx ({His(s) . x }s=1 ) ∈ R (9) Different from simply compressing the historical future into a single vector, we preserve its patch-wise structure.
Projection
Context Example Sample Self Attention
Self Attnetion
TSS Alternating Encoder
Pre-feature Encoding Context Example Target
...
���1�
����1
���2�
...
����2
Spatial Self Attention
���� ...
Time Self Attention
����
Context Example Target
����
Token Construction Prompt Injection
���
Unified Token Sequence
����
Patch-Embedding
Historical Prediction Prompt
Multi-Variate Time-Series Data
Multivariate Time Series
���1�
Sample #1
����
����1
���1�
���
zero grad (c) Spatial Self Attention
����2
���2�
Sample #2
...
...
Sample #N
����
����
Context Example
����
����
����
Linear
����1
...
����
����
zero grad (b) Time Self Attention
(a) overall
TSS Alternating Encoder
Target
����2
���2�
...
���
Target ���� ���
(d) Sample Self Attention
+
Future outcomes Historical look-back segment
Figure 2: Overview of the Sample-Aware Transformer architecture. Our model is composed of three complementary attention mechanisms—Temporal, Spatial, and Sample-Level attention—which are applied in an alternating and hierarchical manner. Temporal attention captures dependencies across time steps, spatial attention models correlations among variables, and samplelevel attention learns conditional relationships between historical inputs and future outputs. (s)
Specifically, the historical future segment Hisy is encoded by a separate future patch embedding layer: S B×V ×S×Py ×D Hisy = PatchEmbedy ({His(s) , y }s=1 ) ∈ R (10) where Py denotes the number of future tokens. These future tokens act as historical prediction prompts, providing patchwise supervision signals that indicate how each historical lookback window evolves into its future. Before constructing the final token sequence, we apply lightweight self-attention-based pre-encoders to improve intra-window representations:
g = PreEncx (Cur), Cur
(11)
g x = PreEncx (Hisx ), His
g y = PreEncy (Hisy ). His (12) The two pre-encoders operate along the patch dimension and are used to enhance local lookback and future-prompt representations before cross-window interaction. For each historical window, we concatenate the encoded lookback tokens and future-prompt tokens: (s)
(s)
(s) g x , His g y ] ∈ RB×V ×(Px +Py )×D . Histok = [His
(13)
For the current window, since the future is unknown, we append Py learnable query tokens: 1×V ×Py ×D
Qy ∈ R
.
The current tokens are then constructed as g Qy ] ∈ RB×V ×(Px +Py )×D . Curtok = [Cur,
(14) (15)
The learnable query tokens play the role of future placeholders. During sample-level retrieval, they query historical
future-prompt tokens and absorb useful input–output mappings from previous windows. Finally, all historical tokens and the current tokens are stacked along the sample/window dimension: (1)
(S)
Z (0) = [Histok , . . . , Histok , Curtok ] ∈ RB×V ×(S+1)×(Px +Py )×D . (16) For notation simplicity, we denote the total number of windows as S̄ = S +1 and the token length within each window as P = Px + Py .
Temporal-Spatial-Sample Alternating Encoder The constructed tensor Z (0) ∈ RB×V ×S̄×P ×D contains three types of dependencies: temporal dependencies along the patch dimension, spatial dependencies among variables, and sample-level dependencies among historical windows and the current query window. Directly applying full attention over all axes is computationally expensive and introduces uncontrolled interactions. Therefore, we design a Temporal-Spatial-Sample alternating encoder, which decomposes the modeling process into axis-wise attention operations. Each encoder block contains three modules: spatial attention, temporal attention, and sample attention. The overall update can be written as (k)
(k)
(k)
Z (k) = Fsample ◦ Ftemp ◦ Fspat (Z (k−1) ),
k = 1, . . . , K. (17)
Spatial Attention. The spatial module models dependencies among variables at each window and patch position. Given Z ∈ RB×V ×S̄×P ×D , we reshape it as Zspat = reshape(Z) ∈ R(B S̄P )×V ×D .
(18)
Non-causal self-attention is then applied along the variable dimension: bspat = AttnV (Zspat ). Z (19) The output is reshaped back to RB×V ×S̄×P ×D . This operation allows each variable to dynamically aggregate information from other variables at the same sample and patch position. For dataset-specific supervised training with long contexts, we optionally apply a zero-gradient strategy to reduce memory cost. Specifically, spatial attention for historical windows is first computed under no grad, while the current window is recomputed with gradient tracking and written back to the output. In this setting, historical windows are treated as contextual evidence, and parameter updates are mainly driven by the current prediction loss. This option is not required in all training settings and is disabled or adjusted when full-gradient optimization is needed. Temporal Attention. The temporal module captures within-window patch dependencies for each variable and sample. The tensor is reshaped as Ztemp = reshape(Z) ∈ R(BV S̄)×P ×D .
(20)
We apply causal self-attention along the patch dimension: btemp = Attncausal Z (Ztemp ). P
(21)
The causal mask prevents later tokens from being accessed by earlier tokens. When the optional zero-gradient strategy is enabled for long-context supervised training, historical windows are processed under no grad, whereas the current window is recomputed with gradients and replaces its corresponding position in the output tensor. Otherwise, the temporal module can be optimized in a standard full-gradient Sample Attention. The sample module is designed to retrieve useful historical input–output mappings for the current prediction. Unlike conventional sample attention that only uses a single CLS token, our implementation performs patch-wise sample retrieval over the future-query tokens. Let the last Py tokens of each window be denoted as R = Z[:, :, :, −Py :, :] ∈ RB×V ×S̄×Py ×D .
(22)
For historical windows, we take Rhist = R[:, :, 1 : S, :, :] ∈ R
After reshaping it into R(BV Py )×1×D , we use it as the query and use the aggregated historical tokens as keys and values: bcur = CrossAttn Rcur , R bhist , R bhist . R (27) Only the current future-query tokens are updated and written back to Z: bcur . Z[:, :, −1, −Py :, :] ← R (28) This realizes a history-first aggregation and current-only retrieval mechanism. It avoids bidirectional mixing between historical and current windows, while allowing the current prediction tokens to selectively retrieve relevant historical input–output patterns.
Prediction Head After the TSS alternating encoder, we extract all tokens from the current window: H = Z (K) [:, :, −1, :, :] ∈ RB×V ×(Px +Py )×D . (29) Instead of using only a single CLS representation, we flatten all current-window tokens for each variable: h = Flatten(H) ∈ RB×V ×((Px +Py )D) . (30) A shared linear head maps the representation of each variable to the prediction horizon: Ŷ = Head(h) ∈ RB×T ×V . (31) When probabilistic forecasting is used, the head outputs Q quantiles: Ŷ = Head(h) ∈ RB×T ×V ×Q . (32) For point forecasting, Q = 1 or the median quantile is used as the final prediction. Following common practice in non-stationary time-series forecasting, we normalize each input sequence before tokenization and apply inverse normalization to the prediction: X −µ X̄ = , Ŷ = σ · Ȳˆ + µ, (33) σ where µ and σ are computed from the input lookback sequence. For deterministic forecasting, the model is optimized by minimizing the mean squared error: B T V 2 1 XXX Ŷb,t,v − Yb,t,v . (34) L= BT V t=1 v=1 b=1
B×V ×S×Py ×D
.
(23)
For quantile forecasting, this objective can be replaced by the standard pinball loss over all quantile levels.
The historical future-prompt tokens are then rearranged as Rhist → R(BV Py )×S×D ,
(24)
so that sample self-attention is independently performed for each variable and each future-patch position: bhist = AttnS (Rhist ). R
(25)
This step aggregates historical windows and produces a memory bank of historical future-prompt representations. Next, the current future-query tokens are extracted: Rcur = R[:, :, −1, :, :] ∈ RB×V ×Py ×D .
(26)
Time-Series Structural Causal Model Generator To provide structured synthetic tasks for PFN-style timeseries forecasting, we introduce a Time-Series Structural Causal Model (TS-SCM) generator. Different from GPbased synthetic priors that mainly define a distribution over temporal functions, TS-SCM defines a distribution over multivariate causal dynamical systems. Each generated task is specified by a directed graph, heterogeneous node mechanisms, edge-level lag functions, noise processes, and optional mechanism drift.
Graph and node sampling. TS-SCM first samples a set of root variables and affected variables. Root variables act as exogenous driving processes and can be generated from periodic functions, Fourier components, trends, Gaussianprocess-like priors, square-wave patterns, or chaotic systems. Affected variables are generated by node-wise mechanisms that receive delayed inputs from their parent variables. The supported node mechanisms include linear functions, bounded nonlinear unary functions, kernel RBF functions, and other nonlinear transformations. For an affected node xi , the basic update can be written as xi (t) = αi xi (t−1)+βi fi m1i (t), . . . , m|P a(i)|i (t) +ϵi (t), (35) where P a(i) denotes the parent set of node i, fi (·) is the node-specific mechanism, mji (t) is the message delivered from parent xj to child xi , αi controls state persistence, βi controls input strength, and ϵi (t) denotes noise. Edge-level causal messages. Each directed edge eji : xj → xi specifies how information from the parent variable is transformed and delivered to the child variable. The message on an edge is defined by an edge weight, an optional nonlinear transform, a lag function, a reduce policy, and a fill policy. Abstractly, the delivered message can be written as mji (t) = ρji ({wji (t) · ψji (xj (t′ )) | t′ + τji (t′ ) = t}) , (36) where ψji (·) is the edge transform, wji (t) is the possibly drifted edge weight, τji (·) is the lag function, and ρji (·) aggregates multiple events arriving at the same time. The reduce policy can be sum, mean, max, first, or last. When no event arrives at a time step, TS-SCM supports different fill policies, including zero filling, forward filling, and linear interpolation. Dynamic lag scheduling. A central design of TS-SCM is that lags are defined at the edge level rather than by a global delay matrix. For each edge, the lag can be fixed, state-dependent, history-dependent, sampled from a valuedependent interval, or defined by a user-specified expression: τji (t) = gji (xj (t), t, Hj (t)) , (37) where Hj (t) denotes the history of the parent variable. This formulation allows different causal effects to propagate with different and time-varying delays. For example, an edge can have a larger delay when the parent value is large, or switch between different delay ranges according to the current state. Push-based event calendar. Instead of letting the child node pull a parent value from a fixed past index, TS-SCM uses a push-based event calendar. When a parent value xj (t) is generated, each outgoing edge computes a future arrival trajectory sji (t) = t + τji (t). (38) Whenever the continuous trajectory sji (t) crosses an integer time bucket k, an event is scheduled to arrive at k. The parent value attached to that event is linearly interpolated between consecutive parent states: x̃j (k) = (1 − λ)xj (t − 1) + λxj (t),
(39)
where λ is determined by the crossing location. The event is then stored in the calendar of edge eji and consumed by the child node at its arrival time. This design avoids the crude rounding of continuous lags and enables smooth delayed propagation under time-varying lag functions. Zero-lag and delayed recurrent dependencies. TS-SCM distinguishes synchronous dependencies from delayed dependencies. Zero-lag edges are allowed only when explicitly specified, and the zero-lag subgraph is constrained to be acyclic so that nodes at the same time step have a valid computation order. In contrast, positive-lag edges can form recurrent feedback loops, including self-loops and reversedirection effects, because their messages are delivered to future time steps. This separation allows TS-SCM to model both instantaneous interactions and delayed feedback while avoiding ill-defined simultaneous cycles. Drift, noise, and stability control. To simulate nonstationary real-world systems, TS-SCM supports piecewise mechanism drift. The drift engine can modify node parameters, global linear scales, or edge weights over time. Noise can be Gaussian, Student-t, autoregressive, or mixture noise, allowing both regular disturbance and heavy-tailed perturbation. To prevent randomly sampled recurrent systems from becoming unstable, TS-SCM applies stability-aware graph sampling. In particular, incoming edge gains are controlled by a child-wise budget, and self-feedback edges use a stricter gain budget than cross-variable delayed edges. This produces diverse yet learnable synthetic systems rather than arbitrary unstable trajectories. Forecasting episode construction. Finally, each generated trajectory is converted into forecasting episodes. Given a long trajectory, TS-SCM constructs multiple historical lookback–future pairs and a current query window: n o (1) (S) (S) E = (His(1) x , Hisy ), . . . , (Hisx , Hisy ), Curx . (40) This episode format is directly aligned with our threedimensional attention architecture: historical lookback– future pairs provide sample-level evidence, while the current query window retrieves relevant input–output mappings for prediction. Therefore, TS-SCM is not only a trajectory generator, but also a forecasting-oriented task generator.
Experiments Experimental Settings We evaluate Trio on three groups of datasets: synthetic delayed-dependency datasets, and public forecasting benchmarks. The synthetic datasets are used to isolate delayed temporal dependencies and variable-selection uncertainty. The public benchmarks include ETTm1, ETTm2, Weather, and Electricity. For public benchmarks, we follow the standard (Wang et al. 2024b) train/validation/test splits used in prior longterm forecasting studies. All methods are evaluated under the same prediction horizons. For our model, the input context is organized into historical lookback-future pairs and
Table 1: Comparison and ablation results on two synthetic datasets. Lower values indicate better performance. (a) Synthetic Dataset #1, where Y is an M -step delayed version of X, with M ∼ U (20, 60).
(b) Synthetic Dataset #2, where Y is an M -step delayed version of one randomly selected variable from {X1 , . . . , X5 }, with M ∼ U(20, 30).
Method
MSE ↓
MAE ↓
Method
MSE ↓
MAE ↓
Standard baselines TimesNet Crossformer TimeXer
0.826 0.980 0.736
0.725 0.503 0.684
Standard baselines TimesNet Crossformer TimeXer
0.981 0.795 0.956
0.789 0.710 0.779
Ablation on sample attention Base Model (w/o Sample Attn.) Base Model + Sample Attn.
0.706 0.298
0.670 0.431
Ablation on sample attention Base Model (w/o Sample Attn.) Base Model + Sample Attn.
1.011 0.644
0.802 0.638
Table 2: Comparison results on benchmark datasets under different forecasting horizons. Metric is Mean Squared Error (MSE); lower is better. Dataset
Horizon Trio TimeMixer++ Patch-wise TIMEKAN MOIRAI TimesFM TimeMixer iTransformer PatchTST DLinear
ETTm1
96 192 336 720 Avg
0.289 0.339 0.377 0.428 0.358
0.310 0.348 0.376 0.440 0.369
0.326 0.374 0.410 0.472 0.396
0.322 0.357 0.382 0.445 0.377
0.353 0.376 0.399 0.432 0.390
0.345 0.374 0.397 0.436 0.388
0.320 0.361 0.390 0.454 0.381
0.334 0.390 0.426 0.491 0.410
0.352 0.374 0.421 0.462 0.402
0.346 0.382 0.415 0.473 0.404
ETTm2
96 192 336 720 Avg
0.156 0.219 0.285 0.360 0.255
0.170 0.229 0.303 0.373 0.269
0.175 0.242 0.304 0.401 0.281
0.174 0.239 0.301 0.395 0.277
0.189 0.247 0.295 0.372 0.276
0.263 0.309 0.349 0.415 0.334
0.175 0.237 0.298 0.391 0.275
0.180 0.250 0.311 0.412 0.288
0.183 0.255 0.309 0.412 0.290
0.193 0.284 0.382 0.558 0.354
96 192 Electricity 336 720 Avg
0.133 0.156 0.172 0.195 0.164
0.135 0.147 0.164 0.212 0.165
0.146 0.161 0.174 0.208 0.172
0.174 0.182 0.197 0.236 0.197
0.152 0.171 0.192 0.236 0.188
– – – – –
0.153 0.166 0.185 0.225 0.182
0.148 0.162 0.178 0.225 0.178
0.190 0.199 0.217 0.258 0.216
0.210 0.210 0.223 0.258 0.225
96 192 336 720 Avg
0.149 0.212 0.270 0.364 0.249
0.155 0.201 0.237 0.312 0.226
0.167 0.219 0.274 0.353 0.253
0.162 0.207 0.263 0.338 0.243
0.177 0.219 0.277 0.365 0.260
– – – – –
0.163 0.208 0.251 0.339 0.240
0.174 0.221 0.278 0.358 0.258
0.186 0.234 0.284 0.356 0.265
0.195 0.237 0.282 0.345 0.265
Weather
one current query window according to the indexing rule described in Sec. . For baselines, we use the context lengths and hyperparameters reported in their original papers whenever available, and otherwise tune them on the validation set.
All experiments are implemented in PyTorch and trained with the Adam optimizer. The initial learning rate is set to 10−4 . We use the mean squared error loss for deterministic forecasting and apply early stopping based on the validation loss. Unless otherwise specified, results are reported using MSE and MAE, where lower values indicate better performance. For experiments involving randomness, we use fixed random seeds and report the best validation-selected checkpoint on the test set.
Comparison Methods We compare Trio with representative forecasting baselines, including TimeMixer++ (Wang et al. 2025), Patch-wise Structural (Kudrat et al. 2025), TIMEKAN (Huang et al. 2025), MOIRAI (Woo et al. 2024), TimesFM (Das et al. 2024a), TimeMixer (Wang et al. 2024a), iTransformer (Liu et al. 2024), PatchTST (Nie et al. 2022), DLinear (Zeng et al. 2023). These baselines cover Transformer-based, patchbased, decomposition-based, MLP-based, and foundationstyle forecasting methods. All results are reported under multiple prediction horizons for a comprehensive comparison.
Results on Synthetic Scenarios We first validate the proposed sample-level modeling strategy on two synthetic datasets. In Synthetic Dataset #1, the
target series Y is generated as an M -step delayed version of the input series X, where the delay is sampled from M ∼ U (20, 60). This setting is used to test whether a model can capture variable temporal delays. In Synthetic Dataset #2, Y is generated as an M -step delayed version of one randomly selected variable from the multivariate input {X1 , . . . , X5 }, where M ∼ U(20, 30). This setting is more challenging because the model must identify not only the delay length but also the relevant source variable. As shown in Table 1, standard forecasting baselines show limited performance under these delayed-dependency settings. In contrast, adding Sample Attention consistently improves the base model. On Synthetic Dataset #1, the MSE decreases from 0.706 to 0.298. On Synthetic Dataset #2, the MSE decreases from 1.011 to 0.644. The improvement is more pronounced in the second setting, suggesting that sample-level retrieval is particularly useful when the model needs to infer both temporal delay and variable relevance. These results support our motivation that historical lookback–future pairs provide useful evidence for current prediction.
Table 3: Comparison of ICTSP and Trio on ETTm1 and ETTm2 datasets with different context lengths.
2000 3000 4000
ETTm1 ICTSP Trio Mean Best Mean Best 0.381 0.358 0.371 0.370 0.359 0.358 0.370 0.365
Context / Horizon
MSE ↓
MAE ↓
MSE ↓
MAE ↓
ETTm1
1056 / 96 1536 / 192 2256 / 336 4176 / 720
0.323 0.374 0.444 0.512
0.379 0.414 0.450 0.521
0.306 0.347 0.369 0.424
0.361 0.394 0.409 0.446
ETTm2
1056 / 96 1536 / 192 2256 / 336 4176 / 720
0.191 0.294 0.345 0.394
0.284 0.352 0.392 0.434
0.175 0.230 0.291 0.383
0.270 0.310 0.349 0.434
Table 5: Average performance across all prediction horizons under different context lengths with the default input length of 96. Lower values indicate better performance. ETTm1
ETTm2
Weather
ECL
ETTh1
ETTh2
MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE
As shown in Table 2, Trio achieves the best average MSE on three out of four datasets, including ETTm1, ETTm2, and Electricity. Specifically, Trio obtains average MSE scores of 0.358, 0.255, and 0.164 on ETTm1, ETTm2, and Electricity, respectively. On Weather, Trio achieves an average MSE of 0.249, which is not the best overall result; TimeMixer++ obtains the lowest average MSE of 0.226 on this dataset. Nevertheless, Trio remains competitive on Weather and achieves the best result at the short prediction horizon H = 96. These results suggest that Trio can effectively exploit historical input–output correspondences on most benchmarks, while its advantage may vary across datasets with different temporal characteristics.
Dataset
Base + Sample Attn
Base Model
Dataset
Context Length (L)
Results on Public Benchmarks
Context Length
Table 4: Ablation results on ETTm1 and ETTm2 datasets. We compare the base model performance with and without Sample Attention across various context and prediction lengths. Metric is Mean Squared Error (MSE) and Mean Absolute Error (MAE); lower is better.
ETTm2 ICTSP Trio Mean Best Mean Best 0.265 0.255 0.262 0.262 0.256 0.255 0.266 0.266
We further compare our model with ICTSP (Lu, Sun, and Yang 2024) under different context lengths in Table 3. ICTSP also attempts to use long historical context for forecasting, but it does not explicitly organize historical segments as input–output examples. In contrast, our model uses historical lookback–future pairs and sample-level retrieval to model the relationship between past observations and future outcomes. As a result, our method achieves better best performance on both ETTm1 and ETTm2. This comparison supports the advantage of explicitly modeling historical input–output mappings rather than simply increasing the context length.
96 1000 2000 3000 4000
0.387 0.356 0.358 0.359 0.365
0.391 0.378 0.385 0.385 0.393
0.281 0.263 0.255 0.256 0.266
0.320 0.318 0.318 0.315 0.320
0.252 0.249 0.248 0.247 0.242
0.274 0.273 0.275 0.276 0.278
0.167 0.164 0.167 0.168 0.172
0.259 0.450 0.433 0.376 0.397 0.257 0.440 0.434 0.353 0.389 0.264 0.477 0.452 0.366 0.404 0.264 – – – – 0.268 – – – –
Ablation Study Definition of the Base Model. In all ablation studies, the Base Model refers to the same backbone as Trio except that the Sample Attention module is removed. Specifically, it keeps the same patch embedding layers, pre-encoders, temporal attention, spatial attention, prediction head, normalization scheme, model depth, hidden dimension, and training protocol. The only difference is that the current future-query tokens do not retrieve historical future prompts through the sample dimension. Therefore, the comparison between the Base Model and Base Model + Sample Attention isolates the contribution of sample-level historical input–output retrieval. Effectiveness of Sample Attention. Table 4 evaluates the contribution of Sample Attention on ETTm1 and ETTm2. Across all tested horizons, adding Sample Attention consistently improves the base model. For example, on ETTm1 with horizon 720, the MSE decreases from 0.512 to 0.424. On ETTm2 with horizon 192, the MSE decreases from 0.294 to 0.230. These improvements demonstrate that retrieving relevant historical lookback–future pairs helps the model capture long-range and delayed temporal dependencies. Scalability to Long Contexts. Tables 5 and 6 report the average performance across all prediction horizons under different context lengths, with default input lengths of 96 and 336, respectively. Increasing the context length generally improves forecasting performance in both settings, showing the benefit of using longer historical contexts. However, the best context length varies across datasets and input
Table 6: Average performance across all prediction horizons under different context lengths with the default input length of 336. Lower values indicate better performance. Context Length (L) 336 2000 3000 4000 5000
ETTm1
ETTm2
ETTh1
ETTh2
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.371 0.357 0.347 0.362 0.380
0.388 0.384 0.379 0.386 0.395
0.281 0.264 0.257 0.263 0.266
0.324 0.324 0.320 0.322 0.321
0.439 0.447 0.515 – –
0.437 0.445 0.481 – –
0.387 0.353 0.360 – –
0.404 0.391 0.401 – –
Table 7: Zero-shot forecasting performance of models trained on different synthetic data generators. All methods are evaluated on real-world ETT benchmarks without targetdataset fine-tuning. Lower MSE and MAE indicate better performance. Synthetic Generator CausalDynamics LMC Synth Ours
ETTh1
ETTh2
ETTm1
ETTm2
MSE ↓
MAE ↓
MSE ↓
MAE ↓
MSE ↓
MAE ↓
MSE ↓
MAE ↓
0.569 0.789 0.425
0.525 0.571 0.427
0.156 0.179 0.162
0.261 0.279 0.251
0.548 0.690 0.512
0.503 0.506 0.474
0.106 0.093 0.073
0.212 0.204 0.180
settings. For the default input length of 96, moderate context lengths such as L = 1000 or L = 2000 already provide strong performance, while further increasing L does not consistently improve the results. For the default input length of 336, ETTm1 and ETTm2 obtain the best average performance around L = 3000, but performance drops when L is further increased. This may be because longer input windows reduce the number of available training samples and may also introduce redundant or noisy historical information. Therefore, moderate long-context modeling is useful, but blindly increasing the context length is not always beneficial. Full horizon-wise results are provided in Appendix.
Exploratory Zero-Shot Transfer with TS-SCM We further conduct an exploratory zero-shot transfer evaluation to examine whether TS-SCM-generated tasks can provide transferable forecasting priors. In this experiment, the forecasting model is trained only on synthetic tasks and directly evaluated on real-world ETT benchmarks without target-dataset fine-tuning. We emphasize that this setting is intended to assess the transfer potential of different synthetic data generation strategies, rather than to claim a complete solution to zero-shot time-series forecasting. For evaluation, we construct each test instance using seven context samples, where each context sample contains a lookback window of length 96 and a future window of length 48. Given the target input sequence of length 96, the model is required to predict the next 48 time steps. We report the average MSE as the main evaluation metric. All compared models use the same evaluation protocol. We compare TS-SCM with two representative synthetic data generation strategies, including Causal Dynamics (Herdeanu et al. 2025) and LMC Synth (Taga, Ildiz, and Oymak 2025). Table 7 reports the zero-shot forecasting results on the ETT benchmarks. The proposed TS-SCM generator achieves the best MAE on all four datasets and the best MSE
on three out of four datasets, including ETTh1, ETTm1, and ETTm2. On ETTh2, CausalDynamics obtains a slightly lower MSE, while TS-SCM still achieves the best MAE. These results suggest that the proposed forecasting-oriented synthetic task construction provides more transferable temporal priors than the compared synthetic generators in most evaluated settings. Nevertheless, the advantage is not uniform across all metrics. In particular, the ETTh2 MSE result indicates that different synthetic priors may capture different aspects of real-world temporal dynamics. Therefore, we interpret these zero-shot results as encouraging evidence for the usefulness of TS-SCM-generated tasks, rather than as conclusive proof of a fully general zero-shot time-series forecasting model.
Conclusion We presented Trio, a sample-aware time-series forecasting architecture that decomposes multivariate reasoning into temporal, spatial, and sample-level attention. By representing long histories as explicit lookback-future pairs, the model can retrieve historical input-output mappings for current prediction instead of treating history as a flat context. We further introduced TS-SCM, a forecasting-oriented synthetic task generator with dynamic lags, edge-level event scheduling, noise, feedback, and drift. Experiments on synthetic, industrial, and public benchmarks validate the effectiveness of the proposed architecture. Exploratory zero-shot results suggest that TS-SCM-generated tasks may provide useful transfer cues, but they do not yet establish a generalpurpose PFN-style forecasting model. Future work will focus on stronger zero-shot evaluation, synthetic-data realism diagnostics, efficiency improvements, and broader comparisons with temporal causal priors.
References Auer, A.; Podest, P.; Klotz, D.; Böck, S.; Klambauer, G.; and Hochreiter, S. 2025. TiRex: Zero-Shot Forecasting Across Long and Short Horizons with Enhanced In-Context Learning. In The Thirty-Ninth Annual Conference on Neural Information Processing Systems. Cai, S.; Sun, X.; and Zhong, H. 2025. Explore the Time Series Forecasting Potential of TabPFN Leveraging the Intrinsic Periodicity of Data. In 1st ICML Workshop on Foundation Models for Structured Data. Cao, F.; Dai, L.; Han, J.; and Xiong, H. 2026. Enhancing multivariate time series forecasting with global temporal retrieval. arXiv preprint arXiv:2602.10847. Das, A.; Kong, W.; Sen, R.; and Zhou, Y. 2024a. A decoderonly foundation model for time-series forecasting. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org. Das, A.; Kong, W.; Sen, R.; and Zhou, Y. 2024b. A decoderonly foundation model for time-series forecasting. In Fortyfirst International Conference on Machine Learning. Grinsztajn, L.; Flöge, K.; Key, O.; Birkel, F.; Jund, P.; Roof, B.; Manium, M.; Bin, S.; Bühler, M.; Garg, A.; et al. 2026. TabPFN-3: Technical Report. arXiv preprint arXiv:2605.13986.
Herdeanu, B.; Nathaniel, J.; Roesch, C.; Buch, J.; Ramien, G.; Haux, J.; and Gentine, P. 2025. CausalDynamics: A large-scale benchmark for structural discovery of dynamical causal models. arXiv preprint arXiv:2505.16620. Hollmann, N.; Müller, S.; Eggensperger, K.; and Hutter, F. 2022. Tabpfn: A transformer that solves small tabular classification problems in a second. arXiv preprint arXiv:2207.01848. Hollmann, N.; Müller, S.; Purucker, L.; Krishnakumar, A.; Körfer, M.; Hoo, S. B.; Schirrmeister, R. T.; and Hutter, F. 2025. Accurate predictions on small data with a tabular foundation model. Nature, 637(8045): 319–326. Hoo, S. B.; Müller, S.; Salinas, D.; and Hutter, F. 2025. From Tables to Time: Extending TabPFN-v2 to Time Series Forecasting. arXiv preprint arXiv:2501.02945. Hu, Y.; Zhang, G.; Liu, P.; Lan, D.; Li, N.; Cheng, D.; Dai, T.; Xia, S.-T.; and Pan, S. 2025. TimeFilter: Patch-specific spatial-temporal graph filtration for time series forecasting. arXiv preprint arXiv:2501.13041. Huang, S.; Zhao, Z.; Li, C.; and BAI, L. 2025. TimeKAN: KAN-based Frequency Decomposition Learning Architecture for Long-term Time Series Forecasting. In The Thirteenth International Conference on Learning Representations. Jung, Y.-M.; and Kim, L. 2026. MambaSL: Exploring Single-Layer Mamba for Time Series Classification. In The Fourteenth International Conference on Learning Representations. Kudrat, D.; Xie, Z.; Sun, Y.; Jia, T.; and Hu, Q. 2025. Patchwise Structural Loss for Time Series Forecasting. In Fortysecond International Conference on Machine Learning. Liu, Y.; Hu, T.; Zhang, H.; Wu, H.; Wang, S.; Ma, L.; and Long, M. 2024. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting. In The Twelfth International Conference on Learning Representations. Lu, J.; Sun, Y.; and Yang, S. 2024. In-context time series predictor. arXiv preprint arXiv:2405.14982. Lu, J.; Sun, Y.; and Yang, S. 2025. In-context Time Series Predictor. In The Thirteenth International Conference on Learning Representations. Nie, Y.; H. Nguyen, N.; Sinthong, P.; and Kalagnanam, J. 2023. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. In International Conference on Learning Representations. Nie, Y.; Nguyen, N. H.; Sinthong, P.; and Kalagnanam, J. 2022. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730. Taga, E. O.; Ildiz, M. E.; and Oymak, S. 2025. TimePFN: Effective multivariate time series forecasting with synthetic data. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 20761–20769. Tire, K.; Taga, E. O.; Ildiz, M. E.; and Oymak, S. 2024. Retrieval augmented time series forecasting. arXiv preprint arXiv:2411.08249. Wang, S.; Li, J.; Shi, X.; Ye, Z.; Mo, B.; Lin, W.; Ju, S.; Chu, Z.; and Jin, M. 2025. Timemixer++: A general time series
pattern machine for universal predictive analysis. In International Conference on Learning Representations, volume 2025, 16980–17016. Wang, S.; Wu, H.; Shi, X.; Hu, T.; Luo, H.; Ma, L.; Zhang, J.; and Zhou, J. 2024a. Timemixer: Decomposable multiscale mixing for time series forecasting. In International conference on learning representations, volume 2024, 38626– 38652. Wang, Y.; Wu, H.; Dong, J.; Liu, Y.; Long, M.; and Wang, J. 2024b. Deep Time Series Models: A Comprehensive Survey and Benchmark. Wang, Y.; Wu, H.; Dong, J.; Liu, Y.; Qiu, Y.; Zhang, H.; Wang, J.; and Long, M. 2024c. Timexer: Empowering transformers for time series forecasting with exogenous variables. Advances in Neural Information Processing Systems. Woo, G.; Liu, C.; Kumar, A.; Xiong, C.; Savarese, S.; and Sahoo, D. 2024. Unified Training of Universal Time Series Forecasting Transformers. In Forty-first International Conference on Machine Learning. Wu, H.; Xu, J.; Wang, J.; and Long, M. 2021. Autoformer: Decomposition transformers with auto-correlation for longterm series forecasting. Advances in neural information processing systems, 34: 22419–22430. Zeng, A.; Chen, M.; Zhang, L.; and Xu, Q. 2023. Are transformers effective for time series forecasting? In Proceedings of the AAAI conference on artificial intelligence, volume 37, 11121–11128. Zhou, Q.; Pei, C.; Sun, F.; Han, J.; Gao, Z.; Pei, D.; Zhang, H.; Xie, G.; and Li, J. 2024. KAN-AD: Time series anomaly detection with Kolmogorov-Arnold networks. arXiv preprint arXiv:2411.00278.
Appendix Overview This appendix provides additional materials that support the main paper. Appendix presents extended discussions of related work, including deep learning methods for time-series forecasting, in-context learning, retrieval-augmented forecasting, and temporal causal priors. Appendix provides the complexity analysis of the proposed TSS encoder and compares the factorized attention design with flattened full attention. We then provide more details about the proposed TSSCM generator. Appendix explains the fractional interpolation strategy used for continuous time-shifted causal events, and Appendix illustrates several hand-crafted SCM examples, including fixed-lag responses, variable-delay propagation, data drift, switching feedback, and multi-delay effects. Finally, we report additional experimental results. Appendix gives the full horizon-wise results of the longcontext ablation study. We also include results on industrial datasets, an ablation study of the zero-gradient strategy, an ablation study of quantile-based context selection, and qualitative zero-shot visualizations comparing models trained with different synthetic data generation strategies. The appendix concludes with a discussion of current limitations and future directions.
Related Works Deep Learning in Time Series Forecasting In recent years, advances in machine learning have significantly accelerated progress in time series forecasting. Wu et al. (Wu et al. 2021) first proposed decomposing time series into trend and seasonal components and modeling periodic patterns efficiently via autocorrelation, enabling more stable long-horizon forecasting. Subsequently, inspired by Transformer architectures, Wu et al. (Nie et al. 2023) explored partitioning time series into temporal blocks and applying self-attention to capture dependencies across blocks, achieving improved predictive performance. More recently, Wang et al. and Liu et al. (Liu et al. 2024) have incorporated exogenous variables to further enhance forecasting accuracy, while Wang et al. (Wang et al. 2024c) additionally investigated multi-scale representations to better capture temporal patterns at different resolutions. Despite these advances in modeling temporal and spatial characteristics, most existing methods struggle to effectively exploit very long input sequences, which limits their performance in long-range forecasting scenarios.
In-context Learning In recent years, several studies have begun to explore the use of in-context learning (ICL) to extend the amount of historical context available to time series models. Auer et al. (Auer et al. 2025) explored the use of xLSTM to enhance statetracking capabilities in long sequences. Das et al. (Das et al. 2024b) proposed leveraging large-scale pretrained decoders to improve generalization in time series forecasting, while Lu et al. (Lu, Sun, and Yang 2025) formulated the problem by constructing a series of (lookback, future) pairs within the input tokens. However, these approaches primarily focus on increasing model capacity or expanding the scale of contextual inputs, without explicitly modeling the correspondence between lookback segments and their associated futures within historical data. As a result, historical information is often utilized as stacked context rather than as conditionally retrievable evidence for guiding current predictions.
Retrieval-augmented and Example-based Time-Series Forecasting A related line of work enhances time-series forecasting by retrieving similar historical patterns or using previous segments as contextual evidence. Such methods are motivated by the observation that many real-world time series contain recurring motifs, delayed responses, and repeated local dynamics. Compared with purely parametric forecasting models, retrieval-based methods can explicitly reuse historical examples and are often effective when similar patterns reappear over time. Our method shares the intuition that history should not be treated as a homogeneous sequence. However, instead of performing external nearest-neighbor search or retrieving only similar lookback windows, Trio organizes long histories into explicit lookback-future pairs. The future part of each historical pair is encoded as patch-wise prediction
prompts, and the current future-query tokens retrieve information from these prompts through learned sample attention. This allows the model to learn soft, task-dependent retrieval in the representation space, rather than relying on a fixed distance metric over raw windows.
Temporal Causal Priors and Synthetic Time-Series Generators Synthetic task generation is central to PFN-style learning. Existing time-series priors often rely on Gaussian-process kernels, linear coregionalization, VAR-like dynamics, or hand-crafted temporal patterns. These generators are useful for producing smooth or correlated sequences, but they do not always expose models to directed delayed mechanisms, state-dependent lags, feedback, and distributional changes. Recent temporal causal simulators and benchmarks study richer dynamical systems, including structural causal priors, learned causal simulators, and causal discovery benchmarks. For example, causal-discovery-oriented benchmarks focus on recovering underlying temporal graphs from generated trajectories, while learned simulators aim to match realdata distributions through data-driven simulator selection. Our goal is different: TS-SCM is designed as a forecastingoriented synthetic task prior. It directly constructs lookbackfuture episodes that match the input format of sample-level forecasting attention. Compared with prior synthetic time-series generators, TS-SCM emphasizes three mechanisms. First, delayed effects are defined at the edge level instead of through a global lag matrix. Second, dynamic lags are implemented by a push-based event calendar, where parent effects are scheduled to future arrival times. Third, the generated trajectory is converted into structured forecasting episodes, allowing the forecasting model to learn from historical input-output mappings.
Complexity Analysis Let B be the batch size, V the number of variables, S̄ = S + 1 the number of windows including the current window, P = Px + Py the number of tokens per window, and D the hidden dimension. A full attention operation over all variable, sample, and patch dimensions would require attention over V S̄P tokens, resulting in O B(V S̄P )2 D , (41) which is prohibitive for long-context forecasting. The proposed TSS encoder factorizes this operation into three axis-wise attention modules. The spatial module attends over variables: O B S̄P V 2 D . (42) The temporal module attends over patch tokens: O BV S̄P 2 D .
(43)
The sample module attends over historical windows for each variable and future-query position: O BV Py S 2 D . (44)
Thus, one TSS block has complexity O B S̄P V 2 D + BV S̄P 2 D + BV Py S 2 D .
(45)
This factorization avoids the quadratic cost of flattened full attention over all axes. Nevertheless, the sample module still scales quadratically with the number of historical windows S. In the current implementation, we keep S moderate and use the zero-gradient strategy to reduce memory usage. For much longer histories, top-k sample selection, memory compression, or low-rank sample attention could be incorporated to further reduce the S 2 cost. We leave these scalable retrieval variants for future work.
Compared with direct rounding, the proposed interpolation strategy better preserves the continuous temporal structure of the shifted parent signal. It avoids assigning a delayed event to an arbitrary nearest bin and instead distributes events according to the actual fractional trajectory. As a result, the generated child sequence can better reflect smooth delayed responses, phase shifts, and non-stationary temporal propagation patterns.
Illustrative Examples of TS-SCM Dynamics.
The corresponding parent value at position k is then obtained by linear interpolation:
To illustrate how TS-SCM constructs diverse synthetic forecasting tasks, we provide several hand-crafted examples of structural causal dynamics in Fig. 3. Each example consists of a small structural causal graph and the corresponding generated time series. The nodes represent different variables, while directed edges specify temporal causal dependencies with configurable delay, weight, and transformation functions. By changing these edge-level mechanisms, TS-SCM can generate diverse temporal patterns, including fixed-lag responses, variable-delay propagation, distribution drift, switching feedback, and multi-delay causal effects. In the fixed-lag example, the target variable receives a delayed signal from its parent variable with a constant time lag and edge weight. This setting simulates common delayedresponse patterns where the effect of one variable appears after a fixed number of time steps. The variable-delay example further allows the delay to change over time, producing non-stationary phase shifts between the parent and child variables. The data-drift example introduces changes in the underlying data-generating parameters, such as noise scale or distributional parameters, leading to non-stationary temporal behavior. The switching-signal example uses an additional control variable to switch between different feedback regimes, allowing the generated sequence to exhibit regimedependent dynamics. Finally, the multi-delay example combines multiple delayed causal paths between variables, producing richer temporal dependencies with both short-term and long-term effects. These examples show that TS-SCM does not rely on a single fixed synthetic pattern. Instead, it defines synthetic tasks through compositional structural mechanisms, where different temporal dependencies can be created by combining causal edges, time delays, feedback, switching signals, and distribution shifts. Such diversity is important for exposing the forecasting model to a broader range of temporal structures during synthetic pretraining.
x̂k = (1 − β)x0 + βx1 .
More Experimental Results
More information about TS-SCM Interpolation for Fractional Time-shifted Events In our synthetic causal time-series generation process, a child variable may receive a delayed or shifted signal from its parent variable. When the time shift is continuous, the shifted arrival position is not necessarily aligned with an integer time index. A naive implementation would round the arrival position to the nearest integer time step. However, such a rounding operation may introduce artificial discontinuities and temporal misalignment, especially when the delay changes over time. To avoid this issue, we use a fractional interpolation strategy to redistribute the parent signal onto integer time bins. Specifically, let the fractional arrival position at time step t be s(t) = t + dt , (46) where dt denotes the time-varying delay. For two consecutive parent observations, we record their fractional arrival positions and values as s0 = (t − 1) + dt−1 ,
x0 = xparent [t − 1],
(47)
x1 = xparent [t].
(48)
and s1 = t + dt ,
If the interval [s0 , s1 ] crosses one or more integer time bins, we assign events to these crossed integer positions. For each crossed integer position k, we compute its relative location between s0 and s1 : β=
k − s0 , s1 − s0
β ∈ [0, 1].
(49)
(50)
This interpolated value is used as the event value arriving at the integer time bin k. When s1 > s0 , the crossed integer positions are given by k ∈ {⌊s0 ⌋ + 1, . . . , ⌊s1 ⌋} .
(51)
When s1 < s0 , which can occur under abrupt changes in the time-varying delay, the same procedure is applied in the reverse direction. This ensures that all integer bins crossed by the fractional trajectory are considered.
Results on Industrial Datasets Table 8 reports the results on three industrial datasets. Compared with existing forecasting models, the proposed Trio model achieves the best MSE on all three datasets. It also obtains the best MAE on A and B. Compared with the base model, Trio reduces the MSE from 0.076 to 0.072 on A, from 0.134 to 0.117 on B, and from 0.068 to 0.066 on C. These results indicate that explicitly exploiting historical input–output correspondences is beneficial for real industrial
Figure 3: Illustrative examples of TS-SCM-generated structural causal dynamics. Each panel shows a hand-crafted structural causal graph together with the generated time series. By configuring edge-level delay, weight, feedback, switching signals, and distribution drift, TS-SCM can produce diverse temporal patterns such as fixed-lag responses, variable-delay propagation, data drift, switching feedback, and multi-delay effects. Table 8: Performance comparison (MSE and MAE) on A, B, and C datasets. Bold indicates the best result. A
Methods
B
C
MSE
MAE
MSE
MAE
MSE
MAE
TimesNet PatchTST iTransformer TimeXer PAttn
0.090 0.090 0.092 0.131 0.088
0.130 0.144 0.132 0.154 0.136
0.129 0.122 0.140 0.134 0.124
0.120 0.111 0.115 0.114 0.109
0.086 0.120 0.072 0.085 0.118
0.167 0.174 0.151 0.168 0.174
Baseline Trio
0.076 0.072
0.117 0.116
0.134 0.117
0.109 0.111
0.068 0.066
0.143 0.140
time series. This is consistent with the characteristics of industrial systems, where the current state often depends on delayed responses, long-range temporal patterns, and interactions among multiple variables. Although the MAE improvement is not always large, the consistent MSE reduction suggests that the proposed architecture is effective at reducing large prediction deviations.
Compared with the retrieval-based methods Table 9 compares Trio with GTR (Cao et al. 2026) and RAFT (Tire et al. 2024) on the overlapping benchmark datasets. Trio achieves the best MSE on three out of four datasets, including ETTm1, ETTm2, and Electricity, while
Table 9: Comparison with GTR and RAFT on overlapping benchmark datasets. Metric is MSE; lower is better. Dataset
Trio
GTR
RAFT
ETTm1 ETTm2 Electricity Weather
0.358 0.255 0.164 0.249
0.367 0.268 0.166 0.239
0.381 0.281 0.175 0.270
3
1
0
Count
GTR performs better on Weather. These results suggest that Trio is competitive with recent retrieval-augmented forecasting methods, especially on ETT and electricity forecasting benchmarks.
Full Results of Long-Context Ablation The averaged results are summarized in Tables 5 and 6 in the main text, while the complete horizon-wise results are reported here. Tables 10 and 11 correspond to the default input lengths of 96 and 336, respectively. For each setting, we vary the context length L and evaluate the model under prediction horizons H ∈ {96, 192, 336, 720}. As shown in Table 10, when the default input length is 96, increasing the context length generally improves the
Table 10: Full performance comparison across different context lengths (L) and prediction horizons (H) with the default input length of 96. Lower values indicate better performance Context Length (L)
ETTm1
ETTm2
Weather
ECL
ETTh1
Table 11: Full performance comparison across different context lengths (L) and prediction horizons (H) with the default input length of 336. Lower values indicate better performance.
ETTh2
MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE MSE MAE
Context Length (L) Prediction Horizon H = 96 96 1000 2000 3000 4000
0.313 0.289 0.289 0.312 0.294
0.350 0.336 0.341 0.349 0.350
0.173 0.159 0.156 0.155 0.156
96 1000 2000 3000 4000
0.366 0.344 0.339 0.337 0.333
0.379 0.368 0.371 0.372 0.374
0.240 0.232 0.219 0.214 0.212
0.251 0.246 0.249 0.244 0.247
0.157 0.149 0.154 0.156 0.161
0.198 0.193 0.203 0.204 0.209
0.142 0.133 0.134 0.139 0.141
MSE 0.235 0.227 0.231 0.235 0.238
0.386 0.374 0.388 0.484 –
0.396 0.287 0.334 0.393 0.281 0.334 0.398 0.279 0.336 0.403 – – – – –
Prediction Horizon H = 192 0.297 0.298 0.293 0.288 0.288
0.217 0.212 0.210 0.203 0.202
0.255 0.254 0.246 0.250 0.253
0.159 0.156 0.162 0.162 0.163
0.250 0.440 0.427 0.367 0.384 0.248 0.422 0.422 0.361 0.388 0.256 0.433 0.427 0.358 0.391 0.256 – – – – 0.256 –4 – – –
Prediction Horizon H = 336 96 1000 2000 3000 4000
0.399 0.371 0.377 0.368 0.365
0.399 0.389 0.402 0.393 0.392
0.300 0.291 0.285 0.273 0.272
96 1000 2000 3000 4000
0.469 0.419 0.428 0.418 0.469
0.438 0.419 0.428 0.425 0.457
0.409 0.370 0.360 0.384 0.425
0.334 0.337 0.340 0.330 0.329
0.275 0.270 0.281 0.272 0.261
0.296 0.293 0.305 0.301 0.294
0.173 0.172 0.177 0.173 0.180
0.265 0.266 0.276 0.271 0.276
0.478 0.466 0.476 0.484 0.498
0.446 0.418 0.425 0.440 0.380 0.407 0.447 0.400 0.429 0.457 – – 0.464 – –
Prediction Horizon H = 720 0.399 0.392 0.389 0.399 0.416
0.358 0.364 0.347 0.356 0.346
0.348 0.352 0.345 0.352 0.355
0.194 0.195 0.197 0.199 0.205
ETTm1
0.286 0.494 0.465 0.432 0.443 0.288 0.497 0.483 0.392 0.426 0.292 0.610 0.538 0.427 0.461 0.294 – – – – 0.301 – – – –
MAE
ETTm2 MSE
MAE
ETTh1 MSE
ETTh2
MAE
MSE
MAE
0.399 0.400 0.400 – –
0.280 0.281 0.289 – –
0.337 0.336 0.349 – –
0.429 0.439 0.429 – –
0.400 0.347 0.345 – –
0.406 0.382 0.385 – –
0.452 0.451 0.459 – –
0.448 0.377 0.402 – –
0.433 0.404 0.426 – –
0.468 0.491 0.635 – –
0.418 0.407 0.406 – –
0.438 0.441 0.445 – –
Prediction Horizon H = 96 336 2000 3000 4000 5000
0.299 0.298 0.278 0.286 0.289
0.349 0.344 0.335 0.338 0.338
0.164 0.162 0.161 0.157 0.157
0.250 0.254 0.253 0.247 0.251
0.386 0.380 0.381 – –
Prediction Horizon H = 192 336 2000 3000 4000 5000
0.351 0.323 0.325 0.329 0.336
0.385 0.364 0.365 0.364 0.367
0.276 0.227 0.219 0.221 0.222
0.315 0.298 0.294 0.293 0.293
0.426 0.439 0.428 – –
Prediction Horizon H = 336 336 2000 3000 4000 5000
0.378 0.374 0.367 0.366 0.388
0.390 0.396 0.389 0.395 0.399
0.307 0.283 0.277 0.276 0.286
0.341 0.338 0.337 0.334 0.336
0.464 0.465 0.471 – –
Prediction Horizon H = 720
performance compared with the default setting. However, the best context length varies across datasets and prediction horizons. For instance, moderate context lengths such as L = 1000 or L = 2000 often achieve strong performance on ETTm1, ECL, ETTh1, and ETTh2, while longer contexts are more beneficial in some ETTm2 and Weather settings. This suggests that historical information beyond the default input window can provide useful temporal cues, but the optimal context length is dataset-dependent. Table 11 further reports the results when the default input length is 336. Compared with the default setting, longer contexts also improve the average performance on ETTm1 and ETTm2, with L = 3000 achieving the best average results on these two datasets. Nevertheless, further increasing the context length to L = 4000 or L = 5000 does not consistently bring additional gains and may even degrade the performance. One possible reason is that longer input windows reduce the number of available training samples under a fixed-length time series, which may weaken the effective supervision during training. Moreover, excessively long contexts may introduce redundant or noisy historical patterns, making it harder for the model to focus on the most relevant temporal dependencies. Overall, these full results support the conclusion in the main text: long-context modeling is beneficial, but simply increasing the context length is not always optimal. A moderate context length can better balance useful historical information, training sample availability, and noise introduced by overly long inputs.
336 2000 3000 4000 5000
0.455 0.432 0.418 0.465 0.509
0.428 0.432 0.426 0.445 0.474
0.376 0.384 0.372 0.398 0.399
0.390 0.405 0.397 0.414 0.405
0.479 0.502 0.778 – –
Discussion on the Zero-Gradient Strategy The zero-gradient strategy is used only in dataset-specific supervised training with long historical contexts. In this setting, the prediction loss is defined on the current future sequence, while historical windows mainly serve as contextual evidence. Therefore, we process historical windows without gradient tracking and recompute the current window with gradients, which reduces memory consumption and stabilizes optimization. It is important to note that this strategy is not a core component of the proposed sample-attention mechanism, nor is it required by the TS-SCM generator. Instead, it is a practical training option for long-context supervised forecasting on specific datasets. When the model is trained or evaluated under other settings, such as synthetic-task training or exploratory zero-shot transfer, the use of zero-gradient processing can be disabled or adjusted according to the training objective. This strategy may limit the adaptation of historical representations during each update, and should therefore be viewed as an efficiency-oriented approximation rather than a theoretically optimal design. Empirically, Table 12 shows that it generally improves or maintains performance under
long-context supervised training. Exploring partial-gradient alternatives, such as lightweight adapters, memory compression, or curriculum-based gradient schedules for historical windows, is left for future work. Table 12 studies the zero-gradient strategy used for historical windows. This strategy treats historical windows mainly as conditional context and retains gradients only for the current prediction window. The results show that zero grad generally improves or maintains performance, especially when the context length becomes large. For example, on ETTm1 with context length 3000, the MSE decreases from 0.362 to 0.360. On ETTm2 with context length 3000, the MSE decreases from 0.265 to 0.262. These results suggest that the zero-gradient strategy can reduce optimization instability while still allowing the model to benefit from long historical information. Table 12: Ablation study on the optional zero grad strategy for dataset-specific long-context supervised training on ETTm1 and ETTm2. Context Length 1000 2000 3000 4000
ETTm1 w/o zero grad w zero grad MSE↓ MAE↓ MSE↓ MAE↓
ETTm2 w/o zero grad w zero grad MSE↓ MAE↓ MSE↓ MAE↓
0.371 0.372 0.362 0.377
0.272 0.277 0.265 0.272
0.405 0.407 0.405 0.415
0.372 0.367 0.360 0.368
0.405 0.403 0.405 0.409
0.333 0.339 0.332 0.342
0.271 0.265 0.262 0.279
0.331 0.331 0.329 0.339
Ablation on Quantile-based Context Selection. To analyze the effect of the proposed quantile-based context selection strategy, we compare it with an MSE-based selection strategy under the prediction length of 96. For each selection strategy, the baseline directly uses the selected historical contexts, while the Cross-Attention variant further integrates the selected contexts with the current input sequence through cross-attention. As shown in Table 13, the quantile-based strategy achieves consistent improvements after introducing crossattention on most datasets. For example, the MSE is reduced from 0.387 to 0.358 on ETTm1, from 0.281 to 0.255 on ETTm2, from 0.158 to 0.154 on ECL, from 0.450 to 0.440 on ETTh1, and from 0.376 to 0.353 on ETTh2. This demonstrates that quantile-based context selection can provide informative historical patterns, and cross-attention further helps the model exploit these selected contexts. Compared with the MSE-based selection strategy, the quantile-based strategy shows more stable performance. Although the MSE-based strategy can also benefit from crossattention on ETTm1, ETTm2, and Weather, its improvement is less consistent, and the performance on ECL slightly degrades after adding cross-attention. This suggests that directly selecting contexts according to point-wise MSE may overemphasize local numerical similarity, whereas quantilebased selection can better capture distribution-level temporal characteristics.
Visualization Figures 4–6 provide a qualitative comparison of models trained with different synthetic data generation strategies.
Table 13: Ablation study of quantile-based context selection and cross-attention under the prediction length of 96. Lower values indicate better performance. Selection
Quantile
MSE
Method
ETTm1
ETTm2
Weather
ECL
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Baseline
0.387
0.391
0.281
0.320
0.252
0.274
0.158
0.250
Cross-Attention
0.358
0.385
0.255
0.318
0.248
0.275
0.154
0.247
Baseline
0.376
0.396
0.280
0.335
0.272
0.293
0.176
0.272
Cross-Attention
0.367
0.403
0.264
0.327
0.249
0.288
0.180
0.279
Causal Dynamics mainly captures coarse global trends but often fails to reproduce local fluctuations and sharp turning points. LMC Synth produces more stable predictions, yet its outputs are generally over-smoothed and tend to miss highfrequency variations. In contrast, the model trained with our synthetic data better follows local temporal variations and preserves peak-valley structures, indicating stronger shapelevel generalization. Nevertheless, sharp extrema remain challenging, as the predictions are still relatively conservative around abrupt changes.
Limitations This work has several limitations. First, although the proposed framework is motivated by PFN-style learning, most main experiments are still conducted under supervised forecasting settings. The TS-SCM-based zero-shot results are preliminary and should be interpreted as evidence of transfer potential rather than a complete solution to general-purpose time-series forecasting. Second, the zero-gradient strategy for historical windows is an empirical optimization design. While it reduces memory cost and stabilizes training, it may also limit the adaptation of historical representations. More flexible strategies, such as partial gradients, adapters, or curriculum-based historical updates, deserve further investigation. Third, the current TS-SCM generator is hand-designed. Although it supports dynamic lags, event calendars, drift, and heterogeneous node mechanisms, its realism is not fully validated against real-world time-series distributions. Future work should include stronger realism diagnostics, broader comparisons with temporal causal priors, and learned simulator components. Finally, the current evaluation does not fully explore all design choices, such as different future-query lengths, nearest-neighbor retrieval baselines, and interpretability of retrieved historical windows. These analyses are important for understanding when and why sample-level attention is most effective.
Figure 4: Qualitative results of the model trained with Causal Dynamics synthetic data. The blue curves denote the ground-truth sequences, and the orange dashed curves denote model predictions.
Figure 5: Qualitative results of the model trained with LMC Synth synthetic data. The blue curves denote the ground-truth sequences, and the orange dashed curves denote model predictions.
Figure 6: Qualitative results of the model trained with our synthetic data. The blue curves denote the ground-truth sequences, and the orange dashed curves denote model predictions.