Causal Discovery with Inverted Self-attention for Multivariate Time Series Yusen Liu1 , Yong Wang3 , Yifan Yin1 , Tianqing Zhu4 , Xiufeng Liu2 ( Huan Huo1 ( )
), and
School of Computer Science, University of Technology Sydney, Sydney, Australia {Yusen.Liu, Yifan.Yin, Huan.Huo}@uts.edu.au 2 Department of Technology, Management and Economics, Technical University of Denmark, Lyngby, Denmark [email protected] 3 TianFu YongXing Laboratory, Chengdu, China [email protected] 4 Faculty of Data Science, City University of Macau, Macau, China [email protected]
arXiv:2607.28212v1 [cs.CL] 30 Jul 2026
1
Abstract. Causal discovery in multivariate time series data is challenging due to complex interactions, high dimensionality, and nonlinear dependencies among variables. Existing methods often struggle to capture these complexities, resulting in inaccurate causal structures. To address this issue, we propose a novel framework that leverages self-attention mechanisms within the transformer architecture for causal discovery. Our approach introduces a novel inverted causal self-attention mechanism (CSAM) that emphasizes latent and indirect causal relationships by inverting tokens and inducing sparsity in attention scores, focusing on significant causal interactions and reducing spurious correlations. Additionally, we develop a global causal algorithm to identify global causal links, providing a holistic metric for causal influence, along with a causal verification module to ensure robustness in the identified causal relationships, enhancing the reliability of our framework. Experiments on both linear and nonlinear datasets, along with ablation studies and sensitivity analyses, show that our framework outperforms existing methods, demonstrating its potential for causal discovery in complex multivariate time series. Keywords: Causal discovery · Multivariate time series · Attention mechanisms
1
Introduction
Time series analysis is widely applied in fields like economics, environmental science, and healthcare. The temporal dependencies and dynamic nature of time series data create unique challenges, requiring advanced analytical techniques for tasks such as classification and forecasting. Beyond these tasks, understanding causal relationships between variables in multivariate time series holds particular
2
Liu et al.
importance, as it not only provides insights into underlying mechanisms but also improves forecasting and classification performance, supporting more informed decision-making across various domains. However, causal discovery in time series data remains challenging due to the complex interactions between variables over time. A primary difficulty is distinguishing causality from correlation, as temporal dependencies can obscure true causal structures, leading to potential misinterpretations [14]. This challenge is further intensified by the high dimensionality and potential nonlinearity present in multivariate data [13]. Traditional methods often struggle to handle these complexities, which can limit research and applications; inaccurate causal links may result in flawed conclusions and less effective interventions, impacting decision-making in critical areas [17]. Existing methods for causal discovery in time series data, including distancebased [16], index-based [22], and traditional statistical methods [4], primarily capture linear relationships but are often sensitive to noise and may confuse causation with correlation, limiting their effectiveness in dynamic multivariate contexts [2]. Kernel-based approaches, such as Kernel Granger Causality (KGC) [8], extend causal discovery to nonlinear relationships. Constraint-based methods, including Time Series Fast Causal Inference (tsFCI) [3] and PCMCI [15], detect causal links even with latent confounders. Score-based methods, such as NOTEARS [24] and DYNOTEARS [11], offer additional tools for uncovering causal structures in multivariate time series, addressing some limitations of traditional methods. Despite these advances, there has been limited exploration of deep learning’s potential for causal discovery [10], especially given its capacity to model complex, high-dimensional data. To further address high dimensionality and nonlinearity challenges in multivariate time series data, we propose a novel framework that leverages an inverted self-attention mechanism for causal discovery. First, we propose an inverted causal self-attention mechanism (CSAM) that highlights latent and indirect causal relationships. Specifically, we apply Sparsemax in the attention score matrix, inducing sparsity to focus on significant causal interactions and reduce spurious correlations. Then, we develop a global algorithm to integrate attention scores across model components, providing a metric for causal influence that captures complex causal structures in time series data. This approach enables the model to handle nonlinear interactions while remaining robust to noise and confounders, improving interpretability and reducing false positives. Finally, we integrate Permutation Importance (PI) [12] to verify that identified causal relationships are robust, enhancing the reliability of our causal discovery pipeline. In summary, the main contributions of this paper are: We propose a novel framework that leverages an inverted causal self-attention mechanism for causal discovery, enabling the capture of complex, nonlinear causal relationships in multivariate time series. − We introduce a sparsity-inducing modification to the attention score matrix, improving interpretability and reducing spurious correlations. −
Causal Discovery with Inverted Self-attention for Multivariate Time Series
3
We develop a global causal algorithm to identify global causal links, along with a causal verification module to ensure robustness in the identified relationships. − We validate our framework through experiments on high-dimensional and complex time series data, with ablation studies and sensitivity analyses demonstrating the contribution of each component. −
The structure of this paper is as follows: Section 2 reviews related work on causal discovery in time series and attention mechanisms. Section 3 introduces the definitions and assumptions of this paper. Section 4 details our proposed methodology. Section 5 presents the experiments and results. Finally, Section 6 concludes the paper and suggests future research directions.
2
Related Work
Causal Discovery for Time Series. Causal discovery in time series data aims to identify causal relationships, with Granger causality as a foundational method. It posits that if past values of a time series Y improve the prediction of another series X, then Y Granger-causes X [4]. Building on this, KGC [8] extends Granger’s framework to nonlinear relationships via kernel methods. Distancebased methods [16] also capture nonlinear relationships but are sensitive to noise and require careful parameter tuning. Additionally, constraint-based approaches like tsFCI [3] detect causal links even with latent confounders. PCMCI [15] combines the PC algorithm [18] with momentary conditional independence tests for efficient causal structure identification. Score-based methods, such as NOTEARS [24] and DYNOTEARS [11], use continuous optimization to capture temporal and nonlinear dependencies. Further, [19,10] leverage neural networks to infer causality directly. Together, these methods offer a range of tools for analyzing causal relationships in time series. Attention Mechanisms. Attention mechanisms enable models to focus on relevant data segments and are widely used in fields like NLP and computer vision. First introduced by [1] for machine translation, attention improved both model performance and interpretability. The Transformer architecture by [20], relying entirely on attention, set new benchmarks in NLP. In time series analysis, attention mechanisms have shown strong potential in models like Informer [25] and Crossformer [23]. Attention-based models have also been applied in causal discovery, with [10] incorporating attention into convolutional neural networks to improve causal effect estimation. In contrast to these approaches, we propose a novel CSAM to identify latent and nonlinear causal relationships in multivariate time series data, facilitating a deeper understanding of dynamic interactions.
3
Definitions and Assumptions
Definition 1. (Multivariate Time Series) A multivariate time series is defined as an M -dimensional series X = (X1 , X2 , . . . , XM ) of length T , where each component Xi represents an individual time series.
4
Liu et al.
Definition 2. (Causal Graph) Given a multivariate time series X, its causal graph is a directed acyclic graph (DAG) G = (V, E), where V = {1, 2, . . . , M } represents the set of variables (time series), and E is the set of directed edges. An edge (i, j) ∈ E exists if and only if Xi Granger-causes Xj . Assumption 1. (Causal Markov) The causal Markov condition states that each variable is conditionally independent of non-effects given its direct causes, focusing the analysis on direct causal dependencies. Assumption 2. (Faithfulness) Faithfulness assumes that any conditional independencies in the data reflect the true causal structure, ensuring that observed independencies correspond to actual causal relationships. Assumption 3. (Strict Causal Ordering) Strict temporal ordering in the causal graph implies that if Xi causes Xj , then i < j, disallowing contemporaneous causation and simplifying temporal dependency analysis. Assumption 4. (Bounded Dependencies) For each edge (i, j) ∈ E, a constant ϵ > 0 exists such that the minimum absolute difference between the prediction error of Xj with and without Xi as a predictor exceeds ϵ for sufficiently large T , ensuring a minimum "signal strength" for true causal dependencies.
4
Methods
Following our proposed definitions and assumptions, we introduce a novel framework that systematically addresses the identification of Granger causality networks in multivariate time series X. The model aims to delineate causal interactions among series components through matrix A. The full time series Xt is input to a causal self-attention module (CSAM), which differs from traditional self-attention in token handling and internal dynamics. This module’s output feeds into a transformer, generating predictions for each series Xit and identifying causal links with other series during training. Granger causality is then derived from Xt through a global algorithm and verification strategy, as detailed in Sections 4.2 and 4.3. 4.1
Causal Self-attention Mechanism
Transformers effectively capture complex dependencies in sequential data through self-attention mechanisms. To leverage this for causal discovery, we introduce an inverted self-attention mechanism, CSAM, designed to identify causal relationships in multivariate time series data. Traditional self-attention computes attention scores based on the affinity between positions in the input sequence, formulated as: QK T Attention(Q, K, V ) = softmax √ V (1) dk
…
5
Variables
Inverted Tokenization
Multivariate
Time
…
Traditional Tokenization
Time
(a) Traditional Token
Time steps
Variables
Causal Discovery with Inverted Self-attention for Multivariate Time Series
(b) Inverted Token
Fig. 1: Comparison of token representations in (a) traditional self-attention and (b) the proposed causal self-attention mechanism. where Q, K, and V represent the matrices for queries, keys, and values, respectively, and dk is the dimensionality of the keys. In multivariate time series analysis, each token X = {x1 , x2 , x3 , . . .} represents a combination of different variables, as shown in Fig. 1(a). Each token corresponds to a single time point, and a sequence of tokens {X1 , X2 , X3 , . . .} represents consecutive time points. Using these tokens in the self-attention mechanism allows for the computation of inter-temporal relationships. Contrastingly, in Granger causality analysis, the focus is on identifying relationships among different time series rather than across time intervals. To achieve this, we invert the token representation. As shown in Fig. 1(b), each inverted token X̂ consists of sequential observations within a single time series, X̂ = {x1 , x2 , x3 , . . .}, representing one complete time series. A collection of these inverted tokens, {X̂1 , X̂2 , X̂3 , . . .}, represents multiple time series. Using these inverted tokens in the self-attention framework allows for the analysis of crossseries relationships, aligning with Granger causality objectives. This approach maintains the temporal sequence within each series while enabling a detailed exploration of causal dynamics across the multivariate time series. After transposing the tokens, we apply the CSAM to reveal Granger causality among the time series, as illustrated in Fig. 2. The inverted tokens are initially transformed through multiplication with the weight matrices W k̂ , W q̂ , and W v̂ , producing the key (K̂), query (Q̂), and value (V̂ ) matrices: K̂ = X̂ · W k̂ ,
Q̂ = X̂ · W q̂ ,
V̂ = X̂ · W v̂
(2)
It is imperative to note that, diverging from the conventional dimensions observed in standard transformers, which are typically indexed by K ∈ RN ×dk , Q ∈ RN ×dq , V ∈ RN ×dv , the dimensions of our key (K̂), query (Q̂), and value (V̂ ) matrices are structured as K̂ ∈ RT ×dk , Q̂ ∈ RT ×dq , V̂ ∈ RT ×dv , Where N stands for the number of variables and T stands for the time steps. Focusing on a specific time series designated by i, we extract the i-th row from the Q̂ matrix and engage in a matrix multiplication with the transposed KˆT matrix. This operation is instrumental in deriving the initial attention scores ei that encapsulate potential causal influences exerted by all time series on the series of
6
Liu et al.
interest i: ei =
Q̂i × K̂ T √ dk
(3)
To enhance the interpretability of the resultant weight vector, we apply the SparseMax function [9]. This transformation imparts sparsity to the vector, thereby facilitating a more straightforward interpretation by accentuating the most salient causal relationships and diminishing the noise from negligible interactions. The SparseMax function is a differentiable alternative to the softmax function, designed to provide sparser probabilities. Formally, the SparseMax function for a vector z ∈ Rd is defined as the solution to the following optimization problem: 2 (4) SparseMax(z) = argmin ∥p − z∥ p∈∆d−1
where ∆d−1 denotes the (d − 1)-dimensional simplex, i.e., the set of p ∈ Rd such Pd that i=1 pi = 1 and pi ≥ 0 for all i. The SparseMax operation projects the input vector z onto the simplex, resulting in a sparse probability distribution where many elements can be exactly zero. αi = SparseMax (ei )
(5)
After applying SparseMax, the non-zero elements in the sparse vectors αi are added to the causal tensor Cijt as potential causes of the i-variables, where t denotes different tokens. Additionally, αi is duplicated across the dv dimension to form an attention map Ai for the CSAM. The attention map represents the refined causal influence structure, with each element indicating the degree of influence one time series has on another within the multivariate context. Ai = Replicate(αi , dv ) = αi αi · · · αi (6) The final stage performs an element-wise multiplication (Hadamard product) between the attention map Ai and the V matrix, encapsulating the weighted significance of each value element based on the derived causal relationships. This product is then fed into a fully connected layer to produce the final processed token X̂ ′ , which serves as input to the transformer model. X̂ ′ = FC(Ai ⊙ V) 4.2
(7)
Global Causal Algorithm
Algorithm 1 presents the global causal algorithm, which systematically identifies potential causal links in the dataset based on causal strengths derived from the attention mechanism. This approach aggregates causal information from each token during training to output a set of potential causes P. Specifically, the attention mechanism is applied to each token during training, generating a causal tensor Cijt , where t represents different tokens. The algorithm then calculates
Causal Discovery with Inverted Self-attention for Multivariate Time Series
7
𝑲 𝜶𝒊
Attention-size
SparseMax
Time steps
Attention-size
𝑾𝑸
Duplicate
𝑸
Inverted Token
Processed Token
Attention Map
𝑽
𝑭𝑪
☉
Attention-size
𝑾𝑽
Causality Verification & Aggregation
Time steps
𝑾𝑲
Fig. 2: Illustration of the process of our causal self-attention mechanism (CSAM). the causal strengths Hij between each pair of time series i and j by summing Cijt across all tokens: X Hij = Ctij (8) t
Next, the algorithm sorts the causal strengths Hij in descending order and selects a threshold τ based on a predefined parameter k, where τ is set as the k-th largest value in the sorted list. τ = sorted(Hij )[k] (9) We select the threshold τ following the approach in [10]. If the causal strength Hij between any two time series i and j meets or exceeds τ , the algorithm infers a causal relationship from i to j and includes it in the set of potential causes P: P = {(i → j) | Hij ≥ τ }
4.3
(10)
Causal Verification
Our methodology uses Permutation Importance (PI) [12] to validate causal relationships. PI quantifies the effect of variable reordering on prediction error by shuffling the values of a candidate cause variable, disrupting potential causal pathways and assessing the significance of the temporal structure. Given a neural network Nj trained on the complete dataset to predict the target time series Xj , we denote the baseline loss on the original data as LG . For each potential causal factor Xi ∈ P, we create a permuted dataset by randomly rearranging the values of Xi while keeping the rest of the data unchanged. This permutation preserves the original data distribution, allowing us to re-evaluate the prediction for Xj without retraining the network. The perturbed dataset
8
Liu et al.
Algorithm 1: Global Causal Algorithm Data: Causality tensor C, order parameter k Result: potential causes set P 1 Function GlobalCausality(C, k): 2 for i = 1 to N do 3 for j = 1 to PN do 4 Hij ← t Ctij ;
11
hs ← sorted({Hij }); τ ← hs[k]; P ← ∅; for i = 1 to N do for j = 1 to N do if Hij ≥ τ then P ← P ∪ {(i → j)};
12
return P;
5 6 7 8 9 10
produces a new loss metric LI . A significant increase in loss from LG to LI suggests a strong causal influence of Xi on Xj . If the increase is minor, it indicates that the temporal ordering of Xi is not crucial for predicting Xj , questioning its causal role. The set of validated causes Cj for each time series is determined through this process, forming the basis for constructing the Granger causality diagram G(V, E), where V represents the time series nodes and E the directed edges validated by the causal sets.
5
Experiments
5.1
Experimental Setup
Our implementation was based on Python 3.8.18 and PyTorch 1.12.1, with experiments conducted on a server equipped with two NVIDIA A40 GPUs. We employed Mean Squared Error as the loss function and utilized the Adam optimizer with a learning rate of 0.001, modified by a scheduler with a step size of 10 and gamma of 0.98. Hyper parameters were set through preliminary tuning, with a batch size of 32 for 500 epochs. To reduce overfitting, dropout was applied. Input and output window sizes were configured to 10 and 1, respectively, while both causal attention and transformer modules employed 8 heads and 3 layers. Datasets Our experiments used linear and non-linear multivariate time series datasets, as summarized below. Linear Datasets: 1) Hénon Maps: Chaotic time series data from 6 interconnected Hénon maps, totaling 2,048 data points for training [6]. 2) FINANCE: A 25-variable financial time series dataset [5], utilized for evaluating causal discovery algorithms. Non-linear Datasets: 1) Lorenz96 Model: A 10-variable atmospheric dynamics model generating 2,048 samples [7]. 2) fMRI: Simulated BOLD signals derived from dynamic causal modeling for fMRI data, with 1,200 samples for analysis [21].
Causal Discovery with Inverted Self-attention for Multivariate Time Series
9
Table 1: Performance comparison of the proposed method and baselines. Henon Method
Prec.
Rec.
Finance F1
Prec.
Lorenz-96
Rec.
F1
Prec.
Rec.
fMRI F1
Prec.
Rec.
F1
BGGranger 0.238 0.455 0.312 0.041 0.108 0.059 0.295 0.307 0.283 0.250 0.140 0.182 KGC 0.412 0.636 0.500 0.223 0.208 0.215 0.466 0.406 0.548 0.437 0.333 0.378 tsFCI 0.643 0.818 0.720 0.424 0.301 0.356 0.583 0.389 0.467 0.474 0.429 0.450 TCDF 0.875 0.636 0.737 0.521 0.307 0.387 0.735 0.694 0.714 0.800 0.762 0.780 PCMCI 0.889 0.727 0.799 0.624 0.501 0.556 0.703 0.722 0.712 0.704 0.904 0.792 DYNOTEARS 0.845 0.677 0.752 0.653 0.364 0.467 0.699 0.697 0.698 0.547 0.442 0.489 Ours 0.900 0.818 0.857 0.673 0.505 0.586 0.862 0.684 0.769 0.818 0.857 0.837
Cause
Cause TCDF
Effect
Effect
Effect
PCMCI
Cause
Ours
Effect
Cause tsFCI
Cause
KGC
Effect
BGranger
Effect
Effect
Ground Truth
Cause
Cause
Fig. 3: Comparison of causality matrices from various methods on the fMRI dataset, where dark blue indicates true causality and red highlights errors. Baseline Methods We compared our method with six standard models in Granger causal discovery: BGranger [4], KGC [8], tsFCI [3], TCDF [10], PCMCI [15], and DYNOTEARS [11]. These methods represent diverse approaches to causal discovery. BGranger provides a foundational benchmark by testing if past values of one variable improve predictions of another. Constraintbased methods like tsFCI and PCMCI use conditional independence tests to infer causal structures. KGC, a kernel-based method, extends Granger causality to non-linear relationships. TCDF enhances inference with attention mechanisms, and DYNOTEARS, an optimization-based method, applies continuous optimization for dynamic causal discovery.
Evaluation Metrics To evaluate performance, we use Precision (P ), Recall (R), and F1-Score (F1 ) to measure accuracy. Precision represents the fraction of correctly identified causal relationships among those inferred, Recall indicates the fraction of actual causal relationships detected, and F1-Score provides a balanced measure between Precision and Recall.
Liu et al.
1.0 0.8
0.4 0.2
H enon
Finance
Lorenz-96 fMRI
Ours-full Ours-n2n Ours-soft
0.8 0.6
0.6
0.6
0.0
1.0
Ours-full Ours-n2n Ours-soft
0.8
Recall
Precision
1.0
Ours-full Ours-n2n Ours-soft
F1
10
0.4
0.4
0.2
0.2
0.0
(a) Precision
Henon
Finance
Lorenz-96
(b) Recall
fMRI
0.0
Henon
Finance
Lorenz-96
fMRI
(c) F1-Score
Fig. 4: Ablation study results across datasets. 5.2
Overall Performance
The quantitative results in Table 1 demonstrate the effectiveness of our causal discovery framework across different datasets. Our method consistently achieves high precision scores, from 0.673 for the FINANCE dataset to 0.900 for the Hénon dataset, indicating its strong ability to minimize false-positive causal inferences. Furthermore, our approach maintains competitive recall scores, showing its capacity to capture a significant portion of the ground-truth causal relationships. Notably, our method outperforms existing state-of-the-art techniques on the non-linear Lorenz-96 and fMRI datasets, achieving the highest F1-scores of 0.769 and 0.837, respectively. This performance underscores the robustness of our framework in uncovering complex causal structures within non-linear systems, a key requirement for real-world applications. The visual comparison in Fig. 3 further supports the quantitative results. The inferred causality matrix from our method closely matches the ground truth, with minimal false positives and negatives. This accuracy is due to the combined effects of the proposed CSAM, the sparsity-inducing modifications, and the global integration algorithm. CSAM captures complex, non-linear causal dependencies by leveraging self-attention mechanisms within the transformer architecture. The sparsity-inducing modifications enhance the reliability of identified causal relationships by reducing spurious correlations and noise. Finally, the global integration algorithm provides a holistic metric for causal influence, facilitating the discovery of both direct and indirect causal links while accounting for temporal dynamics and inter-dependencies among variables. 5.3
Ablation Study
To evaluate the contributions of our model’s components, we conducted an ablation study focusing on the CSAM and sparsity-inducing mechanism. Our study compares the performance of the complete model (Ours-full) with two variants: Ours-n2n and Ours-soft. The Ours-n2n variant, which predicts all elements of the multivariate time series simultaneously, shows reduced precision compared to Ours-full, indicating that parallel prediction disperses focus and weakens causal detection, as shown in Fig. 4a. The Ours-soft variant, replacing
Causal Discovery with Inverted Self-attention for Multivariate Time Series
11
SparseMax with Softmax for attention scoring, also shows a precision drop across datasets, as Softmax fails to create the desired sparsity, reducing specificity in causal inference. As seen in Fig. 4b, these variants affect recall as well. While Ours-n2n performs similarly to Ours-full in some datasets, it underperforms in complex cases like Lorenz-96 and fMRI. Ours-soft also exhibits inconsistencies, especially in datasets where sparsity is key to accurate causal detection. The F1-scores in Fig. 4c confirm that Ours-full outperforms the variants, demonstrating the importance of both CSAM and sparsity-inducing mechanisms. 5.4 Sensitivity Analysis The CSAM, rooted in the transformer Table 2: Sensitivity analysis. framework, inherently relies on multiple atParameters No. Prec. Rec. F1 tention heads and blocks, both of which 2 0.783 0.698 0.738 are crucial hyper-parameters in the model’s 4 0.769 0.802 0.785 architecture. In our parameter analysis, Attention Head 8 0.818 0.857 0.837 the number of Attention Heads was varied 16 0.747 0.737 0.742 among 2, 4, 8, and 16, with 8 set as the de1 0.779 0.796 0.788 fault value. Similarly, the number of Atten2 0.764 0.834 0.814 Attention Block 3 0.818 0.857 0.837 tion Blocks was adjusted to 1, 2, 3, and 4, 4 0.799 0.854 0.826 with 3 being the default. Number of Attention Heads: In the CSAM, the number of attention heads controls how many attention mechanisms are applied to the input data. Each head processes the sequence separately, allowing the model to capture various dependencies. As shown in Table 2, increasing attention heads to 16 helps capture more complex dependencies but may cause over-fitting. Conversely, reducing heads to 2 simplifies the model, potentially losing important information. Number of Attention Blocks: Attention blocks determine the depth of attention applied across the time series data, with each block adding a layer of processing to refine the model’s understanding. Table 2 shows that increasing blocks to 4 enhances the model’s ability to capture complex dependencies but risks over-fitting, while reducing blocks to 1 may fail to capture sufficient details.
6
Conclusion and Future Work
This study introduces a novel framework for causal discovery in multivariate time series data. Our proposed inverted causal self-attention mechanism (CSAM) provides a unique token representation to analyze causal relationships. By integrating a global causal algorithm with causal verification, the framework effectively handles spurious correlations. Experiments on four benchmarks demonstrate that our method outperforms existing approaches. For future research, we aim to extend the method to complex datasets across different domains and explore approaches focused on identifying true causal frequencies and causal lags. Acknowledgments. This work was supported by the China Scholarships Council (Grant No. 202208410132), and the Tianfu Yongxing Laboratory Organized Research Project Funding (No. 2023CXXM14).
12
Liu et al.
References 1. Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. ICLR (2015) 2. Ebert-Uphoff, I., Deng, Y.: Causal discovery for climate research using graphical models. Journal of Climate 25(17), 5648–5665 (2012) 3. Entner, D., Hoyer, P.O.: On causal discovery from time series data using fci. Probabilistic graphical models pp. 121–128 (2010) 4. Granger, C.W.: Investigating causal relations by econometric models and crossspectral methods. Econometrica: journal of the Econometric Society pp. 424–438 (1969) 5. Kleinberg, S.: Causality, probability, and time. Cambridge University Press (2013) 6. Kugiumtzis, D.: Direct-coupling information measure from nonuniform embedding. Physical Review E 87(6), 062918 (2013) 7. Lorenz, E.N.: Predictability: A problem partly solved. In: Proc. of Seminar on predictability. vol. 1. Reading (1996) 8. Marinazzo, D., Pellicoro, M., Stramaglia, S.: Kernel method for nonlinear granger causality. Physical review letters 100(14), 144103 (2008) 9. Martins, A., Astudillo, R.: From softmax to sparsemax: A sparse model of attention and multi-label classification. In: ICML. pp. 1614–1623. PMLR (2016) 10. Nauta, M., Bucur, D., Seifert, C.: Causal discovery with attention-based convolutional neural networks. Machine Learning and Knowledge Extraction 1(1) (2019) 11. Pamfil, R., Sriwattanaworachai, N., Desai, S., Pilgerstorfer, P., Georgatzis, K., Beaumont, P., Aragam, B.: Dynotears: Structure learning from time-series data. In: AISTATS. pp. 1595–1605. Pmlr (2020) 12. Pereira, J.P., Stroes, E.S., Zwinderman, A.H., Levin, E.: Covered information disentanglement: model transparency via unbiased permutation importance. In: Proc. of the AAAI Conference on Artificial Intelligence. vol. 36, pp. 7984–7992 (2022) 13. Runge, J.: Causal network reconstruction from time series: From theoretical assumptions to practical estimation. Chaos: An Interdisciplinary Journal of Nonlinear Science 28(7) (2018) 14. Runge, J., Bathiany, S., Bollt, E., Camps-Valls, G., Coumou, D., Deyle, E., Glymour, C., Kretschmer, M., Mahecha, M.D., Muñoz-Marí, J., et al.: Inferring causation from time series in earth system sciences. Nature communications 10(1), 2553 (2019) 15. Runge, J., Nowack, P., Kretschmer, M., Flaxman, S., Sejdinovic, D.: Detecting and quantifying causal associations in large nonlinear time series datasets. Science advances 5(11), eaau4996 (2019) 16. Schreiber, T.: Measuring information transfer. Physical review letters 85(2) (2000) 17. Shih, S.Y., Sun, F.K., Lee, H.y.: Temporal pattern attention for multivariate time series forecasting. Machine Learning 108, 1421–1441 (2019) 18. Spirtes, P., Glymour, C.: An algorithm for fast recovery of sparse causal graphs. Social science computer review 9(1), 62–72 (1991) 19. Tank, A., Covert, I., Foti, N., Shojaie, A., Fox, E.B.: Neural granger causality. IEEE TPAMI 44(8), 4267–4279 (2021) 20. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. NeurIPS 30 (2017) 21. Yan, Y., Dahmani, L., Ren, J., Shen, L., Peng, X., Wang, R., He, C., Jiang, C., Gong, C., Tian, Y., et al.: Reconstructing lost bold signal in individual participants using deep machine learning. Nature communications 11(1), 5046 (2020)
Causal Discovery with Inverted Self-attention for Multivariate Time Series
13
22. Zema, S.M.: Directed acyclic graph based information shares for price discovery. Journal of Economic Dynamics and Control 139, 104434 (2022) 23. Zhang, Y., Yan, J.: Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In: ICML (2022) 24. Zheng, X., Aragam, B., Ravikumar, P.K., Xing, E.P.: Dags with no tears: Continuous optimization for structure learning. NeurIPS 31 (2018) 25. Zhou, H., Zhang, S., Peng, J., Zhang, S., Li, J., Xiong, H., Zhang, W.: Informer: Beyond efficient transformer for long sequence time-series forecasting. In: Proc. of the AAAI conference on artificial intelligence. vol. 35, pp. 11106–11115 (2021)